Cloudron makes it easy to run web apps like WordPress, Nextcloud, GitLab on your server. Find out more or install now.


Skip to content
  • Categories
  • Recent
  • Tags
  • Popular
  • Bookmarks
  • Search
Skins
  • Light
  • Cerulean
  • Cosmo
  • Flatly
  • Journal
  • Litera
  • Lumen
  • Lux
  • Materia
  • Minty
  • Morph
  • Pulse
  • Sandstone
  • Simplex
  • Sketchy
  • Spacelab
  • United
  • Yeti
  • Zephyr
  • Dark
  • Cyborg
  • Darkly
  • Quartz
  • Slate
  • Solar
  • Superhero
  • Vapor

  • Default (No Skin)
  • No Skin
Collapse
Brand Logo

Cloudron Forum

Apps | Demo | Docs | Install
  1. Cloudron Forum
  2. Support
  3. Nextcloud Backup without Data

Nextcloud Backup without Data

Scheduled Pinned Locked Moved Support
backupscifshetznerhardlinks
24 Posts 5 Posters 4.1k Views 5 Watching
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • nebulonN Offline
    nebulonN Offline
    nebulon
    Staff
    wrote on last edited by
    #3

    This is indeed often an issue. In your case however you should be able to use hardlinks if you mount the storage box as described at https://cloudron.io/documentation/backups/#cifs

    Generally it is hard to detangle the app's data folder from the other parts (mostly the database backups) as things then risk to go out of sync on a restore and it is hard to predict how apps like Nextcloud handle such cases.

    1 Reply Last reply
    0
    • M Online
      M Online
      msbt
      App Dev
      wrote on last edited by msbt
      #4

      @nebulon when I asked hetzner about a year ago about hardlinks on storage boxes they said:

      auf unseren Storage Boxes kommt ZFS zum Einsatz.
      Hardlinks werden aus Sicherheitsgründen leider nicht unterstützt. Eventuell könnten Sie das Problem der Deduplizierung auch über BorgBackup lösen: https://wiki.hetzner.de/index.php/BorgBackup
      Unsere Storage Boxes unterstützen BorgBackup rsync nativ.
      

      Are you sure those are working now?

      1 Reply Last reply
      0
      • nebulonN Offline
        nebulonN Offline
        nebulon
        Staff
        wrote on last edited by
        #5

        Interesting reply, I am actually using such a storage box myself with samba and hardlinks on Cloudron. So far this has worked fine. I also just did a test hardlink manually on that samba mount and the hardlinks do persist across remounts. What I don't know is if they might actually somewhat duplicate data on the disk itself, but I think in such a case my box would be filled up by now easily.

        Interesting though, I was not aware that they support borgbackup natively.

        Can you maybe create some manual hardlinks on your storage box and check how it behaves for you?

        1 Reply Last reply
        1
        • M Online
          M Online
          msbt
          App Dev
          wrote on last edited by
          #6

          I think when I tried that last year, the first backup went through (as well as manually creating hardlinks), but the subsequent ones were throwing errors. I thought it was due to the amount of data (40gb+ nextcloud instance), but the support said otherwise. When I have time I'll try again and let you know 🙂

          1 Reply Last reply
          1
          • I Offline
            I Offline
            iqweb
            wrote on last edited by iqweb
            #7

            I also asked hetzner support about hardlinks & they said:

            It seems that Samba is transferring the Hard Links correct, but not always showing them correct.
            You could use rsync directly, in order to back up your data to the Storage Box.
            For example:
            -----------------%<-----------------
            rsync --progress -H -e 'ssh -p23' --recursive <local_directory> <username>@<username>.your-storagebox.de:<target_directory>
            -----------------%<-----------------
            

            Also - like msbt - it was working in the beginning & then started giving errors about hardlinks.

            jdaviescoates - Do you mean the borgbackup to Hetzner storagebox - or the Synology pull?

            nebulon - Is there a way to do a cloudron app only -minus nc data backup? I'm only able to keep 2 days of native cloudron backups at the moment & would very much prefer more than that!

            jdaviescoatesJ 1 Reply Last reply
            0
            • girishG Offline
              girishG Offline
              girish
              Staff
              wrote on last edited by
              #8

              @iqweb @msbt Can one of you give me the hard link error you got? I would like to get this fixed.

              AFAIK, CIFS supported hard links since a long time.

              # cd /path/to/storagebox
              # touch foo
              # ln foo bar
              # ln foo baz
              # ls -l foo bar baz 
              

              What happens when you run above?

              1 Reply Last reply
              0
              • M Online
                M Online
                msbt
                App Dev
                wrote on last edited by
                #9
                # cd /var/storagebox/test
                # touch foo
                # ln foo bar
                # ln foo baz
                # ls -l foo bar baz
                -rw-rw---- 3 yellowtent yellowtent 0 May 11 20:44 bar
                -rw-rw---- 3 yellowtent yellowtent 0 May 11 20:44 baz
                -rw-rw---- 3 yellowtent yellowtent 0 May 11 20:44 foo
                

                rather disappointing, seems to work 😉

                1 Reply Last reply
                0
                • girishG Offline
                  girishG Offline
                  girish
                  Staff
                  wrote on last edited by
                  #10

                  @msbt If possible, please trigger a backup as well with hardlinks. I would like to know why it fails. Maybe there is some limit somewhere we are hitting in CIFS.

                  1 Reply Last reply
                  0
                  • M Online
                    M Online
                    msbt
                    App Dev
                    wrote on last edited by
                    #11

                    ok, switched to rsync + hardlinks, testing

                    1 Reply Last reply
                    1
                    • nebulonN Offline
                      nebulonN Offline
                      nebulon
                      Staff
                      wrote on last edited by
                      #12

                      Just to give a bit more information here. I can confirm that sometimes creating hardlinks within a cifs mount on a hetzner storage box fails like:

                      root@my ~ # cp -al /backups/snapshot/app_caafd030-50c4-413d-82b8-733e28d9da01/data/public/ /backups/foo
                      cp: cannot create hard link '/backups/foo/public/css/fonts/element-icons.ttf' to '/backups/snapshot/app_caafd030-50c4-413d-82b8-733e28d9da01/data/public/css/fonts/element-icons.ttf': No such file or directory
                      

                      However as soon as I touch or read/write that failing file, it starts working just fine.

                      If someone else here sees the same issue, maybe you can try to also see if touching a file which is failing fixes the hardlink creating. Given that this seems quite the sporadic issue, it would help to gather more information here to see if this indeed is consistent.

                      1 Reply Last reply
                      1
                      • I Offline
                        I Offline
                        iqweb
                        wrote on last edited by
                        #13

                        I'll switch back to rsync with hardlinks & try touching the error files too

                        1 Reply Last reply
                        0
                        • girishG Offline
                          girishG Offline
                          girish
                          Staff
                          wrote on last edited by
                          #14

                          @iqweb We are monitoring @nebulon 's backups in the meantime, so we can find a workaround. Maybe as simple as a retry or maybe using rsync instead or creating hardlinks manually in node might do the trick.

                          1 Reply Last reply
                          0
                          • M Online
                            M Online
                            msbt
                            App Dev
                            wrote on last edited by
                            #15

                            I've been using hardlinks for a few days now and encountered no problems so far. Would the backup job notify me if some links couldn't be generated? Not sure what happened the last time I tried that.

                            1 Reply Last reply
                            0
                            • nebulonN Offline
                              nebulonN Offline
                              nebulon
                              Staff
                              wrote on last edited by
                              #16

                              Yes you would receive a notification and email if a backup job failed

                              1 Reply Last reply
                              0
                              • I Offline
                                I Offline
                                iqweb
                                wrote on last edited by
                                #17

                                Seems to be working for me now too!

                                1 Reply Last reply
                                0
                                • I iqweb

                                  I also asked hetzner support about hardlinks & they said:

                                  It seems that Samba is transferring the Hard Links correct, but not always showing them correct.
                                  You could use rsync directly, in order to back up your data to the Storage Box.
                                  For example:
                                  -----------------%<-----------------
                                  rsync --progress -H -e 'ssh -p23' --recursive <local_directory> <username>@<username>.your-storagebox.de:<target_directory>
                                  -----------------%<-----------------
                                  

                                  Also - like msbt - it was working in the beginning & then started giving errors about hardlinks.

                                  jdaviescoates - Do you mean the borgbackup to Hetzner storagebox - or the Synology pull?

                                  nebulon - Is there a way to do a cloudron app only -minus nc data backup? I'm only able to keep 2 days of native cloudron backups at the moment & would very much prefer more than that!

                                  jdaviescoatesJ Offline
                                  jdaviescoatesJ Offline
                                  jdaviescoates
                                  wrote on last edited by
                                  #18

                                  @iqweb said in Nextcloud Backup without Data:

                                  jdaviescoates - Do you mean the borgbackup to Hetzner storagebox - or the Synology pull?

                                  Both! Thanks 😊

                                  I use Cloudron with Gandi & Hetzner

                                  1 Reply Last reply
                                  0
                                  • nebulonN Offline
                                    nebulonN Offline
                                    nebulon
                                    Staff
                                    wrote on last edited by
                                    #19

                                    Just a heads up, I have seen some bad behavior with cifs mount and hard-links on my personal Cloudron, which made some backups corrupt.

                                    So please everyone who is using this for now with relevant data, do not use hardlinks.

                                    jdaviescoatesJ 1 Reply Last reply
                                    1
                                    • girishG Offline
                                      girishG Offline
                                      girish
                                      Staff
                                      wrote on last edited by
                                      #20

                                      We have now setup a separate hetzner box to sync large amount of data to debug this issue.

                                      1 Reply Last reply
                                      1
                                      • nebulonN nebulon

                                        Just a heads up, I have seen some bad behavior with cifs mount and hard-links on my personal Cloudron, which made some backups corrupt.

                                        So please everyone who is using this for now with relevant data, do not use hardlinks.

                                        jdaviescoatesJ Offline
                                        jdaviescoatesJ Offline
                                        jdaviescoates
                                        wrote on last edited by
                                        #21

                                        @nebulon said in Nextcloud Backup without Data:

                                        So please everyone who is using this for now with relevant data, do not use hardlinks.

                                        I'm not sure if this applies to me or not. I backup to a Hetzner Cloud Volume (not storage box). Currently I have hardlinks enabled. Should I turn them off?

                                        What should I look for in the logs to see if this is effecting me?

                                        Thanks!

                                        I use Cloudron with Gandi & Hetzner

                                        1 Reply Last reply
                                        0
                                        • nebulonN Offline
                                          nebulonN Offline
                                          nebulon
                                          Staff
                                          wrote on last edited by
                                          #22

                                          It highly depends on how a "disk" is mounted onto the server. The hardlinks issue is only there for cifs mounts and since cifs/samba support hardlinks since a very long time, this seems to be related in particular to the hetzner storage boxes at the moment.

                                          jdaviescoatesJ 1 Reply Last reply
                                          1
                                          Reply
                                          • Reply as topic
                                          Log in to reply
                                          • Oldest to Newest
                                          • Newest to Oldest
                                          • Most Votes


                                          • Login

                                          • Don't have an account? Register

                                          • Login or register to search.
                                          • First post
                                            Last post
                                          0
                                          • Categories
                                          • Recent
                                          • Tags
                                          • Popular
                                          • Bookmarks
                                          • Search