Nextcloud Backup without Data
-
Hi - is there anyway to make cloudron backups for Nextcloud excluding the data folder. I find the internal cloudron backup a bi impractical for large datasets.
I am using a Hetzner smb mounted storagebox share & rsync can't use hardlinks so I have to do full non-incremental backups & as can be expected - quickly run out of space. I am however using borg to backup to the same storagebox & that is deduplicated so uses space efficiently. I also pull a backup of the Nextcloud every night to our Synology NAS.
I would prefer though to still have a cloudron native backup of the Nextcloud app/db minus the data. Is that possible? If not - could it become possible in a future update?
Thanks
-
@iqweb said in Nextcloud Backup without Data:
I am using a Hetzner smb mounted storagebox share & rsync can't use hardlinks so I have to do full non-incremental backups & as can be expected - quickly run out of space. I am however using borg to backup to the same storagebox & that is deduplicated so uses space efficiently. I also pull a backup of the Nextcloud every night to our Synology NAS.
I'd like to do something similar, got any good links to step by step guide of how very such a system in place?
-
This is indeed often an issue. In your case however you should be able to use hardlinks if you mount the storage box as described at https://cloudron.io/documentation/backups/#cifs
Generally it is hard to detangle the app's data folder from the other parts (mostly the database backups) as things then risk to go out of sync on a restore and it is hard to predict how apps like Nextcloud handle such cases.
-
@nebulon when I asked hetzner about a year ago about hardlinks on storage boxes they said:
auf unseren Storage Boxes kommt ZFS zum Einsatz. Hardlinks werden aus Sicherheitsgründen leider nicht unterstützt. Eventuell könnten Sie das Problem der Deduplizierung auch über BorgBackup lösen: https://wiki.hetzner.de/index.php/BorgBackup Unsere Storage Boxes unterstützen BorgBackup rsync nativ.
Are you sure those are working now?
-
Interesting reply, I am actually using such a storage box myself with samba and hardlinks on Cloudron. So far this has worked fine. I also just did a test hardlink manually on that samba mount and the hardlinks do persist across remounts. What I don't know is if they might actually somewhat duplicate data on the disk itself, but I think in such a case my box would be filled up by now easily.
Interesting though, I was not aware that they support borgbackup natively.
Can you maybe create some manual hardlinks on your storage box and check how it behaves for you?
-
I think when I tried that last year, the first backup went through (as well as manually creating hardlinks), but the subsequent ones were throwing errors. I thought it was due to the amount of data (40gb+ nextcloud instance), but the support said otherwise. When I have time I'll try again and let you know
-
I also asked hetzner support about hardlinks & they said:
It seems that Samba is transferring the Hard Links correct, but not always showing them correct. You could use rsync directly, in order to back up your data to the Storage Box. For example: -----------------%<----------------- rsync --progress -H -e 'ssh -p23' --recursive <local_directory> <username>@<username>.your-storagebox.de:<target_directory> -----------------%<-----------------
Also - like msbt - it was working in the beginning & then started giving errors about hardlinks.
jdaviescoates - Do you mean the borgbackup to Hetzner storagebox - or the Synology pull?
nebulon - Is there a way to do a cloudron app only -minus nc data backup? I'm only able to keep 2 days of native cloudron backups at the moment & would very much prefer more than that!
-
-
# cd /var/storagebox/test # touch foo # ln foo bar # ln foo baz # ls -l foo bar baz -rw-rw---- 3 yellowtent yellowtent 0 May 11 20:44 bar -rw-rw---- 3 yellowtent yellowtent 0 May 11 20:44 baz -rw-rw---- 3 yellowtent yellowtent 0 May 11 20:44 foo
rather disappointing, seems to work
-
Just to give a bit more information here. I can confirm that sometimes creating hardlinks within a cifs mount on a hetzner storage box fails like:
root@my ~ # cp -al /backups/snapshot/app_caafd030-50c4-413d-82b8-733e28d9da01/data/public/ /backups/foo cp: cannot create hard link '/backups/foo/public/css/fonts/element-icons.ttf' to '/backups/snapshot/app_caafd030-50c4-413d-82b8-733e28d9da01/data/public/css/fonts/element-icons.ttf': No such file or directory
However as soon as I
touch
or read/write that failing file, it starts working just fine.If someone else here sees the same issue, maybe you can try to also see if
touch
ing a file which is failing fixes the hardlink creating. Given that this seems quite the sporadic issue, it would help to gather more information here to see if this indeed is consistent. -
I'll switch back to rsync with hardlinks & try touching the error files too
-
Seems to be working for me now too!
-
@iqweb said in Nextcloud Backup without Data:
jdaviescoates - Do you mean the borgbackup to Hetzner storagebox - or the Synology pull?
Both! Thanks
-
Just a heads up, I have seen some bad behavior with cifs mount and hard-links on my personal Cloudron, which made some backups corrupt.
So please everyone who is using this for now with relevant data, do not use hardlinks.
8/24