v9 Backups : multiple locations
-
Tentatively getting to grips with new features of v9, specifically multiple backup locations, which is a totally great new feature
-
is there a docs section specifically about multiple locations ?
checked but didn't see anything - maybe still being added -
pre-v9, backups were a blocking process, no other operations like app installs could be done during a backup run. Is this still the case with v9?
-
extending #2, presumably it's necessary to take care to ensure the schedules of each backup location are non-conflicting ? or v9 is so awesome that backups are separate processes which can run to 2 locations concurrently ?? (whoa)
-
seeking clarification about docs on Hetzner Storage Box
- CIFS :
We recommend using SSHFS for Hetzner Storage Box since it is much faster and efficient storage wise compared to CIFS. When using Hetzner Storage Box with CIFS, the Remote Directory is /backup for the main account. For sub accounts, the Remote Directory is /subaccount.
- SSHFS :
When using Hetzner Storage Box, the Remote Directory is /home for the main account. We have found sub accounts to be unreliable with SSHFS. We recommend using CIFS instead if you want to use subaccounts.
- net result is that advice is to use SSHFS for backups to main account and CIFS for sub accounts ?
If so, I'd suggest the docs state that explicitly
Hello @timconsidine
@timconsidine said in v9 Backups : multiple locations:
net result is that advice is to use SSHFS for backups to main account and CIFS for sub accounts ?
If so, I'd suggest the docs state that explicitlyI'd say
SSHFSfor Hetzner Storage Boxes for both main- and sub-accounts.
But, let me get back to you on that after I did some research and validation. -
-
-
There is no doc yet, hopefully the docs will get updated by the end of the year.
-
Right, app backups don't block app operations anymore. App backups are now totally in background
-
Concurrent backups are fine. Just have to make sure the locations don't conflict i.e storageprovider+target+prefix must not eventually be the exact same location. If you have different prefixes, that is fine too. It is also fine if schedules conflict.
-
I think if SSHFS works, I would go with that. It is much faster since we have optimized code paths for it.
@girish whoa, non-blocking AND concurrent backups !
Congratulations, that is truly a huge leap forwards, thank you, thank you, thank you.No problems on the docs, just wanted to check.
I have been happily using scaleway object storage, but separately I pay for a large Hetzner Storage Box, so my plan was to add Hetzner Storage Box as a 2nd location, run backups to both scaleway and Hetzner Storage Box for a while, maybe then remove scaleway from the loop to save some money.
I wasn't completely sure this would be viable but it seems it is
-
-
Hello @timconsidine
@timconsidine said in v9 Backups : multiple locations:
net result is that advice is to use SSHFS for backups to main account and CIFS for sub accounts ?
If so, I'd suggest the docs state that explicitlyI'd say
SSHFSfor Hetzner Storage Boxes for both main- and sub-accounts.
But, let me get back to you on that after I did some research and validation. -
@timconsidine
Okay, soSSHFSand Hetzner Storage Box Sub-Accounts work just fine.
The issue here is the following.
Storage Boxes have a concurrent connections limit of 10.
So if you have e.g. 20 Sub-Accounts and all try to access the Storage Box at the same time 10 will not make it.
But if you keep that in mind and spread out the connections over a timeframe you can use more than 10 Sub Accounts. -
Hello @timconsidine
@timconsidine said in v9 Backups : multiple locations:
net result is that advice is to use SSHFS for backups to main account and CIFS for sub accounts ?
If so, I'd suggest the docs state that explicitlyI'd say
SSHFSfor Hetzner Storage Boxes for both main- and sub-accounts.
But, let me get back to you on that after I did some research and validation. -
@timconsidine
Okay, soSSHFSand Hetzner Storage Box Sub-Accounts work just fine.
The issue here is the following.
Storage Boxes have a concurrent connections limit of 10.
So if you have e.g. 20 Sub-Accounts and all try to access the Storage Box at the same time 10 will not make it.
But if you keep that in mind and spread out the connections over a timeframe you can use more than 10 Sub Accounts. -
@timconsidine make sure to check the "hardlink" option for the backup site. This drastically improves performance after the first full backup run since we have optimized that code to create the hardlinks remotely via ssh.
-
@nebulon I don't see the "hardlink" option.
Is it in the dialogue which appears when you Add a site ?@timconsidine probably only shows up with sshfs
