Backup Strategy Advice
-
@lucidfox Sounds like a solid strategy to me. Certainly right to have a multi-provider & encrypted setup.
One of the biggest risks I see with anything nowadays isn't technical but Ts & Cs and provider lock-out.
Their platforms, their rules, they can change at any time and you can be an accidental victim of bad actors in ways you'll never imagine before it happens.
From what I understand storage boxes have hardware redundancy, so I don't think that would be a point of failure - most backup recovery needs are user or software caused.
In my experience, no-one ever needs a backup past a week old, what is usually needed is the most recent recoverable.
So I think start with shorter intervals to get it all working confidently, then extend to longer intervals and don't store too old for the sake of it if you don't have some regulatory needs to.
This is my personal Cloudron setup now (rsync encrypted to Wasabi) + the 7 daily provider snapshots:
Happy with this for balancing costs, cruft and security.
-
@marcusquinn I don't have any regulatory needs. But it might make sense to reach back into the past, to a reasonable extent, in case a user accidentally deletes files and doesn't realise for a bit.
@lucidfox Haha, yeah, that happens - but some Apps have file-versioning or Trash features too, some allow users to flag a delete but the DB or file system doesn't delete until a sys admin purges. So you might be taking a sledgehammer to crack a nut.
If there's any apps in particular you think that might save you from backup exponentials and you're not sure, ping their names in here and I might or others might have a quicker answer.
Web Design https://www.evergreen.je
Development https://brandlight.org
Life https://marcusquinn.com -
@lucidfox Haha, yeah, that happens - but some Apps have file-versioning or Trash features too, some allow users to flag a delete but the DB or file system doesn't delete until a sys admin purges. So you might be taking a sledgehammer to crack a nut.
If there's any apps in particular you think that might save you from backup exponentials and you're not sure, ping their names in here and I might or others might have a quicker answer.
@marcusquinn I suppose the other thought with rsync is that the retention period isn't so bad on storage space since its all incremental. Tarballs would be though.
So many variables to think through eh! So a worthy post and conversations to have evolve here for others I'm sure.
-
I would recommend encrypted rsync backups. Because of the encryption, you don't have to trust your backup provider.
Not sure but a good option would be Netcups storage spaces together with one of their Cloud vLAN producs to have decent backup speed. Cost would be around 15-25€ for 1TB per month.
Because you said you want another provider maybe within another region, we have to consider other options. The best Idea I had so far is a seperate VPS with enough storage space running Minio. A cheap option (for smaller setups) would be some generic webspace. I also thought about Strato High Drive Business. 1TB for 15€/month but I don't know about traffic and speed.
PS: I'm also looking for a decent cheap backup solution/provider for my 500GB Netcup server ^^
@subven said in Backup Strategy Advice:
Because of the encryption, you don't have to trust your backup provider.
Even though they are encrypted and you can technically put them anywhere doesn't mean just go with any provider (god forbid someone got a hold of your keys to decrypt). You should have reasonable beliefs that the provider you choose wont do anything with your backups (encrypted or otherwise)
I use DigitalOcean for my backups (Dedicated hosting with OVH/Kimsufi) DO is cheap and reliable and I know, even with my encrypted data, they wont try to do anything with it.
-
@lucidfox Sounds like a solid strategy to me. Certainly right to have a multi-provider & encrypted setup.
One of the biggest risks I see with anything nowadays isn't technical but Ts & Cs and provider lock-out.
Their platforms, their rules, they can change at any time and you can be an accidental victim of bad actors in ways you'll never imagine before it happens.
From what I understand storage boxes have hardware redundancy, so I don't think that would be a point of failure - most backup recovery needs are user or software caused.
In my experience, no-one ever needs a backup past a week old, what is usually needed is the most recent recoverable.
So I think start with shorter intervals to get it all working confidently, then extend to longer intervals and don't store too old for the sake of it if you don't have some regulatory needs to.
This is my personal Cloudron setup now (rsync encrypted to Wasabi) + the 7 daily provider snapshots:
Happy with this for balancing costs, cruft and security.
@marcusquinn Marcus do you have any workaround to manage Wasabi deletion policy? I mean, now maximum Cloudron retention policy is "1 month" or "forever".
So in case 1) you are under 3 month Wasabi policy. In second case 2) you should remember each month to delete backups older than 3 months.
@girish Can be interesting to add a "3 month" or "4 month" option?
-
@marcusquinn Marcus do you have any workaround to manage Wasabi deletion policy? I mean, now maximum Cloudron retention policy is "1 month" or "forever".
So in case 1) you are under 3 month Wasabi policy. In second case 2) you should remember each month to delete backups older than 3 months.
@girish Can be interesting to add a "3 month" or "4 month" option?
@p44 My current workaround since figuring the 90-day ruse out is to make a new account and let the old one go
I'm still looking at options but might just made the backup interval longer for now and avoid tarballs.
Agreed, the interval and retention options could have a few more to cover things like this.
-
@p44 My current workaround since figuring the 90-day ruse out is to make a new account and let the old one go
I'm still looking at options but might just made the backup interval longer for now and avoid tarballs.
Agreed, the interval and retention options could have a few more to cover things like this.
@marcusquinn LOL!
But I think it takes time, specially if you've to configure each time sub-accounts... -
so just to update my results here - i tried a hetzner storagebox, but the cifs storage mount was having some stability issues. and it was so painfully slow as to be practically unusable.
so i switched to another option that i've seen mentioned in the forums here, which is to backup to a second cloudron using minio. and i'm happy to report that it's working fine. using an alphavps storage vps it's actually even cheaper, and it's quite convenient to have a further backup if needed.
-
so just to update my results here - i tried a hetzner storagebox, but the cifs storage mount was having some stability issues. and it was so painfully slow as to be practically unusable.
so i switched to another option that i've seen mentioned in the forums here, which is to backup to a second cloudron using minio. and i'm happy to report that it's working fine. using an alphavps storage vps it's actually even cheaper, and it's quite convenient to have a further backup if needed.
-
@lucidfox Thank's a lot for your advices! How many Cloudron instances you've to backup? With minio you use the same token for everyone?
-
@p44 I only have the one cloudron to backup at the moment, but I'd imagine you can use the same token or create other buckets and tokens on the same minio server.
-
@lucidfox Thank's Lucidfox. about cifs storage mount "stability issues", can you tell me more? Because now seems to be very stable and persistent...
@p44 sometimes in cloudron it would say 'this is not a cifs mount' or something similiar. even though it seemed to have been mounted properly, and showed up when checking via ssh. it could just be something to do with mounting it incorrectly though. i figured it was not worth troubleshooting, because of the slow speeds.
-
@p44 sometimes in cloudron it would say 'this is not a cifs mount' or something similiar. even though it seemed to have been mounted properly, and showed up when checking via ssh. it could just be something to do with mounting it incorrectly though. i figured it was not worth troubleshooting, because of the slow speeds.
-
@p44 sometimes in cloudron it would say 'this is not a cifs mount' or something similiar. even though it seemed to have been mounted properly, and showed up when checking via ssh. it could just be something to do with mounting it incorrectly though. i figured it was not worth troubleshooting, because of the slow speeds.
-
@lucidfox I just used the mount code provided by Hetzner here https://docs.hetzner.com/robot/storage-box/access/access-samba-cifs/ and have not had any problems
-
@jdaviescoates Are you using Ubuntu 20?
@ruihildt said in Backup Strategy Advice:
@jdaviescoates Are you using Ubuntu 20?
No. 18.04. I figured why change what's not broken
I'll likely use Ubuntu 20 the next time I do a fresh install on Cloudron, but am keeping my existing Cloudron's that I installed on 18.04 on 18.04.
-
@p44 i just followed the one in the cloudron docs (and also the hetzner docs).
cloudron:
//<server>/<remote_folder> /backups_cifs cifs uid=yellowtent,gid=yellowtent,user=<user>,pass=<pass>,iocharset=utf8,x-systemd.automount 0 0hetzner:
//<username>.your-storagebox.de/backup /mnt/backup-server cifs iocharset=utf8,rw,credentials=/etc/backup-credentials.txt,uid=<system account>,gid=<system group>,file_mode=0660,dir_mode=0770 0 0 -
so just to update my results here - i tried a hetzner storagebox, but the cifs storage mount was having some stability issues. and it was so painfully slow as to be practically unusable.
so i switched to another option that i've seen mentioned in the forums here, which is to backup to a second cloudron using minio. and i'm happy to report that it's working fine. using an alphavps storage vps it's actually even cheaper, and it's quite convenient to have a further backup if needed.
@lucidfox said in Backup Strategy Advice:
so i switched to another option that i've seen mentioned in the forums here, which is to backup to a second cloudron using minio. and i'm happy to report that it's working fine. using an alphavps storage vps it's actually even cheaper, and it's quite convenient to have a further backup if needed.
I also did this and it's working fine. I choose a storage VPS at alphavps with 768MB RAM, 256GB HDD and 1TB bandwidth for 25€ per year. Had to resize the root partition and change the free RAM check at the Cloudron setup to be able to install Cloudron. After that I disabled local backups and installed Minio. Backup speed with rsync is okay for small setups and bandwidth should be suffisant since we're doing incremental backups.
PS: be aware that they freeze your server at bandwidth overuse
Are there any bandwidth overusage fees?
No! We understand how frustrating can bandwidth overusage fees be and we've taken a different approach, to ensure our flat and predictable pricing - when you come close to running out of bandwidth, we will send a warning emails. Should you not respond and upgrade the server, we'll suspend it automatically, until the bandwidth counter is reset. -
@p44 i just followed the one in the cloudron docs (and also the hetzner docs).
cloudron:
//<server>/<remote_folder> /backups_cifs cifs uid=yellowtent,gid=yellowtent,user=<user>,pass=<pass>,iocharset=utf8,x-systemd.automount 0 0hetzner:
//<username>.your-storagebox.de/backup /mnt/backup-server cifs iocharset=utf8,rw,credentials=/etc/backup-credentials.txt,uid=<system account>,gid=<system group>,file_mode=0660,dir_mode=0770 0 0@lucidfox said in Backup Strategy Advice:
@p44 i just followed the one in the cloudron docs (and also the hetzner docs).
cloudron:
//<server>/<remote_folder> /backups_cifs cifs uid=yellowtent,gid=yellowtent,user=<user>,pass=<pass>,iocharset=utf8,x-systemd.automount 0 0hetzner:
//<username>.your-storagebox.de/backup /mnt/backup-server cifs iocharset=utf8,rw,credentials=/etc/backup-credentials.txt,uid=<system account>,gid=<system group>,file_mode=0660,dir_mode=0770 0 0Which one do you use? These above are standards strings. First one on the top should work.
Where is located source datacenter?
-
@lucidfox said in Backup Strategy Advice:
@p44 i just followed the one in the cloudron docs (and also the hetzner docs).
cloudron:
//<server>/<remote_folder> /backups_cifs cifs uid=yellowtent,gid=yellowtent,user=<user>,pass=<pass>,iocharset=utf8,x-systemd.automount 0 0hetzner:
//<username>.your-storagebox.de/backup /mnt/backup-server cifs iocharset=utf8,rw,credentials=/etc/backup-credentials.txt,uid=<system account>,gid=<system group>,file_mode=0660,dir_mode=0770 0 0Which one do you use? These above are standards strings. First one on the top should work.
Where is located source datacenter?
