What is your backup config and why?
-
I thought it’d be interesting to see how people commonly setup the backup config and perhaps even why they set it up that way. In particular I’m curious about frequency of backups and retention periods.
I’ve been going through many different schemes over the last year. Currently my setup is this:
- Frequency: backup twice daily at 7 AM & 7 PM
- Retention: 2 days
I’ve been thinking of changing retention to a week as a “just in case” (especially since object storage is so inexpensive too), but honestly not sure if it’s needed as I’ve so far always been able to restore just fine with a two-day retention.
I used to backup much more frequently (4-6 times a day) back when email was much more critical and changing frequently during the pandemic for my medical clinic clients, however that’s since become less used so I’ve dropped the backup frequency to twice as a result.
Since it’s currently done twice a day, I chose 7 AM and 7 PM to keep them evenly apart and outside of peak usage of the server to minimize any performance impact. Additionally 7 PM is important for me because I have the system auto-update apps on weekdays at 8 PM so having the backups done just prior is an extra safety net in case anything catastrophic happens during a Cloudron update or something too.
What about you? What do you have it configured to do and why? I think it’d be great to learn from others in case they are thinking of things that I’m missing and need to consider, or vice-versa.
-
I backup once a day every day except sunday, where I run updates. Also I use filesystem backups for now until my internet is good enough to not have multi hour S3 backups.
-
I thought it’d be interesting to see how people commonly setup the backup config and perhaps even why they set it up that way. In particular I’m curious about frequency of backups and retention periods.
I’ve been going through many different schemes over the last year. Currently my setup is this:
- Frequency: backup twice daily at 7 AM & 7 PM
- Retention: 2 days
I’ve been thinking of changing retention to a week as a “just in case” (especially since object storage is so inexpensive too), but honestly not sure if it’s needed as I’ve so far always been able to restore just fine with a two-day retention.
I used to backup much more frequently (4-6 times a day) back when email was much more critical and changing frequently during the pandemic for my medical clinic clients, however that’s since become less used so I’ve dropped the backup frequency to twice as a result.
Since it’s currently done twice a day, I chose 7 AM and 7 PM to keep them evenly apart and outside of peak usage of the server to minimize any performance impact. Additionally 7 PM is important for me because I have the system auto-update apps on weekdays at 8 PM so having the backups done just prior is an extra safety net in case anything catastrophic happens during a Cloudron update or something too.
What about you? What do you have it configured to do and why? I think it’d be great to learn from others in case they are thinking of things that I’m missing and need to consider, or vice-versa.
-
I think I am a little too paranoid here. Once I had to recover data deleted from the Gitea instance a month before. Since then I have this backup strategy:
- Frequency: 01:00 AM daily
- Retention: 7 daily, 4 weekly, 12 monthly
- Method: Full snapshot (not incremental)
At all times I use around 23x180GB = ~4TB space only for backup.
-
I think I am a little too paranoid here. Once I had to recover data deleted from the Gitea instance a month before. Since then I have this backup strategy:
- Frequency: 01:00 AM daily
- Retention: 7 daily, 4 weekly, 12 monthly
- Method: Full snapshot (not incremental)
At all times I use around 23x180GB = ~4TB space only for backup.
Daily unencrypted Backup to local filesystem using the rsync backend and the "3 daily, 4 weekly, 6 monthly" schedule.
Daily encrypted backup of the "snapshot" folder via restic to OneDrive (which is in my O365 subscription) with the same Schedule as above.
-
Note to self, need to get around to testing this too: https://community.hetzner.com/tutorials/install-and-configure-borgbackup
-
@atrilahiji - Just curious, how much data needs to be backed and to S3 (when you enable that method) where it takes multiple hours? I use the OVH Object Storage which uses the S3 protocol, and I have approximately 30-35 GB to backup (before compression), and it takes me almost 45-55 minutes each time. I am using tarball/tgz though so some of that time may be the limitations in CPU on my node before it's even uploaded. After it's zipped the 30-35 GB becomes closer to 6 GB. I'm sure OVH Object Storage isn't as performant either as AWS S3 would be.
@p44 - Do you find the 2 PM (presumably that's a "peak" time for your server traffic) impacts performance at all?
@nj - Wow! That's a huge data transfer for the backups and total backup storage! So far I've been lucky in that anytime I need to restore it's always from a day or two (usually just hours) prior so I've kept mine at 2 days retention, but I do fear at some point it'll happen where something may be irrecoverable if I don't have a backup from far enough back. Maybe I'll look into the daily/weekly retention setting instead of 2 days or even 1 week.
@necrevistonnezr - I wish I had enough disk space locally to backup to a file system for the lighter stuff. That's awesome you can do that! Neat idea on the restic sync to OneDrive afterwards too.
-
@atrilahiji - Just curious, how much data needs to be backed and to S3 (when you enable that method) where it takes multiple hours? I use the OVH Object Storage which uses the S3 protocol, and I have approximately 30-35 GB to backup (before compression), and it takes me almost 45-55 minutes each time. I am using tarball/tgz though so some of that time may be the limitations in CPU on my node before it's even uploaded. After it's zipped the 30-35 GB becomes closer to 6 GB. I'm sure OVH Object Storage isn't as performant either as AWS S3 would be.
@p44 - Do you find the 2 PM (presumably that's a "peak" time for your server traffic) impacts performance at all?
@nj - Wow! That's a huge data transfer for the backups and total backup storage! So far I've been lucky in that anytime I need to restore it's always from a day or two (usually just hours) prior so I've kept mine at 2 days retention, but I do fear at some point it'll happen where something may be irrecoverable if I don't have a backup from far enough back. Maybe I'll look into the daily/weekly retention setting instead of 2 days or even 1 week.
@necrevistonnezr - I wish I had enough disk space locally to backup to a file system for the lighter stuff. That's awesome you can do that! Neat idea on the restic sync to OneDrive afterwards too.
@d19dotca
I have about 250GB right now to backup. Mostly from Nextcloud. -
@atrilahiji - Just curious, how much data needs to be backed and to S3 (when you enable that method) where it takes multiple hours? I use the OVH Object Storage which uses the S3 protocol, and I have approximately 30-35 GB to backup (before compression), and it takes me almost 45-55 minutes each time. I am using tarball/tgz though so some of that time may be the limitations in CPU on my node before it's even uploaded. After it's zipped the 30-35 GB becomes closer to 6 GB. I'm sure OVH Object Storage isn't as performant either as AWS S3 would be.
@p44 - Do you find the 2 PM (presumably that's a "peak" time for your server traffic) impacts performance at all?
@nj - Wow! That's a huge data transfer for the backups and total backup storage! So far I've been lucky in that anytime I need to restore it's always from a day or two (usually just hours) prior so I've kept mine at 2 days retention, but I do fear at some point it'll happen where something may be irrecoverable if I don't have a backup from far enough back. Maybe I'll look into the daily/weekly retention setting instead of 2 days or even 1 week.
@necrevistonnezr - I wish I had enough disk space locally to backup to a file system for the lighter stuff. That's awesome you can do that! Neat idea on the restic sync to OneDrive afterwards too.
@d19dotca said in What is your backup config and why?:
@p44 - Do you find the 2 PM (presumably that's a "peak" time for your server traffic) impacts performance at all?
This is a very good question. Performances could be affected but our customers in that time they are in lunch time, so we prefer to keep an updated copy of backup after "morning work". Backup all data it takes around 30minutes.
