What's the fastest backup and restore method?
-
TL;DR: I am assuming that using a local disk and no encryption is the fastest but in production that just wouldn't fly for me
39G apps -> 38G backup ~ 45min via rsync with encryption to remote CIFS location; restore 30min
Currently, I am using the following settings (times are estimates, not hard data):
- Server location: Hetzner (Nuremberg)
- Backup location: Hetzner (Falkenstein)
- Backup type: StorageBox (CIFS)
- Backup method: rsync (with encryption)
- Backup set size: Actually I am not sure as there's no quick and easy way to determine that. Looking at the snapshot folder and the latest backup folder the total appears to be around 38GB with the size of my /home/yellowtent directory being 39G.
I was trying to find out from the logs how long the backup ran for but couldn't locate anything useful. So I ended up looking at the bandwidth logs, which look like they suggest that it took 45min. That's because Cloudron doesn't log this and Hetzner's Cloud console only gives a sh*t resolution. My backups run from 22:00.
I have now installed vnstat to monitor it in more detail and wondered what your experiences were.
To restore, to a new VM in the same location, it took around 30min. Again, this lacks some hard data and I'll see if I can rig up a scientifically more useful test if anyone is interested.
Could this work?
One scenario I think might work, is to mount a block storage volume to the box locally, encrypt it using LUKS full disk encryption, back up to that volume without encryption using rsync and restore from it. Then using rclone with crypt to shovel the backups to a remote location separately using cron.If the box fails, remount the block storage to the new VM (think of it like a USB disk) and restore from it.
How big are your backups, what method do you use and how long does it take?
馃憠 Find our more www.bebraver.online
-
TL;DR: I am assuming that using a local disk and no encryption is the fastest but in production that just wouldn't fly for me
39G apps -> 38G backup ~ 45min via rsync with encryption to remote CIFS location; restore 30min
Currently, I am using the following settings (times are estimates, not hard data):
- Server location: Hetzner (Nuremberg)
- Backup location: Hetzner (Falkenstein)
- Backup type: StorageBox (CIFS)
- Backup method: rsync (with encryption)
- Backup set size: Actually I am not sure as there's no quick and easy way to determine that. Looking at the snapshot folder and the latest backup folder the total appears to be around 38GB with the size of my /home/yellowtent directory being 39G.
I was trying to find out from the logs how long the backup ran for but couldn't locate anything useful. So I ended up looking at the bandwidth logs, which look like they suggest that it took 45min. That's because Cloudron doesn't log this and Hetzner's Cloud console only gives a sh*t resolution. My backups run from 22:00.
I have now installed vnstat to monitor it in more detail and wondered what your experiences were.
To restore, to a new VM in the same location, it took around 30min. Again, this lacks some hard data and I'll see if I can rig up a scientifically more useful test if anyone is interested.
Could this work?
One scenario I think might work, is to mount a block storage volume to the box locally, encrypt it using LUKS full disk encryption, back up to that volume without encryption using rsync and restore from it. Then using rclone with crypt to shovel the backups to a remote location separately using cron.If the box fails, remount the block storage to the new VM (think of it like a USB disk) and restore from it.
How big are your backups, what method do you use and how long does it take?
-
TL;DR: I am assuming that using a local disk and no encryption is the fastest but in production that just wouldn't fly for me
39G apps -> 38G backup ~ 45min via rsync with encryption to remote CIFS location; restore 30min
Currently, I am using the following settings (times are estimates, not hard data):
- Server location: Hetzner (Nuremberg)
- Backup location: Hetzner (Falkenstein)
- Backup type: StorageBox (CIFS)
- Backup method: rsync (with encryption)
- Backup set size: Actually I am not sure as there's no quick and easy way to determine that. Looking at the snapshot folder and the latest backup folder the total appears to be around 38GB with the size of my /home/yellowtent directory being 39G.
I was trying to find out from the logs how long the backup ran for but couldn't locate anything useful. So I ended up looking at the bandwidth logs, which look like they suggest that it took 45min. That's because Cloudron doesn't log this and Hetzner's Cloud console only gives a sh*t resolution. My backups run from 22:00.
I have now installed vnstat to monitor it in more detail and wondered what your experiences were.
To restore, to a new VM in the same location, it took around 30min. Again, this lacks some hard data and I'll see if I can rig up a scientifically more useful test if anyone is interested.
Could this work?
One scenario I think might work, is to mount a block storage volume to the box locally, encrypt it using LUKS full disk encryption, back up to that volume without encryption using rsync and restore from it. Then using rclone with crypt to shovel the backups to a remote location separately using cron.If the box fails, remount the block storage to the new VM (think of it like a USB disk) and restore from it.
How big are your backups, what method do you use and how long does it take?
@3246 said in [ Question ] What's the fastest backup and restore method?:
How big are your backups,
Pretty big, I guess, because I'm currently backing up 237.79 GB of data. Not checked recently how big the backup ends up being
what method do you use
I just use tgz (to a CIFS mounted Hetzer Storage box too) because I like the idea of each backup being a complete backup (and I guess also because most of my backup is Nextcloud which includes loads of small files and rsync takes ages working out what has changed which kinda defeats the purpose of the transfer being quicker because it only backs up those changes. But mostly because I just prefer backups to be standalone complete backups)
and how long does it take?
I'm not actually too sure because I'm asleep when it happens (I only backup daily, not multiple times a day like some people), but I think it takes a good couple of hours.
-
N nebulon marked this topic as a question on
-
TL;DR: I am assuming that using a local disk and no encryption is the fastest but in production that just wouldn't fly for me
39G apps -> 38G backup ~ 45min via rsync with encryption to remote CIFS location; restore 30min
Currently, I am using the following settings (times are estimates, not hard data):
- Server location: Hetzner (Nuremberg)
- Backup location: Hetzner (Falkenstein)
- Backup type: StorageBox (CIFS)
- Backup method: rsync (with encryption)
- Backup set size: Actually I am not sure as there's no quick and easy way to determine that. Looking at the snapshot folder and the latest backup folder the total appears to be around 38GB with the size of my /home/yellowtent directory being 39G.
I was trying to find out from the logs how long the backup ran for but couldn't locate anything useful. So I ended up looking at the bandwidth logs, which look like they suggest that it took 45min. That's because Cloudron doesn't log this and Hetzner's Cloud console only gives a sh*t resolution. My backups run from 22:00.
I have now installed vnstat to monitor it in more detail and wondered what your experiences were.
To restore, to a new VM in the same location, it took around 30min. Again, this lacks some hard data and I'll see if I can rig up a scientifically more useful test if anyone is interested.
Could this work?
One scenario I think might work, is to mount a block storage volume to the box locally, encrypt it using LUKS full disk encryption, back up to that volume without encryption using rsync and restore from it. Then using rclone with crypt to shovel the backups to a remote location separately using cron.If the box fails, remount the block storage to the new VM (think of it like a USB disk) and restore from it.
How big are your backups, what method do you use and how long does it take?
@3246 Local rsync backup, then restic overnight to a remote host. 300 GB of data takes less than 30 minutes (depending on the changes): https://forum.cloudron.io/topic/6928/tutorial-remote-backup-of-local-cloudron-backup-snapshots-with-restic-rclone/1?_=1673172629890
-
TL;DR: I am assuming that using a local disk and no encryption is the fastest but in production that just wouldn't fly for me
39G apps -> 38G backup ~ 45min via rsync with encryption to remote CIFS location; restore 30min
Currently, I am using the following settings (times are estimates, not hard data):
- Server location: Hetzner (Nuremberg)
- Backup location: Hetzner (Falkenstein)
- Backup type: StorageBox (CIFS)
- Backup method: rsync (with encryption)
- Backup set size: Actually I am not sure as there's no quick and easy way to determine that. Looking at the snapshot folder and the latest backup folder the total appears to be around 38GB with the size of my /home/yellowtent directory being 39G.
I was trying to find out from the logs how long the backup ran for but couldn't locate anything useful. So I ended up looking at the bandwidth logs, which look like they suggest that it took 45min. That's because Cloudron doesn't log this and Hetzner's Cloud console only gives a sh*t resolution. My backups run from 22:00.
I have now installed vnstat to monitor it in more detail and wondered what your experiences were.
To restore, to a new VM in the same location, it took around 30min. Again, this lacks some hard data and I'll see if I can rig up a scientifically more useful test if anyone is interested.
Could this work?
One scenario I think might work, is to mount a block storage volume to the box locally, encrypt it using LUKS full disk encryption, back up to that volume without encryption using rsync and restore from it. Then using rclone with crypt to shovel the backups to a remote location separately using cron.If the box fails, remount the block storage to the new VM (think of it like a USB disk) and restore from it.
How big are your backups, what method do you use and how long does it take?
-
TL;DR: I am assuming that using a local disk and no encryption is the fastest but in production that just wouldn't fly for me
39G apps -> 38G backup ~ 45min via rsync with encryption to remote CIFS location; restore 30min
Currently, I am using the following settings (times are estimates, not hard data):
- Server location: Hetzner (Nuremberg)
- Backup location: Hetzner (Falkenstein)
- Backup type: StorageBox (CIFS)
- Backup method: rsync (with encryption)
- Backup set size: Actually I am not sure as there's no quick and easy way to determine that. Looking at the snapshot folder and the latest backup folder the total appears to be around 38GB with the size of my /home/yellowtent directory being 39G.
I was trying to find out from the logs how long the backup ran for but couldn't locate anything useful. So I ended up looking at the bandwidth logs, which look like they suggest that it took 45min. That's because Cloudron doesn't log this and Hetzner's Cloud console only gives a sh*t resolution. My backups run from 22:00.
I have now installed vnstat to monitor it in more detail and wondered what your experiences were.
To restore, to a new VM in the same location, it took around 30min. Again, this lacks some hard data and I'll see if I can rig up a scientifically more useful test if anyone is interested.
Could this work?
One scenario I think might work, is to mount a block storage volume to the box locally, encrypt it using LUKS full disk encryption, back up to that volume without encryption using rsync and restore from it. Then using rclone with crypt to shovel the backups to a remote location separately using cron.If the box fails, remount the block storage to the new VM (think of it like a USB disk) and restore from it.
How big are your backups, what method do you use and how long does it take?
@3246 They should add a way to add 2 backup jobs setup however you like:
For Me:
Job 1 - Backup to Local external disk for fast access in case of emergency
Job 2 - Offsite backup for long-storage.
Both encrypted, and this offers a quick restore (in the event of just needing a quick restore) and one for proper emergencies
Not that you even need to set it up like this, but more backups == better backups even if one is local for emergency "non-natural-disaster" restore.
-
G girish marked this topic as a regular topic on
-
@eddowding said in What's the fastest backup and restore method?:
I have 175GB and it takes about 12h
I guess something is very wrong.Yeah, my backup of about 350GB takes about 2 hours (from a Hetzner Dedicated Server in Finland Helsinki (HEL1) to a Heztner Storage Box in Falkenstein (FSN1))
IMHO your issue is very likely due to those "too good to be true" iDrive offer pricing they had which I guess you went for.
