Cloudron makes it easy to run web apps like WordPress, Nextcloud, GitLab on your server. Find out more or install now.


Skip to content
  • Categories
  • Recent
  • Tags
  • Popular
  • Bookmarks
  • Search
Skins
  • Light
  • Cerulean
  • Cosmo
  • Flatly
  • Journal
  • Litera
  • Lumen
  • Lux
  • Materia
  • Minty
  • Morph
  • Pulse
  • Sandstone
  • Simplex
  • Sketchy
  • Spacelab
  • United
  • Yeti
  • Zephyr
  • Dark
  • Cyborg
  • Darkly
  • Quartz
  • Slate
  • Solar
  • Superhero
  • Vapor

  • Default (No Skin)
  • No Skin
Collapse
Brand Logo

Cloudron Forum

Apps | Demo | Docs | Install
  1. Cloudron Forum
  2. Discuss
  3. What's the fastest backup and restore method?

What's the fastest backup and restore method?

Scheduled Pinned Locked Moved Discuss
backupshetznercifsrsyncencryption
10 Posts 8 Posters 2.2k Views 8 Watching
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • 32463 Offline
      32463 Offline
      3246
      wrote on last edited by nebulon
      #1

      TL;DR: I am assuming that using a local disk and no encryption is the fastest but in production that just wouldn't fly for me 馃槈

      39G apps -> 38G backup ~ 45min via rsync with encryption to remote CIFS location; restore 30min

      Currently, I am using the following settings (times are estimates, not hard data):

      • Server location: Hetzner (Nuremberg)
      • Backup location: Hetzner (Falkenstein)
      • Backup type: StorageBox (CIFS)
      • Backup method: rsync (with encryption)
      • Backup set size: Actually I am not sure as there's no quick and easy way to determine that. Looking at the snapshot folder and the latest backup folder the total appears to be around 38GB with the size of my /home/yellowtent directory being 39G.

      I was trying to find out from the logs how long the backup ran for but couldn't locate anything useful. So I ended up looking at the bandwidth logs, which look like they suggest that it took 45min. That's because Cloudron doesn't log this and Hetzner's Cloud console only gives a sh*t resolution. My backups run from 22:00.

      my-apps-bebraver-cloud-路-Hetzner-Cloud (1).png

      I have now installed vnstat to monitor it in more detail and wondered what your experiences were.

      To restore, to a new VM in the same location, it took around 30min. Again, this lacks some hard data and I'll see if I can rig up a scientifically more useful test if anyone is interested.

      Could this work?
      One scenario I think might work, is to mount a block storage volume to the box locally, encrypt it using LUKS full disk encryption, back up to that volume without encryption using rsync and restore from it. Then using rclone with crypt to shovel the backups to a remote location separately using cron.

      If the box fails, remount the block storage to the new VM (think of it like a USB disk) and restore from it. 馃

      How big are your backups, what method do you use and how long does it take?

      馃憠 Find our more www.bebraver.online

      robiR jdaviescoatesJ necrevistonnezrN timconsidineT murgeroM 5 Replies Last reply
      1
      • 32463 3246

        TL;DR: I am assuming that using a local disk and no encryption is the fastest but in production that just wouldn't fly for me 馃槈

        39G apps -> 38G backup ~ 45min via rsync with encryption to remote CIFS location; restore 30min

        Currently, I am using the following settings (times are estimates, not hard data):

        • Server location: Hetzner (Nuremberg)
        • Backup location: Hetzner (Falkenstein)
        • Backup type: StorageBox (CIFS)
        • Backup method: rsync (with encryption)
        • Backup set size: Actually I am not sure as there's no quick and easy way to determine that. Looking at the snapshot folder and the latest backup folder the total appears to be around 38GB with the size of my /home/yellowtent directory being 39G.

        I was trying to find out from the logs how long the backup ran for but couldn't locate anything useful. So I ended up looking at the bandwidth logs, which look like they suggest that it took 45min. That's because Cloudron doesn't log this and Hetzner's Cloud console only gives a sh*t resolution. My backups run from 22:00.

        my-apps-bebraver-cloud-路-Hetzner-Cloud (1).png

        I have now installed vnstat to monitor it in more detail and wondered what your experiences were.

        To restore, to a new VM in the same location, it took around 30min. Again, this lacks some hard data and I'll see if I can rig up a scientifically more useful test if anyone is interested.

        Could this work?
        One scenario I think might work, is to mount a block storage volume to the box locally, encrypt it using LUKS full disk encryption, back up to that volume without encryption using rsync and restore from it. Then using rclone with crypt to shovel the backups to a remote location separately using cron.

        If the box fails, remount the block storage to the new VM (think of it like a USB disk) and restore from it. 馃

        How big are your backups, what method do you use and how long does it take?

        robiR Offline
        robiR Offline
        robi
        wrote on last edited by
        #2

        @3246 As far as I recall, box logs track time for task duration including backups.

        Conscious tech

        1 Reply Last reply
        1
        • 32463 3246

          TL;DR: I am assuming that using a local disk and no encryption is the fastest but in production that just wouldn't fly for me 馃槈

          39G apps -> 38G backup ~ 45min via rsync with encryption to remote CIFS location; restore 30min

          Currently, I am using the following settings (times are estimates, not hard data):

          • Server location: Hetzner (Nuremberg)
          • Backup location: Hetzner (Falkenstein)
          • Backup type: StorageBox (CIFS)
          • Backup method: rsync (with encryption)
          • Backup set size: Actually I am not sure as there's no quick and easy way to determine that. Looking at the snapshot folder and the latest backup folder the total appears to be around 38GB with the size of my /home/yellowtent directory being 39G.

          I was trying to find out from the logs how long the backup ran for but couldn't locate anything useful. So I ended up looking at the bandwidth logs, which look like they suggest that it took 45min. That's because Cloudron doesn't log this and Hetzner's Cloud console only gives a sh*t resolution. My backups run from 22:00.

          my-apps-bebraver-cloud-路-Hetzner-Cloud (1).png

          I have now installed vnstat to monitor it in more detail and wondered what your experiences were.

          To restore, to a new VM in the same location, it took around 30min. Again, this lacks some hard data and I'll see if I can rig up a scientifically more useful test if anyone is interested.

          Could this work?
          One scenario I think might work, is to mount a block storage volume to the box locally, encrypt it using LUKS full disk encryption, back up to that volume without encryption using rsync and restore from it. Then using rclone with crypt to shovel the backups to a remote location separately using cron.

          If the box fails, remount the block storage to the new VM (think of it like a USB disk) and restore from it. 馃

          How big are your backups, what method do you use and how long does it take?

          jdaviescoatesJ Offline
          jdaviescoatesJ Offline
          jdaviescoates
          wrote on last edited by jdaviescoates
          #3

          @3246 said in [ Question ] What's the fastest backup and restore method?:

          How big are your backups,

          Pretty big, I guess, because I'm currently backing up 237.79 GB of data. Not checked recently how big the backup ends up being

          what method do you use

          I just use tgz (to a CIFS mounted Hetzer Storage box too) because I like the idea of each backup being a complete backup (and I guess also because most of my backup is Nextcloud which includes loads of small files and rsync takes ages working out what has changed which kinda defeats the purpose of the transfer being quicker because it only backs up those changes. But mostly because I just prefer backups to be standalone complete backups)

          and how long does it take?

          I'm not actually too sure because I'm asleep when it happens (I only backup daily, not multiple times a day like some people), but I think it takes a good couple of hours.

          I use Cloudron with Gandi & Hetzner

          1 Reply Last reply
          3
          • nebulonN nebulon marked this topic as a question on
          • 32463 3246

            TL;DR: I am assuming that using a local disk and no encryption is the fastest but in production that just wouldn't fly for me 馃槈

            39G apps -> 38G backup ~ 45min via rsync with encryption to remote CIFS location; restore 30min

            Currently, I am using the following settings (times are estimates, not hard data):

            • Server location: Hetzner (Nuremberg)
            • Backup location: Hetzner (Falkenstein)
            • Backup type: StorageBox (CIFS)
            • Backup method: rsync (with encryption)
            • Backup set size: Actually I am not sure as there's no quick and easy way to determine that. Looking at the snapshot folder and the latest backup folder the total appears to be around 38GB with the size of my /home/yellowtent directory being 39G.

            I was trying to find out from the logs how long the backup ran for but couldn't locate anything useful. So I ended up looking at the bandwidth logs, which look like they suggest that it took 45min. That's because Cloudron doesn't log this and Hetzner's Cloud console only gives a sh*t resolution. My backups run from 22:00.

            my-apps-bebraver-cloud-路-Hetzner-Cloud (1).png

            I have now installed vnstat to monitor it in more detail and wondered what your experiences were.

            To restore, to a new VM in the same location, it took around 30min. Again, this lacks some hard data and I'll see if I can rig up a scientifically more useful test if anyone is interested.

            Could this work?
            One scenario I think might work, is to mount a block storage volume to the box locally, encrypt it using LUKS full disk encryption, back up to that volume without encryption using rsync and restore from it. Then using rclone with crypt to shovel the backups to a remote location separately using cron.

            If the box fails, remount the block storage to the new VM (think of it like a USB disk) and restore from it. 馃

            How big are your backups, what method do you use and how long does it take?

            necrevistonnezrN Offline
            necrevistonnezrN Offline
            necrevistonnezr
            wrote on last edited by necrevistonnezr
            #4

            @3246 Local rsync backup, then restic overnight to a remote host. 300 GB of data takes less than 30 minutes (depending on the changes): https://forum.cloudron.io/topic/6928/tutorial-remote-backup-of-local-cloudron-backup-snapshots-with-restic-rclone/1?_=1673172629890

            1 Reply Last reply
            6
            • 32463 3246

              TL;DR: I am assuming that using a local disk and no encryption is the fastest but in production that just wouldn't fly for me 馃槈

              39G apps -> 38G backup ~ 45min via rsync with encryption to remote CIFS location; restore 30min

              Currently, I am using the following settings (times are estimates, not hard data):

              • Server location: Hetzner (Nuremberg)
              • Backup location: Hetzner (Falkenstein)
              • Backup type: StorageBox (CIFS)
              • Backup method: rsync (with encryption)
              • Backup set size: Actually I am not sure as there's no quick and easy way to determine that. Looking at the snapshot folder and the latest backup folder the total appears to be around 38GB with the size of my /home/yellowtent directory being 39G.

              I was trying to find out from the logs how long the backup ran for but couldn't locate anything useful. So I ended up looking at the bandwidth logs, which look like they suggest that it took 45min. That's because Cloudron doesn't log this and Hetzner's Cloud console only gives a sh*t resolution. My backups run from 22:00.

              my-apps-bebraver-cloud-路-Hetzner-Cloud (1).png

              I have now installed vnstat to monitor it in more detail and wondered what your experiences were.

              To restore, to a new VM in the same location, it took around 30min. Again, this lacks some hard data and I'll see if I can rig up a scientifically more useful test if anyone is interested.

              Could this work?
              One scenario I think might work, is to mount a block storage volume to the box locally, encrypt it using LUKS full disk encryption, back up to that volume without encryption using rsync and restore from it. Then using rclone with crypt to shovel the backups to a remote location separately using cron.

              If the box fails, remount the block storage to the new VM (think of it like a USB disk) and restore from it. 馃

              How big are your backups, what method do you use and how long does it take?

              timconsidineT Offline
              timconsidineT Offline
              timconsidine
              App Dev
              wrote on last edited by
              #5

              @3246 I use scaleway
              Seems to be about 150Gb
              Takes about 30 mins
              Fails occasionally - not sure why - I just do it again - I'm guessing some network connection issue

              1 Reply Last reply
              2
              • 32463 3246

                TL;DR: I am assuming that using a local disk and no encryption is the fastest but in production that just wouldn't fly for me 馃槈

                39G apps -> 38G backup ~ 45min via rsync with encryption to remote CIFS location; restore 30min

                Currently, I am using the following settings (times are estimates, not hard data):

                • Server location: Hetzner (Nuremberg)
                • Backup location: Hetzner (Falkenstein)
                • Backup type: StorageBox (CIFS)
                • Backup method: rsync (with encryption)
                • Backup set size: Actually I am not sure as there's no quick and easy way to determine that. Looking at the snapshot folder and the latest backup folder the total appears to be around 38GB with the size of my /home/yellowtent directory being 39G.

                I was trying to find out from the logs how long the backup ran for but couldn't locate anything useful. So I ended up looking at the bandwidth logs, which look like they suggest that it took 45min. That's because Cloudron doesn't log this and Hetzner's Cloud console only gives a sh*t resolution. My backups run from 22:00.

                my-apps-bebraver-cloud-路-Hetzner-Cloud (1).png

                I have now installed vnstat to monitor it in more detail and wondered what your experiences were.

                To restore, to a new VM in the same location, it took around 30min. Again, this lacks some hard data and I'll see if I can rig up a scientifically more useful test if anyone is interested.

                Could this work?
                One scenario I think might work, is to mount a block storage volume to the box locally, encrypt it using LUKS full disk encryption, back up to that volume without encryption using rsync and restore from it. Then using rclone with crypt to shovel the backups to a remote location separately using cron.

                If the box fails, remount the block storage to the new VM (think of it like a USB disk) and restore from it. 馃

                How big are your backups, what method do you use and how long does it take?

                murgeroM Offline
                murgeroM Offline
                murgero
                App Dev
                wrote on last edited by murgero
                #6

                @3246 They should add a way to add 2 backup jobs setup however you like:

                For Me:

                Job 1 - Backup to Local external disk for fast access in case of emergency

                Job 2 - Offsite backup for long-storage.

                Both encrypted, and this offers a quick restore (in the event of just needing a quick restore) and one for proper emergencies

                Not that you even need to set it up like this, but more backups == better backups even if one is local for emergency "non-natural-disaster" restore.

                --
                https://urgero.org
                ~ Professional Nerd. Freelance Programmer. ~

                1 Reply Last reply
                3
                • girishG girish marked this topic as a regular topic on
                • E Offline
                  E Offline
                  eddowding
                  wrote on last edited by
                  #7

                  I have 175GB and it takes about 12h 馃槥 I guess something is very wrong.

                  jdaviescoatesJ 1 Reply Last reply
                  0
                  • L Offline
                    L Offline
                    LoudLemur
                    wrote on last edited by
                    #8

                    You have probably already considered keeping your backups with a different VPS provider, just in case for some reason the one company suddenly decides to end the service without notification.

                    1 Reply Last reply
                    3
                    • E Offline
                      E Offline
                      eddowding
                      wrote on last edited by
                      #9

                      It's backed up to idrive.

                      1 Reply Last reply
                      0
                      • E eddowding

                        I have 175GB and it takes about 12h 馃槥 I guess something is very wrong.

                        jdaviescoatesJ Offline
                        jdaviescoatesJ Offline
                        jdaviescoates
                        wrote on last edited by jdaviescoates
                        #10

                        @eddowding said in What's the fastest backup and restore method?:

                        I have 175GB and it takes about 12h 馃槥 I guess something is very wrong.

                        Yeah, my backup of about 350GB takes about 2 hours (from a Hetzner Dedicated Server in Finland Helsinki (HEL1) to a Heztner Storage Box in Falkenstein (FSN1))

                        IMHO your issue is very likely due to those "too good to be true" iDrive offer pricing they had which I guess you went for.

                        I use Cloudron with Gandi & Hetzner

                        1 Reply Last reply
                        3
                        Reply
                        • Reply as topic
                        Log in to reply
                        • Oldest to Newest
                        • Newest to Oldest
                        • Most Votes


                          • Login

                          • Don't have an account? Register

                          • Login or register to search.
                          • First post
                            Last post
                          0
                          • Categories
                          • Recent
                          • Tags
                          • Popular
                          • Bookmarks
                          • Search