Cloudron makes it easy to run web apps like WordPress, Nextcloud, GitLab on your server. Find out more or install now.


Skip to content
  • Categories
  • Recent
  • Tags
  • Popular
  • Bookmarks
  • Search
Skins
  • Light
  • Cerulean
  • Cosmo
  • Flatly
  • Journal
  • Litera
  • Lumen
  • Lux
  • Materia
  • Minty
  • Morph
  • Pulse
  • Sandstone
  • Simplex
  • Sketchy
  • Spacelab
  • United
  • Yeti
  • Zephyr
  • Dark
  • Cyborg
  • Darkly
  • Quartz
  • Slate
  • Solar
  • Superhero
  • Vapor

  • Default (No Skin)
  • No Skin
Collapse
Brand Logo

Cloudron Forum

Apps | Demo | Docs | Install
  1. Cloudron Forum
  2. Discuss
  3. Tutorial: remote backup of local Cloudron backup snapshots with restic / rclone

Tutorial: remote backup of local Cloudron backup snapshots with restic / rclone

Scheduled Pinned Locked Moved Discuss
resticrclonebackups
16 Posts 7 Posters 2.9k Views 9 Watching
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • necrevistonnezrN necrevistonnezr

    added command to list all available snapshots to original post

    necrevistonnezrN Offline
    necrevistonnezrN Offline
    necrevistonnezr
    wrote on last edited by necrevistonnezr
    #6

    Note that (very effective) compression using zstd has been added to restic beta recently (beta versions are quite stable) - my repository went from 221 GB down to 180 GB with default settings. I have updated the tutorial to reflect this and also a section for migration of existing repos.

    1 Reply Last reply
    7
    • marcusquinnM Offline
      marcusquinnM Offline
      marcusquinn
      wrote on last edited by
      #7

      Very handy, bookmarking this post for future learning.

      Also handy for Rsync with OSX: https://github.com/rsyncOSX/RsyncOSX

      Web Design https://www.evergreen.je
      Development https://brandlight.org
      Life https://marcusquinn.com

      1 Reply Last reply
      0
      • necrevistonnezrN necrevistonnezr

        As suggested here, I turned my old post into a separate topic:

        This is what I use for remote backups of my local Cloudron backup snapshots (done by rsync) via restic / rclone to Onedrive.

        restic is a robust backup solution for incremental, encrypted, mountable(!) backups to local and remote storage. rclone, an equally robust sync software, is just a "transporter tool" that expands the available remote storages by a lot.

        Maybe it can be a starting point and some inspiration for your personal needs.

        Tools

        • rclone: https://rclone.org/docs/
        • restic: https://restic.readthedocs.io/en/stable/030_preparing_a_new_repo.html#other-services-via-rclone
        • ssmtp: https://wiki.archlinux.org/title/SSMTP

        Installation

        • Install tools above via apt
        • afterwards update to latest version (repo versions are old): sudo restic self-update && sudo rclone selfupdate

        Setup rclone

        • Enter an interactive setup process via rclone config
        • in my case I use Onedrive as it has 1TB of space coming with my Office 365 subscription
        • for the rest of this summary, we assume you gave it the repository name "REPOSITORY"
        • details at https://rclone.org/commands/rclone_config/

        Setup restic

        • set up a backup repository restic -r rclone:REPOSITORY init (for compression support add --repository-version 2- recommended!)
        • for a subfolder on onedrive just use restic -r rclone:REPOSITORY:subfolder init (for compression support add --repository-version 2 - recommended!)
        • save password that you gave the repository in file /home/USER/resticpw
        • details at https://restic.readthedocs.io/en/latest/030_preparing_a_new_repo.html#other-services-via-rclone

        Setup SSMTP

        • for receiving backup results, otherwise not needed
        • See https://wiki.archlinux.org/title/SSMTP

        Cloudron Backup settings

        • Provider: mountpoint
        • Location: /media/CloudronBackup (<-- obviously adjust to your settings)
        • this creates a snapshot at /media/CloudronBackup/snapshot for the current backup
        • Storage Format: rsync
        • Adjust schedule and retention to your liking

        Backup, Prune and Check scripts

        restic-cron-backup.sh: The actual backup
        #!/bin/bash
        d=$(date +%Y-%m-%d)
        if pidof -o %PPID -x “$0”; then
        echo “$(date “+%d.%m.%Y %T”) Exit, already running.”
        exit 1
        fi
        restic -r rclone:REPOSITORY:subfolder backup /media/CloudronBackup/snapshot -p=/home/USER/resticpw 
        restic -r rclone:REPOSITORY:subfolder forget --keep-monthly 12 --keep-weekly 5 --keep-daily 14 -p=/home/USER/resticpw
        restic -r rclone:REPOSITORY:subfolder check --read-data-subset=2% -p=/home/USER/resticpw
        exit
        

        First line does the backup (incremental, encrypted), second line is the backup retention, third line checks a random 2 % of all data for errors.
        Note that I only backup the /snapshot folder as all versioning is done by restic.
        For compression, add --compression auto (or max) to the backup command.

        restic-cron-prune.sh: Pruning unused files in the backup
        #!/bin/bash
        d=$(date +%Y-%m-%d)
        if pidof -o %PPID -x “$0”; then
        echo “$(date “+%d.%m.%Y %T”) Exit, already running.”
        exit 1
        fi
        restic -r rclone:REPOSITORY:subfolder prune -p=/home/USER/resticpw
        exit
        

        removes unused data from the repository, I run this once a week

        restic-cron-check.sh: thorough health check of the backups
        #!/bin/bash
        d=$(date +%Y-%m-%d)
        if pidof -o %PPID -x “$0”; then
        echo “$(date “+%d.%m.%Y %T”) Exit, already running.”
        exit 1
        fi
        restic -r rclone:REPOSITORY:subfolder check --read-data -p=/home/USER/resticpw
        exit
        

        checks all data for errors, I run this once a week

        Crontab

        30 2 * * * sh /home/USER/restic-cron-backup.sh | mailx -s "Restic Backup Results" server@mydomain.com
        1 5 1 * * sh /home/USER/restic-cron-prune.sh | mailx -s "Restic Prune Results" server@mydomain.com
        1 8 1 * * sh /home/USER/restic-cron-check.sh | mailx -s "Restic Full Check Results" server@mydomain.com
        

        Backup daily at 2:30, prune and check once a week. Receive results to specified mail

        Mount backups

        Just to be complete: You can mount restic backups locally like
        restic -r rclone:REPOSITORY:subfolder mount /media/resticmount/ -p=/home/USER/resticpw && cd /media/resticmount
        obviously adjust /media/resticmount/to your settings; allows you to browse and copy from full snapshots for each backup

        List backups

        For listing all available snapshots use
        restic -r rclone:REPOSITORY:subfolder snapshots -p=/home/USER/resticpw

        Migrate existing backups to compressed backups

        For migrating existing repos to compressed repos use these two steps (will take long!)

        • restic -r rclone:REPOSITORY:subfolder migrate upgrade_repo_v2 -p=/home/USER/resticpw
        • restic -r rclone:REPOSITORY:subfolder prune --repack-uncompressed -p=/home/USER/resticpw

        See https://restic.readthedocs.io/en/latest/045_working_with_repos.html#upgrading-the-repository-format-version for details.

        murgeroM Offline
        murgeroM Offline
        murgero
        App Dev
        wrote on last edited by
        #8

        @necrevistonnezr Very in-depth. I don't use Cloudron anymore, but this is gonna be really good for admins looking for something easy to manage.

        --
        https://urgero.org
        ~ Professional Nerd. Freelance Programmer. ~

        necrevistonnezrN 1 Reply Last reply
        0
        • murgeroM murgero

          @necrevistonnezr Very in-depth. I don't use Cloudron anymore, but this is gonna be really good for admins looking for something easy to manage.

          necrevistonnezrN Offline
          necrevistonnezrN Offline
          necrevistonnezr
          wrote on last edited by
          #9

          @murgero said in Tutorial: remote backup of local Cloudron backup snapshots with restic / rclone:

          @necrevistonnezr Very in-depth. I don't use Cloudron anymore, but this is gonna be really good for admins looking for something easy to manage.

          Thanks! Can I ask why you’re not using Cloudron anymore…?

          murgeroM 1 Reply Last reply
          1
          • necrevistonnezrN necrevistonnezr

            @murgero said in Tutorial: remote backup of local Cloudron backup snapshots with restic / rclone:

            @necrevistonnezr Very in-depth. I don't use Cloudron anymore, but this is gonna be really good for admins looking for something easy to manage.

            Thanks! Can I ask why you’re not using Cloudron anymore…?

            murgeroM Offline
            murgeroM Offline
            murgero
            App Dev
            wrote on last edited by
            #10

            @necrevistonnezr No bad reasoning here, just needed a few more features and found something else to use. Cloudron is and always will have a special spot in my heart. Love the Cloudron team, and I'll still be around the forums too

            --
            https://urgero.org
            ~ Professional Nerd. Freelance Programmer. ~

            jdaviescoatesJ 1 Reply Last reply
            0
            • murgeroM murgero

              @necrevistonnezr No bad reasoning here, just needed a few more features and found something else to use. Cloudron is and always will have a special spot in my heart. Love the Cloudron team, and I'll still be around the forums too

              jdaviescoatesJ Offline
              jdaviescoatesJ Offline
              jdaviescoates
              wrote on last edited by
              #11

              @murgero said in Tutorial: remote backup of local Cloudron backup snapshots with restic / rclone:

              just needed a few more features and found something else to use.

              What are you using now?

              I use Cloudron with Gandi & Hetzner

              murgeroM 1 Reply Last reply
              0
              • jdaviescoatesJ jdaviescoates

                @murgero said in Tutorial: remote backup of local Cloudron backup snapshots with restic / rclone:

                just needed a few more features and found something else to use.

                What are you using now?

                murgeroM Offline
                murgeroM Offline
                murgero
                App Dev
                wrote on last edited by
                #12

                @jdaviescoates Technically speaking I still use Cloudron, just not for my email. I should have made that clear.

                Currently using MailCow.

                --
                https://urgero.org
                ~ Professional Nerd. Freelance Programmer. ~

                jdaviescoatesJ 1 Reply Last reply
                1
                • murgeroM murgero

                  @jdaviescoates Technically speaking I still use Cloudron, just not for my email. I should have made that clear.

                  Currently using MailCow.

                  jdaviescoatesJ Offline
                  jdaviescoatesJ Offline
                  jdaviescoates
                  wrote on last edited by
                  #13

                  @murgero said in Tutorial: remote backup of local Cloudron backup snapshots with restic / rclone:

                  Currently using MailCow.

                  Just out of interest, which features does MailCow have that Cloudron's email offering lacks? Thanks!

                  I use Cloudron with Gandi & Hetzner

                  murgeroM 1 Reply Last reply
                  0
                  • jdaviescoatesJ jdaviescoates

                    @murgero said in Tutorial: remote backup of local Cloudron backup snapshots with restic / rclone:

                    Currently using MailCow.

                    Just out of interest, which features does MailCow have that Cloudron's email offering lacks? Thanks!

                    murgeroM Offline
                    murgeroM Offline
                    murgero
                    App Dev
                    wrote on last edited by
                    #14

                    @jdaviescoates domain admins, quotas, oauth for 3rd party apps, and rspamd to name a few

                    --
                    https://urgero.org
                    ~ Professional Nerd. Freelance Programmer. ~

                    marcusquinnM 1 Reply Last reply
                    3
                    • murgeroM murgero

                      @jdaviescoates domain admins, quotas, oauth for 3rd party apps, and rspamd to name a few

                      marcusquinnM Offline
                      marcusquinnM Offline
                      marcusquinn
                      wrote on last edited by
                      #15

                      @murgero Worth logging each of them as feature requests here given your good experience with both?

                      Web Design https://www.evergreen.je
                      Development https://brandlight.org
                      Life https://marcusquinn.com

                      1 Reply Last reply
                      2
                      • ? Offline
                        ? Offline
                        A Former User
                        wrote on last edited by
                        #16

                        This is super helpful. I’m hoping we see something like this as a feature baked in though. Like multiple backup sources with different schedules

                        1 Reply Last reply
                        3
                        • necrevistonnezrN necrevistonnezr referenced this topic on
                        • M michaelpope referenced this topic on
                        • necrevistonnezrN necrevistonnezr referenced this topic on
                        • necrevistonnezrN necrevistonnezr referenced this topic on
                        • necrevistonnezrN necrevistonnezr referenced this topic on
                        Reply
                        • Reply as topic
                        Log in to reply
                        • Oldest to Newest
                        • Newest to Oldest
                        • Most Votes


                        • Login

                        • Don't have an account? Register

                        • Login or register to search.
                        • First post
                          Last post
                        0
                        • Categories
                        • Recent
                        • Tags
                        • Popular
                        • Bookmarks
                        • Search