Tutorial: remote backup of local Cloudron backup snapshots with restic / rclone

[necrevistonnezr]

As suggested here, I turned my old post into a separate topic:

This is what I use for remote backups of my local Cloudron backup snapshots (done by rsync) via restic / rclone to Onedrive.

restic is a robust backup solution for incremental, encrypted, mountable(!) backups to local and remote storage. rclone, an equally robust sync software, is just a "transporter tool" that expands the available remote storages by a lot.

Maybe it can be a starting point and some inspiration for your personal needs.

Tools

• rclone: https://rclone.org/docs/
• restic: https://restic.readthedocs.io/en/stable/030_preparing_a_new_repo.html#other-services-via-rclone (latest snapshot: https://beta.restic.net/latest/ or directly e.g. https://beta.restic.net/latest_restic_linux_amd64)
• ssmtp: https://wiki.archlinux.org/title/SSMTP

Installation

• Install tools above via apt
• afterwards update to latest version (repo versions are old): sudo restic self-update && sudo rclone selfupdate (for latest beta use https://beta.restic.net/latest/ or e.g. https://beta.restic.net/latest_restic_linux_amd64)

Setup rclone

• Enter an interactive setup process via rclone config
• in my case I use Onedrive as it has 1TB of space coming with my Office 365 subscription
• for the rest of this summary, we assume you gave it the repository name "REPOSITORY"
• details at https://rclone.org/commands/rclone_config/

Setup restic

• set up a backup repository restic -r rclone:REPOSITORY init (for compression support add --repository-version 2 if you use https://beta.restic.net/latest/)
• for a subfolder on onedrive just use restic -r rclone:REPOSITORY:subfolder init (for compression support add --repository-version 2 https://beta.restic.net/latest/)
• save password that you gave the repository in file /home/USER/resticpw
• details at https://restic.readthedocs.io/en/latest/030_preparing_a_new_repo.html#other-services-via-rclone

Setup SSMTP

• for receiving backup results, otherwise not needed
• See https://wiki.archlinux.org/title/SSMTP

Cloudron Backup settings

• Provider: mountpoint
• Location: /media/CloudronBackup (<-- obviously adjust to your settings)
• this creates a snapshot at /media/CloudronBackup/snapshot for the current backup
• Storage Format: rsync
• Adjust schedule and retention to your liking

Backup, Prune and Check scripts

restic-cron-backup.sh: The actual backup

#!/bin/bash
d=$(date +%Y-%m-%d)
if pidof -o %PPID -x “$0”; then
echo “$(date “+%d.%m.%Y %T”) Exit, already running.”
exit 1
fi
restic -r rclone:REPOSITORY:subfolder backup /media/CloudronBackup/snapshot -p=/home/USER/resticpw 
restic -r rclone:REPOSITORY:subfolder forget --keep-monthly 12 --keep-weekly 5 --keep-daily 14 -p=/home/USER/resticpw
restic -r rclone:REPOSITORY:subfolder check --read-data-subset=2% -p=/home/USER/resticpw
exit

First line does the backup (incremental, encrypted), second line is the backup retention, third line checks a random 2 % of all data for errors.
Note that I only backup the /snapshot folder as all versioning is done by restic.
For compression, add --compression auto (or max) to the backup command.

restic-cron-prune.sh: Pruning unused files in the backup

#!/bin/bash
d=$(date +%Y-%m-%d)
if pidof -o %PPID -x “$0”; then
echo “$(date “+%d.%m.%Y %T”) Exit, already running.”
exit 1
fi
restic -r rclone:REPOSITORY:subfolder prune -p=/home/USER/resticpw
exit

removes unused data from the repository, I run this once a week

restic-cron-check.sh: thorough health check of the backups

#!/bin/bash
d=$(date +%Y-%m-%d)
if pidof -o %PPID -x “$0”; then
echo “$(date “+%d.%m.%Y %T”) Exit, already running.”
exit 1
fi
restic -r rclone:REPOSITORY:subfolder check --read-data -p=/home/USER/resticpw
exit

checks all data for errors, I run this once a week

Crontab

30 2 * * * sh /home/USER/restic-cron-backup.sh | mailx -s "Restic Backup Results" server@mydomain.com
1 5 1 * * sh /home/USER/restic-cron-prune.sh | mailx -s "Restic Prune Results" server@mydomain.com
1 8 1 * * sh /home/USER/restic-cron-check.sh | mailx -s "Restic Full Check Results" server@mydomain.com

Backup daily at 2:30, prune and check once a week. Receive results to specified mail

Mount backups

Just to be complete: You can mount restic backups locally like
restic -r rclone:REPOSITORY:subfolder mount /media/resticmount/ -p=/home/USER/resticpw && cd /media/resticmount
obviously adjust /media/resticmount/to your settings; allows you to browse and copy from full snapshots for each backup

List backups

For listing all available snapshots use
restic -r rclone:REPOSITORY:subfolder snapshots -p=/home/USER/resticpw

Migrate existing backups to compressed backups

For migrating existing repos to compressed repos use these two steps (will take long!)

• restic -r rclone:REPOSITORY:subfolder migrate upgrade_repo_v2 -p=/home/USER/resticpw
• restic -r rclone:REPOSITORY:subfolder prune --repack-uncompressed -p=/home/USER/resticpw

See https://restic.readthedocs.io/en/latest/045_working_with_repos.html#upgrading-the-repository-format-version for details.

[timconsidine]

@necrevistonnezr nice article.
Great fan of rclone.
I use it with wasabi and scaleway.

Don't know restic so I'm off to learn.

Becoming fan of https://ntfy.sh : great for adding into scripts run by cron to confirm running/script results/server status (e.g. df -h)

[robi]

@timconsidine said in Tutorial: remote backup of local Cloudron backup snapshots with restic / rclone:

Becoming fan of https://ntfy.sh :

Nice find, that's totally self-hostable, you should add it to the list!

We had several instances where we needed a way to have notifications for cloudron apps/events. This would do the trick!

It can also be made a feature request for box notifications to support an API or curl call for admins.

[timconsidine]

@robi I've been meaning to have a go at packaging it.
But I stumbled on something (I forget what).
Will try again.
And will open a wishlist for someone more talented than me to have a crack.

[necrevistonnezr]

added command to list all available snapshots to original post

[necrevistonnezr]

Note that (very effective) compression using zstd has been added to restic beta recently (beta versions are quite stable) - my repository went from 221 GB down to 180 GB with default settings. I have updated the tutorial to reflect this and also a section for migration of existing repos.

[marcusquinn]

Very handy, bookmarking this post for future learning.

Also handy for Rsync with OSX: https://github.com/rsyncOSX/RsyncOSX

[murgero]

@necrevistonnezr Very in-depth. I don't use Cloudron anymore, but this is gonna be really good for admins looking for something easy to manage.

[necrevistonnezr]

@murgero said in Tutorial: remote backup of local Cloudron backup snapshots with restic / rclone:

@necrevistonnezr Very in-depth. I don't use Cloudron anymore, but this is gonna be really good for admins looking for something easy to manage.

Thanks! Can I ask why you’re not using Cloudron anymore…?

[murgero]

@necrevistonnezr No bad reasoning here, just needed a few more features and found something else to use. Cloudron is and always will have a special spot in my heart. Love the Cloudron team, and I'll still be around the forums too

[jdaviescoates]

@murgero said in Tutorial: remote backup of local Cloudron backup snapshots with restic / rclone:

just needed a few more features and found something else to use.

What are you using now?

[murgero]

@jdaviescoates Technically speaking I still use Cloudron, just not for my email. I should have made that clear.

Currently using MailCow.

[jdaviescoates]

@murgero said in Tutorial: remote backup of local Cloudron backup snapshots with restic / rclone:

Currently using MailCow.

Just out of interest, which features does MailCow have that Cloudron's email offering lacks? Thanks!

[murgero]

@jdaviescoates domain admins, quotas, oauth for 3rd party apps, and rspamd to name a few

[marcusquinn]

@murgero Worth logging each of them as feature requests here given your good experience with both?

[A Former User]

This is super helpful. I’m hoping we see something like this as a feature baked in though. Like multiple backup sources with different schedules