Backup: 40 GB, 6+ hours – time to switch to tarball?

[thoresson]

I use Netcup as my host, in one of their datacenters in Germany, and do my backups to Wasabi, also in Germany, using rsync. Even though my Cloudron installation is only 40 GB, backups take at least 6 hours. Six apps installed: Mastodon, Ghost, Adguard, FreshRSS, Uptime Kuma, and Umami. Guess the reason is a bunch of small files.

I went with rsync for incremental backups at less storage space. But I am starting to reconsider that, and perhaps switch to tarball.

What backup strategies do everyone else have?

[matheustav]

Hi.
My Cloudron server, which is hosted by Netcup too, is using at this moment 73GB and it has 14 apps installed.

For backups I'm using IDrive e2. I don't know Wasabi, but e2 is amazing and very cheap. This month for example I used a total of 409GB and they charged me $1.64. I'm using it for a year already.

Each backup is taking about 40 mins and generates a total of 24GB. I have both tarball and encryption enabled.

I think using tarball would reduce a lot of the time to backup your system and the size of the snapshots too.

[necrevistonnezr]

AFAIK Mastodon is difficult and often produces backup hickups (https://forum.cloudron.io/search?term=Mastodon backup error&in=titlesposts).
Mastodon has lots of small files, I believe.
From the docs:

The rsync format uploads individual files to the backup storage. It keeps track of what was copied the last time around, detects what changed locally and uploads only the changed files on every backup. Note that despite uploading 'incrementally', tgz format can be significantly faster when uploading a large number of small files (like source code repositories) because of the large number HTTP requests that need to be made for each file.

[thoresson]

Switched to tarball, and it made a huge difference!