Cloudron Forum

Ah this is good information and indeed a likely a problem for rsync backups with a lot of files (not sure how the rate-limit turns out in the end though)

thanks, fixed in https://git.cloudron.io/cloudron/box/-/commit/1d1a7af48ebd41fa9a54f73c124a708c226e38e8

Found https://rsync.samba.narkive.com/DFt4GEa9/over-stdin-stdout which has some interesting discussion.

@eddowding said in What's the fastest backup and restore method?:

I have 175GB and it takes about 12h 😞 I guess something is very wrong.

Yeah, my backup of about 350GB takes about 2 hours (from a Hetzner Dedicated Server in Finland Helsinki (HEL1) to a Heztner Storage Box in Falkenstein (FSN1))

IMHO your issue is very likely due to those "too good to be true" iDrive offer pricing they had which I guess you went for.

@girish Thanks, that works.

@msbt I also forgot about that option actually, such a long time ago when I wrote all that code 😄

For restic, can you upvote here - https://forum.cloudron.io/topic/1575/backup-improvements-restic-backend ? It should be possible to add a new restic backend like tgz/rsync. Though, we still have to figure out how we can "support" it. But that's a separate topic.

@girish my initial questions are all resolved now. I'm having some new problems though. Namely that it seems the Cloudron installer is installing Gnome which is making my server enter sleep mode. I disabled it now but I'm confused as to why Gnome is being installed.

@girish Sounds good, thanks.

@robi The fsmetadata.json is only used in rsync backups. It's used to keep track of empty directories and 'x' bit of files when using rsync mode. We do this because object storage services (like s3, gcs and friends) cannot track this since they are not a filesystem but an object store.

This is not needed in tgz because tgz format can encode this information inside the tarball.

In essence, fsmetadata.json is usually owned by root. If it's owned by yellowtent, it's probably because maybe you restored the Cloudron in the past or something. The restore logic changes the permission of all files to yellowtent after downloading so that restore can continue as non-root user.

@girish Ah okay, that makes sense on the tgz part. Unfortunately I haven't had much success at all in any quicker times (in fact it's often closer to double the time than tgz, taking between 1-2 hours) when using rsync in my tests for OVH Object Storage. But maybe I just haven't found that 'sweet spot' yet. I'll keep testing. Thanks Girish.