Backup is fine for app, but fails for general cloudron backup

[privsec]

Exactly as it says.
I keep getting backup failed notices but when I manually backup the app it works just fine

[nebulon]

To help here, we need a lot more context, like error logs, storage provider and backup type used...

[ekevu123]

More logs would help indeed, but I had a similar issue recently where I ran into rate limits of the backup provider. So, making the major backup of the whole cloudron helped me get this through successfully and then I trigger the big apps (e.g. Immich) manually at different times of the day.

[nebulon]

If such rate-limits are hit, it may not be the ideal backup provider then

[privsec]

So what do you need and where?

I see this

box:tasks update 11938: {"percent":94.02325581395341,"message":"Uploading backup 27648M@0MBps (pictures.domain.com)"}
Jul 08 16:15:33 box:tasks update 11938: {"percent":94.02325581395341,"message":"Uploading backup 27648M@0MBps (pictures.domain.com)"}
Jul 08 16:15:43 box:tasks update 11938: {"percent":94.02325581395341,"message":"Uploading backup 27648M@0MBps (pictures.domain.com)"}
Jul 08 16:15:53 box:tasks update 11938: {"percent":94.02325581395341,"message":"Uploading backup 27648M@0MBps (pictures.domain.com)"}
Jul 08 16:16:03 box:tasks update 11938: {"percent":94.02325581395341,"message":"Uploading backup 27648M@0MBps (pictures.domain.com)"}
Jul 08 16:16:13 box:tasks update 11938: {"percent":94.02325581395341,"message":"Uploading backup 27648M@0MBps (pictures.domain.com)"}

And I have these settings

Further I am using iDrive

[privsec]

I also see these logs
Jul 10 23:10:39 12:M 11 Jul 2024 05:10:39.017 * 10 changes in 300 seconds. Saving...
Jul 10 23:10:39 12:M 11 Jul 2024 05:10:39.017 * Background saving started by pid 1355
Jul 10 23:10:39 1355:C 11 Jul 2024 05:10:39.068 * DB saved on disk
Jul 10 23:10:39 1355:C 11 Jul 2024 05:10:39.069 * Fork CoW for RDB: current 1 MB, peak 1 MB, average 1 MB
Jul 10 23:10:39 12:M 11 Jul 2024 05:10:39.118 * Background saving terminated with success

[nebulon]

This appears to be the backup stalling issue which we have seen a few times now. A fix is coming with the next Cloudron version.

[girish]

@privsec we have narrowed down the stalling issue. I am testing the fix on various servers (https://git.cloudron.io/cloudron/box/-/commit/1dc6b40a68067f705230f87d26fbd6e391798e93) . The main issue is that the files are changing when we are creating a tar ball. Tarball format requires us to know the size of the file and then stream the file. But if the size of the file changes when we are streaming it, we hit issues. The traditional tar tool usually gives a warning "file changed as we read it" for this situation.

[privsec]

Ahh ok. Interesting.

If this helps, I have two apps for photos. One app doesnt get any updates to it anymore (like new photos) and the other app does. Both are having this issue

What should I do in the mean time as I have now gone a whle without a full system back up

[joseph]

@privsec a quick workaround (until we have a fix) is to stop the app and take a backup. This will most likely succeed. Stopped apps cannot be backed up ...