Cloudron Forum

Unfortunately, we ended up breaking https://forum.cloudron.io/topic/11993/backuptask-crashing-after-8-0-update . But yeah, overall the fix is good.

The issue here had to do with something in JupyterHub container creating a massive number of folders recursively. Think 300k folders or something. Nuking all that stuff, made backups work again.

This is not possible, no. The structure of data of both the apps are quite different. Also, when you backup Cloudron apps, the code is not part of the backup. LAMP app is sort of a special case here since the code is part of the "data" and part of backup .

You should be able to migrate it manually though. Import database and copy over the needed files.

Well, I thought you would say that.

But how about at least make some suitable limits? E.g. the memory limit shouldn't be higher than what I can afford based on my usage. Also, if I can only upload up to three parts simultaneously that require as much memory, why can the memory limit and the upload part size differ? Wouldn't it make more sense to have one setting for both?

@ChristopherMag Wow, what a great story, with a happy ending, too!

Thank you for taking the time to tell us the tale.

@girish Hello, no the backup sometimes fails with this error but I don't have any more information. I suspect the bandwidth because I have several servers which backup at night to the same destinations even if the times are staggered, it can happen that the backups are in parallel.

@girish got it, thanks 🙂

@girish said in Bunny Storage for Backups:

BTW, looks like they said S3 support is coming 2 years ago - https://bunny.net/blog/introducing-edge-storage-sftp-support-s3-next/

There is also a bit of an update here https://bunny.net/blog/whats-happening-with-s3-compatibility/

@girish Thank you very much !

@mic-testing-123 We have replied from there asking for more info. (specifically the backup backend and the backup logs).

@DualOSWinWiz yes, correct. This looks like some kernel related issue or some missing ubuntu package. Can you try to make the NFS mounting work with just the CLI . This will remove Cloudron out of the equation. I think it is possible some missing package or miscompiled kernel.

I can now say safely that the backuptask crashes while working with Immich.

Maybe I should choose different setting? I switched to Backblaze recently and I am currently using rsync. Maybe there is a way to do this more efficiently considering that there are so many files to upload? I was assuming rsync might be more efficient than a tarball because it only syncs the changes, but I might be wrong?

@archos for bugs like Synapse, I am fixing our code to have better logs . Off late, there's been a bunch of backup related issues. It's not clear (yet) whether this is some bug in node or some node module or the upstream provider. The error happens randomly and the logs are not helpful. I will update this post when I have a patch.

@TheMoodBoardz Does it fail with tgz format also? Not sure how to debug this since it is just crashing with no real logs. If you reach out at support@cloudron.io , we can debug further.

"Thank's" is like....
image.png

Ah, thanks! I hadn't seen the app-mail-folders before. 🙂

@girish I tried this one, but i get a sad no-go :
Error put object cloudron-testfile. Message: Socket timed out without establishing a connection HTTP Code: TimeoutError
I'll wait the next update 🙂
Tx for the patch

no sorry. Just coincidentally having a similar issue with backups, at the same time I think. I can see why you'd think that, I kind of jumped into the conversation.

@girish I don't think it requires seal encryption.

I believe using seal encryption causes a different code path to be taken that avoids the underlying bug but it is not the presence or absence of seal encryption in and of itself that I believe fixes the issue.

After a kernel upgrade this works fine without seal encryption though I have now enabled it as it seems like a good idea to have anyway so making it the default is probably a good idea but as long as Ubuntu 22.04's default kernel version is 5.15.0-102 this issue will likely still occur.