Backup issue in rsync non-encrypted filesystem type. Never moves past status of "Removing directory" for one particular app.

[nebulon]

Not sure where exactly to respond with the various topics open. However guessing from the others, is it correct that this folder is cifs mounted? Both the ENAMETOOLONG and possibly the directory removal might be related to the filesystem type. Can you testwise do a backup of that app to the local filesystem to see if the issues persist?

[d19dotca]

This post is deleted!

[d19dotca]

@nebulon Yes, it is CIFS. Per your response in the other thread, you had stated you test with CIFS, not really as much with NFS so you recommended CIFS over NFS. I mounted it with CIFS and now have these issues. I'm not sure I have enough disk space locally to actually do that, it'd fill it up pretty quickly. Although if I can get away with only doing that one app then that would definitely be fine. I'll try that now and see what happens.

[girish]

This seems to be some filesystem related issue and not related to Cloudron. Specifically, we need to figure out why this fails:

rm -rf /cloudron-backups/snapshot/app_a4c34340-d201-4a5e-8d78-d043d717dcfd
rm: cannot remove '/cloudron-backups/snapshot/app_a4c34340-d201-4a5e-8d78-d043d717dcfd/data/_data_/_default_/storage/cfg': Directory not empty

If you cannot remove from the shell, cloudron cannot remove it either. What is in that directory? Is it empty or some NFS/CIFS bug?

[d19dotca]

@nebulon I was just able to test the Rainloop app backup on a local /tmp directory and yes it worked fine. So it's a combination of that app and CIFS it seems.

[d19dotca]

@girish It's possible, however I think it's odd that this works fine for every other app but this one, so I'd be hesitant to say it's only a CIFS issue. I was able to run a test a moment ago and can verify it works fine on local disk, but why would this one app have a problem on a CIFS-mounted disk and none of the other apps? Seems strange, no?

Btw, if Cloudron can't complete an operation like removing a directory, I'd suggest it fail fast. I left it for hours before thinking it might finally do it and it never did. I'm not sure how long that backup would have kept running for.

[d19dotca]

@girish Here is what's in that directory:

ls -alh /cloudron-backups/snapshot/app_a4c34340-d201-4a5e-8d78-d043d717dcfd/data/data/default/storage/cfg/

total 0
drwxr-xr-x 2 yellowtent yellowtent 0 Apr 1 16:39 .
drwxr-xr-x 2 yellowtent yellowtent 0 Apr 1 16:39 ..
drwxr-xr-x 2 yellowtent yellowtent 0 Apr 1 16:39 al
drwxr-xr-x 2 yellowtent yellowtent 0 Apr 1 16:39 au
drwxr-xr-x 2 yellowtent yellowtent 0 Apr 1 16:39 br
drwxr-xr-x 2 yellowtent yellowtent 0 Apr 1 16:39 cl
drwxr-xr-x 2 yellowtent yellowtent 0 Apr 1 16:39 do
drwxr-xr-x 2 yellowtent yellowtent 0 Apr 1 16:39 du
drwxr-xr-x 2 yellowtent yellowtent 0 Apr 1 16:39 he
drwxr-xr-x 2 yellowtent yellowtent 0 Apr 1 16:39 j.
drwxr-xr-x 2 yellowtent yellowtent 0 Apr 1 16:39 mi
drwxr-xr-x 2 yellowtent yellowtent 0 Apr 1 16:39 ne
drwxr-xr-x 2 yellowtent yellowtent 0 Apr 1 16:39 s.

If I keep going into it, it seems to be the different usernames, as the folders then become <username>@<domain>, and then it becomes the aliases setup in Rainloop, by the looks of it.

[girish]

@d19dotca apps do not touch the backups directory. so it's related to CIFS for sure.

Is this CIFS/NFS on your home server btw? Is it an option to run minio? That might be more reliable than all these file permission problems

[d19dotca]

@girish No, these are all hosted in a dedicated backup partition hosted at OVH in Canada. Running minio I don't think is an option although I'll look into it. Ultimately I need to keep all data in Canada, which is one of the reasons I'm now trying to use the filesystem backup method to keep it in Canada since the other API-related options to my knowledge do not host in Canada for their data (I know DigitalOcean for sure doesn't anyways, and AWS does I believe but it's expensive).

[girish]

@d19dotca Have you seen this note about CIFS mount options - https://cloudron.io/documentation/backups/#sambacifs ?

In your directory listing, can files be individually rmed?

[d19dotca]

@girish Yes, they can. And that's the thing, every other app can remove files and directories without issue. That's why I'm not totally convinced this is only a CIFS issue, it seems more like a combination of that app data and CIFS perhaps. Of course you could be right, it may be entirely CIFS, but then I can't understand why this issue only occurs with one app alone and all others work (well all others up until this one anyways, it's about the 7th app backed up, so at least 6 others work fine).

[d19dotca]

Side note: I just learned that OVH seems to have an S3-compatible storage solution, so if I can make sure they store that in Canada then I may try that S3 API option in the Cloudron backups, maybe that'll be better. Doesn't solve the problem but at least gets me a working backup in Canada. Also seems cheaper than what I'm currently doing.

[nebulon]

Maybe one specific file name/path which just happens to be produced by rainloop causes CIFS to fail?