Hetzner StorageBox backup error

[archos]

@girish Hi, I don't want to start a new thread, as it seems there have been multiple issues with backups here. I'm using a Hetzner box with rsync for backups, and after the latest Cloudron update, I'm also facing problems. The backup takes an extremely long time and doesn't complete. According to the logs, the issue is with Nextcloud. If I've posted this in the wrong thread, I apologize.

box:shell saveFsMetadata execArgs: find ["/home/yellowtent/appsdata/496f823e-b62e-4ac3-9083-bb909c6cba89","-type","f","-executable"]
Sep 03 03:39:18 box:shell safeFsMetadata execArgs: find ["/home/yellowtent/appsdata/496f823e-b62e-4ac3-9083-bb909c6cba89","-type","l"]
Sep 03 03:39:18 box:shell saveFsMetadata execArgs: find ["/mnt/data/Nextcloud","-type","d","-empty"]
Sep 03 06:10:19 box:shell saveFsMetadata execArgs: find ["/mnt/data/Nextcloud","-type","f","-executable"]
Sep 03 07:39:16 box:shell safeFsMetadata execArgs: find ["/mnt/data/Nextcloud","-type","l"]

[girish]

@archos I think the error here is related to Exoscale SOS and not backups in general. Can you create a new thread (don't hesitate to create new threads, it's fine) with your issue? Your logs don't actually show any error though, it seems fine.

[girish]

Nevermind, I forked the topic. But @archos is the issue that it just gets "stuck" above? How big are your backups? I suspect it's really, really big and that's why the find command stalls...?

[archos]

@girish Thank you very much for the quick response. My backups are around 200 GB, so I don't think that's too much. Since I moved to a new server and removed Matrix Synapse, I haven't had any issues. I tried running the backup manually, and it completed in two hours. I just checked the logs, so I'm not sure if everything is okay.

[girish]

@archos so /mnt/data/Nextcloud is a volume I assume. Is that also a Hetzner Storage Box volume ?

[archos]

@girish Hi! Yes, /mnt/data/Nextcloud is a separate volume that I use for Nextcloud data. However, it is not a Hetzner Storage Box, it's local storage on the server. I only use the Hetzner Storage Box for backups.

[girish]

@archos ok, so I am not sure why it's getting stuck. We will have to debug this further on the set up to understand why. From what I can make out from the logs, the find commands are getting stuck for some reason. Maybe you can try to run those commands manually on the server and check if they succeed (or why they fail). Otherwise, you have to write to support@cloudron.io and we have to debug on the serve.r

[archos]

@girish Thank you for the information. I think it would be safer to write to your support team, as I’m not really sure where to look myself. Thanks again for your help!

[girish]

As an update, it seems that find command is very slow on the server on the server. It just hangs essentially even though hard disk seems OK. I guess this falls outside purview of Cloudron. The issue is not storage box related.

[archos]

@girish Thank you very much for looking into this. I have moved the Nextcloud data to an SSD, and it seems like the backups are now working fine. The find command also completed without any issues. However, I still see a lot of errors in the log like:

box:storage/filesystem Error: ENOENT: no such file or directory, unlink '/mnt/cloudronbackup/my...

I understand that this is not a fatal error, but I’m a bit concerned that it might cause problems with restoring backups in the future. I know this isn't a Cloudron-specific issue, but do you have any idea where the problem might be?

The server has 64 GB of RAM and a 12-core Intel(R) Core(TM) i7-8700 CPU @ 3.20GHz. It was purchased from a Hetzner auction, so I believe it’s not a slow machine. I also checked the permissions on the HDD, which are as follows:

drwxrwxrwx 8 root root 4096 Sep 7 10:47 /mnt/data

The permissions seem fine to me, or should the owner be set to a different user?
Thank you very much, I will be grateful for any information. I understand that this is not a Cloudron issue.

[girish]

@archos Good catch with the warning. I have removed that debug - https://git.cloudron.io/cloudron/box/-/commit/1cd577cc65cc488b31c742f90376a37f33e45f78 . They are not errors.

[archos]

@girish Ok, great! Thank you very much for the information. I really appreciate your time. Thanks again!