Cloudron Forum

Awesome, I couldn't get NFS to mount, so just set the tarball option instead, thanks.

@daixiwen ah ok. I would ignore the errors. It's normal for upstream services to go down once in a while... I just checked https://status.backblaze.com/ and unfortunately no history of uptime here. But it says "Backblaze performs regular maintenance every Thursday from 11:30 am to 1:30 p.m. Pacific Time with the aim to consistently improve our systems and services. "

Just today I got a notification about an erroneous backup of the server where I have Nextcloud installed (150GB size) and I backup once a week.. I should add that the previous backups were successful. The provider is Backblaze B2. Oct 06 01:02:32 box:storage/s3 Upload progress: {"loaded":1073741824,"part":2,"key":"102/snapshot/app_50d07945-bf1b-4345-8e8c-c146e86c12a7.tar.gz.enc"} Jan 01 01:24:00 node:events:496 [no timestamp] throw er; // Unhandled 'error' event [no timestamp] ^ [no timestamp] [no timestamp] write EPIPE [no timestamp] at WriteWrap.onWriteComplete [as oncomplete] (node:internal/stream_base_commons:94:16) [no timestamp] 'error' event on TLSSocket instance at: [no timestamp] at emitErrorNT (node:internal/streams/destroy:169:8) [no timestamp] at emitErrorCloseNT (node:internal/streams/destroy:128:3) [no timestamp] at process.processTicksAndRejections (node:internal/process/task_queues:82:21) { [no timestamp] errno: -32, [no timestamp] code: 'EPIPE', [no timestamp] syscall: 'write' [no timestamp] } [no timestamp] [no timestamp] v20.12.2 Oct 06 01:02:32 box:shell backup-snapshot/app_50d07945-bf1b-4345-8e8c-c146e86c12a7: /usr/bin/sudo -S -E --close-from=4 /home/yellowtent/box/src/scripts/backupupload.js snapshot/app_50d07945-bf1b-4345-8e8c-c146e86c12a7 tgz {"localRoot":"/home/yellowtent/appsdata/50d07945-bf1b-4345-8e8c-c146e86c12a7","layout":[]} errored BoxError: backup-snapshot/app_50d07945-bf1b-4345-8e8c-c146e86c12a7 exited with code 1 signal null [no timestamp] at ChildProcess.<anonymous> (/home/yellowtent/box/src/shell.js:122:19) [no timestamp] at ChildProcess.emit (node:events:518:28) [no timestamp] at ChildProcess.emit (node:domain:488:12) [no timestamp] at ChildProcess._handle.onexit (node:internal/child_process:294:12) { [no timestamp] reason: 'Shell Error', [no timestamp] details: {}, [no timestamp] code: 1, [no timestamp] signal: null [no timestamp] } Oct 06 01:02:32 box:taskworker Task took 152.05 seconds Oct 06 01:02:32 box:tasks setCompleted - 2443: {"result":null,"error":{"stack":"BoxError: Backuptask crashed\n at runBackupUpload (/home/yellowtent/box/src/backuptask.js:164:15)\n at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n at async uploadAppSnapshot (/home/yellowtent/box/src/backuptask.js:361:5)\n at async backupAppWithTag (/home/yellowtent/box/src/backuptask.js:383:5)\n at async fullBackup (/home/yellowtent/box/src/backuptask.js:504:29)","name":"BoxError","reason":"Internal Error","details":{},"message":"Backuptask crashed"}} Oct 06 01:02:32 box:tasks update 2443: {"percent":100,"result":null,"error":{"stack":"BoxError: Backuptask crashed\n at runBackupUpload (/home/yellowtent/box/src/backuptask.js:164:15)\n at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n at async uploadAppSnapshot (/home/yellowtent/box/src/backuptask.js:361:5)\n at async backupAppWithTag (/home/yellowtent/box/src/backuptask.js:383:5)\n at async fullBackup (/home/yellowtent/box/src/backuptask.js:504:29)","name":"BoxError","reason":"Internal Error","details":{},"message":"Backuptask crashed"}} [no timestamp] Backuptask crashed [no timestamp] at runBackupUpload (/home/yellowtent/box/src/backuptask.js:164:15) [no timestamp] at process.processTicksAndRejections (node:internal/process/task_queues:95:5) [no timestamp] at async uploadAppSnapshot (/home/yellowtent/box/src/backuptask.js:361:5) [no timestamp] at async backupAppWithTag (/home/yellowtent/box/src/backuptask.js:383:5) [no timestamp] at async fullBackup (/home/yellowtent/box/src/backuptask.js:504:29) On other servers the backups are successful.

Yeah, I guessed the same after reading the debianforum wiki Results after running it a couple of times: sudo hdparm -t /dev/sdb1 /dev/sdb1: Timing buffered disk reads: 192 MB in 3.00 seconds = 63.92 MB/sec Timing buffered disk reads: 200 MB in 3.00 seconds = 66.58 MB/sec Timing buffered disk reads: 224 MB in 3.01 seconds = 74.44 MB/sec Timing buffered disk reads: 236 MB in 3.01 seconds = 78.31 MB/sec Timing buffered disk reads: 248 MB in 3.01 seconds = 82.33 MB/sec Timing buffered disk reads: 234 MB in 3.01 seconds = 77.69 MB/sec

Switching to hetzner feels good enough for now . Full backup takes 30min for 21 apps. I've cancelled contabo storage service. Hope this stays so snappy for the long run

The multi-part copy was just hanging. If someone hits the same issue, you can just apply this one liner https://git.cloudron.io/platform/box/-/commit/b4d58f0609b1c252fa44799eccbbd9c691c7c2ac . The file is /home/yellowtent/box/src/storage/s3.js

@girish Ok, great! Thank you very much for the information. I really appreciate your time. Thanks again!

Wonderful. Thank you @Joseph, that change to the endpoint URL did the trick. The configuration saved successfully.

@joseph said in Keep Getting Backuptask crashed: If one of you can reach out to support@cloudron.io , maybe we can debug on your server as to what the issue is. Was a solution found, or can this topic be closed for now?

@Beardmancer said in Failed backups and email config issue since 8.0.3 upgrade: I decided to go ahead and try adding this to /etc/systemd/resolved.conf since it's easy enough to comment out. DNS=8.8.8.8 1.1.1.1 After doing that and restarting the service with systemctl restart systemd-resolved, things seem to be working correctly. The file /etc/resolv.conf lists nameservers now and a manually-initiated backup just completed successfully. Thanks for the hlep @nebulon ! In the future, is there anything I should look out for to avoid this? I don't know what caused it. Maybe it was the 8.0.3 upgrade, or maybe it was an Ubuntu update? I think I rebooted Cloudron for the latter recently. If there's a configuration I might have setup wrong, I'd love to correct it.

It did turn out to be a DNC issue which Joseph at Clourdon support fixed and then created a backup.

@girish One thing to note, all stopped apps where not imported through the restore method even tho they had backups retained with them. We are still trying to find and recover those apps atm, but they must be buried in a random 2024-04-08-030000-684 folder. Never the less. Is it possible to recover data from the snapshot folder? what is that for if backups pull from the other folders? I think our stopped apps are in there but Im not sure how to recover them through the front end that way. One thing that would have made this process so much more seamless is just including that restore config file in the snapshot folder, and the corresponding configs for each app backup, would have saved us hours of finding the correct backup id manually for each method, since Im assuming they are slightly different?

Thanks for the report, this is fixed for the next release.

@girish yeah, I understand there is nothing about Cloudron here - just wanted to check your experience / understanding here. Thanks for sharing!

Since this question seems to come up often, I wonder if we should just show the timezone in brackets everywhere in the UI. @nebulon ?

What is your use-case to restore the whole Cloudron from Cloudron itself? I can see if Cloudron and thus the dashboard is not working, a restore might be required, but then it cannot restore itself in such cases.

No, it was a one-time issue so far. I posted it more as a bug report. Version 7.7.2, by the way.

@matix131997 My freescout app is the one that crashed I think and it's around 1GB only. I have backups disabled for my Nextcloud because I'm pretty sure it would never finish the job within a reasonable time even if it didn't crash.

Hey @girish, @nebulon, thanks for coming back to me so quickly! Happy to report that the fix worked wonders, and the backup completed successfully. As for our setup, we set up the backup server a while ago now, my memories are not the freshest. But IIRC we are using OpenSSH Server for Windows Server. And from what I can see Windows shell uses mklink to create hard links. But I have never used it.

@girish said in WordPress app container stuck on updating from package version 3.7.2 to 3.7.3: This happens with apps which contains files that change their size when the backup is going on. ... Maybe you have some plugin that writes to somewhere periodically? Thank you @girish Great advice and just the clue we needed. We figured out what that was, stopped it and then we could take a WP backup manually; and then we were also able to update to the latest packaged WP version 3.8.0 Thank you once again; your prompt help is much appreciated You can mark this ticket as resolved; cheers!