@falkfyre so did it work out?

@overholt said in Upgrade Cloudron on Linode: Which is the best way to do this? Personally I'd go with starting with a fresh server rather than upgrading Ubuntu I always do a dry run backup restore to check everything is working before updating DNS, see https://docs.cloudron.io/backups/#dry-run Note, when installing Cloudron on the new server be sure not to miss this Looking to restore link at the bottom here: [image: looking-to-restore.png] (I've missed it before and had to start all over again )

Just today I got a notification about an erroneous backup of the server where I have Nextcloud installed (150GB size) and I backup once a week.. I should add that the previous backups were successful. The provider is Backblaze B2. Oct 06 01:02:32 box:storage/s3 Upload progress: {"loaded":1073741824,"part":2,"key":"102/snapshot/app_50d07945-bf1b-4345-8e8c-c146e86c12a7.tar.gz.enc"} Jan 01 01:24:00 node:events:496 [no timestamp] throw er; // Unhandled 'error' event [no timestamp] ^ [no timestamp] [no timestamp] write EPIPE [no timestamp] at WriteWrap.onWriteComplete [as oncomplete] (node:internal/stream_base_commons:94:16) [no timestamp] 'error' event on TLSSocket instance at: [no timestamp] at emitErrorNT (node:internal/streams/destroy:169:8) [no timestamp] at emitErrorCloseNT (node:internal/streams/destroy:128:3) [no timestamp] at process.processTicksAndRejections (node:internal/process/task_queues:82:21) { [no timestamp] errno: -32, [no timestamp] code: 'EPIPE', [no timestamp] syscall: 'write' [no timestamp] } [no timestamp] [no timestamp] v20.12.2 Oct 06 01:02:32 box:shell backup-snapshot/app_50d07945-bf1b-4345-8e8c-c146e86c12a7: /usr/bin/sudo -S -E --close-from=4 /home/yellowtent/box/src/scripts/backupupload.js snapshot/app_50d07945-bf1b-4345-8e8c-c146e86c12a7 tgz {"localRoot":"/home/yellowtent/appsdata/50d07945-bf1b-4345-8e8c-c146e86c12a7","layout":[]} errored BoxError: backup-snapshot/app_50d07945-bf1b-4345-8e8c-c146e86c12a7 exited with code 1 signal null [no timestamp] at ChildProcess.<anonymous> (/home/yellowtent/box/src/shell.js:122:19) [no timestamp] at ChildProcess.emit (node:events:518:28) [no timestamp] at ChildProcess.emit (node:domain:488:12) [no timestamp] at ChildProcess._handle.onexit (node:internal/child_process:294:12) { [no timestamp] reason: 'Shell Error', [no timestamp] details: {}, [no timestamp] code: 1, [no timestamp] signal: null [no timestamp] } Oct 06 01:02:32 box:taskworker Task took 152.05 seconds Oct 06 01:02:32 box:tasks setCompleted - 2443: {"result":null,"error":{"stack":"BoxError: Backuptask crashed\n at runBackupUpload (/home/yellowtent/box/src/backuptask.js:164:15)\n at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n at async uploadAppSnapshot (/home/yellowtent/box/src/backuptask.js:361:5)\n at async backupAppWithTag (/home/yellowtent/box/src/backuptask.js:383:5)\n at async fullBackup (/home/yellowtent/box/src/backuptask.js:504:29)","name":"BoxError","reason":"Internal Error","details":{},"message":"Backuptask crashed"}} Oct 06 01:02:32 box:tasks update 2443: {"percent":100,"result":null,"error":{"stack":"BoxError: Backuptask crashed\n at runBackupUpload (/home/yellowtent/box/src/backuptask.js:164:15)\n at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n at async uploadAppSnapshot (/home/yellowtent/box/src/backuptask.js:361:5)\n at async backupAppWithTag (/home/yellowtent/box/src/backuptask.js:383:5)\n at async fullBackup (/home/yellowtent/box/src/backuptask.js:504:29)","name":"BoxError","reason":"Internal Error","details":{},"message":"Backuptask crashed"}} [no timestamp] Backuptask crashed [no timestamp] at runBackupUpload (/home/yellowtent/box/src/backuptask.js:164:15) [no timestamp] at process.processTicksAndRejections (node:internal/process/task_queues:95:5) [no timestamp] at async uploadAppSnapshot (/home/yellowtent/box/src/backuptask.js:361:5) [no timestamp] at async backupAppWithTag (/home/yellowtent/box/src/backuptask.js:383:5) [no timestamp] at async fullBackup (/home/yellowtent/box/src/backuptask.js:504:29) On other servers the backups are successful.

RESOLVED: It's likely that cloudflare was having some internal issue, but upon checking https://www.cloudflarestatus.com/ I was left more confused is there was any incident on their end in my region. It's buried under scheduled maintanaces and other stuff. I still not sure if this was the correct resolution, but it started working after executing these steps. sudo apt-get install -y systemd sudo nano /etc/systemd/resolved.conf Added the DNS of Cloudflare and google # This file is part of systemd. # # systemd is free software; you can redistribute it and/or modify it under the # terms of the GNU Lesser General Public License as published by the Free # Software Foundation; either version 2.1 of the License, or (at your option) # any later version. # # Entries in this file show the compile time defaults. Local configuration # should be created by either modifying this file, or by creating "drop-ins" in # the resolved.conf.d/ subdirectory. The latter is generally recommended. # Defaults can be restored by simply deleting this file and all drop-ins. # # Use 'systemd-analyze cat-config systemd/resolved.conf' to display the full config. # # See resolved.conf(5) for details. [Resolve] # Some examples of DNS servers which may be used for DNS= and FallbackDNS=: # Cloudflare: 1.1.1.1 #cloudflare-dns.com 1.0.0.1#cloudflare-dns.com 2606:4700:4700::1111 #cloudflare-d># Google: 8.8.8.8 #dns.google 8.8.4.4 #dns.google 2001:4860:4860::8888 #dns.google 2001:4860:4860::8> # Quad9: 9.9.9.9#dns.quad9.net 149.112.112.112 #dns.quad9.net 2620:fe::fe #dns.quad9.net 2620:fe::> DNS=1.1.1.1 8.8.8.8 sudo systemd-resolve --flush-caches # didn't work systemd-resolve --status # wasn't even running. So maybe the config change was irrelevant Reboot and started working... and I am not sure how this is possible... Anyway that for the help. This issue was plaguing me for a few days.

@jtippetts atleast the dashboard seems up now. Not sure why the machine shutdown itself though.

Awesome, I couldn't get NFS to mount, so just set the tarball option instead, thanks.

Things work like a charm indeed. I'll likely move the vps over soon or later once answering the question of where I should go. Also looking for a good option for CI/CD runner and to mitigate most issues faced this last month with Contabo.

Switching to hetzner feels good enough for now . Full backup takes 30min for 21 apps. I've cancelled contabo storage service. Hope this stays so snappy for the long run

So wildcard is only for subdomains of a domain. The domain.com record is not covered by wildcard. But yes the instructions probably sound like both *.domain.com and domain.com are in fact managed by Coudron, but all I can tell you that unless an app is using a domain, the certs will not be renewed. Maybe we can be smarter about this in the future, but to solve your problem this is what is required. Note a redirect to an existing app will also work.

We were able to resolve this. The root cause was the missing unbound-anchor ubuntu package. After intstalling this it worked. More info on that issue is at https://forum.cloudron.io/topic/12496/unbound-anchor-not-found-in-ubuntu-24-04?_=1727881329798

@daixiwen ah ok. I would ignore the errors. It's normal for upstream services to go down once in a while... I just checked https://status.backblaze.com/ and unfortunately no history of uptime here. But it says "Backblaze performs regular maintenance every Thursday from 11:30 am to 1:30 p.m. Pacific Time with the aim to consistently improve our systems and services. "

@nebulon said in All Offline adter update or db can't restart.: by avoiding disabling by avoiding disabling at least I think that's what you mean?

Hi @girish, I did that step and did a server reboot as the systemctl restart unbound did not work and when it came back up, all was good again! Thank you!

@girish said in Cert renewing failing: @visuex are you the same as @cpa ? Nope, not me!

Yeah, I guessed the same after reading the debianforum wiki Results after running it a couple of times: sudo hdparm -t /dev/sdb1 /dev/sdb1: Timing buffered disk reads: 192 MB in 3.00 seconds = 63.92 MB/sec Timing buffered disk reads: 200 MB in 3.00 seconds = 66.58 MB/sec Timing buffered disk reads: 224 MB in 3.01 seconds = 74.44 MB/sec Timing buffered disk reads: 236 MB in 3.01 seconds = 78.31 MB/sec Timing buffered disk reads: 248 MB in 3.01 seconds = 82.33 MB/sec Timing buffered disk reads: 234 MB in 3.01 seconds = 77.69 MB/sec

I think @nebulon means host -t NS vysvr.org . This should resolve to your nameservers.

I think we have to go back to the original question. What's the original issue? Using tools like df in /var/lib/docker/overlay2 won't work.

@joseph , Running 'cloudron-support --recreate-containers' fixed the issue. Thank you.

Great! Thank you! BTW, it seems in retrospect that the start of the DNS problems coincided with me beginning to use the OpenVPN app. Take that for whatever it's worth.

@andreasdueren said in Cloudron can't uninstall apps after access to domain got denied: Sep 26 10:49:40 box:dns unregisterLocation: Error unregistering location A. retryable: false. message: Invalid access token statusCode: 403 code:9109 The directory message is just a warning can be ignored. The above message is the real error. Are you sure you are not using any other domain in the app which in turn uses some DNS provider? Are all the DNS providers manual ?