Cloudron Dashboard Offline, Cannot start manually
-
My Cloudron dashboard is offline, and returns an app not responding message. It seems the Cloudron app wasn't running. I tried to restart it manually, but the script got stuck. Here's the log, please assist.
PS:All other apps are running fine; only the dashboard is inaccessible.Now all apps show an INVALID CERTIFICATE (Self Signed) error after a reboot;
root@myserver:~# /home/yellowtent/box/setup/start.sh 2021-07-09T05:17:02 ==> start: Cloudron Start media:x:500: 2021-07-09T05:17:02 ==> start: Configuring docker Synchronizing state of apparmor.service with SysV service script with /lib/systemd/systemd-sysv-install. Executing: /lib/systemd/systemd-sysv-install enable apparmor Error response from daemon: network with name cloudron already exists 2021-07-09T05:17:03 ==> start: Ensuring directories 2021-07-09T05:17:03 ==> start: Configuring journald 2021-07-09T05:17:03 ==> start: Setting up unbound 2021-07-09T05:17:04 ==> start: Adding systemd services Synchronizing state of unbound.service with SysV service script with /lib/systemd/systemd-sysv-install. Executing: /lib/systemd/systemd-sysv-install enable unbound Synchronizing state of cron.service with SysV service script with /lib/systemd/systemd-sysv-install. Executing: /lib/systemd/systemd-sysv-install enable cron 2021-07-09T05:17:07 ==> start: Configuring sudoers 2021-07-09T05:17:07 ==> start: Configuring collectd 2021-07-09T05:17:07 ==> start: Configuring logrotate 2021-07-09T05:17:07 ==> start: Adding motd message for admins 2021-07-09T05:17:07 ==> start: Configuring nginx 2021-07-09T05:17:08 ==> start: Starting mysql mysqladmin: [Warning] Using a password on the command line interface can be insecure. Warning: Since password will be sent to server in plain text, use ssl connection to ensure password safety. mysql: [Warning] Using a password on the command line interface can be insecure. mysql: [Warning] Using a password on the command line interface can be insecure. 2021-07-09T05:17:08 ==> start: Migrating data cp: -r not specified; omitting directory '/home/yellowtent/boxdata/certs/tmp' [ERROR] Error: Command failed: cp /home/yellowtent/boxdata/certs/* /home/yellowtent/platformdata/nginx/cert cp: -r not specified; omitting directory '/home/yellowtent/boxdata/certs/tmp' at checkExecSyncError (child_process.js:616:11) at Object.execSync (child_process.js:652:15) at /home/yellowtent/box/migrations/20210505223829-blobs-migrate-certs.js:43:27 at wrapper (/home/yellowtent/box/node_modules/async/dist/async.js:268:20) at replenish (/home/yellowtent/box/node_modules/async/dist/async.js:435:29) at iterateeCallback (/home/yellowtent/box/node_modules/async/dist/async.js:424:21) at /home/yellowtent/box/node_modules/async/dist/async.js:321:20 at /home/yellowtent/box/node_modules/async/dist/async.js:2955:19 at wrapper (/home/yellowtent/box/node_modules/async/dist/async.js:268:20) at replenish (/home/yellowtent/box/node_modules/async/dist/async.js:435:29) at iterateeCallback (/home/yellowtent/box/node_modules/async/dist/async.js:424:21) at /home/yellowtent/box/node_modules/async/dist/async.js:321:20 at /home/yellowtent/box/node_modules/async/dist/async.js:2953:17 at tryCatcher (/home/yellowtent/box/node_modules/bluebird/js/release/util.js:16:23) at Promise.successAdapter [as _fulfillmentHandler0] (/home/yellowtent/box/node_modules/bluebird/js/release/nodeify.js:23:30) at Promise._settlePromise (/home/yellowtent/box/node_modules/bluebird/js/release/promise.js:601:21) 2021-07-09T05:17:08 ==> start: DB migration failed
-
It seems the latest update 6.3.4 completely broke my system. This is the log that shows that the update script was broken. Currently, no user can log in (says password incorrect, even if it's correct because I use a password manager)
None of the installed apps have had a valid certificate; all apps have Self-Signed certificates. I cannot do a password reset because the email isn't working.
-
I cannot restore from backup because automatic backups were failing for the last few days, (no apps were backed up) so I will lose the last three days' worth of data. Surprisingly, all apps were working till now, but when I restarted the system the apps lost their certificates.
-
I am 100% sure the culprit is this file:
20210505223829-blobs-migrate-certs.js
because looking at the logs, it failed to run thecp
command to copy certs from/home/yellowtent/boxdata/certs/*
to/home/yellowtent/platformdata/nginx/cert
and failed because of the missing-r
flag. -
Finally, I fixed the issue by modifying the migration file. Adding
-r
to thecp
command, and then manually restarting Cloudron/home/yellowtent/box/setup/start.sh
fixed the issue for me. I'm surprised that no other Cloudron instances had this issue.File:
/home/yellowtent/box/migrations/20210505223829-blobs-migrate-certs.js
Line43
:child_process.execSync(`cp -r ${OLD_CERTS_DIR}/* ${NEW_CERTS_DIR}`); // this way we copy the non-migrated ones like .host, .user etc as well
This thread is a copy of the emails I sent to the support statff. I posted here so it might help someone else in this situation.
-
@nj great that you figured this by your own then. Strange that you had a
tmp
folder there causing the issue, I guess you had created that in the past. Generally it is not recommended to create custom folder structures in/home/yellowtent/boxdata/
or/home/yellowtent/platformdata/
since we cannot really test such cases for updates.Is there any specific use-case to have that folder there, which we should consider?
-
@girish I don't remember creating any folder myself, so I can't tell how it got created there. None of my other Cloudron instances had this issue so it might have happened somehow. I don't have any custom nginx config.
Maybe removing the
tmp
folder could have solved the issue without the-r
flag. The error cp: -r not specified; omitting directory '/home/yellowtent/boxdata/certs/tmp' meant it couldn't copy thetmp
folder because-r
flag was missing. Also, I added-r
because the error message hinted me to do so and it fixed the issue. -
@nj yes with the folder in place there adding the
-r
to solve this makes sense, however the initial issue is that this folder should not be there in the first place. As the name already suggests, I guess this was just some intermediate manual action to stash certs. Essentially if you don't actively use files in that folder then just delete it to solve this for future releases.
1/9