Cloudron dash is stuck in reconnecting loop

[nebulon]

Actually on that domain, the cert looks fine to me from my browser. nginx appears fine also, however the box backend service is not working. Check the logs at /home/yellowtent/platformdata/logs/box.log

[mpeterson0418]

@nebulon

This is what I'm getting and it keeps repeating -

2024-09-06T15:04:36.771Z box:server ==========================================
2024-09-06T15:04:36.772Z box:server            Cloudron 8.0.3
2024-09-06T15:04:36.772Z box:server ==========================================
2024-09-06T15:04:36.772Z box:platform initialize: start platform
2024-09-06T15:04:36.792Z box:tasks stopAllTasks: stopping all tasks
2024-09-06T15:04:36.792Z box:shell stopTask /usr/bin/sudo -S /home/yellowtent/box/src/scripts/stoptask.sh all
All tasks stopped
Error: Error starting server: {"name":"BoxError","reason":"Database Error","details":{},"message":"ER_NOT_SUPPORTED_AUTH_MODE: Client does not support authentication protocol requested by server; consider upgrading MySQL client","nestedError":{"code":"ER_NOT_SUPPORTED_AUTH_MODE","errno":1251,"sqlMessage":"Client does not support authentication protocol requested by server; consider upgrading MySQL client","sqlState":"08004","fatal":true},"code":"ER_NOT_SUPPORTED_AUTH_MODE","sqlMessage":"Client does not support authentication protocol requested by server; consider upgrading MySQL client"}
    at main (/home/yellowtent/box/box.js:48:41)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

Circled back to your previous update and ran this command -

/home/yellowtent/box/setup/start.sh

Now I have a screen asking me to setup cloudron again. I'm guessing I should be able to restore my instance with this interface right?

If so.... what options would you recommend for me here?

[joseph]

@mpeterson0418 You should not see the setup screen. It's almost like the database is entirely gone. I recommend contacting us at support@cloudron.io , so we can check what has happened.

[mpeterson0418]

@joseph

Thanks I just sent an email to support@cloudron.io

If that's the case.... is there something on the backend here for my account which would contain a backup I can recover from? I'm a bit confused on what's going on here

Aside from a MySQL DB recovery, this is not what I expected to see. Is my website completely hosed at this point?

[joseph]

@mpeterson0418 the app database is totally separate from cloudron's own database. But we have to see what is the state of the server to assess the situation.

[mpeterson0418]

@joseph

Ok well that makes more sense.... I guess lol

To be honest from a website perspective I didn't touch a thing on here. So whatever is going on at this point I'd be very curious to know. Honestly only other thing I could imagine here would be some potential disk corruption of some sort..... but hopefully you can give some better insight on that after looking things over

[mpeterson0418]

@joseph

FYI the SSH access has been fixed. You should be able to remotely connect to the server for review

[joseph]

@mpeterson0418 so the issue was that mysql, docker, containerd were all corrupt. I am not sure what happened to the disk or the server. I restore mysql from a previous backup. container/docker (i.e /var/lib/docker and /var/lib/containerd) had to be deleted entirely and re-created .

[mpeterson0418]

@joseph

Thank you for all your assistance. Everything looks great now!

Send you a couple other questions in the support case I raised. If you don't mind following up with me on it.... and then I believe we can close this one out

[joseph]

@mpeterson0418 I found that the root file system was either corrupt or "reset" . I don't know and unable to guess what happened to the disk (maybe power outage?), but I can explain what I did to recover.

• I found host MySQL to be empty. The box database had tables but was empty. The fact that it had tables meant that migration scripts were run on this . Maybe /home/yellowtent/box/setup/start.sh was run at some point on an empty (i.e no tables even) database.

  • Usually, the last well known mysql dump is located /home/yellowtent/boxdata/box/box.mysqldump . One can also find the same inside the latest backup (inside the box.tar.gz) .
  • To "restore", mysql -uroot -ppassword < /home/yellowtent/boxdata/box/box.mysqldump

• Then I found, docker won't start. In Cloudron, containers are throw away infrastructure since data is stored outside the containers. I tried cloudron-support --recreate-docker but this did not work. On investigating further, it seems /var/lib/containerd was also corrupt . Corrupt here means no docker commands work. So, I removed that directory as well. After that, cloudron-support --recreate-docker worked.

The best way to migrate in Cloudron is not to do the above but instead just restore from backups. See https://docs.cloudron.io/backups/#move-cloudron-to-another-server .