Cloudron stuck "starting service, this may take a while"

[JC49]

Hello, I am currently having an issue with Cloudron in which all apps are stuck restarting. Error message is "Starting services, this can take a while" and I have tried the following steps but they have not worked.

Recovered a backup from this morning before the issue presented itself
Cleared mysql logs and restarted the service as well as box

/home/yellowtent/platformdata/logs/box.log show:

2023-05-02T16:20:42.018Z box:locker Lock unreleased platform_start
2023-05-02T16:25:42.019Z box:locker Lock unreleased platform_start

[nebulon]

Not sure how it is already in that state but it could take some time. Further are specific services not starting up? Check the services view in the dashboard for this.

[JC49]

@nebulon All apps were stuck in the same state. So far they have all come up except for 2. One of which is not responding and the other is accessible via the domain but still shows "Restarting" as a status.

2023-05-02T17:37:35.998Z box:tasks startTask: 6845 completed with code 0
2023-05-02T17:37:35.999Z box:apps scheduleTask: task 6845 of 670373b1-9e60-4e70-93a5-ab68d009fbe5 completed
2023-05-02T17:37:35.999Z box:locker Recursive lock released : apptask. Depth : 2
2023-05-02T17:37:35.999Z box:scheduler resumeJobs: 670373b1-9e60-4e70-93a5-ab68d009fbe5
2023-05-02T17:37:35.999Z box:tasks startTask: 6845 done. error: null
2023-05-02T17:37:39.279Z box:shell startTask (stderr): Finished with result: success
Main processes terminated with: code=exited/status=0
Service runtime: 10.502s

2023-05-02T17:37:39.283Z box:shell startTask (stdout): Service box-task-6846 finished with exit code 0

2023-05-02T17:37:39.284Z box:tasks startTask: 6846 completed with code 0
2023-05-02T17:37:39.285Z box:apps scheduleTask: task 6846 of ecb51306-1b9d-48a6-a1e3-61983d4ee3a6 completed
2023-05-02T17:37:39.285Z box:locker Recursive lock released : apptask. Depth : 1
2023-05-02T17:37:39.285Z box:scheduler resumeJobs: ecb51306-1b9d-48a6-a1e3-61983d4ee3a6
2023-05-02T17:37:39.285Z box:tasks startTask: 6846 done. error: null
2023-05-02T17:37:40.009Z box:apphealthmonitor setHealth: ecb51306-1b9d-48a6-a1e3-61983d4ee3a6 (dev.int.socialmobile.com) waiting for 319.997 to update health
2023-05-02T17:37:40.038Z box:apphealthmonitor app health: 4 running / 0 stopped / 2 unresponsive
2023-05-02T17:37:48.054Z box:locker Lock unreleased apptask
2023-05-02T17:37:50.012Z box:apphealthmonitor setHealth: ecb51306-1b9d-48a6-a1e3-61983d4ee3a6 (dev.int.socialmobile.com) waiting for 309.994 to update health
2023-05-02T17:37:50.115Z box:apphealthmonitor app health: 4 running / 0 stopped / 2 unresponsive

I will be updating tonight to v7.4.2 to see if that helps.

[girish]

@JC49 We replied from support, looks like everything is running again already? What did you have to do?

[cyberfreakde]

Having the same notification. Apps finally start after long time but the notification is still there.

[JC49]

@girish Unfortunately it was just waiting. It took about an hour or so for all the apps to start up. It seemed to have started from a stuck Gitlab update but that also stopped backups completely. Is this normal? Stuck updates should not stop backups completely.

[girish]

@cyberfreakde If the notification is there it means that something has not updated completely internally. Can you double check by refreshing the browser and checking if the notification is still there? If it is, can you then check /home/yellowtent/platformdata/logs/box.log for any errors?

[girish]

@JC49 Currently, there is a simple lock in place so things don't trample each other. When an app is updating, the other tasks "wait" for it to finish. For example, we don't want backup task to backup an app which is updating because it can be in the middle of a migration etc.

[clouwou]

Having the same issue on a fresh install. "Lock unreleased platform_start".

Installing my first up got stuck with "Starting services..."

I see that there is an update (hoping this brings a fix), but I can't update because the system is waiting to complete the app installation which, in turn, seems to be waiting for something else to happen.

Unfortunately, I also can't stop the installation or uninstall.

So it seems I am completely stuck

[girish]

@clouwou Can you check the output of /home/yellowtent/platformdata/logs/box.log ? One or more internal services are not starting up properly. There will be an error in the logs.

[clouwou]

There was an error in the logs about some keys that resembled another post here. The recommendation was to delete two files (don't remember which ones) and after doing so, the installation went through and after that I was able to update. The issue has been resolved.