Cloudron Forum

Today our cloudron instance (on a hetzner dedicated machine) became unresponsive, both to web and ssh requests

Power cycling did not help. Booting into recovery mode, the last lines of the box.log are as follows:

2024-03-18T05:29:50.217Z box:apphealthmonitor app health: 43 running / 2 stopped / 0 unresponsive
2024-03-18T05:30:00.005Z box:disks checkDiskSpace: checking disk space
2024-03-18T05:30:00.042Z box:janitor Cleaning up expired tokens
2024-03-18T05:30:00.044Z box:eventlog cleanup: pruning events. creationTime: Tue Dec 19 2023 05:30:00 GMT+0000 (Coordinated Universal Time)
2024-03-18T05:30:00.046Z box:tasks startTask - starting task 6699 with options {}. logs at /home/yellowtent/platformdata/logs/tasks/6699.log
2024-03-18T05:30:00.046Z box:shell startTask spawn: /usr/bin/sudo -S -E /home/yellowtent/box/src/scripts/starttask.sh 6699 /home/yellowtent/platformdata/logs/tasks/6699.log 0 400
2024-03-18T05:30:00.054Z box:janitor Cleaned up 0 expired tokens
2024-03-18T05:30:00.126Z box:shell startTask (stderr): Running as unit: box-task-6699.service

2024-03-18T05:30:00.400Z box:disks checkDiskSpace: disk space checked. out of space: no
2024-03-18T05:30:00.431Z box:apphealthmonitor app health: 43 running / 2 stopped / 0 unresponsive
2024-03-18T05:30:10.217Z box:apphealthmonitor app health: 43 running / 2 stopped / 0 unresponsive
2024-03-18T05:30:20.210Z box:apphealthmonitor app health: 43 running / 2 stopped / 0 unresponsive
2024-03-18T05:30:30.215Z box:apphealthmonitor app health: 43 running / 2 stopped / 0 unresponsive
2024-03-18T05:30:40.208Z box:apphealthmonitor app health: 43 running / 2 stopped / 0 unresponsive
2024-03-18T05:30:50.202Z box:apphealthmonitor app health: 43 running / 2 stopped / 0 unresponsive
2024-03-18T05:31:00.211Z box:apphealthmonitor app health: 43 running / 2 stopped / 0 unresponsive
2024-03-18T05:31:10.202Z box:apphealthmonitor app health: 43 running / 2 stopped / 0 unresponsive
2024-03-18T05:31:20.227Z box:apphealthmonitor app health: 43 running / 2 stopped / 0 unresponsive
2024-03-18T05:31:25.823Z box:shell startTask (stderr): Finished with result: success
Main processes terminated with: code=exited/status=0
Service runtime: 1min 25.708s
CPU time consumed: 6.591s

2024-03-18T05:31:25.829Z box:shell startTask (stdout): Service box-task-6699 finished with exit code 0

2024-03-18T05:31:25.831Z box:tasks startTask: 6699 completed with code 0
2024-03-18T05:31:25.833Z box:tasks startTask: 6699 done. error: null
2024-03-18T05:31:30.205Z box:apphealthmonitor app health: 43 running / 2 stopped / 0 unresponsive
2024-03-18T05:31:40.216Z box:apphealthmonitor app health: 43 running / 2 stopped / 0 unresponsive
2024-03-18T05:31:50.204Z box:apphealthmonitor app health: 43 running / 2 stopped / 0 unresponsive
2024-03-18T05:32:00.209Z box:apphealthmonitor app health: 43 running / 2 stopped / 0 unresponsive
2024-03-18T05:32:10.220Z box:apphealthmonitor app health: 43 running / 2 stopped / 0 unresponsive
2024-03-18T05:32:20.208Z box:apphealthmonitor app health: 43 running / 2 stopped / 0 unresponsive
2024-03-18T05:32:30.216Z box:apphealthmonitor app health: 43 running / 2 stopped / 0 unresponsive

df output:

Any idea how we can further troubleshoot?

Finally got things working again after taking the following steps:

1. Asking Hetzner to replace the machine (not drives) - although this didn't fix the issue, and may have been unnecessary

2. Opening the rescue system and running fsck -fy /dev/md2 per this guide: https://docs.hetzner.com/robot/dedicated-server/troubleshooting/filesystem-check

This seemed to fix a load of disk issues and now everything is running fine.

Actually - I was able to fix this by taking two actions:

1. Increase memory to 640mb
2. Use Firefox instead of Brave

API keys now generate successfully.

Edit: it's definitely jitsi in Matrix, this is what we were presented with for our 3 way (successful) video call, with screen sharing:

Was just stock cloudron matrix, using the stock cloudron element web client on the customer side of the call.

We could not do a successful 3-way video call with this same customer, on the same browser, using the stock standalone cloudron jitsi instance.

We're on cloudron 7.2.5

We would love to see the following:

• TURN server support for jitsi (to enable enterprise calls over 443)
• Well Known support - to enable arbitrary well-known configs (not just a predefined list)

We have started getting random repositories / users appear in our gitea instance, eg "AccidentInjuryLawyers". Before that, we had a sofa company. It looks like spam, I have to keep deleting them. How to prevent such signups?

https://git.datacontroller.io/explore/repos

@oj the problem with the cloudron jitsi install is the lack of a TURN server, which means it is not possible to have video calls with customers on enterprise platforms (that require traffic over 443)

So, any major customer basically.

Thankyou! This fixed it up.