Cloudron stops working periodically

girish

This is weird but here it goes :-).

@tadeas I have validated your mail here. We got your email to support but it does not have a working email (the FROM address is not gmail) and it's thus bouncing. Please send us an email from your gmail account. Thanks.

tadeas

Thanks (I used to have g. apps for my domain for many years, which is still working to send emails from, even after I migrated to cloudron)

Anyway, here's the issue ---

I've been using cloudron for about 6 months now. Usually every 4 weeks I need to login via ssh and manually reboot, because the server stops responding (mail, calendar, etc.). I didn't have time to troubleshoot that.

Now however, the mail&calendar became inaccessible and the my.* interface was saying update in progress for many hours. I rebooted and now I'm only getting an error page from the cloudron.

Logging into the server via ssh, I see mysql is not working and also does not restart.

Apr 22 20:30:49 my..net systemd[1]: Starting MySQL Community Server...
Apr 22 20:30:50 my..net systemd[1]: mysql.service: Main process exited, code=exited, status=1/FAILURE
Apr 22 20:31:19 my..net systemd[1]: Failed to start MySQL Community Server.
Apr 22 20:31:19 my..net systemd[1]: mysql.service: Unit entered failed state.
Apr 22 20:31:19 my..net systemd[1]: mysql.service: Failed with result 'exit-code'.
Apr 22 20:31:20 my..net systemd[1]: mysql.service: Service hold-off time over, scheduling restart.
Apr 22 20:31:20 my.*.net systemd[1]: Stopped MySQL Community Server.

root@my:~# systemctl status mysql

mysql.service - MySQL Community Server
Loaded: loaded (/lib/systemd/system/mysql.service; enabled; vendor preset: enabled)
Active: activating (start-post) (Result: exit-code) since Sun 2018-04-22 21:30:42 UTC; 17s ago
Process: 21980 ExecStart=/usr/sbin/mysqld (code=exited, status=1/FAILURE)
Process: 21964 ExecStartPre=/usr/share/mysql/mysql-systemd-start pre (code=exited, status=0/SUCCESS)
Main PID: 21980 (code=exited, status=1/FAILURE); : 21988 (mysql-systemd-s)
Tasks: 2
Memory: 320.0K
CPU: 278ms
CGroup: /system.slice/mysql.service
-control |-21988 /bin/bash /usr/share/mysql/mysql-systemd-start post-22390 sleep 1

Apr 22 21:30:42 my..net systemd[1]: Stopped MySQL Community Server.
Apr 22 21:30:42 my..net systemd[1]: Starting MySQL Community Server...
Apr 22 21:30:43 my.*.net systemd[1]: mysql.service: Main process exited, code=exited, status=1/FAILURE

Unbound service also not working, but starts if I restart it.

I don't know how other services are doing.

This happened by itself, I was not doing anything with the server. I can of course wipe the server, reinstall and load a backup, but I would prefer not to have this happen again.

I have now wiped the server and am restoring from backup, I can have it offline for longer.

But I will leave this here for future reference.

Thanks, T.

girish

@tadeas It's hard to tell what is happening, are you able give us SSH access to the server? SSH keys are here - https://cloudron.io/documentation/support/#remote-support

Can you also tell us:

Does the server have sufficient disk space? If it ran out of disk space, please use the procedure here - https://cloudron.io/documentation/server/#recovery-after-disk-full
What version of Cloudron are you running?
Can you paste the output of "ls -lh /var/lib/mysql/box"? If you see the eventlog to be very large (like in gigs), that is probably the root of all the problems.

Thanks,

Girish

tadeas

@girish Thanks for the response. I have wiped the server and did a restore, so I can't do what you suggest atm.

There is/was plenty of space (42gb used out of 490), and ram (1gb out of 6gb).

I was/am running 2.0.1.

I'll check the event log size once the problem comes in again and will report here.