cloudron hangs up needing a restart at least every few days

tadeas

Hi, I've had this problem almost the whole time using cloudron, but it has intensified the last few days and weeks - cloudron "hangs" - I can log in via ssh, but no apps are working, and neither is the my.* cloudron interface. This used to happen a few days inbetween, now almost every other day. Running latest cloudron version.

I have tried backing up, reinstalling ubuntu (I am using contabo vps) and restoring, but the problem is still occuring.

I can provide access to the machine if you want to inspect the logs. I could not find anything special in the cloudron logs myself.

T.

nebulon

Actually this is not the first time this happens on customers on Contabo. If it happens, does a systemctl restart docker help?

muewe96

Same here. I'm running cloudron on a contabo vps. Every few days cloudron hangs. To connect again I need to restart the whole vps. Just restarting docker does not work.

When I try to login to nextcloud i get this error:

Interner Serverfehler
Der Server konnte die Anfrage nicht fertig stellen.

Sollte dies erneut auftreten, senden Sie bitte die nachfolgenden technischen Einzelheiten an Ihren Server-Administrator.

Weitere Details können im Server-Protokoll gefunden werden.

Technische Details
Entfernte Adresse: 172.18.0.1

nebulon

Thanks for the hint, this looks like in such a case, the nginx reverse proxy cannot connect to the docker subnetwork, in which the apps are spawned. I wonder if contabo has some custom networking or firewall scripts, which from time to time overwrite the configuration Cloudron expects.

If this happens again, could you attempt to ping the gateway with ping 172.18.0.1?

Also since this appears to be contabo specific, maybe try to contact their support in such a situation as well, possibly they can take a look and give more insights on the case.

tadeas

I never got such an error in nextcloud and could not figure out the cause from the logs. Restarting docker sent the whole VPS into unreachable state.

Since then I have migrated to Hetzner (a bit less storage space, but fully SSD) and have only booted once there with no more restarts. Contabo is probably also running some heavy optimization to be able to offer the VPSes for such a low price.

muewe96

@nebulon I can ping the ip 172.18.0.1

ping 172.18.0.1

PING 172.18.0.1 (172.18.0.1) 56(84) bytes of data.
64 bytes from 172.18.0.1: icmp_seq=1 ttl=64 time=0.248 ms
64 bytes from 172.18.0.1: icmp_seq=2 ttl=64 time=0.092 ms

Is there something else I should test?

nebulon

We have found out that Contabo has many random issues running Cloudron. The conclusion for now is, that we cannot reliably support Contabo as is, maybe the situation gets better with running ubuntu 18.04 there. I will close this thread so far as there is nothing we can currently do really.

I just want to ask if the things are better now with Contabo and using Cloudron.
Contabo itself seems to be very reliable hosting company and I plan to switch to them which makes no sense if Cloudron is struggling on their VPS.

nebulon

I don't believe we have new information here regarding Contabo and if it is now performing better.

ruihildt

@Captain-Kirk I have been using Contabo for a long time, with several Cloudron.

What we found is that the cheapest VPS would get random errors, ever changing and not fixable. When moving to an upper plan, no problem would be noticed.

So I would suggest to go with mid-level VPS at first, and if you encounter error, try another plan.

Thanks @ruihildt. So the VPS L SSD plan should be fine?

ruihildt

@Captain-Kirk That's what I have indeed, I don't remember if I tried the one below, but at that price point, it's a no brainer anyway.

robi

Another option that is more stable is the link in my signature for SSDnodes.

It seems their default server routing places you in a busy area, so if you notice issues in the first few days, an email to support to ask they move you to another part of the DC and you'll be fine.

Can't beat the price.

@robi thanks, looks amazing. The only thing is the GDPR. So I have to work with European (German) providers which host in German data centers.

robi

Yep, I think they're in Amsterdam but not Germany yet.

jdaviescoates

@staff I wonder if this thread should have contabo added to its title and/ or be tagged contabo to help people find it when considering VPS hosts?

rmdes

Just to Confirm that Contabo VPS L SSD plan has no issue with Cloudron

robi

@jdaviescoates search will find it.

scooke

@Captain-Kirk If you are still talking about ssdnodes.com they do have a Frankfurt location which I've been on for over a year. There is a sale in fact right now for that location!

image url)

@scooke That's right. But the company is based in CA. American companies are subject to the Freedom Act (formerly: Patriot Act) and are therefore obliged to allow US authorities potential access to the (cloud) data of their customers.

I am leaving my comfortable G-Suite and will fully comply with data protection regulations with my hosted solutions. Do it once, do it right

scooke

@Captain-Kirk Ah, I understand now. Sorry about that. What company will you choose? I'm also looking for an EU-based host.

@scooke I wait until Cloudron Version 6 is released, where Ubuntu 20.04 is supported. Then I will move to Contabo. A lot of the memebers here trust in them.
https://contabo.de/?show=vps

scooke

@Captain-Kirk Pretty good prices actually. Thanks.

msbt

Sorry to hijack this topic for contabo related questions, but there seem to be a lot of customers here that could maybe shed some light on my issue. I ordered a VPS 1400 and setting it up worked fine (although being slower than anything I have from hetzner, from connecting via ssh - this takes a few seconds sometimes - to installing things), but after 2-3 days I wanted to reinstall Nextcloud and realized that it couldn't resolve any domain names. I've already contacted support, but maybe this is a known issue and there's already a solution for it.

Basically the name resolution stopped working, so Cloudron can't download the docker images or get a read from api/appstore. Restart didn't solve it and I didn't change anything. Any thoughts?

Cheers, M

girish

@msbt I would start by debugging at the server level.

Check /etc/resolv.conf . This gives the DNS server the system uses. In Cloudron, this should be set to 127.0.0.1. This is where unbound is listening.
systemctl status unbound to see if unbound is running
host www.cloudron.io 127.0.0.1 this queries unbound to resolve the domain name
host www.cloudron.io 8.8.8.8 this queries google DNS to resolve the domain name

Do all the above look good?

subven

@msbt I had this exact issue while installing Cloudron an a Contabo VPS lately. The installer was unable to pull docker containers because of errors like this:

Get https://registry-1.docker.io/v2/: dial tcp: lookup registry-1.docker.io on 192.168.65.1:53: no such host

The problem was that Contabos Ubuntu image uses Contabos own nameservers and enforces DNSSEC. I can post you my configuration changes if you need them.

PS: I have also set host entries for docker.io (before I found the real issue) but that should only be a temporary fix because their IPs can change.

msbt

@subven that would be nice, I just "solved" the issue with a complete shutdown and reboot, for whatever reason it started working again. If you have a more permanent solution, would appreciate if you could share it.

thanks @girish, I'll try that the next time, at the moment it's working again.

subven

@msbt SSH into your server and check if systemd-resolved is running

Cloudron makes it easy to run web apps like WordPress, Nextcloud, GitLab on your server. Find out more or install now.

Solved cloudron hangs up needing a restart at least every few days