Cannot Access Cloudron

[A Former User]

I have recently cloned my VPS to a more powerful server due to one or more apps running out of memory. After the migration, I am no longer able to access my Cloudron admin dashboard at my.<domain>.

I am almost certain that the problem is the memory limits imposed on the app(s), but I cannot access the admin dashboard to increase the memory limits. The Cloudron CLI interface also fails to log in, throwing error 502.

My apps are showing "This app is currently not responding. Please try refreshing the page in a few minutes."

Is there a way to increase the memory size of Cloudron/Cloudron admin dashboard, so I can regain access to my server? I have critical data in my apps and the backups from my old server are no longer accessible. I have backups of my apps before the memory limit was reached, but I cannot access those, either, due to lack of access to admin dashboard.

[girish]

@inference If you migrated to a new server, did you change the DNS entry of my.<domain> to point to the new location manually? That's most likely the issue. Otherwise, can you give check systemctl status box ?

[A Former User]

@girish systemctl status box shows that the process/service is running.

Yes, I manually changed the DNS record to point to my new server. I changed both the my.<domain> and <app>.<domain> (my apps run on their own subdomains).

When I try to connect to my.<domain>, it states that Cloudron is offline and is reconnecting; this remains on screen, permanently.

[girish]

@inference Can you also check systemctl status nginx ? You can also check /home/yellowtent/platformdata/logs/box.log . The last few lines should say the apps are alive / dead.

[A Former User]

@girish systemctl status nginx shows that the process/service is also running.

The log file you mentioned shows:

2021-10-08T01:56:37.344Z box:server ==========================================
2021-10-08T01:56:37.345Z box:server            Cloudron 6.3.6  
2021-10-08T01:56:37.345Z box:server ==========================================
2021-10-08T01:56:37.397Z box:settings initCache: pre-load settings
2021-10-08T01:56:37.429Z box:tasks stopTask: stopping all tasks
2021-10-08T01:56:37.430Z box:shell stopTask spawn: /usr/bin/sudo -S /home/yellowtent/box/src/scripts/stoptask.sh all
2021-10-08T01:56:37.462Z box:dockerproxy startDockerProxy: started proxy on port 3003

[girish]

@inference it is supposed to "start up" after that. It seems to be stuck for some reason. You can check if systemctl restart box makes it go ahead. Otherwise, do you see lots of containers running when you do docker ps ?

How did you migrate btw? With a backup or using VM image / snapshot & restore ?

[A Former User]

@girish docker ps shows Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get http://%2Fvar%2Frun%2Fdocker.sock/v1.24/containers/json: dial unix /var/run/docker.sock: connect: permission denied.

When I run the command as root, it hangs without any output.

I migrated by cloning my previous VPS (2 shared CPU cores, 4 GiB RAM). My previous VPS is now unavailable. New VPS has 4 shared CPU cores, 8 GiB RAM.

[girish]

@inference said in Cannot Access Cloudron:

When I run the command as root, it hangs without any output.

Ah ok, this is the root cause then. Cloudron is trying to check status on services on start up (the services are docker containers) and looks like docker is not responding. Does systemctl restart docker help ? You can also try jounalctl -u docker to see docker logs.

[A Former User]

@girish systemctl restart docker outputs A dependency job for docker.service failed. See 'journalctl -xe' for details.

journalctl -u docker outputs

Oct 08 02:04:30 localhost systemd[1]: docker.service: Job docker.service/start failed with result 'dependency'.
Oct 08 02:04:35 localhost systemd[1]: Dependency failed for Docker Application Container Engine.

[A Former User]

@inference @girish After searching the internet, I tried the /usr/bin/dockerd -H unix:// command; it outputs (IDs are container ID and network ID, which I have removed due to privacy/security concerns)

ERRO[2021-10-08T03:30:06.719778335+01:00] failed to start container                     container=<container ID> error="network <network ID> not found

[girish]

@inference mm, not sure what that message means. you shouldn't have to run dockerd manually. maybe you can try a server reboot for good measure to see if that helps run docker again. Is there anything in journalctl -xe ?

[A Former User]

@girish I restarted the server, then restarted both docker.service and docker-socket, but no change,

journalctl -xe outputs

-- The unit UNIT has successfully entered the 'dead' state.
Oct 08 16:32:09 localhost systemd[1402]: var-lib-docker-overlay2-383347feb2d94ef4018a64c1ec85641766201d0e624b311fa83d82c652355b05-merged.mount: Succeeded.
-- Subject: Unit succeeded
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support

and other similar messages.

I noticed that the Ubuntu MOTD has a connection error, too, but my IP address and domain is reachable.

[nebulon]

@inference looks like something more fundamental is wrong with this instance, since you mentioned initially that you had previous Cloudron installation on a small instance, maybe you can do a fresh clone of this and see if this error is the same. Also which VPS provider is this? Regarding the motd message, are there any other outbound connection issues you have seen?

[A Former User]

@nebulon My VPS provider is Linode. I think the issue is 1 of 2 things; either a network issue due to the cloned VPS having a different IP address, or a memory issue of the Cloudron apps, which ran out of memory and I have not been able to increase the memory limits due to lack of access to the Cloudron admin dashboard.

I have not noticed any other outbound issues.

Is there a way to recover my backups from Cloudron, so I could create a new VPS, then restore from an earlier backup before the issues occurred?

[girish]

@inference the issue is docker itself is not running (for some reason). app memory is not an issue because apps are run as containers using docker.

What I can suggest is:

• spin up another new VPS using the VM image you have. check if docker runs then
• for backups, do you remember how you configured backups in Cloudron ? By default, the backups are in /var/backups of the server. Do you see many timestamped directories in that folder?