Cloudron Service Not available after Server Reboot

[senthilkumaran]

I am using Cloudron on AWS LightSail. I rebooted my instance, stopped/ started. The services are coming up fine, but my dashboard is not loading.

My Dashboard URL : https://my.sourashtraassociation.org

Services

• Box

ubuntu@ip-172-26-4-82:~$ sudo systemctl status box
● box.service - Cloudron Admin
     Loaded: loaded (/etc/systemd/system/box.service; enabled; vendor preset: enabled)
     Active: active (running) since Sat 2023-06-10 17:32:07 UTC; 7min ago
   Main PID: 1035 (node)
      Tasks: 11 (limit: 4710)
     Memory: 80.2M (max: 400.0M)
     CGroup: /system.slice/box.service
             └─1035 node /home/yellowtent/box/box.js

Jun 10 17:32:07 ip-172-26-4-82 systemd[1]: Started Cloudron Admin.
Jun 10 17:32:11 ip-172-26-4-82 sudo[1376]: pam_unix(sudo:session): session opened for user root by (uid=0)
Jun 10 17:32:11 ip-172-26-4-82 sudo[1376]: pam_unix(sudo:session): session closed for user root
Jun 10 17:32:11 ip-172-26-4-82 sudo[1396]: pam_unix(sudo:session): session opened for user root by (uid=0)
Jun 10 17:32:11 ip-172-26-4-82 sudo[1396]: pam_unix(sudo:session): session closed for user root
Jun 10 17:32:41 ip-172-26-4-82 sudo[3407]: pam_unix(sudo:session): session opened for user root by (uid=0)
Jun 10 17:32:41 ip-172-26-4-82 sudo[3407]: pam_unix(sudo:session): session closed for user root
ubuntu@ip-172-26-4-82:~$ 

• Nginx

ubuntu@ip-172-26-4-82:~$ sudo systemctl status nginx
● nginx.service - nginx - high performance web server
     Loaded: loaded (/lib/systemd/system/nginx.service; enabled; vendor preset: enabled)
    Drop-In: /etc/systemd/system/nginx.service.d
             └─cloudron.conf
     Active: active (running) since Sat 2023-06-10 17:32:05 UTC; 10min ago
       Docs: http://nginx.org/en/docs/
    Process: 742 ExecStart=/usr/sbin/nginx -c /etc/nginx/nginx.conf (code=exited, status=0/SUCCESS)
   Main PID: 852 (nginx)
      Tasks: 3 (limit: 4710)
     Memory: 11.7M
     CGroup: /system.slice/nginx.service
             ├─ 852 nginx: master process /usr/sbin/nginx -c /etc/nginx/nginx.conf
             ├─3412 nginx: worker process
             └─3413 nginx: worker process

Jun 10 17:32:03 ip-172-26-4-82 systemd[1]: Starting nginx - high performance web server...
Jun 10 17:32:05 ip-172-26-4-82 systemd[1]: nginx.service: Can't open PID file /run/nginx.pid (yet?) after start: Operation not permitted
Jun 10 17:32:05 ip-172-26-4-82 systemd[1]: Started nginx - high performance web server.
ubuntu@ip-172-26-4-82:~$ 

• Unbound

ubuntu@ip-172-26-4-82:~$ sudo systemctl status unbound
● unbound.service - Unbound DNS Resolver
     Loaded: loaded (/etc/systemd/system/unbound.service; enabled; vendor preset: enabled)
     Active: active (running) since Sat 2023-06-10 17:32:03 UTC; 10min ago
   Main PID: 670 (unbound)
      Tasks: 1 (limit: 4710)
     Memory: 9.6M
     CGroup: /system.slice/unbound.service
             └─670 /usr/sbin/unbound -d

Jun 10 17:32:03 ip-172-26-4-82 systemd[1]: Starting Unbound DNS Resolver...
Jun 10 17:32:03 ip-172-26-4-82 unbound[670]: [670:0] notice: init module 0: subnet
Jun 10 17:32:03 ip-172-26-4-82 unbound[670]: [670:0] notice: init module 1: validator
Jun 10 17:32:03 ip-172-26-4-82 unbound[670]: [670:0] notice: init module 2: iterator
Jun 10 17:32:03 ip-172-26-4-82 systemd[1]: Started Unbound DNS Resolver.
Jun 10 17:32:03 ip-172-26-4-82 unbound[670]: [670:0] info: start of service (unbound 1.9.4).
Jun 10 17:32:04 ip-172-26-4-82 unbound[670]: [670:0] info: generate keytag query _ta-4f66. NULL IN
Jun 10 17:37:13 ip-172-26-4-82 unbound[670]: [670:0] info: generate keytag query _ta-4f66. NULL IN

• Docker

ubuntu@ip-172-26-4-82:~$ sudo systemctl status docker
● docker.service - Docker Application Container Engine
     Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset: enabled)
    Drop-In: /etc/systemd/system/docker.service.d
             └─cloudron.conf
     Active: active (running) since Sat 2023-06-10 17:32:59 UTC; 10min ago
TriggeredBy: ● docker.socket
       Docs: https://docs.docker.com
   Main PID: 873 (dockerd)
      Tasks: 50
     Memory: 140.7M
     CGroup: /system.slice/docker.service
             ├─ 873 /usr/bin/dockerd -H fd:// --log-driver=journald --exec-opt native.cgroupdriver=cgroupfs --storage-driver=overlay2 --experimental --ip6tables
             ├─1799 /usr/bin/docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 222 -container-ip 172.18.0.5 -container-port 22
             ├─1819 /usr/bin/docker-proxy -proto tcp -host-ip :: -host-port 222 -container-ip fd00:c107:d509::8 -container-port 22
             └─2343 /usr/bin/docker-proxy -proto tcp -host-ip 127.0.0.1 -host-port 2003 -container-ip 172.18.0.6 -container-port 2003

Jun 10 17:40:00 ip-172-26-4-82 dockerd[873]: time="2023-06-10T17:40:00.463587881Z" level=info msg="ignoring event" container=7cc174cf515c225153365d18e42944a7e3f85c1757e6f>
Jun 10 17:40:07 ip-172-26-4-82 dockerd[873]: time="2023-06-10T17:40:07.284798767Z" level=info msg="ignoring event" container=7cc174cf515c225153365d18e42944a7e3f85c1757e6f>
Jun 10 17:40:56 ip-172-26-4-82 dockerd[873]: time="2023-06-10T17:40:56.855906990Z" level=info msg="ignoring event" container=1d27a105cae7bb27609489759ce4b8d9b333f0e591fe4>
Jun 10 17:41:01 ip-172-26-4-82 dockerd[873]: time="2023-06-10T17:41:01.053620336Z" level=info msg="ignoring event" container=7cc174cf515c225153365d18e42944a7e3f85c1757e6f>
Jun 10 17:42:20 ip-172-26-4-82 dockerd[873]: time="2023-06-10T17:42:20.688972713Z" level=info msg="ignoring event" container=252a4cd3c4a51a9f7c1ac13beb240286103d6395d9fdd>
Jun 10 17:42:21 ip-172-26-4-82 dockerd[873]: time="2023-06-10T17:42:21.621378971Z" level=info msg="ignoring event" container=85b87238bf6f44f11932cd849224487546baa866b43aa>
Jun 10 17:42:28 ip-172-26-4-82 dockerd[873]: time="2023-06-10T17:42:28.609166794Z" level=info msg="ignoring event" container=7cc174cf515c225153365d18e42944a7e3f85c1757e6f>
Jun 10 17:42:30 ip-172-26-4-82 dockerd[873]: time="2023-06-10T17:42:30.305650814Z" level=info msg="ignoring event" container=7cc174cf515c225153365d18e42944a7e3f85c1757e6f>
Jun 10 17:42:56 ip-172-26-4-82 dockerd[873]: time="2023-06-10T17:42:56.637321501Z" level=info msg="ignoring event" container=7cc174cf515c225153365d18e42944a7e3f85c1757e6f>
Jun 10 17:43:13 ip-172-26-4-82 dockerd[873]: time="2023-06-10T17:43:13.689058221Z" level=info msg="ignoring event" container=7cc174cf515c225153365d18e42944a7e3f85c1757e6f>

• tail of the logs

tail -f /home/yellowtent/platformdata/logs/box.log
2023-06-10T17:43:50.020Z box:apphealthmonitor setHealth: 41b888a8-3bbe-4f98-bed0-88de0b07f97c (polltam.sourashtraassociation.org) waiting for 509.98 to update health
2023-06-10T17:43:51.067Z box:apphealthmonitor app health: 4 running / 3 stopped / 2 unresponsive

• Tail shows the apps are running.
• I am not sure why my dashboard and apps are not accessible.

Could it be that ip address of the instance changed the service is not able to resolve to the new ip address?

FWIW, I am using Google Cloud DNS.

Any troubleshooting suggestions?

Thank you,
Senthil

[senthilkumaran]

Could it be that ip address of the instance changed the service is not able to resolve to the new ip address?

The AWS LightSail IP-Address had changed, I had to manually go to my Google Cloud DNS, and update the ip-address to new ip-address and do a DNS Sync for other apps to update the ip-address.

I am bit surprised that DNS sync wasn't automatic when the host ip-address had changed. Am I missing anything?