App status stuck on "Starting..."

[shrey]

The app status (after a resize operation) of one of my Directus apps has gotten stuck at
Starting...

---

Logs (post restarting the app):

Feb 18 12:20:24box:taskworker Starting task 8152. Logs are at /home/yellowtent/platformdata/logs/ed500c4a-37ab-4ad4-9a35-a6dff0f6f2ae/apptask.log
Feb 18 12:20:24box:apptask run: startTask installationState: pending_restart runState: running
Feb 18 12:20:24box:tasks update 8152: {"percent":10,"message":"Starting app services"}
Feb 18 12:20:24box:tasks update 8152: {"percent":20,"message":"Restarting container"}
Feb 18 12:20:35==> Enabling postgis extension
Feb 18 12:20:35box:tasks update 8152: {"percent":80,"message":"Configuring reverse proxy"}
Feb 18 12:20:35box:reverseproxy providerMatchesSync: subject=CN = <my-app-domain> domain=<my-app-domain> issuer=C = US, O = Let's Encrypt, CN = R3 wildcard=false/false prod=true/true issuerMismatch=false wildcardMismatch=false match=true
Feb 18 12:20:35box:reverseproxy expiryDate: subject=CN = <my-app-domain> notBefore=Feb 16 05:10:59 2024 GMT notAfter=May 16 05:10:58 2024 GMT daysLeft=87.9308137037037
Feb 18 12:20:35box:reverseproxy needsRenewal: false. force: false
Feb 18 12:20:35box:reverseproxy ensureCertificate: <my-app-domain> acme cert exists and is up to date
Feb 18 12:20:35NOTICE: extension "postgis" already exists, skipping
Feb 18 12:20:35CREATE EXTENSION
Feb 18 12:20:35==> DB Migration
Feb 18 12:20:35box:reverseproxy writeAppLocationNginxConfig: writing config for "<my-app-domain>" to /home/yellowtent/platformdata/nginx/applications/ed500c4a-37ab-4ad4-9a35-a6dff0f6f2ae/<my-app-domain>.conf with options {"sourceDir":"/home/yellowtent/box","vhost":"<my-app-domain>","hasIPv6":true,"ip":"172.18.20.66","port":8055,"endpoint":"app","redirectTo":null,"certFilePath":"/home/yellowtent/platformdata/nginx/cert/<my-app-domain>.cert","keyFilePath":"/home/yellowtent/platformdata/nginx/cert/<my-app-domain>.key","robotsTxtQuoted":"\"# Disable search engine indexing\\n\\nUser-agent: *\\nDisallow: /\"","cspQuoted":null,"hideHeaders":[],"proxyAuth":{"enabled":false,"id":"ed500c4a-37ab-4ad4-9a35-a6dff0f6f2ae","location":"/"},"upstreamUri":"","ocsp":true,"hstsPreload":true}
Feb 18 12:20:35box:shell reload spawn: /usr/bin/sudo -S /home/yellowtent/box/src/scripts/restartservice.sh nginx
Feb 18 12:20:35box:shell reload (stderr): sudo: unable to resolve host apps: Name or service not known
Feb 18 12:20:37box:tasks update 8152: {"percent":100,"message":"Done"}
Feb 18 12:20:37box:taskworker Task took 12.754 seconds
Feb 18 12:20:37box:tasks setCompleted - 8152: {"result":null,"error":null}
Feb 18 12:20:37box:tasks update 8152: {"percent":100,"result":null,"error":null}
Feb 18 12:20:41[06:50:40.433] INFO: Running migrations...
Feb 18 12:20:41[06:50:40.442] INFO: Database up to date
Feb 18 12:20:41==> Starting Directus
Feb 18 12:20:46[06:50:45.932] INFO: Server started at http://0.0.0.0:8055

---

What could be the issue and how to resolve it?

[girish]

@shrey atleast per the logs, the app is running. What happens if you visit the directus app URL directly on the browser?

[girish]

So you should see a ping line as well, like this:

Feb 18 10:52:12[09:52:12.363] INFO: Running migrations...
Feb 18 10:52:12[09:52:12.368] INFO: Database up to date
Feb 18 10:52:16[09:52:15.943] INFO: Server started at http://0.0.0.0:8055
Feb 18 10:52:21[09:52:20] GET /server/ping 200 8ms

If not, this means the box code is not doing healtchecks. Can you try systemctl restart box ?

[shrey]

@girish said in App status stuck on "Starting...":

So you should see a ping line as well, like this:

Nope, not seeing it.

---

Also, systemctl restart box doesn't seem to be working either:

root@ed500c4a-37ab-4ad4-9a35-a6dff0f6f2ae:/app/code# systemctl restart box
System has not been booted with systemd as init system (PID 1). Can't operate.
Failed to connect to bus: Host is down
root@ed500c4a-37ab-4ad4-9a35-a6dff0f6f2ae:/app/code# 

[shrey]

@girish
But, now that i've enabled & then disabled the 'App Recovery Mode', it seems to be working fine again.

Although, when the app started, i did notice this in the logs:

Feb 19 07:52:18npm notice Run npm install -g npm@10.4.0 to update!
Feb 19 07:52:18npm notice
Feb 19 07:52:18==> Starting Directus
Feb 19 07:52:20=> Healtheck error: Error: connect ECONNREFUSED 172.18.20.66:8055
Feb 19 07:52:23[02:22:23.022] INFO: Server started at http://0.0.0.0:8055

Don't know if it's of any relevance to this case.

[girish]

@shrey said in App status stuck on "Starting...":

Also, systemctl restart box doesn't seem to be working either:

Sorry, this was meant to be run via SSH and not the Web Terminal!

[girish]

@shrey said in App status stuck on "Starting...":

ECONNREFUSED

This is normal. When the app is still starting up, it's not responding to health check requests and refusing connection. As long as it becomes 'Running' eventually, all is good.