PostgreSQL Service Stuck in a Restart Loop - Health Check Failing
-
Hello,
My server recently experienced a severe CPU overload. I have resolved the overload, but my PostgreSQL service is now stuck in a constant restart loop, which prevents apps like n8n from starting.
I have done troubleshooting and have confirmed the following:
- docker ps | grep postgres shows the container is permanently in a Restarting state.
- A full server reboot, even with all other apps stopped, did not resolve the issue.
- The docker logs postgresql command does not show any fatal crash errors. Instead, it shows a repeating loop where the database successfully performs automatic recovery, reports database system is ready to accept connections, and then is terminated a few seconds later.
- This suggests the PostgreSQL service is running correctly, but is failing Cloudron's internal health check and being killed.
- Because the database is never online for more than a few seconds, apps that depend on it (like n8n) cannot start and fail with a DNS lookup error (getaddrinfo EAI_AGAIN postgresql).
The core server health is now good (htop is low, disk space is fine), but this PostgreSQL loop is preventing me from using my apps.
Could you please advise on how to diagnose the failing health check or how to force the PostgreSQL service into a stable state?
-
Hello @Obi79
Can you please try to give the PostgreSQL service more memory in the Cloudron Dashboard?
This might already fix it.
@robi said in PostgreSQL Service Stuck in a Restart Loop - Health Check Failing:
Have you rebooted the server yet?
Maybe the topic was edited after you wrote your message, since @Obi79 stated:
@Obi79 said in PostgreSQL Service Stuck in a Restart Loop - Health Check Failing:
A full server reboot, even with all other apps stopped, did not resolve the issue.
-
Thank you for your feedback.
I have increased the memory to 2GB and rebootet several times. Here's the looping log:
Oct 13 12:19:48 server started Oct 13 12:19:48 ==> Waiting for Postgres Oct 13 12:19:48 pg_ctl: server is running (PID: 15) Oct 13 12:19:48 /usr/lib/postgresql/16/bin/postgres "-D" "/var/lib/postgresql/16/main" Oct 13 12:19:48 ==> Postgres is running Oct 13 12:19:48 ==> Resetting root password Oct 13 12:19:48 ALTER ROLE Oct 13 12:19:48 ==> Upgrading pgvectors Oct 13 12:19:48 2025-10-13 10:19:48.537 UTC [30] root@postgres ERROR: database "cloudronpgvectorupdate" already exists Oct 13 12:19:48 2025-10-13 10:19:48.537 UTC [30] root@postgres STATEMENT: CREATE DATABASE cloudronpgvectorupdate Oct 13 12:19:48 ERROR: database "cloudronpgvectorupdate" already exists Oct 13 12:20:40 ==> Detected existing installation Oct 13 12:20:40 ==> Copying updated config Oct 13 12:20:40 ==> Updating existing postgresql Oct 13 12:20:40 waiting for server to start....2025-10-13 10:20:40.763 UTC [15] LOG: starting PostgreSQL 16.8 (Ubuntu 16.8-0ubuntu0.24.04.1) on x86_64-pc-linux-gnu, compiled by gcc (Ubuntu 13.3.0-6ubuntu2~24.04) 13.3.0, 64-bit Oct 13 12:20:40 2025-10-13 10:20:40.765 UTC [15] LOG: listening on IPv4 address "0.0.0.0", port 5432 Oct 13 12:20:40 2025-10-13 10:20:40.766 UTC [15] LOG: listening on IPv6 address "::", port 5432 Oct 13 12:20:40 2025-10-13 10:20:40.777 UTC [15] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432" Oct 13 12:20:40 2025-10-13 10:20:40.813 UTC [18] LOG: database system was interrupted; last known up at 2025-10-13 10:19:48 UTC Oct 13 12:20:45 ....2025-10-13 10:20:45.582 UTC [18] LOG: database system was not properly shut down; automatic recovery in progress Oct 13 12:20:45 2025-10-13 10:20:45.599 UTC [18] LOG: redo starts at 6/546D6180 Oct 13 12:20:45 2025-10-13 10:20:45.599 UTC [18] LOG: invalid record length at 6/546D7CF8: expected at least 24, got 0 Oct 13 12:20:45 2025-10-13 10:20:45.599 UTC [18] LOG: redo done at 6/546D7CA0 system usage: CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s Oct 13 12:20:45 2025-10-13 10:20:45.640 UTC [16] LOG: checkpoint starting: end-of-recovery immediate wait Oct 13 12:20:45 .2025-10-13 10:20:45.682 UTC [16] LOG: checkpoint complete: wrote 3 buffers (0.0%); 0 WAL file(s) added, 0 removed, 0 recycled; write=0.010 s, sync=0.006 s, total=0.047 s; sync files=4, longest=0.003 s, average=0.002 s; distance=6 kB, estimate=6 kB; lsn=6/546D7CF8, redo lsn=6/546D7CF8 Oct 13 12:20:45 2025-10-13 10:20:45.712 UTC [15] LOG: database system is ready to accept connections Oct 13 12:20:45 done Oct 13 12:20:45 server started```
-
Since the last 3 lines state:
Oct 13 12:20:45 2025-10-13 10:20:45.712 UTC [15] LOG: database system is ready to accept connections Oct 13 12:20:45 done Oct 13 12:20:45 server started
And it still restarts after that? Just to make sure, maybe increase the memory limit even more.
-
Since the last 3 lines state:
Oct 13 12:20:45 2025-10-13 10:20:45.712 UTC [15] LOG: database system is ready to accept connections Oct 13 12:20:45 done Oct 13 12:20:45 server started
And it still restarts after that? Just to make sure, maybe increase the memory limit even more.
@james said in PostgreSQL Service Stuck in a Restart Loop - Health Check Failing:
Since the last 3 lines state:
Oct 13 12:20:45 2025-10-13 10:20:45.712 UTC [15] LOG: database system is ready to accept connections Oct 13 12:20:45 done Oct 13 12:20:45 server started
And it still restarts after that? Just to make sure, maybe increase the memory limit even more.
Yes, it starts again over and over. Increasing memory to 4GB did not change anything.
-
Hello @Obi79
We have looked into your problem.
Please ensure you have a full backup going forward.
If the following steps are too technical, you can enable remote support and write a mail to support@cloudron.io and we will take care of it.- Put the service in recovery mode. https://docs.cloudron.io/services/#configure
- ssh into the server and run
docker exec -ti postgresql /bin/bash
- edit file
/app/code/start.sh
lines 106 and 107 to:psql -Uroot --dbname=postgres -c "CREATE DATABASE IF NOT EXISTS cloudronpgvectorupdate" psql -Uroot --dbname=cloudronpgvectorupdate -c "CREATE EXTENSION IF NOT EXISTS vectors"
- run
/app/code/start.sh
. This should ideally work and database will be running - if step 4 worked, you can disable the recovery mode
-
J james has marked this topic as solved