Nextcloud becomes fully unresponsive - seemingly triggered by scheduled cronjob

[whitespace]

I am running the latest version of Nextcloud on a Cloudron instance. Periodically the app gets unresponsive. It is shown as "Running" in the dashboard but is not functional on front end and sync. The only thing I found in the logs is the following. This coincides with the unresponsiveness. As these messages are not very descriptive, I am not sure what to do here. I already switched off all unused nextcloud plug ins. The Instance is small and only used by a handful of people. No heavy load, mostly syncing text files between 6 machines. CPU and RAM are rarely fully utilized.

It does not fix itself. The app needs to be restarted manually.

=> Run cron job
ldap undefined
ldap undefined
localstorage undefined
localstorage undefined
postgresql undefined
postgresql undefined
redis undefined
redis undefined
scheduler undefined
scheduler undefined
sendmail undefined
sendmail undefined
turn undefined
turn undefined

[girish]

@whitespace those undefined logs can be ignored and are not the source of the error. They are spurious logs caused by a console.log in our code which we left in by mistake. Sorry, it's misleading!

Back to the original issue, when the app goes unresponsive, are you able to open a Web terminal and do a ps aux ? I wonder if apache is getting stuck or something.

[whitespace]

@girish ps aux gives me this. I have a hard time interpreting all values admittedly but in my interpretation there is nothing extraordinary.

[girish]

@whitespace if the dashboard says 'Running', it means that it is responding internally to healthchecks. By any chance is nextcloud behind some proxy/cloudflare etc? Just want to rule out any network issues before we try something.

I would bump the memory limit for a start and see if that helps.

[necrevistonnezr]

Are you running Nextcloud-Office or something like that by any chance?
I had a similar issue recently - Nextcloud was "running" but unresponsive - in the end it was memory and "MaxRequestWorkers", for "tuning" Nextcloud see https://forum.cloudron.io/post/68145

[whitespace]

@girish The nextcloud is not behind a proxy or Cloudflare. Just a domain mapped via DNS API. Nothing extraordinary. I renewed certs and synched DNS already. In the meantime I have asked my VPS host if there are any known networking issues. Waiting for a response there.

The concrete things that are not running as soon as the problem occurs are 1. the Nextcloud frontend and 2. the Nextcloud sync via desktop clients. As far as I have tested, this does not go away automatically. Only a restart via dashboard fixes it.

[whitespace]

@necrevistonnezr This was my first thought too. I was able to log-in and specifically deactivated all nextcloud applications not needed. The only active apps are Files, Activity and Calendar.

I looked into the thread and gave the Nextcloud more Memory and ramped up max-request workers. Testing right now.

[whitespace]

@whitespace @girish @necrevistonnezr

So it happened again.

at 15:45 the log says

Jun 21 15:45:09 => Run cron job

At the exact same time my online monitor shows the Nextcloud as being offline.

So I looked into the scheduled cron job which contains

exec /usr/local/bin/gosu www-data:www-data php -f /app/code/cron.php

Now I'd be at the point of restarting the app manually.
Does this give any clue?

---

EDIT:

It happened again independently from the scheduled cron task. The last log entries were regular stuff of loading the activity page in a browser via GET requests.

[girish]

@whitespace can you reach out to us at support@cloudron.io, I can debug on the server further.

[girish]

One part of the issue was fixed with https://git.cloudron.io/cloudron/box/-/commit/6259849958a41af3f08bdb01e5ffe4367ef6df34 . The health check monitor was holding on to connections incorrectly and clogging up apache instances.

[whitespace]

@girish Uptime is normalizing again. No problems in the last 23 hours. In addition we are upgrading the server from 1 vCPU to a 4 vCPUs since more clients are using it than before. Thank you so much for the care!