Timeout in ESPOcrm, need to restart Webservice

[A Former User]

Hi,
our problem is that espocrm stops responding. We got a red Timeout message in the espocrm UI.
Using supervisorctl restart web in the Terminal resolves the problem.

I did not find any hints in the logs (cloudron and espo)

[girish]

@jensbee4 I published a new package now where we delete the apache pid file on a restart. I am not 100% sure if that is the source of the problem, but it's worth trying.

If it still gets stuck, can you capture the output of supervisorctl status web and ps aux? Thanks!

[A Former User]

While stuck:

root@XXX693:/app/code/5.9.4# ps aux
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root         1  0.0  0.1  64068  9672 pts/0    Ss+  Sep25   3:17 /usr/bin/python /usr/bin/supervisord --configuration /etc/supervisor/supervisord.conf --nodaemon -i EspoCRM
www-data    49  0.0  0.2 554456 22940 pts/0    Sl   Sep25   0:09 /usr/bin/php /app/code/current/websocket.php
root      1899  0.0  0.0  18508  2716 pts/1    Ss+  Sep30   0:00 /bin/bash
root      1915  0.0  0.0  46628  5632 pts/0    S    Sep30   0:02 /usr/bin/python /usr/bin/pidproxy /var/run/apache2/apache2.pid /bin/bash -c APACHE_CONFDIR='' source /etc/apache2/envvars && rm
root      1916  0.0  0.4 695032 34364 pts/0    S    Sep30   0:06 /usr/sbin/apache2 -DFOREGROUND
root      1920  0.0  0.0   4672   700 pts/0    S    Sep30   0:00 /bin/cat
root      1921  0.0  0.0   4672   712 pts/0    S    Sep30   0:00 /bin/cat
www-data  2132  0.0  0.4 698144 35320 pts/0    S    06:52   0:00 /usr/sbin/apache2 -DFOREGROUND
www-data  2133  0.0  0.5 782308 45880 pts/0    Sl   06:55   0:00 /usr/sbin/apache2 -DFOREGROUND
www-data  2326  0.0  0.4 698200 38348 pts/0    S    07:13   0:00 /usr/sbin/apache2 -DFOREGROUND
www-data  2338  0.0  0.4 698136 39396 pts/0    S    07:15   0:00 /usr/sbin/apache2 -DFOREGROUND
www-data  2348  0.0  0.4 698196 39700 pts/0    S    07:17   0:00 /usr/sbin/apache2 -DFOREGROUND
www-data  2592  0.1  0.5 780220 42248 pts/0    Sl   07:48   0:00 /usr/sbin/apache2 -DFOREGROUND
root      2601  0.8  0.0  18508  3376 pts/2    Ss   07:56   0:00 /bin/bash
root      2616  0.0  0.0  34404  2888 pts/2    R+   07:56   0:00 ps aux

Next output

root@XXXa693:/app/code/5.9.4# supervisorctl status web
web                              RUNNING   pid 1915, uptime 1 day, 0:20:56

Restarting Service:
web: stopped
web: started

[girish]

@jensbee4 Ah thanks. I think the issue is that apache gets "stuck". I have seen this happen when I run stress on the PHP apps as well. For a start, are you able to give the EspoCRM app more memory under Resources? How much memory have you given it right now? I think that will mitigate the issue.

[robi]

@girish does it make sense to replace apache with nginx or lighttpd (many other options that are more modern, reliable, lightweight, faster and consume less memory)?

[girish]

Maybe, if we can't get apache to work. Usually upstream always recommends apache, so it's easier for us.

[A Former User]

Memory Limit is set to 1GB ( CPU Shares : 75 %)
Will go to 2GB. I will report when it happend again.

Regards Jens

[A Former User]

ESPO hangs again
ps aux:

USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root         1  0.0  0.0  63948  7312 pts/0    Ss+  Oct02   2:10 /usr/bin/python /usr/bin/supervisord --configuration /etc/supervisor/supervisord.conf --nodaemon -i EspoCRM
root        37  0.0  0.0  46620  5016 pts/0    S    Oct02   0:10 /usr/bin/python /usr/bin/pidproxy /var/run/apache2/apache2.pid /bin/bash -c APACHE_CONFDIR='' source /etc/apache2/envvars && rm -f ${A
www-data    38  0.0  0.2 554456 23084 pts/0    Sl   Oct02   0:03 /usr/bin/php /app/code/current/websocket.php
root        41  0.0  0.3 695028 24060 pts/0    S    Oct02   0:25 /usr/sbin/apache2 -DFOREGROUND
root        45  0.0  0.0   4672   740 pts/0    S    Oct02   0:00 /bin/cat
root        46  0.0  0.0   4672   688 pts/0    S    Oct02   0:01 /bin/cat
root        69  0.0  0.0  18508  2304 pts/1    Ss+  Oct02   0:00 /bin/bash
www-data  1065  0.0  0.4 698148 34536 pts/0    S    05:50   0:00 /usr/sbin/apache2 -DFOREGROUND
www-data  1068  0.0  0.4 698156 34496 pts/0    S    06:02   0:00 /usr/sbin/apache2 -DFOREGROUND
www-data  1078  0.0  0.3 697832 30292 pts/0    S    06:27   0:00 /usr/sbin/apache2 -DFOREGROUND
www-data  1088  0.0  0.4 698148 32612 pts/0    S    06:40   0:00 /usr/sbin/apache2 -DFOREGROUND
www-data  1091  0.2  0.4 698172 36008 pts/0    S    06:58   0:00 /usr/sbin/apache2 -DFOREGROUND
www-data  1102  0.6  0.4 698144 35748 pts/0    S    07:00   0:00 /usr/sbin/apache2 -DFOREGROUND
root      1103  0.0  0.0  18508  3404 pts/2    Ss   07:01   0:00 /bin/bash
root      1123  0.0  0.0  34404  2908 pts/2    R+   07:02   0:00 ps aux

supervisorctl status web
web RUNNING pid 37, uptime 3 days, 22:12:03

[girish]

@jensbee4 Thanks. In the latest EspoCRM package (1.14.0), you can now go to File Manager and edit apache/mpm_preform.conf. What you can do is to bump the MaxRequestWorkers to say 20 or so and then restart the app.

[A Former User]

@girish
OK, will try to change the config.

I got following:

root@bcd346e4-22d8-49d4-93fc-89f39a77a693:/app/code/6.0.0# supervisorctl restart webnt size.
web: stopped
web: ERROR (spawn error)
root@bcd346e4-22d8-49d4-93fc-89f39a77a693:/app/code/6.0.0# supervisorctl restart web
web: stopped
web: ERROR (spawn error)
root@bcd346e4-22d8-49d4-93fc-89f39a77a693:/app/code/6.0.0# supervisorctl restart web
web: stopped
web: started
root@bcd346e4-22d8-49d4-93fc-89f39a77a693:/app/code/6.0.0#

Is the a log to see something about the spawn error?

[girish]

@jensbee4 said in Timeout in ESPOcrm, need to restart Webservice:

Is the a log to see something about the spawn error?

Yes, it's all redirected to stdout. So you can see it in the cloudron logs view.

[marcusquinn]

We still have the same problem too.

[nebulon]

Most likely this was also related to the apache max worker count setting https://httpd.apache.org/docs/2.4/mod/mpm_common.html#maxrequestworkers

For next release we will put this as the default to 256 while one can still configure a higher value if needed in the prefork config in /app/data

[p44]

Dear all, it seems that I'm experiencing this problem... I can only surf to homepage... I'm on v7.6.3 (Ubuntu 20.04.2 LTS), Espo CRM 8.1.0

[girish]

@p44 anything in the logs?

[p44]

@girish
Jan 11 15:59:37=> Healtheck error: Error: Timeout of 7000ms exceeded
Jan 11 15:59:47=> Healtheck error: Error: Timeout of 7000ms exceeded
Jan 11 15:59:57=> Healtheck error: Error: Timeout of 7000ms exceeded
Jan 11 16:00:07=> Healtheck error: Error: Timeout of 7000ms exceeded
Jan 11 16:00:17=> Healtheck error: Error: Timeout of 7000ms exceeded
Jan 11 16:00:27=> Healtheck error: Error: Timeout of 7000ms exceeded

[robi]

@p44 that means it's not running.

[p44]

@robi yes, but it was in "running" state... I increased resources to a VPS and now seems to be stable... very strange...

[robi]

@p44 check the App memory graphs

[p44]

@robi All Memory graphs about Cpu and memory allocations was all ok... even VPS graphs...