Nextcloud Talk CHAT very slow

[i.fritz]

Hello!
We have the problem that Cloudron Nextcloud Talk Chats are very slowly. Entered text is shown 20s-30s later. We use HPF-backend for video, this works well.
I searched Google for that and they say it's a problem with http2 and Apache mpm_prefork. We had to use Apache mpm_event instead (or maybe disable http2). Is there a way to change Apache mode to mpm_event? I just see a Apache config for mpm_prefork where I highered values to this Nextcloud (> 100 user).

I don't think it makes sense to do changes on command line if next docker version overwrites all setttings.

Bye,
Ingo

[joseph]

@i.fritz said in Nextcloud Talk CHAT very slow:

I searched Google for that and they say it's a problem with http2 and Apache mpm_prefork

Do you have this link? What could be the problem with http2 and mpm_prefork be? http2 is everywhere by now .

[i.fritz]

https://github.com/nextcloud/spreed/issues/2211

http2: "After disabling http2, messaging is possible nearly in real time"

nickvergessen2 (Nextcloud Talk developer at Nextcloud GmbH) 08.05.2025:
"It's a misconfiguration on your side, nothing we can fix software side. If you run PHP FPM you need to run the Apache "event" module. FPM + "prefork" will never work"

FPM + Apache prefork is exactly, what Cloudron Nextcloud is working. So should be changed to mpm_event.

[james]

I will take a look at it.

[james]

Hello @i.fritz

@i.fritz said in Nextcloud Talk CHAT very slow:

Entered text is shown 20s-30s later.

Could you go into a little more detail here?
A: You write some text, send it and have to wait 20s -> 30s until the message appears in your local chat
B: You write some text, send it, the receiver gets the message 20s -> 30s seconds late

I could not reliably reproduce this. Also, I don't have access to >100 User Nextcloud for testing more close yto your setup.

Maybe a small video of the problem could help to understand it.
If you make one, please have the browser console open so the console logs are viewable.

[i.fritz]

It took a while to get a video from customer:
https://nxt.md-nord.de/s/FWg4S6tGWZFw73X

I meanwhile modified Apache mpm prefork settings again. I think last update to v31 changed them back:

<IfModule mpm_prefork_module>
    # On startup, start these many servers
	StartServers	50

    # At any given time, keep atleast these many servers
	MinSpareServers	20

    # At any given time, keep atmost these many idle servers (this is always >= MinSpareServers+1)
	MaxSpareServers 20

    # Maximum number of servers at any given instant. Requests will be queued after this
	MaxRequestWorkers	  200

    # Recycle process after handling these many requests. This protected against accidental memory leaks
	MaxConnectionsPerChild   1000
</IfModule>

Server has a 16 Core AMD Ryzen 9 5950 CPU, 128 GB RAM and 4 TB NVMe (RAID1). Load is about 2,5-3,0 and I see CPU at a lot of Apache processes.

Thank you!
Bye,
Ingo

[james]

Hello @i.fritz
Thanks for providing these details.

Inside you config I can see something shady:

MinSpareServers	20
MaxSpareServers 20

But MaxSpareServers should be MaxSpareServers >= MinSpareServers+1 so at least 21.
Just in case this could cause any issues.

@i.fritz said in Nextcloud Talk CHAT very slow:

I see CPU at a lot of Apache processes

As expected when you start StartServers 50 and a minimum of MinSpareServers 20 so at least 70 processes.
So maybe this config is already a bit shady.

<!-- Disclaimer - the following contains partly AI generated content --!>

I've done my due diligance to confirm that everything is correct and sane.
Made the AI format my sloppy text into something readable.

<!-- End of disclaimer --!>

---

For a Nextcloud setup with 100+ users, the Apache prefork MPM settings should balance performance, stability, and resource efficiency. Below are recommended values based on typical production environments, alongside key considerations:

---

Recommended Apache Prefork Settings

<IfModule mpm_prefork_module>
    StartServers             5
    MinSpareServers          5
    MaxSpareServers          10
    MaxRequestWorkers        100   # Previously "MaxClients"
    MaxConnectionsPerChild   5000
</IfModule>

---

Explanation & Rationale

1. StartServers 5

   • Initial processes to handle traffic spikes at startup.
   • Why? Avoids delay in spawning processes during sudden load.

2. MinSpareServers 5
MaxSpareServers 10

   • Maintains 5–10 idle processes to absorb traffic surges.
   • Why? Prevents latency by keeping workers ready. For 100+ users, higher spares reduce wait times during concurrency peaks.

3. MaxRequestWorkers 100

   • Maximum simultaneous requests.
   • Why?
     • Assumes 10–20% concurrent active users (10–20 requests at peak).
     • Each user may open 2–5 connections (pages, syncs, uploads).
     • Adjust based on RAM:
       • Estimate RAM/process:
         • Minimal: 30–50 MB (optimized PHP).
         • Typical: 70–150 MB (with PHP modules).
       • Formula:
         MaxRequestWorkers = (Available RAM for Apache) / (Average PHP process size)
       • Example: 8 GB RAM server → Reserve 4 GB for Apache → 4000 MB / 50 MB ≈ 80 workers.
     • Start with 100, monitor, and adjust.

4. MaxConnectionsPerChild 5000

   • Recycle processes after 5000 requests.
   • Why? Prevents memory leaks without excessive process recycling.
     • Avoid values <1000 (causes frequent restart overhead).
     • Values >10k are acceptable if leaks are minimal.

---

Critical Considerations

• PHP Memory:
  Set php.ini values:

  max_execution_time = 300 ; For large uploads/ops
  

Cloudron sets this for PHP by default:

RUN crudini --set /etc/php/8.3/apache2/php.ini PHP upload_max_filesize 5G && \
    crudini --set /etc/php/8.3/apache2/php.ini PHP post_max_size 5G && \
    crudini --set /etc/php/8.3/apache2/php.ini PHP memory_limit 512M && \
    crudini --set /etc/php/8.3/apache2/php.ini opcache opcache.enable 1 && \
    crudini --set /etc/php/8.3/apache2/php.ini opcache opcache.enable_cli 1 && \
    crudini --set /etc/php/8.3/apache2/php.ini opcache opcache.interned_strings_buffer 32 && \
    crudini --set /etc/php/8.3/apache2/php.ini opcache opcache.max_accelerated_files 10000 && \
    crudini --set /etc/php/8.3/apache2/php.ini opcache opcache.memory_consumption 128 && \
    crudini --set /etc/php/8.3/apache2/php.ini opcache opcache.save_comments 1 && \
    crudini --set /etc/php/8.3/apache2/php.ini opcache opcache.revalidate_freq 1 && \
    crudini --set /etc/php/8.3/apache2/php.ini Session session.save_path /run/nextcloud/sessions && \
    crudini --set /etc/php/8.3/apache2/php.ini Session session.gc_probability 1 && \
    crudini --set /etc/php/8.3/apache2/php.ini Session session.gc_divisor 100

---

Adjustment Workflow

1. Start with recommended values.
2. Simulate load with tools like ab:

   ab -n 1000 -c 50 https://your-nextcloud.com/
   

Note: Do not start the load test from your desktop machine at home or the Nextcloud server itself.
It should be a VM with enough power. Maybe rent a bigger Hetzner vServer for 30–60 minutes to do the load test against your Nextcloud.

3. Monitor:
   • RAM usage (avoid >70% total usage).
   • Idle workers (should rarely hit 0).
   • Queue length (via mod_status; requests waiting for workers).
4. Scale incrementally:
   • Increase MaxRequestWorkers if queues form.
   • Reduce MaxSpareServers if idle processes waste RAM.

With these setting your Nextcloud should at least be assigned 8GB RAM, but since you have 128 GB why not give it 12GB.
This might already help.