Cloudron v9: huge disk I/O is this normal/safe/needed?

[imc67]

@Joseph isn't it strange that you set this topic to solved without checking?

@girish & @nebulon today I spend an awful lot of time to analyse this issue together with Claude.ai and this is the result:

Cloudron health checker triggers excessive MySQL disk I/O via Matomo LoginOIDC plugin

I want to report a bug that causes massive MySQL disk write I/O on servers running Matomo with the LoginOIDC plugin (which is the default Cloudron OIDC integration).

The problem

The Cloudron health checker calls the root URL / of the Matomo app every 10 seconds. When Matomo's LoginOIDC plugin is active, every single health check request causes PHP to create a new session in MySQL containing a Login.login nonce and a LoginOIDC.nonce — even though no user is logging in.

This results in exactly 360 new MySQL session rows per hour, 24 hours a day, on every server running Matomo with OIDC enabled.

Evidence

Session count per hour over a full day (consistent across 3 separate servers):

00:00 → 360 sessions
01:00 → 360 sessions
02:00 → 360 sessions
... (identical every hour, including 3am)
23:00 → 360 sessions

360 sessions/hour = exactly 1 per 10 seconds = the Cloudron health check interval.

Decoding a session row from the MySQL session table confirms the content:

a:3:{
  s:11:"Login.login"; a:1:{s:5:"nonce"; s:32:"44e6599e05b0e829ec469459a413fc11";}
  s:4:"__ZF"; a:2:{
    s:11:"Login.login"; a:1:{s:4:"ENVT"; a:1:{s:5:"nonce"; i:1772890030;}}
    s:15:"LoginOIDC.nonce"; a:1:{s:4:"ENVT"; a:1:{s:5:"nonce"; i:1772890030;}}
  }
  s:15:"LoginOIDC.nonce"; a:1:{s:5:"nonce"; s:32:"7456603093600c7a3686d560bc61acd1";}
}

These are unauthenticated OIDC handshake sessions — not real users.

Sessions have a lifetime of 1,209,600 seconds (14 days), so they accumulate without being cleaned up. On my 3 servers this resulted in 113,000–121,000 session rows per Matomo instance, causing continuous MySQL InnoDB redo log writes and buffer pool flushes of 2.5–4 MB/s disk I/O.

Today's actual visitor count in Matomo: 22 visits across 10 sites. Today's sessions created in MySQL: 4,320+.

Root cause

The Cloudron health checker calls GET / on the Matomo app. This URL triggers the LoginOIDC plugin to initialize an OIDC authentication flow and write a session to MySQL — even for a non-browser health check request with no user interaction.

Suggested fix

The Cloudron health checker should call a static or session-free endpoint instead of /, for example:

• matomo.js or piwik.js (static JavaScript file, no PHP session)
• A dedicated /health or /ping endpoint

This would eliminate the session creation entirely without requiring any changes to Matomo or its plugins.

Environment

• Cloudron v9.1.3 (9.0.17)
• Ubuntu 22.04.5 LTS
• Matomo 5.8.0 with LoginOIDC plugin
• Reproduced on 3 separate Cloudron Pro instances

[joseph]

@imc67 not sure I remember why Does this mean that if you disable matomo temporarily, the disk usage goes down a lot?

Seems easy to fix now that we know the root cause

[imc67]

@joseph I’m pretty sure that more apps suffer from this issue since the introduction of OIDC, I see EspoCRM and FreeScout, also has a Healthcheck to root/ (where the OIDC login is), didn’t check the sessions.

[joseph]

I have to test, but it seems like a matomo bug here (if this is all true). There is no need to create an OIDC session when visiting '/' . You have to only create OIDC session when OIDC login button is clicked.

[luckow]

My two cents: as soon as #28 is correct, this should happen with every Cloudron instance that has Matomo (and OIDC enabled). I looked at one of my instances that met the criteria. One of the Matomo instances had about 300 sessions stored in MySQL. The oldest entry is from Feb 26.
So maybe #28 isn't correct, or it's something that only happens on this instance.

[imc67]

Maybe because the three installs are 5-6 years old and had many many updates/upgrades etc?

can you check how many sessions per hour are being created? Run this query:
sql

SELECT HOUR(FROM_UNIXTIME(modified)) AS hour, COUNT(*) AS sessions
FROM `<your_matomo_db>`.session
WHERE DATE(FROM_UNIXTIME(modified)) = CURDATE() - INTERVAL 1 DAY
GROUP BY hour ORDER BY hour;

On my instances this shows exactly 360 per hour = 1 per 10 seconds = health check interval. If yours shows much less, the health checker behaves differently on your setup.

[luckow]

@imc67 one app instance (4y old)

+------+----------+
| hour | sessions |
+------+----------+
|    0 |        2 |
|    2 |        1 |
|    7 |        2 |
|    8 |        1 |
|    9 |        1 |
|   13 |        3 |
|   15 |        1 |
|   17 |        3 |
|   19 |        1 |
|   20 |        3 |
|   21 |        4 |
|   22 |        1 |
+------+----------+

different app instance (7y old)

+------+----------+
| hour | sessions |
+------+----------+
|    3 |        1 |
|    5 |        2 |
|   15 |        4 |
|   18 |        2 |
|   19 |        2 |
|   20 |        2 |
|   21 |        4 |
|   22 |        2 |
+------+----------+

health check is every 10 sec.

Mar 07 18:00:50 - - - [07/Mar/2026:17:00:50 +0000] "GET / HTTP/1.1" 302 - "-" "Mozilla (CloudronHealth)"
Mar 07 18:00:50 172.18.0.1 - - [07/Mar/2026:17:00:50 +0000] "GET / HTTP/1.1" 302 299 "-" "Mozilla (CloudronHealth)"
Mar 07 18:01:00 - - - [07/Mar/2026:17:01:00 +0000] "GET / HTTP/1.1" 302 - "-" "Mozilla (CloudronHealth)"
Mar 07 18:01:00 172.18.0.1 - - [07/Mar/2026:17:01:00 +0000] "GET / HTTP/1.1" 302 299 "-" "Mozilla (CloudronHealth)"
Mar 07 18:01:10 - - - [07/Mar/2026:17:01:10 +0000] "GET / HTTP/1.1" 302 - "-" "Mozilla (CloudronHealth)"
Mar 07 18:01:10 172.18.0.1 - - [07/Mar/2026:17:01:10 +0000] "GET / HTTP/1.1" 302 299 "-" "Mozilla (CloudronHealth)"
Mar 07 18:01:20 - - - [07/Mar/2026:17:01:20 +0000] "GET / HTTP/1.1" 302 - "-" "Mozilla (CloudronHealth)"
Mar 07 18:01:20 172.18.0.1 - - [07/Mar/2026:17:01:20 +0000] "GET / HTTP/1.1" 302 299 "-" "Mozilla (CloudronHealth)"
Mar 07 18:01:30 - - - [07/Mar/2026:17:01:30 +0000] "GET / HTTP/1.1" 302 - "-" "Mozilla (CloudronHealth)"
Mar 07 18:01:30 172.18.0.1 - - [07/Mar/2026:17:01:30 +0000] "GET / HTTP/1.1" 302 299 "-" "Mozilla (CloudronHealth)"

[imc67]

Second issue found: box.tasks table accumulates completed tasks indefinitely

While investigating the disk I/O issue further, I found a second contributor to the high host MySQL write activity.

The box.tasks table contains tens of thousands of completed tasks (completed=1, pending=0) that are never cleaned up. These go back years:

Server 1 (running since ~2021): 17,752 completed tasks, oldest from 2021
Server 2 (running since ~2019): 22,509 completed tasks, oldest from 2019
Server 3 (running since ~2019): 26,972 completed tasks, oldest from 2019

Breakdown per server 3 as an example:

type                    count   oldest
cleanBackups            9,628   2019-04-19
backup_xxx              7,054   2019-04-19
app                     4,765   2019-10-06
checkCerts              2,239   2021-07-01
reneWcerts              1,611   2019-04-19
updateDiskUsage         1,107   2022-12-03

This large table causes continuous InnoDB buffer pool activity and redo log writes on the host MySQL, contributing to the baseline disk I/O of ~2-3 MB/s — independently of any app issues.

Query to check on your own server:

SELECT type, COUNT(*) as aantal,
MIN(creationTime) as oudste,
MAX(creationTime) as nieuwste
FROM box.tasks
WHERE completed=1 AND pending=0
GROUP BY type
ORDER BY aantal DESC
LIMIT 10;

Questions:

• Is there a safe way to manually clean up old completed tasks?
• Should Cloudron implement automatic cleanup of completed tasks older than X days?
• Is this a known issue or intentional behavior?

[imc67]

We also found some huge MySQL tables from a Wordpress-app with dedicated MainWP due to incorrect retention settings, after correction and deletion the 1 minute iotop -aoP -d 5 is still:

• Docker MySQL: 70 MB
• Host MySQL: 33 MB
• go-carbon: 6.7 MB
• jbd2: 9.9 MB
• Total: ~103 MB per minute

To put this in perspective:

• 103 MB/min = 6.2 GB/hour
• 6.2 GB/hour = 148 GB/day
• 148 GB/day = 4.4 TB/month

This is on a server with relatively low visitors across 10 sites. The vast majority of this write activity is caused by the issues identified above (Matomo health checker sessions, box.tasks accumulation, and app-level retention misconfigurations) — not by actual user traffic.

Note: these are cumulative iotop counters, not sustained rates. The actual average write speed shown by Cloudron's dashboard is ~2.5-4 MB/s, which still translates to 216-345 GB/day of unnecessary disk writes on a lightly loaded server.

[imc67]

Update on the Matomo issue https://forum.cloudron.io/post/121389

I disabled the OIDC plugin in Matomo and then the sessions are still created every 10 seconds, so it's not that plugin. Next when looking into the Matomo settings - System check it says:

Errors below may be due to a partial or failed upload of Matomo files.
--> Try to reupload all the Matomo files in BINARY mode. <--

File size mismatch: /app/code/misc/user/index.html (expected length: 172, found: 170)

The file contains:

/app/code# cat /app/code/misc/user/index.html
This directory stores the custom logo for this Piwik server. Learn more: <a href="https://matomo.org/faq/new-to-piwik/faq_129/">How do I customise the logo in Piwik?</a>

This means nothing serious.

Then I installed a Matomo app on the Cloudron demo server and was able to replicate the every 10 seconds session table increase, within 1 minute 8 extra sessions:

mysql> SELECT COUNT(*), MIN(FROM_UNIXTIME(modified)), MAX(FROM_UNIXTIME(modified)) 
    -> FROM session;
+----------+------------------------------+------------------------------+
| COUNT(*) | MIN(FROM_UNIXTIME(modified)) | MAX(FROM_UNIXTIME(modified)) |
+----------+------------------------------+------------------------------+
|       47 | 2026-03-08 10:58:00          | 2026-03-08 11:05:20          |
+----------+------------------------------+------------------------------+
1 row in set (0.00 sec)

mysql> SELECT COUNT(*), MIN(FROM_UNIXTIME(modified)), MAX(FROM_UNIXTIME(modified))  FROM session;
+----------+------------------------------+------------------------------+
| COUNT(*) | MIN(FROM_UNIXTIME(modified)) | MAX(FROM_UNIXTIME(modified)) |
+----------+------------------------------+------------------------------+
|       55 | 2026-03-08 10:58:00          | 2026-03-08 11:06:40          |
+----------+------------------------------+------------------------------+
1 row in set (0.00 sec)

[imc67]

Third Bug report: Roundcube also creates a new MySQL session on every health check

The same issue we found with Matomo also affects Roundcube. The Cloudron health checker calls GET / every 10 seconds, which causes Roundcube to create a new unauthenticated session in MySQL each time.

Decoded session data from the latest entries:

temp|b:1;
language|s:5:"en_US";
task|s:5:"login";
skin_config|a:7:{...}

This is a pure unauthenticated login page session — no user involved, just the health checker hitting the front page.

Measured growth rate: exactly 6 new sessions per minute per Roundcube instance (= 1 per 10 seconds = health check interval). With 5 Roundcube instances on this server that is 30 new sessions per minute, 43,200 per day.

Suggested fix: Change the health check endpoint from GET / to a static asset that does not trigger PHP session creation, for example:

• A static file like /favicon.ico or /robots.txt
• Or Roundcube's own /index.php/api if available

For reference, WordPress handles this cleanly: GET /wp-includes/version.php returns HTTP 200 with empty output (Wordfence hides the version) without touching the database or creating any session.

It would be great if Cloudron could define a session-free health check endpoint per app type, similar to how it is done for WordPress.