Cloudron Forum

Hello Cloudron,

Thank you for fixing the Solr OOM issue, sadly that was just hiding another issue with users with large mailboxes and large attachments. Some users like to send enormous CAD and spreadsheets to each other and documents the size of war and peace.

Issue:
Full-text search indexing fails on mailboxes containing emails with large attachments due to a hardcoded 60-second HTTP timeout in Dovecot's FTS Solr communication.

Evidence:

• Mailbox A: 120,545 emails → 100% indexed successfully (small emails, batches complete in 1-3 seconds)

• Mailbox B: 35,212 emails → Only 48% indexed (emails with large attachments cause slow batches)

Root Cause:
When Tika processes large email attachments during indexing, some batches of 200 emails take longer than 60 seconds to complete. Dovecot's HTTP client times out and aborts the transaction, even though Solr successfully completes the processing.
Example from logs:

doveadm(user): Error: fts_solr: Indexing failed: Request timed out (Request queued 61.852 secs ago, 60.144 in http ioloop)
doveadm(user): Error: Mailbox [name]: Transaction commit failed: FTS transaction commit failed

Solr logs show a batch completed in 74 seconds, but Dovecot gave up at 60 seconds.
Requested Fix:
Make the HTTP timeout dynamic based on mailbox size (similar to how SOLR_HEAP was made dynamic in 9.1.6).

Suggested implementation:

• Environment variable: DOVECOT_FTS_TIMEOUT (default: 60, recommended: 180-300 for large mailboxes)
• Or alternatively: DOVECOT_FTS_BATCH_SIZE to reduce batch size from 200 to 50 which avoids calculating mailbox sizes.

Workaround:
None available - /app/code/dovecot-config/dovecot-fts.conf is read-only and resets on container updates.
Related:
This issue was discovered after the SOLR_HEAP fix in version 9.1.6 resolved OOM crashes. The timeout issue was previously masked by the memory errors.

cloudron-support --troubleshoot
Vendor: QEMU Product: Standard PC (i440FX + PIIX, 1996)
Linux: 5.15.0-176-generic
Ubuntu: jammy 22.04
Execution environment: kvm
Processor: AMD EPYC Processor (with IBPB) x 10
RAM: 61714212KB
Disk: /dev/sda3       1.2T
[OK]	node version is correct
[OK]	IPv6 is enabled and public IPv6 address is working
[OK]	docker is running
[OK]	docker version is correct
[OK]	MySQL is running
[OK]	netplan is good
[OK]	DNS is resolving via systemd-resolved
[OK]	unbound is running
[OK]	nginx is running
[OK]	dashboard cert is valid
[OK]	dashboard is reachable via loopback
[OK]	No pending database migrations
[OK]	Service 'mysql' is running and healthy
[OK]	Service 'postgresql' is running and healthy
[OK]	Service 'mongodb' is running and healthy
[OK]	Service 'mail' is running and healthy
[OK]	Service 'graphite' is running and healthy
[OK]	Service 'sftp' is running and healthy
[OK]	box v9.1.6 is running
[OK]	Dashboard is reachable via domain name
[OK]	Domain xxxxxx is valid and has not expired

Just a quick note here. Route53 supports TLSA records now.

I would be interested in Storj as a backup options, been having the usual ongoing issues with Backblaze and I have never got a full NextCloud backup to run. It dies just after halfway. Which is critical for me to back up. Currently running VM snapshots as a backup method which is very sub-optimal.

Not sure if this is the place to ask but when is 9.2 due for release? I am waiting for a fix to Full-text search, I was told it should go into 9.2?

I went with Wasabi in the end.

@jdaviescoates Yes I did some digging and found the root cause at least for me. My users have some huge mailboxes. I am just waiting for the fix so I can enable the feature again.

Issue:
The mail container's Solr instance is repeatedly crashing with OOM
errors when attempting to index a large mailbox (hundreds of thousands
of emails). The JVM is hardcoded to 512MB heap and crashes during XML
parsing with OutOfMemoryError: Java heap space.
Evidence:

Continuous SIGABRT crashes every 1-2 seconds visible in supervisord logs
JVM crash dumps showing OOM during XMLLoader.readDoc
I can supply crash logs from /run/solr/logs/ on request

Root Cause:
The 512MB heap limit is insufficient for large mailboxes. Solr's
bin/solr script supports SOLR_HEAP and SOLR_JAVA_MEM environment
variables, but there's no way to configure this in Cloudron.
Request:
Please add configuration support for SOLR_HEAP (either via UI setting or
/app/data/solr/config.ini) so it can be increased to 2048m for
production mailboxes.