Port 25 inbound connection timeout and missing email

[james]

Hello @p44

I would assume this is not only a CPU/RAM issue but also an IOPS issue.
A very high system load can also occur without the CPU and RAM being used too much.
In this case it is almost always the IOPS. So the system storage can't keep up.
Depending on your server setup and provider this might be worth looking into if your server is using HDD, SSD, NVMe drives or a shared storage from the provider.
We had multiple reports of vServers showing this load / IOPS behaviour when there were underlying issues with the whole provider node.
One time it was a netcup node failing RAID system that caused all vServers on that node to have massive IOPS and load issues.
Which was also first denied by the first level support and only after providing detailed performance measurements they acknowledged that something must be wrong with their storage raid.

Port 25 not being inaccessible could also hint towards the mail service crashing or running out of memory.
Please check your mail service in Cloudron if you might need to increase the RAM and also check the logs of the mail service for crash reports.

[p44]

@james Thanks a lot, I didn’t have any crash of Mail service and also RAM is ok. I’ll take a proper look to IOPS. Thanks for your advices.

[p44]

Hi @james I’m still managing this problem. There is a way to manage Haraka timeout to avoid this Connection timed out issue, eg. making longer timeout?

[james]

Hello @p44
Have you looked into the the IOPS stats?

[p44]

@james Thanks a lot for your question.

IOPS it seems to be good...

Block Size | 4k            (IOPS) | 64k           (IOPS)
  ------   | ---            ----  | ----           ---- 
Read       | 510.82 MB/s (127.7k) | 870.43 MB/s  (13.6k)
Write      | 512.17 MB/s (128.0k) | 875.01 MB/s  (13.6k)
Total      | 1.02 GB/s   (255.7k) | 1.74 GB/s    (27.2k)
           |                      |                     
Block Size | 512k          (IOPS) | 1m            (IOPS)
  ------   | ---            ----  | ----           ---- 
Read       | 595.93 MB/s   (1.1k) | 1.49 GB/s     (1.4k)
Write      | 627.60 MB/s   (1.2k) | 1.59 GB/s     (1.5k)
Total      | 1.22 GB/s     (2.3k) | 3.09 GB/s     (3.0k)

[james]

Hello @p44
A momentary view of the IOPS might not be good enough to evaluate this.
Since you mentioned when the system is under high load the issue is present, you should observe / log the IOPS while the system is under load.

[p44]

Ok @james. I’ll do more accurate research

[p44]

Thanks again @james for your cooperation.

I can confirm that after removing custom Spamassassin rules from Email > Settings > Spam Filtering > Custom Spamassassin rules problem has been solved.

However, further testing would be needed to determine which rules interacted negatively with emails, potentially causing false positives.

[james]

Hello @p44
That is an interesting find.
Could save the logs of the mail service when it was failing somewhere so we can request it later from you?

[p44]

@james Yes, this is a great find. Thanks a lot for your cooperation.

In email service’s logs no any log...

Whenever you want, I can reproduce issue and I can send you raw log.

Keep me posted.