SOLVED Too many concurrent SMTP connections

[tomw]

Hi everyone,

I am using the Cloudron built-in SMTP server for the domain Mattermost sends email from and I just noticed in my logs that when sending a notification email to many users at once (e.g. resulting from an @all) , lots of them fail with an error like the following:

{"level":"error","ts":1614543114.932735,"caller":"app/notification_email.go:105","msg":"Error while sending the email","user_email":"[DELETED]","error":"SendMail: Failed to open TLS connection., 421 Too many concurrent connections"}

It looks like Mattermost tries to send all the emails at once and there is no retry logic, so the result of this is that only a fraction of the users will receive the notification emails that are expected. There is an upstream bug to add a configurable maximum number of connections that has been open since 2016.

I guess when using an external mail service, the large number of concurrent connections may not be a problem. For example, Sendgrid states that up to 10,000 concurrent connections from a single server are permitted.

However if possible I would like to stick with the built-in SMTP server so that it is not necessary to pass this data through a third party. In my case there are around 60 people in the Mattermost team, so that should be the theoretical maximum number of emails being sent simultaneously.

I am wondering if it would be practical to bump up the number of simultaneous connections allowed by the built-in SMTP server to 60 without causing problems? Or if anyone has any other ideas for a way around this problem?

Tom

[girish]

@tomw the concurrency is set quite low at the moment - just 5. There's no issue bumping this value, it was just a very conservative setting.

As a temporary fix:

• docker exec -ti mail /bin/bash
• Edit the file /run/haraka/config/limit.ini. There is a concurrency section under which max is set to 5. Make is say 200.
• supervisorctl restart haraka

Does that make things work? The configuration does not survive server restarts, but I will bump it in the next release.

[tomw]

@girish said in Too many concurrent SMTP connections:

Does that make things work?

Brilliant, thanks @girish! I have made the change and will report back next time someone sends an @all message.

[tomw]

Update: We have successfully triggered emails with an @all to 46 people without errors, so the fix seems to have worked.

[girish]

@tomw thanks for the update! I will make the default value higher in the next release.