Too many concurrent SMTP connections
-
Hi everyone,
I am using the Cloudron built-in SMTP server for the domain Mattermost sends email from and I just noticed in my logs that when sending a notification email to many users at once (e.g. resulting from an
@all) , lots of them fail with an error like the following:{"level":"error","ts":1614543114.932735,"caller":"app/notification_email.go:105","msg":"Error while sending the email","user_email":"[DELETED]","error":"SendMail: Failed to open TLS connection., 421 Too many concurrent connections"}It looks like Mattermost tries to send all the emails at once and there is no retry logic, so the result of this is that only a fraction of the users will receive the notification emails that are expected. There is an upstream bug to add a configurable maximum number of connections that has been open since 2016.
I guess when using an external mail service, the large number of concurrent connections may not be a problem. For example, Sendgrid states that up to 10,000 concurrent connections from a single server are permitted.
However if possible I would like to stick with the built-in SMTP server so that it is not necessary to pass this data through a third party. In my case there are around 60 people in the Mattermost team, so that should be the theoretical maximum number of emails being sent simultaneously.
I am wondering if it would be practical to bump up the number of simultaneous connections allowed by the built-in SMTP server to 60 without causing problems? Or if anyone has any other ideas for a way around this problem?
Tom
-
Hi everyone,
I am using the Cloudron built-in SMTP server for the domain Mattermost sends email from and I just noticed in my logs that when sending a notification email to many users at once (e.g. resulting from an
@all) , lots of them fail with an error like the following:{"level":"error","ts":1614543114.932735,"caller":"app/notification_email.go:105","msg":"Error while sending the email","user_email":"[DELETED]","error":"SendMail: Failed to open TLS connection., 421 Too many concurrent connections"}It looks like Mattermost tries to send all the emails at once and there is no retry logic, so the result of this is that only a fraction of the users will receive the notification emails that are expected. There is an upstream bug to add a configurable maximum number of connections that has been open since 2016.
I guess when using an external mail service, the large number of concurrent connections may not be a problem. For example, Sendgrid states that up to 10,000 concurrent connections from a single server are permitted.
However if possible I would like to stick with the built-in SMTP server so that it is not necessary to pass this data through a third party. In my case there are around 60 people in the Mattermost team, so that should be the theoretical maximum number of emails being sent simultaneously.
I am wondering if it would be practical to bump up the number of simultaneous connections allowed by the built-in SMTP server to 60 without causing problems? Or if anyone has any other ideas for a way around this problem?
Tom
@tomw the concurrency is set quite low at the moment - just 5. There's no issue bumping this value, it was just a very conservative setting.
As a temporary fix:
docker exec -ti mail /bin/bash- Edit the file
/run/haraka/config/limit.ini. There is aconcurrencysection under whichmaxis set to 5. Make is say 200. supervisorctl restart haraka
Does that make things work? The configuration does not survive server restarts, but I will bump it in the next release.
-
@tomw the concurrency is set quite low at the moment - just 5. There's no issue bumping this value, it was just a very conservative setting.
As a temporary fix:
docker exec -ti mail /bin/bash- Edit the file
/run/haraka/config/limit.ini. There is aconcurrencysection under whichmaxis set to 5. Make is say 200. supervisorctl restart haraka
Does that make things work? The configuration does not survive server restarts, but I will bump it in the next release.
-
@girish said in Too many concurrent SMTP connections:
Does that make things work?
Brilliant, thanks @girish! I have made the change and will report back next time someone sends an
@allmessage. -
Update: We have successfully triggered emails with an
@allto 46 people without errors, so the fix seems to have worked.
Hello! It looks like you're interested in this conversation, but you don't have an account yet.
Getting fed up of having to scroll through the same posts each visit? When you register for an account, you'll always come back to exactly where you were before, and choose to be notified of new replies (either via email, or push notification). You'll also be able to save bookmarks and upvote posts to show your appreciation to other community members.
With your input, this post could be even better 💗
Register Login