Secondary mail sending service

[crazybrad]

@NCKNE Sorry to share this bad news but we are seeing the same (delayed delivery) behavior using another (not Cloudron, not self-hosted) SMTP service (Postmark). So far it just seems to be just Yahoo and AOL for us. But I'm not sure this idea would work given how inbox providers are treating email even from large SMTP services like Cloudron. These emails are categorized as "broadcast" messages based on the "same message to many" criteria, but are informing customers of important changes. We are not marketing anything at all.

[NCKNE]

We had some good experiences with self hosted mail servers and using SMTP relays (CSA listed). I just don’t want to send all mail messages through an SMTP relay (costs, logs not on Cloudron, etc.), just the ones that stay in the queue for too long. Shouldn’t be too difficult to implement, but I don’t know if there is a need for it for others. We‘ve had all messages delivered so far (even to Gmail, etc.), but I am annoyed by the delays because of 4xx rejections from the large providers.

[d19dotca]

It would be nice to have a failover, for sure. Every now and then (maybe once a year) Microsoft mail services will reject all mail from my server which means I have to file a blocklist removal each time, and immediately (and manually) intervene to use Mailgun or some other SMTP service so my users can still send mail without rejection. I could just continue to use those SMTP services but I like to keep everything localized where possible, and the inconvenience of it is only brief but can still be an issue (what if I’m on vacation for example). Having a backup that’s automatically used when it gets certain error codes back would be nice for sure.

[crazybrad]

@d19dotca Interesting. Do you think this might happen even if you were using Mailgun 100% of the time? Did one of your users misbehave and that's why they blocked your IP? I was operating under the assumption that "pay to play" with a large SMTP service would avoid all this hassle. And to avoid unhappy users I was willing to incur the added cost. But perhaps my logic is flawed. Under those circumstances it might be wise to have multiple SMTP paths, with a simple dropdown to select {Cloudron Native, Mailgun, Postmark, etc.} Then we can all go on vacation!

[d19dotca]

@crazybrad - Microsoft was never able to give me an answer. I think it was more likely that the IP range my server was stuck in was blocked rather than my IP specifically. This is because when I’d apply for an IP removal it would immediately email me back saying it wasn’t blocked at all, and then I’d have to respond again to get human support and it’d take them once or twice to have it removed. I host with OVH which as much as they’re one of the better server hosts in Canada IMO, they don’t have the best reputation for using clean IP addresses. That’s my best guess to why it happens at least.

I’d love to have the ability to have two SMTP servers set, one for primary and one as a failover for whenever there are certain status codes resolved back. It would make automating that much easier when issues occur.

[NCKNE]

Thanks guys, thant‘s exactly the use case I was thinking about. The Hakara MTA used by cloudron seems to be very flexible in regards to the outbound configuration (https://haraka.github.io/core/Outbound). While I am no expert on this subject, it might not be too difficult to implement a failback using an SMTP relay.

[guyds]

I completely agree with this, but the thing is that even if your mails get delivered to those mail providers they might (and probably will) still end up in the junk folder of the recipient(s). Yes, I'm looking at you Microsoft.

So while I agree that a secondary / backup relay would be helpful in case of temporary delivery errors, it won't solve the problem of getting your emails in the recipients inbox in all cases.

Test case:
I did some extensive testing by sending a correctly formatted test mail from a Cloudron server that's being used for +3 years for sending out mails from several clients' addresses without any issues.
The test mail was correctly delivered to the inbox of a MS 365 account, Google Workspace account, GMail account and several other accounts, but was consistently marked as junk in hotmail, live and outlook.com accounts.
The ip address of the sending mailserver isn't on any blacklist, should have a good reputation in the meantime and mail-tester gave a 10/10 for the content of the test mail.
But even when I used a relay server (smtp2go and mailjet) the result was exactly the same.
So as a final test I decided to send the test mail from my MS 365 account and even then it was marked as spam in hotmail, live and outlook.com addresses!
In other words, they don't even trust their own mail servers.

Just as an FYI...

[crazybrad]

@guyds Thank you for posting your test results. I am wondering if email addresses from certain domains should be "blocked" by software developers for account registration. If a user tried to sign up using {hotmail, live, outlook, Yahoo? AOL? etc?} then registration is rejected with a message: "due to unreliable delivery practices at these domains, we can not accept your registration since we can not properly service you. Please choose another inbox provider and we will gladly accept your registraion"

[humptydumpty]

@crazybrad that would be awesome but your average person is getting thrown under the bus if actually implemented. Most people don't know which email provider to sign up with, whom to trust, how to self-host, set up a relay, and whatnot. But yeah, we need to fight back by educating and raising awareness. So far, after years of preaching about online privacy, I can barely get my own family to abide by basic online security rules.

[crazybrad]

@humptydumpty Nothing more difficult than getting family to accept your help and acknowledge that perhaps you know something that they don't:) To your point, perhaps during signup a warning about mail delivery to these inbox providers and a suggestion to use another email address. Caveat emptor.