Email deliverability to Microsoft email servers

[jdaviescoates]

Every now and again I have this issue too and it's always with M$ owned domains (outlook, hotmail, live).

In the past I've tried jumping through the various hoops at places like https://sendersupport.olc.protection.outlook.com/pm/troubleshooting.aspx generally all to no avail.

Most recently I've received emails bounces that look like this:

Intended Recipients: <>
somone@hotmail.com Failure Reason: Error: 550 5.7.1 Unfortunately, messages from [89.58.52.168] weren't sent. Please contact your Internet service provider since part of their network is on our block list (S3150). You can also refer your provider to . [DB3EUR04FT039.eop-eur04.prod.protection.outlook.com 2023-03-21T21:59:11.453Z 08DB29C1C7D2FFBB]
http://mail.live.com/mail/troubleshooting.aspx#errors

The only thing that has ever worked for me (that is, until it happens again) is asking my VPS provider (in this case it was Netcup) to resolve the issue. They then ask some standard questions like this:

I will be happy to pass on your request to the relevant department in order to apply for delisting from the relevant provider. But first we need from you:

• your explicit confirmation that you do not send mass mails (this includes newsletters)
• a list of the sender domains

Please also check that the mail server settings are correct and the DNS records are set correctly. Otherwise, our efforts may be protracted or lead to a negative result.

And I then reply with my answers and then they contact M$ and M$ sorts it out. Until it happens again.

[marcusquinn]

@jdaviescoates Yeah, that's the issue! I really want a permanent solution. Still researching.

In the meantime, have hooked up https://warmupinbox.com. So far it's showing 100% delivery and no spam, so it is all a mystery!

[jdaviescoates]

@marcusquinn said in Email deliverability to Microsoft email servers:

a permanent solution.

I think perhaps only a reputable relay like Postmark would do the job (and even then, perhaps not)

Either that or somehow obtaining a very large range of IP addresses which you can somehow ensure and totally clean and reputable.

I think the issue (as alluded to in the bounce email example I pasted above) is that "part of their network is on our block list" - and it's basically impossible to control the whole VPS provider' network - especially on the good value ones, someone somewhere is going to be spamming.

So not sure there is really a good permanent self hosting solution. Unless maybe people sell guaranteed clean IP ranges and you buy those and also colocate you're own physical machine in a highly reputable data centre or something

[marcusquinn]

@jdaviescoates Yeah. Frustrating AF! Why can't we just have nice things and manage our own small corners of the internet any more?!?

[jdaviescoates]

I was just reading these:

https://postmarkapp.com/guides/how-to-improve-domain-reputation-for-better-email-deliverability
https://postmarkapp.com/dedicated-ips
https://postmarkapp.com/guides/dedicated-vs-shared-ips-for-email-when-to-use-each#what-to-know-about-shared-ip-addresses

It sounds like a "pristine shared IP pool will have the best results". But how to achieve/ do that when self-hosting and not using a relay?

[marcusquinn]

@jdaviescoates For now, for this client, I'm just moving everything to Namecheap's Private Email Ultimate package. Will see how that does, build the reputation for the domain and emails with their premium servers and IP monitoring, and maybe migrate again later once I'm more confident that I can pivot around more easily.

Not ideal, but still better value and independence from Google / Microsoft. Going to keep researching this, so will share any findings here. Thanks all for your contributions, I love having a team of virtual system admins to bounce off, wouldn't get this much collaboration in many enterprises

[marcusquinn]

@jdaviescoates A relay option I've been happy in the past with is Elastic Email. They do also offer dedicated IP addresses, but as others here say, you can get caught out with being a part of an IP range being relegated.

This is a very valuable area for business privacy, so definitely going to keep researching and testing solutions.

[marcusquinn]

@jdaviescoates Interesting. Perhaps the issue is so few emails being sent from these Cloudron VPS dedicated IP addresses!

[marcusquinn]

@here I've not used mail relays with a Cloudron email service setup before (only used them for email newsletter sending).

Anyone know if the sent mail is still stored on the IMAP server before it's sent to the relay to deliver?

[marcusquinn]

Yeah, email now sending with no Microsoft bounce-backs, using Namecheap, so at least there's one reasonable option for others having issues to consider as an alternative to using Cloudron email services.

I do think I'll move this one back at some point, but going to test email relay on another domain, first.

It might just be that having a dedicated IP and low email volumes is just not going to achieve the same email deliverability as these services with shared IPs, as long as they are managing their IP reputation across all clients.

Let's see. Keep sharing the knowledge, we move a lot faster with many more R&D testing opportunities as a community!

[marcusquinn]

@marcusquinn said in Email deliverability to Microsoft email servers:

Anyone know if the sent mail is still stored on the IMAP server before it's sent to the relay to deliver?

OK, thinking this through Sent email is being stored from the email client to the server using IMAP, and SMTP to Cloudron credentials then either sends it directly from the server IP, or relays, to whichever relay is setup.

I think the knowledge base here could do with something to explain that Sent emails will still be stored before relaying, here:

• https://docs.cloudron.io/email/#relay-outbound-mails

From what the relay options being compared, it seems that Elastic Email might remain my preferred for bang for bucks. It's pretty difficult to maintain decent online review ratings in an angry world, but these seem to be reassuring:

• https://uk.trustpilot.com/review/elasticemail.com

[marcusquinn]

Found this very useful!

• https://www.mail-tester.com/

@girish @nebulon Recommend adding this mail-tester.com link to the support documents, and any email canned responses on deliverability, should help a lot for self-diagnosis, and I've now got 10/10 deliverability scores on all my Cloudrons from the couple of things I spotted from using this.

I still think more obscure domain extensions are going to take a while from new to be trusted.

I also found this free mail warmup tool, that will send and check 5 emails a day for deliverability, so it's worth anyone having issues with that to setup a separate email box on their Cloudron, just for having this continuous sending flow from their domain to increase the trust ranking of their server's dedicated IP address.

• https://www.mailflow.io/

For higher volume sending, this one is the cheapest of the bunch that does the job for me:

• https://app.warmupinbox.com

Right, think that's a long thread and research but with some useful conclusions for me, and to share. My work here is done for the day (4:30am), the life of system admin, eh!

[necrevistonnezr]

I worked at a company before who offered transactional email services for business customers (we‘re talking like 1-2 million mails per day, e.g. banks that send out notifications or password reset links etc.). Believe me, email reputation was a 24/7 job.
I would, therefore, rather rely on a pro for relaying email and keeping reputation.
And I don’t blame it so much on Microsoft/Google but all these little a**holes who sign up a VPS and start sending their shit. You just can’t have nice things anymore….

[marcusquinn]

@necrevistonnezr Yeah, I learned a few things from the research, so it was a bit of commentary on that to see what other experience and successes we might find.

I think I misunderstood how the relaying was implemented, now I know that, and a couple of other things, much happier that there are solutions for all occasions. Hopefully some tips to hep others here, and additions to the documentation should help, too.

[michaelpope]

@marcusquinn said in Email deliverability to Microsoft email servers:

Anyone know if the sent mail is still stored on the IMAP server before it's sent to the relay to deliver?

This depends on your email client settings actually. Basically, most email clients save a copy of the email you are sending in the 'Sent' folder, but they can be configured not to. The saved copy is uploaded to your actual 'mailbox' via IMAP - but this is not equivalent to sending an email, this is just saving a copy that doesn't leave your mailbox. IMAP is basically what allows you to organize and upload and download to that mailbox - and it doesn't have anything to do with actually sending of the email.

Actual outgoing email goes through a different protocol called SMTP - and it doesn't touch your actual 'mailbox' at all except for when email is delivered to you. SMTP is used by your email client to send your email to your own email server, and then SMTP is used again from your server to the receiver's email server (and sometimes multiple chains if it's configured that way). So it's basically just a protocol for sending email.

Anyways, best of luck with all of this . Sorry it's being so painful to get it working.

[marcusquinn]

@michaelpope Yeah, one of those things I should have known, but have used relays in every other context but this over the years, so just was leaning on the experience here for a sanity check

Now I'm sure, it actually makes far more sense to just use for all servers. I'd also fallen into the thinking that my dedicated IPs were better for controlling deliverability, but had never thought that lower volumes might be a negative factor.

I'm good with pain, it's just the price of filling knowledge gaps, and it it's not obvious or common knowledge, it's probably more valuable experience, too.

Think I have a happy solution now, either using Elastic Mail for pure relaying, or Namecheap or any other mass-market provider SMTP relay for outbound.

Still not ideal for 100% privacy, but you'd hope most are good to their word of not storing data beyond retention periods for logs, and the inbound for things like 2FA can still remain as private as the Cloudron instance can be kept.

Maybe there's an add-on service idea for Cloudron in also offering its own relay.

Ultimately, with non-E2EE, you have to trust some man in the middle, andI feel we have a good measure of trust ethics here.

[crazybrad]

FWIW, I have found the best email delivery using Postmark, a REASONABLE, paid service provider. We have used Mailgun (shared IP) and periodically needed to switch IP addresses once the IP address got "poisoned" by someone else. Dedicated IP works, but is very expensive. If you do not send a large volume of emails, it may be difficult to earn a good reputation.

For all those reasons, we have moved most of our outbound email to Postmark. Deliverability is excellent. You can (and should) establish individual "servers" for different domains/email streams (marketing vs. transactional).
We get confirmation that emails were delivered to the recipient's mail server. That's how we found out that some of our emails were being "blackholed" by a large inbox provider (guess who?)

For an expenditure of $US20/month we can send 10K/month and have delivery records for 90 days. Totally worth it.

[jdaviescoates]

@crazybrad said in Email deliverability to Microsoft email servers:

send 10K/month

Yeah, if I need to send large volumes of mail I'd certainly give Postmark a try given how many people here seem to be recommending it.

[crazybrad]

@jdaviescoates I send nowhere near that limit. I use it for business and personal email accounts because of the deliverability issue. The monthly cost is trivial compared to the time wasted trying to troubleshoot issues.

[foliovision]

Deliverability to Microsoft and Google are a real challenge, that's for sure. Full DMARC with discard/delete options does help.

If we could just override the default Cloudron email configuration on a per app basis, it would be much easier to deal with

We're happy using our Cloudron Digital Ocean hosted droplet foliovision.net SMTP to deliver internal emails but can't use it for client emails (not reliable enough). Unfortunately Cloudron is missing a per application option not to hijack the app's email settings. This oversight caused us serious issues with InvoiceNinja where we had endless trouble with invoice deliverability due to the SMTP settings being silently erased on every restart.