Reading a cloudron mailbox using GMail

[wu-lee]

We're using Cloudron to replace some of our company's pre-existing services, notably email. (And of course to add new ones.)

To date, mail for the company's domain has just been routed to a single GMail account, to which everyone who needs access shares log-in details for.

With Cloudron we have the ability to create personal and shared mailboxes. However, for the time being we want to keep the official company email on the GMail account, simply because everyone's used to that and we don't want to disrupt our main communication channel. Later when Cloudron has proved itself, we might persuade people to shift.

However, in setting this up we have encountered more obstacles than I expected.

The top-level question I have is, how have other people solved this?

Following are some of the angles we've looked at. I think my favourite would be the second one, if it were possible.

1. Configuring personal emails via an MX on a subdomain, leave the top-level domain as-is

Currently we have an MX record which points to the domain registrar's MTA, which then forwards everything to the one GMail account.

So the company domain would normally be configured by Cloudron to have itself as the primary MX server, if it is set to "Automatic" mode. This clobbers the MX record and the redirect that implements, so we're deferring that and keeping it on "Manual" until we resolve our problem.

One workaround might be to have Cloudron manage only the MX on a subdomain. I think I see how to do that using the "Change Dashboard Domain" panel, but it means our new personal emails have to use that subdomain, and I'd rather avoid that if possible.

2. Add a Cloudron shared IMAP inbox for the company email as an external GMail account

Another angle we've tried is to add a catch-all shared Cloudron mailbox to GMail as an external account. GMail used to support this, I think. Now I find it only supports external accounts which are hosted on:

1. POP3 servers, or
2. Special IMAP services which are supported by their "GMailify" service.

Arbitrary IMAP servers like Cloudron servers are apparently not supported by "GMailify" (see link below); and Cloudron does not support a POP3 service, which is the only other option GMail offers.

https://support.google.com/mail/thread/55959852?hl=en
https://forum.cloudron.io/post/13096

3. Forwarding the company email address to the GMail account

Maybe we could create a Cloudron mailing list and use that to forward email sent to the company address on to the GMail account's address...

However, I'm a bit concerned that GMail will then see all the spam and then blacklist our Cloudron server. I've experienced this previously in other cases, where a domain's email was naiively forwarded to a GMail mailbox. It seems a bit risky, even if Cloudron does set up the SPF, DMARC and DKIM, because mail to our main email address is then at the mercy of Google's algorithms.

Google's support document on this does not really reassure me:

https://support.google.com/mail/answer/175365?hl=en

4. Set Google as the MX for our domain

There is the option to defer outgoing delivery services to Google. I've not yet seen one to allow Google to set as the MX for Cloudron's domain. However, I would prefer not to, as part of the point is to move away from Google and to use open-source software. But this would be one of our final options.

[nebulon]

Hi, actually Cloudron does not require any MX to be set, nor does it do that automatically, unless you enable "Incoming Email" for a domain. The default should be off.

This means the Cloudron and the apps will still be able to send out emails.

Maybe I misunderstood your question though, but generally one does not have to move all mailboxes or mail address to a Cloudron at all, this is optional.

[wu-lee]

Thanks @nebulon -

I think if I don't enable "Incoming Email", I can't set up any mailboxes for users, correct?

We'd like our Cloudron users to have email accounts, but keep the one company email working on GMail, as described. This means we can transition gradually to no GMail, or not, based on our experiences.

[girish]

@wu-lee said in Reading a cloudron mailbox using GMail:

We'd like our Cloudron users to have email accounts, but keep the one company email working on GMail, as described. This means we can transition gradually to no GMail, or not, based on our experiences.

Email works at the domain level and not at the mailbox level. Meaning, it's not possible to say "xx@domain.com" goes to gmail but "yy@domain.com" goes to Cloudron. So, from what I understand of your question, it's not possible to have some mailboxes on gmail and some on Cloudron. It's all or nothing.

[robi]

1. is not possible

2. is possible, if you can get GMail to "check a Cloudron mailbox (shared or otherwise)"

3. is possible, but at that point you may as well create a new shared mailbox on Cloudron that everyone checks (and if you wait for the next release, you get multiple users having access to a mailbox w/o checking a separate account.)

4. I think you answered this one yourself.

[wu-lee]

@robi said in Reading a cloudron mailbox using GMail:

1. is not possible

I believe it is possible to set an MX for a subdomain separately from the top-level domain.

However, I don't really want a Cloudron email on a separate subdomain. So no, this doesn't work for me.

2. is possible, if you can get GMail to "check a Cloudron mailbox (shared or otherwise)"

Apparently GMail won't do this unless it is POP3. Or supported "Gmailify", which Cloudron isn't. This was my go-to option, I was surprised that it wasn't possible.

3. is possible, but at that point you may as well create a new shared mailbox on Cloudron that everyone checks (and if you wait for the next release, you get multiple users having access to a mailbox w/o checking a separate account.)

Yes, this seems the best remaining option, but I think I am risking having my own MX server blacklisted as a source of spam by GMail.

4. I think you answered this one yourself.

Are there any others?

[robi]

Looking for something else, I came across this.

msbt Oct 19, 2018, 4:14 PM

A workaround to have a unified inbox is the mail app in nextcloud. You can add multiple imap accounts that can use a single inbox to send and receive mails.

[wu-lee]

So digging around further on Option 3, the problem with forwarding to GMail is that with simple email forwarding, they spot that the SPF header (if it is present for the original sender) doesn't permit delivery from your own domain (from which the email was re-delivered). This then jacks up the email's spam score. Or possibly has it rejected outright.

There is a technique called Sender Rewriting Scheme (SRS), which rewrites the email headers to keep the SPF (and DMARK/DKIM?) headers consistent with the source: your own domain. These seems to be the recommended way to avoid being rejected. For example:

https://superuser.com/questions/1192322/mail-forwarding-do-i-need-to-concern-myself-with-the-spf-fail-header-from-googl

https://www.jwz.org/blog/2015/03/google-seems-to-have-broken-email-forwarding/

Happily the Cloudron docs say this is implemented for mailing groups and sieve filters:

https://docs.cloudron.io/email/#mailing-group
https://docs.cloudron.io/email/#forward-all-emails

However, the problem then: if the forwarded email stream contains spam, the source is now interpreted as your own domain and you may find GMail adds your domain to a DNS blacklist for that (and then your spam score everywhere gets jacked up).

Therefore, to avoid this the server implementing SRS (Cloudron) needs to do decent spam-filtering before forwarding.

I see that Cloudron can do spam filtering - but either it requires training by marking the spam as junk (not possible in this case), or manual configuration. The documentation doesn't mention DNSBLs, although perhaps that can be configured manually via SpamAssassin rules (although I'm yet to discover how).

So here is a more specific question: can anyone give me a pointer for how would I implement spam filtering adequately on a forwarded email address to avoid becoming blacklisted by GMail?

[robi]

You can do a few forms of whitelisting in Gmail.

[wu-lee]

That's useful, thank you @robi. I didn't yet find an option to do that in GMail which seemed guaranteed to work.

[wu-lee]

Actually, on a second look: using those methods, it seems like I'd either have to whitelist sender addresses on a case-by-case basis (which seems infeasible to do reliably for a public contact address), or whitelist everything, which would then disable Google's spam filtering.

Whereas I think what I need to do is to whitelist our MX server from being DNS-blacklisted, whilst still allowing Google to do spam filtering on individual emails from that server, based on their sender etc. I wonder if that's possible.

[rmdes]

I have been forwarding emails to Gmail without any issue, so I'm not sure what's the challenge here...simply added a filter in roundcube to forward all incoming emails (cloudron hosted email/domain previously a google suite domain/email) to a gmail, I can see in the logs that most emails get forwarded just fine and often cloudron spot spam and don't forward them, am I missing something ?

[wu-lee]

@rmdes

Basically, if SRS is being done (which I think it is for Cloudron itself, not sure about Roundcube), you're over the first hurdle: GMail won't reject the forwarded mail as having invalid SPF/DMARC headers.

But there's another problem which might occur, just not predictably: GMail could decide to add your Cloudron server to one of its blacklists (see reasons above). Or worse, a public DNSBL. Then you've then got the unpleasant job of reassuring your users it's gonna be fixed soon, whilst you try and convince GMail etc. to un-blacklist your mail server. And also find an alternative to forwarding.

I know this can happen because something similar actually happened to me some time ago in a different circumstance. Mail forwarding from a domain's addresses to GMail was working for a while, then suddenly it was being rejected.

If this second problem hasn't happened to you, or it could just be that you don't get a lot of spam. But how can you be sure it won't happen at some point later?

(I assume you've read the links in post 5067 above, to conversations about this elsewhere.)

[girish]

@wu-lee said in Reading a cloudron mailbox using GMail:

Basically, if SRS is being done (which I think it is for Cloudron itself, not sure about Roundcube), you're over the first hurdle: GMail won't reject the forwarded mail as having invalid SPF/DMARC headers.

Yes, Cloudron does SRS, by default. You are also right that this is done at the mail server level (and not specific to an app like roundcube/rainloop).

For the forwarding, are you thinking about the case where Cloudron gets lots of spam and forwarding spam to gmail will cause issues? I think that's a valid concern (I remember they have some article saying you have to filter spam before forwarding). I guess this is a feature we have to implement in Cloudron.

[wu-lee]

@girish said in Reading a cloudron mailbox using GMail:

are you thinking about the case where Cloudron gets lots of spam and forwarding spam to gmail will cause issues?

Yes, exactly that.

[girish]

@wu-lee Yes, we have to implement outbound spam filtering for this. We only do inbound spam filtering right now. Feel free to open a feature request in the forum section.

[jdaviescoates]

This seems relevant here

https://twitter.com/sneakdotberlin/status/1317734739537653760?s=20

Gmail now rewrite links even in email pulled in via IMAP

[robi]

I use a browser extension that removes all those as well as utm link data.

[eddowding]

added a Feature Request for #2 -- please upvote https://forum.cloudron.io/topic/5027/pop3-gmail-polling-support