The SNDS page/service is useless. It's good to tell if you're blocked or not and that's it. I signed up and added my server IP +8 months ago and I have yet to receive a single report from them. Anyway, I found the right page to apply for delisting if your IP is blocked: https://sender.office.com.

@girish said in Anyone else see many connections denied due to "Mail from domain <domain> is not allowed from your host" repeatedly from spammy IPs?:

A bit of a wild guess: mail from is usually <> for bounce mail. So, this seems like some poor denial of service or maybe those IPs know that some mail software misbehaves with such carefully crafted mail.

Ah very interesting, I appreciate that insight. It was definitely strange when I saw it happening - so many requests at once. I'll keep an eye out for it. Sounds like it's all good then as far as Cloudron is concerned. 🙂 Thanks Girish.

Latest round of SpamAssassin rules I'm using, if anyone is interested.

The highlights here are just a couple of things:

A few new sources which come with new rules Slight scoring tweaks on just a few rules

Of course, as they say... YMMV. 😉

# scoring DNSBLs (blocklists & allowlists) score RCVD_IN_BL_SPAMCOP_NET 2.5 score RCVD_IN_DNSWL_HI -5.0 score RCVD_IN_DNSWL_LOW -0.5 score RCVD_IN_DNSWL_MED -2.5 score RCVD_IN_DNSWL_NONE 0.5 score RCVD_IN_GBUDB 4.5 score RCVD_IN_IADB_DK -0.5 score RCVD_IN_IADB_DOPTIN_GT50 -0.5 score RCVD_IN_IADB_DOPTIN_LT50 -0.5 score RCVD_IN_IADB_EDDB -0.5 score RCVD_IN_IADB_EPIA -0.5 score RCVD_IN_IADB_GOODMAIL -0.5 score RCVD_IN_IADB_LISTED -0.5 score RCVD_IN_IADB_LOOSE -0.5 score RCVD_IN_IADB_MI_CPEAR 0 score RCVD_IN_IADB_MI_CPR_30 0 score RCVD_IN_IADB_MI_CPR_MAT 0.0 score RCVD_IN_IADB_NOCONTROL -0.5 score RCVD_IN_IADB_OOO -0.5 score RCVD_IN_IADB_OPTIN -0.5 score RCVD_IN_IADB_OPTIN_GT50 -0.5 score RCVD_IN_IADB_OPTIN_LT50 -0.5 score RCVD_IN_IADB_OPTOUTONLY -0.5 score RCVD_IN_IADB_RDNS -0.5 score RCVD_IN_IADB_SENDERID -0.5 score RCVD_IN_IADB_SPF -0.5 score RCVD_IN_IADB_UNVERIFIED_1 -0.5 score RCVD_IN_IADB_UNVERIFIED_2 -0.5 score RCVD_IN_IADB_UT_CPEAR 0 score RCVD_IN_IADB_UT_CPR_30 0 score RCVD_IN_IADB_UT_CPR_MAT 0 score RCVD_IN_JMF_BL 2.5 score RCVD_IN_MSPIKE_BL 0.0 score RCVD_IN_MSPIKE_H2 0.0 score RCVD_IN_MSPIKE_H3 -0.5 score RCVD_IN_MSPIKE_H4 -2.0 score RCVD_IN_MSPIKE_H5 -3.0 score RCVD_IN_MSPIKE_L2 1.5 score RCVD_IN_MSPIKE_L3 3.5 score RCVD_IN_MSPIKE_L4 4.5 score RCVD_IN_MSPIKE_L5 5.0 score RCVD_IN_MSPIKE_WL 0.0 score RCVD_IN_MSPIKE_ZBI 4.0 score RCVD_IN_PBL 3.5 score RCVD_IN_SBL 3.5 score RCVD_IN_SBL_CSS 3.5 score RCVD_IN_SEM_BACKSCATTER 1.5 score RCVD_IN_SEM_BLACK 3.5 score RCVD_IN_SEM_NET_BLACK 2.5 score RCVD_IN_SORBS_BLOCK 2.5 score RCVD_IN_SORBS_DUL 2.5 score RCVD_IN_SORBS_HTTP 2.5 score RCVD_IN_SORBS_MISC 2.5 score RCVD_IN_SORBS_SMTP 2.5 score RCVD_IN_SORBS_SOCKS 2.5 score RCVD_IN_SORBS_SPAM 2.5 score RCVD_IN_SORBS_WEB 2.5 score RCVD_IN_SORBS_ZOMBIE 2.5 score RCVD_IN_SPAMRATS 2.5 score RCVD_IN_XBL 3.5 score RCVD_IN_ZEN_BLOCKED 0.0 score RCVD_IN_ZEN_BLOCKED_OPENDNS 0.0 # scoring URIBLs score URIBL_ABUSE_SURBL 4.0 score URIBL_BLACK 4.5 score URIBL_CR_SURBL 4.0 score URIBL_CSS 2.0 score URIBL_CSS_A 2.0 score URIBL_DBL_ABUSE_BOTCC 3.5 score URIBL_DBL_ABUSE_MALW 3.5 score URIBL_DBL_ABUSE_PHISH 3.5 score URIBL_DBL_ABUSE_REDIR 3.5 score URIBL_DBL_ABUSE_SPAM 3.5 score URIBL_DBL_BLOCKED 0.0 score URIBL_DBL_BLOCKED_OPENDNS 0.0 score URIBL_DBL_BOTNETCC 3.5 score URIBL_DBL_ERROR 3.5 score URIBL_DBL_MALWARE 3.5 score URIBL_DBL_PHISH 3.5 score URIBL_DBL_SPAM 3.5 score URIBL_GREY 1.0 score URIBL_MW_SURBL 4.0 score URIBL_PH_SURBL 4.0 score URIBL_RED 1.5 score URIBL_RHS_DOB 2.0 score URIBL_SBL 2.0 score URIBL_SBL_A 2.0 score URIBL_SEM 3.0 score URIBL_SEM_FRESH30 1.5 score URIBL_WS_SURBL 3.0 score URIBL_ZEN_BLOCKED 0.0 score URIBL_ZEN_BLOCKED_OPENDNS 0.0 # scoring DKIM & SPF score DKIM_INVALID 1.5 score DKIM_SIGNED 0.0 score DKIM_VALID 0.0 score DKIM_VALID_AU 0.0 score DKIM_VALID_EF 0.0 score DKIM_VERIFIED 0.0 score DKIMWL_BL 3.0 score DKIMWL_WL_HIGH -3.5 score DKIMWL_WL_MED -1.5 score DKIMWL_WL_MEDHI -2.5 score FORGED_SPF_HELO 3.0 score SPF_FAIL 1.5 score SPF_HELO_FAIL 1.5 score SPF_HELO_NEUTRAL 1.0 score SPF_HELO_NONE 0.5 score SPF_HELO_PASS 0.0 score SPF_HELO_SOFTFAIL 1.5 score SPF_NEUTRAL 0.5 score SPF_NONE 0.5 score SPF_PASS 0.0 score SPF_SOFTFAIL 1.5 # scoring BAYES score BAYES_00 -3.0 score BAYES_05 -1.5 score BAYES_20 0.5 score BAYES_40 1.5 score BAYES_50 2.0 score BAYES_60 3.0 score BAYES_80 4.0 score BAYES_95 4.5 score BAYES_99 5.0 score BAYES_999 1.5 # scoring HTML score HTML_FONT_LOW_CONTRAST 0.5 score HTML_IMAGE_ONLY_04 1.5 score HTML_IMAGE_ONLY_08 2.0 score HTML_IMAGE_ONLY_12 2.0 score HTML_IMAGE_ONLY_16 2.0 score HTML_IMAGE_ONLY_20 2.0 score HTML_IMAGE_ONLY_24 2.5 score HTML_IMAGE_ONLY_28 2.5 score HTML_IMAGE_ONLY_32 3.0 score HTML_IMAGE_RATIO_02 0.0 score HTML_IMAGE_RATIO_04 0.0 score HTML_IMAGE_RATIO_06 0.0 score HTML_IMAGE_RATIO_08 0.0 score HTML_MESSAGE 0.0 # scoring HEADER & MISSING score HEADER_FROM_DIFFERENT_DOMAINS 0.5 score HEADER_SPAM 2.5 score MISSING_DATE 3.0 score MISSING_FROM 1.5 score MISSING_HB_SEP 0.0 score MISSING_HEADERS 1.5 score MISSING_MID 1.0 score MISSING_MIMEOLE 2.0 score MISSING_SUBJECT 2.0 # scoring FREEMAIL score FORGED_GMAIL_RCVD 2.5 score FORGED_YAHOO_RCVD 2.5 score FREEMAIL_ENVFROM_END_DIGIT 0.5 score FREEMAIL_FORGED_REPLYTO 1.5 score FREEMAIL_FROM 0 score FREEMAIL_REPLY 1.0 score FREEMAIL_REPLYTO 1.0 score FREEMAIL_REPLYTO_END_DIGIT 0.5 score MALFORMED_FREEMAIL 4.0 # additional scoring tweaks score BILLION_DOLLARS 2.0 score BODY_URI_ONLY 1.5 score EMPTY_MESSAGE 1.5 score HELO_DYNAMIC_SPLIT_IP 2.0 score HK_RANDOM_ENVFROM 0.5 score HK_RANDOM_FROM 0.5 score LOTS_OF_MONEY 0.5 score MPART_ALT_DIFF 1.0 score MPART_ALT_DIFF_COUNT 1.5 score NO_DNS_FOR_FROM 0.5 score PDS_TONAME_EQ_TOLOCAL 0.5 score PDS_TONAME_EQ_TOLOCAL_VSHORT 0.5 score RDNS_NONE 1.5 score REPLYTO_WITHOUT_TO_CC 2.5 score UNPARSEABLE_RELAY 0.5 score URI_DQ_UNSUB 2.0 # add GDUB TRUNCATE DNSBL header RCVD_IN_GBUDB eval:check_rbl('gbudb', 'truncate.gbudb.net.') describe RCVD_IN_GBUDB Listed in truncate.gbudb.net tflags RCVD_IN_GBUDB net # add JMF-Black DNSBL header RCVD_IN_JMF_BL eval:check_rbl('jmf', 'black.junkemailfilter.com.') describe RCVD_IN_JMF_BL Listed in black.junkemailfilter.com tflags RCVD_IN_JMF_BL net # add Spamrats DNSBL header RCVD_IN_SPAMRATS eval:check_rbl('spamrats', 'all.spamrats.com.') describe RCVD_IN_SPAMRATS Sender listed in all.spamrats.com tflags RCVD_IN_SPAMRATS net # add SpamEatingMonkey backscatter DNSBL header RCVD_IN_SEM_BACKSCATTER eval:check_rbl('sem', 'backscatter.spameatingmonkey.net') tflags RCVD_IN_SEM_BACKSCATTER net describe RCVD_IN_SEM_BACKSCATTER Received from an IP listed by SEM-BACKSCATTER # add SpamEatingMonkey network blacklist DNSBL header RCVD_IN_SEM_NET_BLACK eval:check_rbl('sem', 'netbl.spameatingmonkey.net') tflags RCVD_IN_SEM_NET_BLACK net describe RCVD_IN_SEM_NET_BLACK Received from an IP listed by SpamEatingMonkeys # add SpamEatingMonkey blacklist DNSBL header RCVD_IN_SEM_BLACK eval:check_rbl('sem', 'bl.spameatingmonkey.net') tflags RCVD_IN_SEM_BLACK net describe RCVD_IN_SEM_BLACK Received from an IP listed by SpamEatingMonkeys # add SpamEatingMonkey URIBL urirhssub URIBL_SEM uribl.spameatingmonkey.net. A 2 body URIBL_SEM eval:check_uridnsbl('URIBL_SEM') describe URIBL_SEM Contains a URI listed by SpamEatingMonkeys tflags URIBL_SEM net # add SpamEatingMonkey fresh domain URIBL urirhssub URIBL_SEM_FRESH30 fresh30.spameatingmonkey.net. A 2 body URIBL_SEM_FRESH30 eval:check_uridnsbl('URIBL_SEM_FRESH30') describe URIBL_SEM_FRESH30 From a domain registered less than 30 days ago tflags URIBL_SEM_FRESH30 net

Okay... I may be on the side of this working properly again. lol. Maybe I've been wrong this whole time in thinking it wasn't working correctly.

So coincidentally I was checking the mail server logs and saw another example of the same message go through to the same recipient from the same mail server, it was listed in the logs as "just now" so I quickly checked mxtoolbox and found that only 4 at that time had been listed, none of which were ones I was using.

Here is how it looked at the very moment I checked when it was "just now" in the logs:

69bc5a02-12ca-420e-958a-27405c21f7ed-image.png

07b937c4-4840-4c14-887b-7513acc87251-image.png

Edit: Checking about 6 minutes later, I see the blocklists have aleady been updated for more (Spamhaus Zen in this case would have caught it if it were about 5 minutes earlier):

4522d168-dc21-498f-845b-885cfe0a73a1-image.png

So I guess we can probably mark this as resolved, as I now see conclusive evidence that the various blocklists used just didn't have it listed until a few minutes after the message was received. I guess in order for it to adapt so quickly this spam attack on one of my users from those mail servers must be right at the beginning of a spam wave. Kind of neat actually to see how real-time these lists are. haha.

Nevermind, I did not realize sendgrid setup was only for sending email, I see the other parts to setup for receiving email now. Please disregard.

@girish Will do

I really like the blackllist checking being built-in. Frankly, I'd also be a fan of getting notifications about it. I suppose UX-wise, perhaps this is the appropriate sort of thing to trigger a yellow status on email.

@girish Ok

@marcusquinn Thank's Marcus, I'll follow your advice to check the reputation BEFORE, and if reputation is poor, just delete and purchase new one.

Should be interesting that ISP's do themself this control and tell BEFORE the IP reputation.

Personally, I'm migrating away from GMail, and wouldn't recommend it, although I appreciate it's difficult to cut those ties for some.

Another possible solution: would be to forward to any other email service that does support POP3 but doesn't have spam filtering. That way, you're not sending email to GMail, it's pulling and filtering. Appreciate that a 3 mailserver setup is odd though, but then so is GMail.

I think it's worth just having a POP3 post in Feature Requests though, purely as a way to allow for any other 3rd party email someone wanted to do this with. I can see privacy advantages in POP3 as well, if one didn't want emails stored on their Cloudron and just locally. I wouldn't like to see POP3 ignored as without advantage, when if privacy is a concern, that could be a legitimate use-case.

I use a browser extension that removes all those as well as utm link data.

(still an issue - also possibly related to https://forum.cloudron.io/topic/4921/bayes_00-rule-set-for-some-mail-to-mailing-list-no-mailbox/1 ?)

@girish said in Spam filter doesn't always learn:

rspamd is better here but it does require more resources

rspamd require less training to work, and less complicated configuration setup.
but without clamav is pretty much equal to spamassasin with a little bit of training, and tweak.

AntiSpam in general are really havy to run, and complicated, if you take a plesk or cpanel antispam out of the box are similar to cloudron on.
The only solution is to implement a more complete stack in general, with blacklist, signature, DNSBL in a collaborative way, so that every cloudron can report spam and especially ham.

@imc67 if u want to manually improve you can add barracuda (b.barracudacentral.org) DNSBL to start. (this guide should work; backup your config file before edit it).
The issue of cloudron are mostly resources, because an efficient antispam will take to much ram, and cpu, so we need to find alternative solution, and my company is try to help cloudron to improve on this.

OK, with @NCKNE 's help we got this figured out. Cloudron has a anti-spoof check where we don't allow external servers to send email with FROM address set to any incoming domain. In this case, a backup MX is relaying email to Cloudron and it is correctly detected as spoof-ed email.

The workaround is to simply whitelist the MX's IP in the SPF record. With this Cloudron has the "authorization" that the server is allowed to relay such email and accepts the mail. I have added a section in our doc here - https://cloudron.io/documentation/email/#alternate-mx

@iamthefij Well the you can make it so you're not discoverable unless you message the person. Thats pretty anonymous

@d19dotca Currently, there is no way. The SA settings are set in stone. I can open up our mail addon git repo tomorrow so you can get a better idea of how we have implemented it.

But overall, our idea was that customers don't have to deal with implementation details like SA (for example, maybe we use rspamd later) and that spam detection should just work. If you have SA expertise, I am happy to incorporate your suggestions into the default settings.

@girish I just wanted to add another reason why we need better spam control exposed to admins... I just had a client who had their home IP address blacklisted by Spamhaus in their PBL because of the range of IP it was found in, even though they were not spamming at all (since the PBL blocks IP ranges). I have asked for it to be delisted but it'd be awesome if in the future I could just whitelist the IP address or something like that to quickly resolve it for them.

Update: Just found https://cloudron.io/documentation/email/#blacklist-domains-and-ips which is for blacklisting but I extrapolated from that and found the related whitelist files too, so used that. This needs to be added to the docs. I will see if I can submit that change in the git for you guys. 🙂 With that said though, it'd still be nice to see this exposed in the UI so if an admin is on the road / unable to SSH to the server, it can be easily whitelisted still.

@necrevistonnezr said in How does spam work with special mail folders?:

@girish Wouldn't it make sense to create hardlinks instead of symlinks for these folders so that you don't have duplicate folders? Ar am I overlooking something?

The cloudron backup logic does not understand hardlinks. It would then end up taking backup of your junk folder twice. This is unlike symlinks which is ignored by the backup code. Neither the symlink nor the contents of a symlink are backed up.

This does bring up an important point that if you move your mail server to a different server, you have to remember to create this symlink by hand again!