Sharing custom SpamAssassin Rules

[necrevistonnezr]

@timconsidine said in Sharing custom SpamAssassin Rules:

@necrevistonnezr I avoid fussing about GDPR as I consider it a total mess but one important distinction to bear in mind is that while an IP address is indeed capable of indicating identity, the core issue is whether it is “PID” : personally identifiable data.

As I said, Identifiers can be problematic in combination. And please think of false positives, where totally legit IPs are combined with your server domain - think a private domain and an IP to abortion information, stuff like that. BTW companies are not protected, only persons.

And while GDPR is exhausting (mostly, if not really understood) and needs improvements, it has lead to many companies thinking privacy when developing a product, not just afterwards. Or how they need to secure private data with technical and organizational measures (Art 32 GDPR) - hardcoding passwords or keeping them unencrypted for example (think the current Twitter scandal) can lead to fines and liability. It’s a new mind set.

[jdaviescoates]

@d19dotca said in Sharing custom SpamAssassin Rules:

@necrevistonnezr Yes, you can simply copy & paste the entire thing.

Even though you link to the docs methinks it's worth explicitly adding this to the top post too, perhaps also with a mention to scroll down to find your latest tweaks) i.e.

Just copy the latest set of rules (scroll down the thread to find the most recent) and then paste them into the Custom Spamassassin Rules box that pops up when you press the little pencil edit icon in Email -> Setting -> Spam filtering:

Thanks again (if you can count my previous upvote as a thanks ) for sharing this!

[d19dotca]

@jdaviescoates said in Sharing custom SpamAssassin Rules:

Even though you link to the docs methinks it's worth explicitly adding this to the top post too, perhaps also with a mention to scroll down to find your latest tweaks)

I'd like to do that but unfortunately (I guess too much time has passed so) it won't let me edit the original post anymore. All I can do is post updates to this one. Maybe @girish or somebody with admin privileges can allow me to do that or make the edit for me?

[girish]

I have elongated the post editing time frame.

[A Former User]

@girish, But do we get updates when @d19dotca just edits the post?

[humptydumpty]

@wirkaholic you should get a notification on the forum but you can make sure you're "watching" this topic as shown below.

Also, in your account settings, you can have the forum email you when you have notifications. Go to Profile > Settings > Notifications > When a post is edited in a topic you are watching > Set it to " Notification & Email".

[A Former User]

This post is deleted!

[A Former User]

@humptydumpty Yes, I can see! Thanks for your prompt reply!

[d19dotca]

Updated SpamAssassin Rules list for anyone wanting to use it or compare against the previous versions. Realized many rules weren't ever triggered and some lists were rarely ever used (GDUBD for example was removed as it only was triggered on 2 emails out of hundreds, so didn't seem worthwhile to keep using).

Highlights include:

• Removed a few of the outdated/never used DNSBLs
• Added in more SpamRATS and JunkEmailFilter HostKarma lists for better control over scoring (specifically added a new HostKarma allowlist too from JunkEmailFilter rather than only using the blocklist)
• Renamed a few of the SpamRATS and JunkEmailFilter HostKarma rules to reflect the service providers recommended names
• Small tweaks to various scores

# scoring BAYES
score BAYES_00 -5.0
score BAYES_05 -4.0
score BAYES_20 0.5
score BAYES_40 1.5
score BAYES_50 2.0
score BAYES_60 2.5
score BAYES_80 3.0
score BAYES_95 3.5
score BAYES_99 4.0
score BAYES_999 1.0

# scoring DNSBLs & DNSWLs
score RCVD_IN_BL_SPAMCOP_NET 2.0
score RCVD_IN_DNSWL_BLOCKED 0.0
score RCVD_IN_DNSWL_HI -5.0
score RCVD_IN_DNSWL_LOW -2.0
score RCVD_IN_DNSWL_MED -3.0
score RCVD_IN_DNSWL_NONE -0.5
score RCVD_IN_HOSTKARMA_BL 3.0
score RCVD_IN_HOSTKARMA_BR 0.5
score RCVD_IN_HOSTKARMA_W -5.0
score RCVD_IN_MSPIKE_BL 1.0
score RCVD_IN_MSPIKE_H2 0.0
score RCVD_IN_MSPIKE_H3 -0.5
score RCVD_IN_MSPIKE_H4 -1.0
score RCVD_IN_MSPIKE_H5 -3.0
score RCVD_IN_MSPIKE_L2 1.5
score RCVD_IN_MSPIKE_L3 2.5
score RCVD_IN_MSPIKE_L4 3.5
score RCVD_IN_MSPIKE_L5 4.5
score RCVD_IN_MSPIKE_WL -2.0
score RCVD_IN_MSPIKE_ZBI 4.0
score RCVD_IN_PBL 3.5
score RCVD_IN_SBL 3.5
score RCVD_IN_SBL_CSS 3.5
score RCVD_IN_SORBS_BLOCK 2.5
score RCVD_IN_SORBS_DUL 2.5
score RCVD_IN_SORBS_HTTP 2.5
score RCVD_IN_SORBS_MISC 2.5
score RCVD_IN_SORBS_SMTP 2.5
score RCVD_IN_SORBS_SOCKS 2.5
score RCVD_IN_SORBS_SPAM 2.5
score RCVD_IN_SORBS_WEB 2.5
score RCVD_IN_SORBS_ZOMBIE 2.5
score RCVD_IN_SPAMRATS_DYNA 3.0
score RCVD_IN_SPAMRATS_NOPTR 2.0
score RCVD_IN_SPAMRATS_SPAM 1.0
score RCVD_IN_XBL 3.5
score RCVD_IN_ZEN_BLOCKED 0.0
score RCVD_IN_ZEN_BLOCKED_OPENDNS 0.0

# scoring URIBLs
score URIBL_ABUSE_SURBL 4.5
score URIBL_BLACK 4.5
score URIBL_CR_SURBL 3.5
score URIBL_CSS 2.0
score URIBL_CSS_A 2.0
score URIBL_DBL_ABUSE_BOTCC 3.5
score URIBL_DBL_ABUSE_MALW 3.5
score URIBL_DBL_ABUSE_PHISH 3.5
score URIBL_DBL_ABUSE_REDIR 1.0
score URIBL_DBL_ABUSE_SPAM 3.0
score URIBL_DBL_BLOCKED 0.0
score URIBL_DBL_BLOCKED_OPENDNS 0.0
score URIBL_DBL_BOTNETCC 3.0
score URIBL_DBL_ERROR 0.0
score URIBL_DBL_MALWARE 3.5
score URIBL_DBL_PHISH 3.5
score URIBL_DBL_SPAM 3.5
score URIBL_GREY 1.0
score URIBL_MW_SURBL 3.5
score URIBL_PH_SURBL 3.5
score URIBL_RED 0.5
score URIBL_RHS_DOB 2.0
score URIBL_SBL 1.5
score URIBL_SBL_A 1.5
score URIBL_ZEN_BLOCKED 0.0
score URIBL_ZEN_BLOCKED_OPENDNS 0.0

# scoring DKIM & SPF
score DKIM_INVALID 1.5
score DKIM_SIGNED 0.0
score DKIM_VALID 0.0
score DKIM_VALID_AU 0.0
score DKIM_VALID_EF 0.0
score DKIM_VERIFIED 0.0
score DKIMWL_BL 3.0
score DKIMWL_WL_HIGH -3.5
score DKIMWL_WL_MED -2.5
score DKIMWL_WL_MEDHI -3.0
score FORGED_SPF_HELO 3.0
score SPF_FAIL 1.5
score SPF_HELO_FAIL 1.5
score SPF_HELO_NEUTRAL 1.0
score SPF_HELO_NONE 0.5
score SPF_HELO_PASS 0.0
score SPF_HELO_SOFTFAIL 1.5
score SPF_NEUTRAL 0.5
score SPF_NONE 0.5
score SPF_PASS 0.0
score SPF_SOFTFAIL 1.5
score USER_IN_DEF_DKIM_WL -5.0

# scoring HTML
score HTML_FONT_LOW_CONTRAST 0.5
score HTML_IMAGE_ONLY_04 0.5
score HTML_IMAGE_ONLY_08 0.5
score HTML_IMAGE_ONLY_12 1.0
score HTML_IMAGE_ONLY_16 1.0
score HTML_IMAGE_ONLY_20 2.0
score HTML_IMAGE_ONLY_24 2.0
score HTML_IMAGE_ONLY_28 2.5
score HTML_IMAGE_ONLY_32 3.0
score HTML_IMAGE_RATIO_02 0.0
score HTML_IMAGE_RATIO_04 0.0
score HTML_IMAGE_RATIO_06 0.0
score HTML_IMAGE_RATIO_08 0.0
score HTML_MESSAGE 0.0

# scoring HEADER & MISSING
score HEADER_FROM_DIFFERENT_DOMAINS 0.5
score HEADER_SPAM 2.5
score MISSING_DATE 3.0
score MISSING_FROM 1.5
score MISSING_HB_SEP 0.0
score MISSING_HEADERS 1.5
score MISSING_MID 1.0
score MISSING_MIMEOLE 1.0
score MISSING_SUBJECT 1.0

# scoring FREEMAIL
score FORGED_GMAIL_RCVD 1.5
score FORGED_YAHOO_RCVD 1.5
score FREEMAIL_ENVFROM_END_DIGIT 0.5
score FREEMAIL_FORGED_REPLYTO 0.5
score FREEMAIL_FROM 0
score FREEMAIL_REPLY 0.5
score FREEMAIL_REPLYTO 0.5
score FREEMAIL_REPLYTO_END_DIGIT 0.5
score MALFORMED_FREEMAIL 4.0

# additional scoring tweaks
score BILLION_DOLLARS 2.0
score BODY_URI_ONLY 1.5
score EMPTY_MESSAGE 1.5
score HELO_DYNAMIC_SPLIT_IP 2.0
score HK_RANDOM_ENVFROM 0.5
score HK_RANDOM_FROM 0.5
score LOTS_OF_MONEY 0.5
score MPART_ALT_DIFF 0.5
score MPART_ALT_DIFF_COUNT 1.0
score NO_DNS_FOR_FROM 0.5
score PDS_TONAME_EQ_TOLOCAL 0.5
score PDS_TONAME_EQ_TOLOCAL_VSHORT 0.5
score RDNS_NONE 1.5
score REPLYTO_WITHOUT_TO_CC 2.5
score UNPARSEABLE_RELAY 0.5
score URI_DQ_UNSUB 2.0
score T_FILL_THIS_FORM_SHORT 0.5

# add JunkEmailFilter HostKarma DNSBL & DNSWL
header __RCVD_IN_HOSTKARMA eval:check_rbl('HOSTKARMA-lastexternal','hostkarma.junkemailfilter.com.')
describe __RCVD_IN_HOSTKARMA Sender listed in JunkEmailFilter
tflags __RCVD_IN_HOSTKARMA net
header RCVD_IN_HOSTKARMA_W eval:check_rbl_sub('HOSTKARMA-lastexternal','127.0.0.1')
describe RCVD_IN_HOSTKARMA_W Sender listed in HOSTKARMA-WHITE
tflags RCVD_IN_HOSTKARMA_W net nice
header RCVD_IN_HOSTKARMA_BL eval:check_rbl_sub('HOSTKARMA-lastexternal','127.0.0.2')
describe RCVD_IN_HOSTKARMA_BL Sender listed in HOSTKARMA-BLACK
tflags RCVD_IN_HOSTKARMA_BL net
header RCVD_IN_HOSTKARMA_BR eval:check_rbl_sub('HOSTKARMA-lastexternal','127.0.0.4')
describe RCVD_IN_HOSTKARMA_BR Sender listed in HOSTKARMA-BROWN
tflags RCVD_IN_HOSTKARMA_BR net

# add Spamrats DNSBL
header __RCVD_IN_SPAMRATS eval:check_rbl('spamrats-lastexternal','all.spamrats.com.')
describe __RCVD_IN_SPAMRATS SPAMRATS: sender is listed in SpamRats
tflags __RCVD_IN_SPAMRATS net
reuse __RCVD_IN_SPAMRATS
header RCVD_IN_SPAMRATS_DYNA eval:check_rbl_sub('spamrats-lastexternal','127.0.0.36')
describe RCVD_IN_SPAMRATS_DYNA RATS-Dyna: sent directly from dynamic IP address
tflags RCVD_IN_SPAMRATS_DYNA net
reuse RCVD_IN_SPAMRATS_DYNA
header RCVD_IN_SPAMRATS_NOPTR eval:check_rbl_sub('spamrats-lastexternal','127.0.0.37')
describe RCVD_IN_SPAMRATS_NOPTR RATS-NoPtr: sender has no reverse DNS
tflags RCVD_IN_SPAMRATS_NOPTR net
reuse RCVD_IN_SPAMRATS_NOPTR
header RCVD_IN_SPAMRATS_SPAM eval:check_rbl_sub('spamrats-lastexternal','127.0.0.38')
describe RCVD_IN_SPAMRATS_SPAM RATS-Spam: sender is a spam source
tflags RCVD_IN_SPAMRATS_SPAM net
reuse RCVD_IN_SPAMRATS_SPAM

[doodlemania2]

@d19dotca This is cool! How do I "implement" it?

[d19dotca]

@doodlemania2 You simply add it to the Cloudron Email function under Spam Filtering > Custom SpamAssassin Rules. It's documented here: https://docs.cloudron.io/email/#custom-spam-filtering-rules

[d19dotca]

While I noticed some improvements in my last set of rules I also saw a few extras getting through to my inbox too, so I think the last update was a "one step forward, two steps back" update, so I apologize if anyone saw a decrease in effectiveness if using the latest list. I immediately made some tweaks and have noticed this seems to be more effective. Let me know if you have any issues though.

# scoring BAYES
score BAYES_00 -5.0
score BAYES_05 -4.0
score BAYES_20 1.0
score BAYES_40 2.0
score BAYES_50 2.5
score BAYES_60 3.0
score BAYES_80 3.5
score BAYES_95 4.0
score BAYES_99 4.5
score BAYES_999 1.0

# scoring DNSBLs & DNSWLs
score RCVD_IN_BL_SPAMCOP_NET 2.0
score RCVD_IN_DNSWL_BLOCKED 0
score RCVD_IN_DNSWL_HI -6.0
score RCVD_IN_DNSWL_LOW -2.0
score RCVD_IN_DNSWL_MED -4.0
score RCVD_IN_DNSWL_NONE 0.5
score RCVD_IN_HOSTKARMA_BL 2.0
score RCVD_IN_HOSTKARMA_BR 0.5
score RCVD_IN_HOSTKARMA_W -5.0
score RCVD_IN_MSPIKE_BL 0.0
score RCVD_IN_MSPIKE_H2 -0.5
score RCVD_IN_MSPIKE_H3 -0.5
score RCVD_IN_MSPIKE_H4 -2.0
score RCVD_IN_MSPIKE_H5 -3.0
score RCVD_IN_MSPIKE_L3 0.5
score RCVD_IN_MSPIKE_L4 2.0
score RCVD_IN_MSPIKE_L5 3.0
score RCVD_IN_MSPIKE_WL 0.0
score RCVD_IN_MSPIKE_ZBI 2.0
score RCVD_IN_PBL 3.0
score RCVD_IN_SBL 3.0
score RCVD_IN_SBL_CSS 3.0
score RCVD_IN_SPAMRATS_DYNA 2.0
score RCVD_IN_SPAMRATS_NOPTR 2.0
score RCVD_IN_SPAMRATS_SPAM 3.0
score RCVD_IN_XBL 3.0
score RCVD_IN_ZEN_BLOCKED 0.0
score RCVD_IN_ZEN_BLOCKED_OPENDNS 0.0

# scoring URIBLs
score URIBL_ABUSE_SURBL 4.5
score URIBL_BLACK 4.5
score URIBL_CR_SURBL 3.5
score URIBL_CSS 2.0
score URIBL_CSS_A 2.0
score URIBL_DBL_ABUSE_BOTCC 3.0
score URIBL_DBL_ABUSE_MALW 3.0
score URIBL_DBL_ABUSE_PHISH 3.0
score URIBL_DBL_ABUSE_REDIR 1.0
score URIBL_DBL_ABUSE_SPAM 3.0
score URIBL_DBL_BLOCKED 0.0
score URIBL_DBL_BLOCKED_OPENDNS 0.0
score URIBL_DBL_BOTNETCC 3.0
score URIBL_DBL_ERROR 0.0
score URIBL_DBL_MALWARE 3.0
score URIBL_DBL_PHISH 3.0
score URIBL_DBL_SPAM 3.0
score URIBL_GREY 1.0
score URIBL_MW_SURBL 3.5
score URIBL_PH_SURBL 3.5
score URIBL_RED 0.5
score URIBL_RHS_DOB 2.0
score URIBL_SBL 3.0
score URIBL_SBL_A 3.0
score URIBL_ZEN_BLOCKED 0.0
score URIBL_ZEN_BLOCKED_OPENDNS 0.0

# scoring DKIM & SPF
score DKIM_INVALID 1.5
score DKIM_SIGNED 0.0
score DKIM_VALID 0.0
score DKIM_VALID_AU 0.0
score DKIM_VALID_EF 0.0
score DKIM_VERIFIED 0.0
score DKIMWL_BL 3.0
score DKIMWL_WL_HIGH -3.5
score DKIMWL_WL_MED -2.5
score DKIMWL_WL_MEDHI -3.0
score FORGED_SPF_HELO 3.0
score SPF_FAIL 1.5
score SPF_HELO_FAIL 1.5
score SPF_HELO_NEUTRAL 1.0
score SPF_HELO_NONE 0.5
score SPF_HELO_PASS 0.0
score SPF_HELO_SOFTFAIL 1.5
score SPF_NEUTRAL 0.5
score SPF_NONE 0.5
score SPF_PASS 0.0
score SPF_SOFTFAIL 1.5
score USER_IN_DEF_DKIM_WL -5.0

# scoring HTML
score HTML_FONT_LOW_CONTRAST 0.5
score HTML_IMAGE_ONLY_04 1.0
score HTML_IMAGE_ONLY_08 1.0
score HTML_IMAGE_ONLY_12 1.0
score HTML_IMAGE_ONLY_16 1.5
score HTML_IMAGE_ONLY_20 1.5
score HTML_IMAGE_ONLY_24 2.0
score HTML_IMAGE_ONLY_28 2.5
score HTML_IMAGE_ONLY_32 3.0
score HTML_IMAGE_RATIO_02 0.0
score HTML_IMAGE_RATIO_04 0.0
score HTML_IMAGE_RATIO_06 0.0
score HTML_IMAGE_RATIO_08 0.0
score HTML_MESSAGE 0.0
score HTML_MIME_NO_HTML_TAG 0.5
score HTML_SHORT_LINK_IMG_1 2.5
score HTML_SHORT_LINK_IMG_2 1.5
score HTML_SHORT_LINK_IMG_3 0.5

# scoring HEADER & MISSING
score HEADER_FROM_DIFFERENT_DOMAINS 0.5
score MISSING_DATE 3.0
score MISSING_FROM 1.5
score MISSING_HEADERS 2.0
score MISSING_SUBJECT 1.0

# scoring FREEMAIL
score FREEMAIL_ENVFROM_END_DIGIT 0.5
score FREEMAIL_FORGED_REPLYTO 1.0
score FREEMAIL_FROM 0
score FREEMAIL_REPLY 0.5
score FREEMAIL_REPLYTO 0.5
score FREEMAIL_REPLYTO_END_DIGIT 0.5

# additional scoring tweaks
score HELO_DYNAMIC_SPLIT_IP 3.0
score LOTS_OF_MONEY 0.5
score MPART_ALT_DIFF 0.5
score MPART_ALT_DIFF_COUNT 0.5
score RDNS_NONE 0.5
score T_FILL_THIS_FORM_SHORT 0.5
score UNPARSEABLE_RELAY 0.5

# add JunkEmailFilter HostKarma DNSBL & DNSWL
header __RCVD_IN_HOSTKARMA eval:check_rbl('HOSTKARMA-lastexternal','hostkarma.junkemailfilter.com.')
describe __RCVD_IN_HOSTKARMA Sender listed in JunkEmailFilter
tflags __RCVD_IN_HOSTKARMA net
header RCVD_IN_HOSTKARMA_W eval:check_rbl_sub('HOSTKARMA-lastexternal','127.0.0.1')
describe RCVD_IN_HOSTKARMA_W Sender listed in HOSTKARMA-WHITE
tflags RCVD_IN_HOSTKARMA_W net nice
header RCVD_IN_HOSTKARMA_BL eval:check_rbl_sub('HOSTKARMA-lastexternal','127.0.0.2')
describe RCVD_IN_HOSTKARMA_BL Sender listed in HOSTKARMA-BLACK
tflags RCVD_IN_HOSTKARMA_BL net
header RCVD_IN_HOSTKARMA_BR eval:check_rbl_sub('HOSTKARMA-lastexternal','127.0.0.4')
describe RCVD_IN_HOSTKARMA_BR Sender listed in HOSTKARMA-BROWN
tflags RCVD_IN_HOSTKARMA_BR net

# add Spamrats DNSBL
header __RCVD_IN_SPAMRATS eval:check_rbl('spamrats-lastexternal','all.spamrats.com.')
describe __RCVD_IN_SPAMRATS SPAMRATS: sender is listed in SpamRats
tflags __RCVD_IN_SPAMRATS net
reuse __RCVD_IN_SPAMRATS
header RCVD_IN_SPAMRATS_DYNA eval:check_rbl_sub('spamrats-lastexternal','127.0.0.36')
describe RCVD_IN_SPAMRATS_DYNA RATS-Dyna: sent directly from dynamic IP address
tflags RCVD_IN_SPAMRATS_DYNA net
reuse RCVD_IN_SPAMRATS_DYNA
header RCVD_IN_SPAMRATS_NOPTR eval:check_rbl_sub('spamrats-lastexternal','127.0.0.37')
describe RCVD_IN_SPAMRATS_NOPTR RATS-NoPtr: sender has no reverse DNS
tflags RCVD_IN_SPAMRATS_NOPTR net
reuse RCVD_IN_SPAMRATS_NOPTR
header RCVD_IN_SPAMRATS_SPAM eval:check_rbl_sub('spamrats-lastexternal','127.0.0.38')
describe RCVD_IN_SPAMRATS_SPAM RATS-Spam: sender is a spam source
tflags RCVD_IN_SPAMRATS_SPAM net
reuse RCVD_IN_SPAMRATS_SPAM

[murgero]

@d19dotca This looks pretty good. I am testing it now