Sharing custom SpamAssassin Rules

[d19dotca]

@sponch said in Sharing custom SpamAssassin Rules:

@d19dotca great and thanks! for abusix I just have to put in the api key without <>, right?
Done but don't get queries shown in the dashboard (though I sent some mails).
Using zen.spamhaus.org as DNSBL

That’s correct, no angle brackets. The full URL to use is shown in the Abusix dashboard but it’s really just the API key plus the subdomain parts.

I didn’t see queries until the following day I think, if I’m remembering correctly. So maybe give it another day or two? Also maybe make sure you don’t have any spaces or blank characters in the DNSBL just in case that’s throwing off the DNS queries to it.

Also I saw you mentioned that you didn’t see on the dashboard “though [you] sent some mails”… just to clarify, the queries will be done when you receive mail rather than send mail. I’m sure you knew that, but just in case, I thought I should clarify that part.

If you don’t see anything in a couple of days on the dashboard then let me know, and I can try to help. If it’s set correctly in Cloudron though then it could be something more on the Abusix side, maybe something needs to get confirmed or activated first (I don’t remember having to do that though but I’ve been using it for a while so I can’t remember the full on-boarding workflow).

[sponch]

hey @d19dotca thanks again. Working now- I can see the first queries in abusix.
With "sent" I meant sent to the specific mailbox looking forward to the next days filtering result :_)

[BrutalBirdie]

7 days recap after applying your rules.
I believe not one spam mail has hit my spam folder or inbox so far.
normally I'd get ~20x+ spam mails a day since my Inbox also redirects my old legacy mailboxes from web.de which have been leaked and abused over and over again.

I must say, this feels very good.

[crazybrad]

@humptydumpty So I asked my favorite tool for some help on your list of "bad Gmail actors" and here is a detailed analysis for your consideration: https://www.perplexity.ai/search/please-review-the-attached-gma-BjXGrt4qR_er6c45dse5Vw .

I found myself curious as to whether those email addresses even exist. Unfortunately Gmail does not have a "finger" API and there are limited options within Spam Assassin for handling this directly. There were some ideas on combining Spam Assassin's rule-based tagging with a Sieve filter. Here are the details for your consideration: https://www.perplexity.ai/search/does-gmail-have-the-ability-to-jePfq628TDeod5jDVoYU2Q

[humptydumpty]

@crazybrad That was an interesting read! I'm going to test the gmail spam rules and see how it goes. I'll add my gmail based clients to the whitelist to be on the safe side though. TYVM!

[sponch]

@BrutalBirdie still get them in my spam folder but at least not in my inbox

[murgero]

@d19dotca Heyo! Finally got around to applying this - do I need to add anything to Mail ACL or just to custom spamassassin rules?

[d19dotca]

@murgero just in the SpamAssassin rules part is all you need. Nothing else. But let me know if that doesn’t seem to work for you at all and I can try to help with some screenshots later too.

[marcusquinn]

I've been getting a LOT of spam lately.

@girish Any chances we can have this implemented but the core app? Save everyone having to discoverer this thread and do the same.

[necrevistonnezr]

Alternatively: turn it into a community guide, link to the guide in the documentation

[msbt]

Thanks a bunch for the list @d19dotca! Quick question about the rest of the setup though: Do you still have entries in the Email ACL DNSBL Zones or is that empty because everything is handled in the custom rules? Like those:

zen.spamhaus.org
bl.mailspike.net
noptr.spamrats.com
dnsbl.sorbs.net

Or is that empty on your side?

[d19dotca]

@msbt Great question! So for me personally I use the following one in there: {APIKey}.exploit.mail.abusix.zone. The reason being is that seems to be 100% accurate in terms of 0 false positives. The goal is to get to 0 false positives and then tag the rest as either ham or spam so the users can decide from there if anything is incorrect. That way they don’t risk losing any mail that may be important.

I have also been tinkering with the spam rules again the past month, testing some things out. I’ll go into more detail with that soon with updated scores that I’m using. I wanted to do a bit more analysis of it today actually to make sure it’s in the right direction before sharing it, but I’ll likely be in a position to share it pretty soon.

[BrutalBirdie]

@d19dotca said in Sharing custom SpamAssassin Rules:

I have also been tinkering with the spam rules again the past month, testing some things out. I’ll go into more detail with that soon with updated scores that I’m using. I wanted to do a bit more analysis of it today actually to make sure it’s in the right direction before sharing it, but I’ll likely be in a position to share it pretty soon.

[humptydumpty]

@d19dotca The last rules you provided are working great for me. I still get spam sent via the major mail providers like Gmail and Outlook though. They seem to target my info@ mailboxes. I wonder if there is anything we can do in that regards other than using keyword filtering?

[d19dotca]

@humptydumpty That's something I'd like to look into too, although I have a feeling the only thing that can really work its magic there is the Bayesian learning, so running the SpamAssassin learn commands. I've been running a script (with the help of ChatGPT, lol) like one below in case this helps as I find the Bayesian learning in Cloudron seems to be really manual or inconsistent at running (I think they've admitted that too in a post I saw somewhere the other month), and it's improved IMO with running this often. Personally I run this manually for now just because I wanted to make sure it was working, but I'll probably consider throwing this in a cron job soon enough.

1. sudo docker exec -ti mail /bin/bash

2. Run this script in the mail container:

nohup bash -c '
MAILDIR="/app/data/vmail"; SPAMD_DIR="/app/data/spamd";
for user in $(ls "$MAILDIR"); do
    MAILBOX="$MAILDIR/$user/mail"; BAYES_PATH="$SPAMD_DIR/$user";
    mkdir -p "$BAYES_PATH"; chown -R cloudron:cloudron "$BAYES_PATH"; chmod 700 "$BAYES_PATH";
    echo "🔄 Training SpamAssassin for $user..." | tee -a /app/data/spamd/train.log;

    # Train spam from .Spam and .Junk folders (including subfolders)
    find "$MAILBOX/.Spam" "$MAILBOX/.Junk" -type d -name "cur" 2>/dev/null | while read folder; do
        echo "📂 Training SPAM from: $folder" | tee -a /app/data/spamd/train.log;
        sa-learn --spam --dbpath "$BAYES_PATH" --dir "$folder" | tee -a /app/data/spamd/train.log;
    done

    # Train ham from Inbox and Archive, but EXCLUDE Junk, Spam, Trash, Sent, and Drafts
    find "$MAILBOX" -type d -name "cur" 2>/dev/null | grep -Ev "/(\.Trash|\.Deleted Messages|\.Sent|\.Sent Messages|\.Drafts|\.Junk|\.Spam)/" | while read folder; do
        echo "📂 Training HAM from: $folder" | tee -a /app/data/spamd/train.log;
        sa-learn --ham --dbpath "$BAYES_PATH" --dir "$folder" | tee -a /app/data/spamd/train.log;
    done

    echo "✔ Completed training for $user! BAYES files stored in $BAYES_PATH" | tee -a /app/data/spamd/train.log;
done;
echo "🎉 SpamAssassin training completed for all mailboxes." | tee -a /app/data/spamd/train.log;
' > /app/data/spamd/train.log 2>&1 &

It creates that train.log file and writes all the output to it so you can see it learning across all mailboxes for the Inbox and Archive folder as ham and the Junk/Spam folder as spam for all users. It's neat to see it saying it learned ham from 34 messages or something like that for each mailbox, haha.

I think my latest spam rules are doing well the past week, so I'll likely be posting them here soon.