Ability for spam messages to not be forward via mailing list

d19dotca

I'd like to request the ability for Cloudron to not forward on mail to a mailing list recipient if it's identified it as spam.

Context behind feature request:

I have a particular recipient who checks their email on iCloud.com but has a domain I host for them too which is simply a bunch of mailing lists that forward to their iCloud.com accounts.

They’ve been getting a lot of spam lately. Thankfully Apple blocks most of it so they don’t really see the extra spam, but in my mail server logs I see that there are frequently messages sent which the system identifies as spam (and obvious spam at that - see log below), but still forwards it on anyway. This then causes Apple to send a bounce which my mail server forwards on and basically goes in a bit of loop.

For multiple reasons, I’d like the system to not forward messages to mailing lists when it it identifies the message as spam. Ideally also the ability to set a threshold number to avoid false-positives. Some reasons include:

unnecessary resource usage of the mail server
risk of being blocked by receiving mail server as sending spam (this is the biggest reason IMO)

{
  "ts": 1615659016646,
  "type": "queued",
  "direction": "inbound",
  "uuid": "080BB72D-E208-4C11-8CA0-080450DC03E6.1",
  "remote": {
    "ip": "162.215.212.45",
    "port": 45066,
    "host": "162-215-212-45.unifiedlayer.com",
    "info": "162-215-212-45.unifiedlayer.com",
    "closed": false,
    "is_private": false,
    "is_local": false
  },
  "authUser": null,
  "mailFrom": "<SRS0=326c=IM=worldmap2100.info=WORLDMAP210020210313@<domain>.<tld>>",
  "rcptTo": [
    "<<username>@<domain>.<tld>>"
  ],
  "details": {
    "spamStatus": "Yes, score=20.8 required=5.0 tests=BAYES_99,BAYES_999,  \tCK_HELO_DYNAMIC_SPLIT_IP,DYN_RDNS_AND_INLINE_IMAGE,  \tFREEMAIL_FORGED_REPLYTO,FREEMAIL_REPLYTO_END_DIGIT,FROM_LOCAL_DIGITS,  \tFROM_LOCAL_HEX,GB_FREEMAIL_DISPTO,GB_FREEMAIL_DISPTO_NOTFREEM,  \tHELO_MISC_IP,HTML_IMAGE_ONLY_04,HTML_MESSAGE,HTML_SHORT_LINK_IMG_1,  \tIMG_ONLY_FM_DOM_INFO,KHOP_HELO_FCRDNS,MIME_HTML_ONLY,  \tRCVD_IN_BL_SPAMCOP_NET,RCVD_IN_MSPIKE_BL,RCVD_IN_MSPIKE_L5,  \tRCVD_IN_RP_RNBL,RDNS_DYNAMIC,SPF_HELO_NONE,TVD_RCVD_IP,T_SPF_TEMPERROR,  \tURI_HEX autolearn=no autolearn_force=no version=3.4.4",
    "message": "Message Queued (080BB72D-E208-4C11-8CA0-080450DC03E6.1) (080BB72D-E208-4C11-8CA0-080450DC03E6.1)"
  }
}

girish

@d19dotca to continue our discussion from the 6.3 release thread, I wanted to understand this a bit more. When I was fixing this, I realized that a user could simply have created a "closed" mailing list i.e a member only list that we already support. Then understanding your context a bit more here, it seems that you are using the mailing list feature to essentially work purely as a mailbox forwarder. Did I understand that correctly? If so, is there a reason why you don't setup sieve rules for this? Is this because it is too painful to set it up or something else?

On a side note, the sieve forwarding filters were not working on Cloudron. This has actually been fixed now in 6.3. I think sieve filters can also be used to check X-Spam-Status.

d19dotca

@girish Yes, it was more for a matter of maintenance that I chose to use the Mailing List functionality instead of a full mailbox. I suppose that's a workaround (though sounds like forwarding wasn't even working anyways until fixed in 6.3?), it's just not as "clean" or "convenient" to do. I think most people would agree that spam should never be sent to a mailing list (or at least not forwarded on from the mailing list), so I'd hope for a cleaner less-overhead manner to prevent spam from being forwarded to mailing lists. Hopefully that makes sense, but let me know if I can clarify at all.

d19dotca

I performed a test, and believe the following sieve rules will work for anyone who wants to achieve this using a mailbox instead of mailing list.

# rule:[Discard spam messages on arrival]
if allof (header :contains "x-spam-status" "Yes,")
{
	discard;
	stop;
}
# rule:[Forward non-spam messages]
if allof (not header :contains "x-spam-status" "Yes,")
{
	redirect "<externalEmailAddress>";
}

For future visitors... To use the sieve filter above:

Login to Roundcube webmail app
Go to Settings > Filters
Click Actions > Edit Filter Set
Paste the above snippet, being sure to replace <externalEmailAddress> with the actual recipient's external email address
Save

Ideally though, I think there still needs to be a filter to discard/delete all spam messages sent to a mailing list, as spam should never arrive on a mailing list (regardless if it's used simply for forwarding or for an actual mailing list).

robi

@d19dotca great, now just document how it's applied and enabled for everyone else.

d19dotca

@robi said in Ability for spam messages to not be forward via mailing list:

@d19dotca great, now just document how it's applied and enabled for everyone else.

Edited with steps now

girish

@d19dotca to answer your question in the other thread

I think that's fine for now, but would it be possible to only affect mailing lists perhaps (assuming this change is mostly just temporary and probably will only really be used by me for the time being). I ask because I think if I were to just drop all messages marked as spam from all mail delivery completely, it'd be a detriment to my hosted mailboxes. The only place I want to disable sending spam mail is for mailing lists alone, if possible.

yes, correct. this filter/reject is only for the mailing lists.

d19dotca

I was checking in on my emails today that I had setup the other day as mailboxes with sieve rules to try and filter out spam. However it doesn’t actually appear to be working; which is odd because I tested them myself and they seemed correct. I’m going to make a small tweak and test again. Originally I had “Yes” and then I changed it to “Yes,” to remove false-positives in case any of the rules contained “yes” in their character set, but maybe that comma is throwing it off.

d19dotca

I can update that the first rule doesn't seem to work in my sieve filter, it's not discarding the spam messages but is still saving them in spam instead. However the main use-case of only forwarding on non-spam messages seems to work well. I'm still trying to figure out the first one, because basically I don't want any email saved in the mailbox when it's being used for forwarding only.

d19dotca

@girish - this is not possible to do unfortunately in Cloudron it seems, but kindly tell me if I'm doing something wrong here.

The redirect works for filtering out spam, in other words only non-spam messages get through to the endpoint email using a filter like this:

# rule:[Forward non-spam messages]
if allof (not header :contains "x-spam-status" "Yes,")
{
	redirect "<externalEmailAddress>";
}

However, all messages marked as spam still save in the mailbox regardless of any filters I set to discard them. In my case, I want the spam messages to be discarded/deleted automatically as there's no sense in them remaining in the "forwarding only" mailbox. It seems maybe there's a priority filter going on elsewhere in Cloudron that's overriding my sieve filter?

Using a filter like this:

# rule:[Discard spam messages on arrival]
if allof (header :contains "x-spam-status" "Yes,")
{
	discard;
	stop;
}

... will still save the email in the Spam folder. I've tried variations of it too but no-dice. Basically I can't get any filters to work when it's an identified spam message, only filters when it's NOT spam.

Here's a way to reproduce it:

Set this filter in Roundcube:

# rule:[Discard spam messages on arrival]
if allof (header :contains "x-spam-status" "Yes,")
{
	discard;
	stop;
}

Send a message to the email account using the GTUBE string which SpamAssassin automatically marks with 1000 points, it's basically a test for spam filters: XJS*C4JDBQADN1.NSBN3*2IDNEN*GTUBE-STANDARD-ANTI-UBE-TEST-EMAIL*C.34X
See if the message is really discarded per the sieve filter or if it's still arriving in the Spam folder. In my cases, they still arrive in the Spam folder which seems to be incorrect behaviour.

Edit: I wrote a dedicated bug for this instead: https://forum.cloudron.io/topic/5189/can-t-discard-spam-messages-on-arrival-using-sieve-filters/1

Cloudron makes it easy to run web apps like WordPress, Nextcloud, GitLab on your server. Find out more or install now.

Solved Ability for spam messages to not be forward via mailing list