Is there a way to add in more DNSBL / RBL sources?
-
@d19dotca The mailspike is a great find. It seems very professionally done - https://www.mailspike.net/usage.html . Spamrats is also updated - https://spamrats.com/lists.php
-
@girish Yeah Mailspike is a great one. I completely forgot about it until all of this recent testing, haha. I am about 95% sure after visiting it's website that I used that on my email server before Cloudron too and it served really well. It was so many years ago now that I basically forgot which lists I used before, I think I'm basically going through all the same tests as I did about 4 years ago, lol.
For anyone who doesn't want Mailspike actually denying connections though, it should also work great in SpamAssassin as they already gave all the needed headers for it too so at least it'll help in identifying spam better too, using the text below:
header RCVD_IN_MSPIKE_BL eval:check_rbl('mspike-lastexternal', 'bl.mailspike.net.') tflags RCVD_IN_MSPIKE_BL net score RCVD_IN_MSPIKE_BL 3.5 header RCVD_IN_MSPIKE_WL eval:check_rbl('mspike-lastexternal', 'wl.mailspike.net.') tflags RCVD_IN_MSPIKE_WL net score RCVD_IN_MSPIKE_WL -2.1 -
@d19dotca said in Is there a way to add in more DNSBL / RBL sources?:
spam will never be 100% blocked and there will always be false-positives too
Yes, I totally agree with that.
By big question here is how much is useful to add feature and complexity to cloudron, when there are solutions that are specifically built for that.
About mailspike and Spamhaus, they both have SpamAssassin module, and if cloudron provides the possibility to disable SA, I think is a grate idea to have them.
But not as simple DNSBL, but as SA module.For spamrats idk, having to many DNSBL or SpamAssassin module will slow down the server a lot, remember for every incoming email, you need to call unbound that check his cache and if doesn't have the record call the DNSBL.
This happens for every email, and every external check you have, and maybe they are slow at that moment and the request take more time than usual, ... and so on
Resources usage need to be taken into count. -
@d19dotca said in Is there a way to add in more DNSBL / RBL sources?:
That is an interesting approach, and I'll consider it.
I know that is not the best solution out there.
But you can automate it, with API and Sieve is an open protocol.But is a start for now, and yes having a mail filter also for mail fwd is a good option, and if you don't need it you should just be able to disable SpamAssassin from the server.
I really don't like to w8st resources
-
@moocloud_matt I think it's safe to say from our conversations in this post and several others related to email improvements that you and I have different views overall on how to run our own mail servers. And that's totally okay!
That's actually the point... that there is no one-size-fits-all approach to running a mail server. Thus, you should be able to run your mail server how you best see fit for your clients, and how you run your mail server may not be the best way for me to run mine for my clients. Options are the key here, and that's what I'm trying to make sure is understood. It should all be optional and there should be no changes to the defaults on a new install, IMO. In fact I'd go to far to say that even Spamhaus shouldn't be used by default to block at the MTA level.In my case as an example, I've had a lot of email denied now at the MTA level by adding in the extra DNSBLs, and they've worked perfectly with zero false-positives so far in nearly 48 hours of running it on a very actively used mail server. So I see that as a big win for me and my clients. And of course I can always reconsider if I start seeing too many false-positives (or heck, even one false-positives for that matter - which I've yet to see so far in my testing), and that's kind of another example of the point I'm making here too... that we need options so we can enable and disable at will. In other words, none of what I'm asking for or proposing should be enabled by default, and none of it should be hard-coded as it currently is - we should be exposing this to users so they can make the decisions that best suits their own needs.
I know you seem worried about "complexity to cloudron" which is a fair concern always on how to keep Cloudron user-friendly, however I don't quite share that one this time because the data is already in Cloudron (it already uses Spamhaus Zen for blocking at MTA level and as we've seen is easily configurable already - it just doesn't survive restarts the way we can manually do it now), so the existing functionality just needs to be exposed in the UI. It's really no different than how they had recently exposed the SpamAssassin configuration too. It just needs a new little line on the Mail page with the other settings for message sized and such, a toggle and a box to list new DNSBLs. Of course it's more complicated than that in the backend, but from a user perspective this should not add any additional complexities. And I'd assume even in the backend once it's done it's done and shouldn't really need to be maintained at all.
All to say... we should have options available to us to benefit all of us uniquely as running a mail server is a complex task where it is not feasible to use a one-size-fits-all approach, IMO.
-
Latest update. I've been trialling a few different DNSBLs for use here at the MTA level for denying connections. So far I am very happy to say that I've had zero false-positives in over 3 days now on a very active mail server. This makes me very comfortable that this is a very safe configuration, but at the same itme would not necessarily recommend we make these default at all (in fact I don't even know if Spamhaus should be enabled by default to be totally honest- I really think that should be up to the mail admin).
The zone I've settled on and most recently been using...
black.junkemailfilter.com;bl.mailspike.net;all.spamrats.com;zen.spamhaus.orgTo give a bit of context to each of them...
-
black.junkemailfilter.comwas one I added the other day because of a clearly-spam message getting through and I had seen it in the "just now" timeframe so quickly checked mxtoolbox.com to see which DNSBL had it listed, and four of them did. This was one of them. The other was Barracuda which requires registration that I didn't want to try yet, and the other two were UCEPROTECT-2 and UCEPROTECT-3 which I didn't want to use (see side note at bottom). It’s caught a fair bit of spam the others didn’t catch earlier. This is known as "JMF-Black" on the Intra2Net list and has zero false-positives. -
bl.mailspike.netis one that seems very useful and is very accurate, has helped block a lot that Spamhaus Zen didn't catch earlier (before I had the zen.spamhaus.org earlier in the list meaning it should be checked first so we'd know if anything else blocked it that Spamhaus Zen didn't have yet). This blocked a good amount of spam. Zero false-positives. -
all.spamrats.comis also an excellent one and I'd say blocked as many as Spamhaus Zen did, it was catching spam frequently. Earlier I was using thenoptr.spamrats.comwhich worked very well too but later learned of the broaderall.spamrats.comand started using that yesterday with continued success and zero false-positives still. -
zen.spamhaus.orgneeds no introduction, it's probably the most popular DNSBL ever created. Much like the others, it's highly accurate and hasn't been seen to have any false-positives.
For all of the above and more, I'd recommend checking out the Intra2Net service which monitors the accuracy of the various DNSBLs. My recommendation is to stick to ones if blocking from the MTA level that has a 0% or at least no higher than a 0.05% chance of false-positives as they'd be considered safe. All the other DNSBLs that are more aggressive should really just go a step down to the SpamAssassin level for scoring metrics there (which I later did too for the URIBLs instead and they've been great so far too). I did have two false-positives in very early tests with the
dnsbl.sorbs.netone and also withbl.spamcop.netone. I'd suggest avoiding those for denying connections but can be used in SpamAssassin instead.I hope the above is a good test report for people, and others will hopefully find this helpful. Certainly there is no one-size-fits-all approach here, and I'd argue that none of these should even be enabled by default, however I believe them all to be "safe" based on my own experience and was glad to see the spam cut down further than it was prior to adding in the additional DNSBLs. What works for me may not work for you of course, depends likely on a lot of different factors, so "your mileage may vary" as they say. I've been watching the logs like a hawk all week and been checking every single "denied" entry and happy to report no false-positives in my testing so far over the last few days using the four listed above.
Here’s a quick screenshot:
Side note regarding UCEPROTECT DNSBLs: I strongly discourage use of the UCEPROTECT-* lists except possibly UCEPROTECT-1, because the level 2 and 3 seem to just blacklist large IP ranges that affect entire providers such as DigitalOcean, OVH, and more and basically demand fees for "express delisting" which doesn't even guarantee anything as it can be re-listed the next day. I question the ethics of that particular DNSBL provider as they seem to "extort" money from large network providers, and there is also this article I found that pretty much strips them apart line by line and explains why they may not be good to trust or use. My advice is to stay away from the UCEPROTECT DNSBLs based on the above plus they'd surely have a fairly high false-positive rate (you can see from the link above that the UCEPROTECT-3 has a whopping 17% inaccuracy rate.
@girish - Hopefully the above report will be useful for you and @nebulon when discussing some of the mail changes that may be coming in 6.3 there.
Update - March 26, 2021: Out of thousands of emails over the last week, I've only found two false-positives (thankfully non-critical emails, one was a Snapchat newsletter for example). That is a very impressive result to me and my users and I'm pleased with that as that seems to be within the reasonable threshold when weighing the pros and cons.
With that said, I have started a second test which involves removing one of the DNSBLs which made the false-positive result, and then added instead to the SpamAssassin side of things to at least help with identifying spam better to avoid the inbox. While this has led to more spam processing on my server, it seems to still be working well to achieving the ultimate goal of keeping spam messages out of my users inboxes. Here is my current DNSBL list zones in effect:
zen.spamhaus.org;bl.mailspike.net;noptr.spamrats.com(notice I removed theblack.junkemailfilter.comand changed fromall.spamrats.comback tonoptr.spamrats.com)So far the results are good. This however means unfortunately some of my clients who have mailing lists on the server that forward to their personal accounts elsewhere are receiving a bit more spam again until the new feature request is implemented to prevent external spam messages from being sent.
Depending on the results of the above tests, I may either stick to the current implementation or go back to how it was last week.
-
-
Wanted to write a quick update: Anyone wanting to enable an RBL can now do so very easily in the new 7.x Cloudron version! Big thanks to the Cloudron team for implementing that feature!
Since many visit this thread (it's even linked in the documentation now too!) for the list of the various RBLs and experience with them, reviewing them, etc... I wanted to add one more to the list which I've been testing out for a little bit and so far seems great, blocking spam from bad IPs which even hours later still isn't on some of the other popular blacklists when I've been checking manually to verify things.
Abusix is a premium service, however they do have a free tier which offers a rather large 5,000 queries per day - and I suspect most of us are not close to that amount in a single day, many likely not even over the course of a week - effectively meaning we can get premium-level spam filtering for free. They have several different lists they manage, but the recommended one to use is their combined.mail.abusix.zone zone which checks three separate lists of theirs out of the several. It is their "recommended" one for production servers offering a good balance of more checks and performance using one single lookup zone without being too overbearing as to include false-positives, this way it greatly limits any false-positives (of which I've seen zero so far!).
The only downside is a very minor cosmetic issue in Cloudron with it as the Abusix list is something like
<UUID>.combined.mail.abusix.zonesince it's premium so it's a unique URL to every user, and as such it's a very long URL due to the UUID which means some of the log entries in Cloudron's UI for denied messages get pretty long looking. I may file a feature request later for us to perhaps try naming our zones how owe want them to so we can avoid really long named ones in the logs, but overall it's just a cosmetic issue and nothing else.So just to summarize, the ones I'm using with great success so far are the following:
<UUID>.combined.mail.abusix.zone zen.spamhaus.org bl.mailspike.net -
In case you’re wondering: https://docs.cloudron.io/email/#dnsbl
Thanks to @d19dotca and the Cloudron Team!
-
@necrevistonnezr Ah thank you! I should have included that. Edited my earlier comment to include that link too now.
-
@d19dotca said in Is there a way to add in more DNSBL / RBL sources?:
black.junkemailfilter.com;bl.mailspike.net;all.spamrats.com;zen.spamhaus.org
interesting, I never eared about mailspike before
mainly I use [dbl and zen]spamhaus + barracudacentral.orgAlso on wikipedia they have a great listing with nice description:
https://en.wikipedia.org/wiki/Comparison_of_DNS_blacklists -
-
@necrevistonnezr The DNSBL IP check is done as the very first thing as soon as the sender connects. So, the mail is not even received and thus not stored anywhere.
-
@necrevistonnezr I found Abusix to be superior to Spamhaus Zen, and use that as my only DNSBL at the moment with everything else going through the SpamAssassin rules.
-
@d19dotca
We use abusix with rspamd, and after the implementation on the new signature base list, the quality improvement.But abusix with spamassasin, is good but still got a lot of spam passing, because SA don't support signature base abusix list.
