Cloudron makes it easy to run web apps like WordPress, Nextcloud, GitLab on your server. Find out more or install now.


    Cloudron Forum

    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular

    Is there a way to add in more DNSBL / RBL sources?

    Discuss
    6
    40
    2080
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • d19dotca
      d19dotca last edited by d19dotca

      I know Cloudron already checks against Spamhaus and rejects if the sending server is listed... is there an ability to add in other sources at all? I don't think there is but wanted to double-check. If not, then I'll file a feature request.

      --
      Dustin Dauncey
      www.d19.ca

      MooCloud_Matt 1 Reply Last reply Reply Quote 2
      • d19dotca
        d19dotca last edited by d19dotca

        Example: Recently had an IP of 81.30.27.198 send clear spam to my server, and it's blocked on several lists already but not yet on Spamhaus.

        Having an additional one for maybe SpamCop or SORBS (though in my experience SORBS tends to be over-aggressive - though I could be mixing that up with SpamCop, one of them is over aggressive lol).

        https://mxtoolbox.com/SuperTool.aspx?action=blacklist%3A81.30.27.198&run=toolpage

        9e073a66-c531-4731-abe2-0672e8899617-image.png

        If this isn't possible (I suspect it's not) then I'll file a feature request. I have a bunch of mail server improvements I hope we can see in 7.0 as there's so much room for improvement in the mail server functionality, some of which has been requested for quite a long time now.

        --
        Dustin Dauncey
        www.d19.ca

        1 Reply Last reply Reply Quote 1
        • girish
          girish Staff last edited by

          The list is hardcoded - https://git.cloudron.io/cloudron/box/-/blob/master/src/mail.js#L371 . The list is derived from https://raw.githubusercontent.com/jawsome/node-dnsbl/master/list.json . One danger of adding lesser known lists is that they suddenly disappear and get replaced by ad pages. I am happy to add any missing popular provider though.

          d19dotca 1 Reply Last reply Reply Quote 0
          • d19dotca
            d19dotca @girish last edited by d19dotca

            @girish Ah interesting, thanks for sharing that link. A couple of things then:

            • I've only ever seen mail blocked in the logs by Spamhaus Zen, never any others. Curious if that's seen by others in there too. Wondering if there's an issue there at all.

            • I'm wondering why this message got through for example when it's listed on so many blocklists (seems 3 or 4 of which from the screenshot above are present in the Cloudron configuration then). I suppose it's technically possible it came in before being added to the blocklists, but I really doubt this is the case as I looked this up just minutes after it was in the logs and it already showed all those blocklists with it.

            I'm starting to wonder if there's a possible defect here where the other lists aren't actually being checked? Mostly because this message still passed, and also I've yet to ever see anything blocked by anything other than Spamhaus Zen in the mail logs, so I'm thinking the others may not be working properly.

            Example of recent "blacklisted by" items in the logs:

            cbb27b11-2e7c-4eb3-8e0c-7b1a36d30c16-image.png

            I think we may have uncovered a possible bug here.

            PS - I agree being hesitant to add any lesser-known ones as they tend to be "fly by night", but the UCEPROTECT, SORBS, and SpamCop have been around forever for example, and thankfully you have those added in Cloudron which I didn't realize before, but I still haven't seen any of those in the logs for any messages blocked from being processed further in Cloudron (see image above). I'm actually happy to see the current configuration you have as it's a solid list (if anything I'd have actually only enabled two or three to avoid false-positives, haha, so I think your configuration is even more aggressive than I was wanting, but that's okay for now since I haven't seen any false-positives yet - though again I haven't even seen the others actually work yet either).

            --
            Dustin Dauncey
            www.d19.ca

            girish 2 Replies Last reply Reply Quote 0
            • girish
              girish Staff @d19dotca last edited by

              @d19dotca Ah, I see your confusion. The list is posted is only for the DNSBL checks in the UI (mail -> status). It doesn't affect mail delivery, it's really just a check to inform the user and nothing else.

              As for accepting mail, we only check against Spamhaus Zen in the mail server. If it's black listed, then we delay the rejection until the user logs in. Sometimes, you will see in the logs that the IP is blacklisted but then it proceeds to accept email . This is because the user logged in and when a user logs in, we ignore the spamhaus checks (because most residential and user IPs are blacklisted anyways).

              Zen has worked out pretty well so far since it's very actively maintained. I don't really know much about the lists but if we add it to the mail server, then we have to be 500% sure it's reliable because atleast for support rejecting email is way more work than accepting some junk πŸ™‚

              d19dotca 1 Reply Last reply Reply Quote 1
              • girish
                girish Staff @d19dotca last edited by

                @d19dotca Wow, your server is really hot property for incoming spam! Is the issue that despite the check, some spam slips through?

                d19dotca 1 Reply Last reply Reply Quote 0
                • d19dotca
                  d19dotca @girish last edited by

                  @girish Oh, I see, so the others are checked for adding in headers to identify it as spam, but only Spamhaus Zen is actually used for blacklisting it completely, right?

                  So I guess that's back to my original question then (but I guess you've answered it now), is how to add more than just Spamhaus Zen for the connection blocking.

                  Ultimately on a related topic... I'm focusing on my mail server and see so many instances of clear abuse of the spam and the Cloudron server doesn't seem to be doing enough, IMO here as the list of sources to check for denying access isn't customizable for example, and plus the other area I've seen and filed a feature request for already which is to block messages identified as spam from being forward on to mailing list recipients. Basically my server is handling far more spam that it should be, IMO, and it'd be great to see more customizations here. I'll file a feature request for it though πŸ™‚

                  Thanks for clarifying the current behaviour, Girish.

                  --
                  Dustin Dauncey
                  www.d19.ca

                  1 Reply Last reply Reply Quote 0
                  • d19dotca
                    d19dotca @girish last edited by d19dotca

                    @girish A LOT slip through, haha, yeah. I mean most get filed correctly to the Spam folder and maybe 5% passes to the inbox, which is reasonable IMO, but there are too many instances where the mail server is processing clear spam and passing it forward still when I'd prefer that it just be outright denied. My bigger concern is for the mailing list ones as I'm fearful Apple and Google and Microsoft will eventually blocklist my own server IP from all the mail that's being forwarded to their addresses which are clear spam. There are instances already of Google not blocking but greylisting my server because of the spam volume. There's lot of instances where I'm looking at the IP for a spam assault on my server haha and they're blacklisted, but not in Spamhaus Zen yet so they pass through still.

                    --
                    Dustin Dauncey
                    www.d19.ca

                    girish 1 Reply Last reply Reply Quote 0
                    • girish
                      girish Staff @d19dotca last edited by

                      @d19dotca good points. I think allowing setting custom bl servers is complicated. I prefer adding them to the code itself if those lists are reliable. Haraka supports adding multiple servers, so we can easily add it if we can identify another reliable source. For example, I thought spamcop was very reliable and in fact just 2-3 months ago, the site disappeared (and then re-appeared).

                      But I will look into not forwarding emails marked as spam. For some reason, I thought we already did this since I remember reading gmail policy about this.

                      MooCloud_Matt 1 Reply Last reply Reply Quote 1
                      • girish
                        girish Staff last edited by

                        I think https://support.google.com/mail/answer/175365?hl=en was the article. But I recall some other article explicitly recommending not forwarding email without spamassassin checks.

                        1 Reply Last reply Reply Quote 0
                        • MooCloud_Matt
                          MooCloud_Matt @girish last edited by

                          @girish
                          +1 for not let add additional DNSBL.

                          i think that DNSBL check is good, but for example, spamhaus.org have multiple modules that you can use, some DNSBL use custom score, so I think that let people add list as they like just create more mess then use just a good one.

                          And you should add them also to the unbound server so that he can cache the requests.

                          Matteo. R.
                          Founder and Tech-Support Manager.
                          MooCloud MSP
                          Swiss Managed Service Provider

                          1 Reply Last reply Reply Quote 0
                          • MooCloud_Matt
                            MooCloud_Matt @d19dotca last edited by

                            @d19dotca
                            i advise you to update regularly your custom rules on SpamAssassin to add new words used and improve your filtering base on your own experience with spam.

                            You can also use a Domain provider that protects your email and contact data on whois, which is the main cause of spam.

                            Matteo. R.
                            Founder and Tech-Support Manager.
                            MooCloud MSP
                            Swiss Managed Service Provider

                            d19dotca 1 Reply Last reply Reply Quote 0
                            • girish
                              girish Staff last edited by

                              I have created https://git.cloudron.io/cloudron/box/-/issues/776 to track this

                              1 Reply Last reply Reply Quote 1
                              • d19dotca
                                d19dotca @MooCloud_Matt last edited by d19dotca

                                @moocloud_matt I think you may misunderstand a couple of items here in my messages. I'll try to clarify below.

                                update regularly your custom rules on SpamAssassin to add new words used and improve your filtering base on your own experience with spam

                                The SpamAssassin rules are fine, they properly identify spam and the vast majority of spam make it into the spam box away from the inbox. Identifying spam inside of SpamAsssin isn’t the issue in this case.

                                You can also use a Domain provider that protects your email and contact data on whois, which is the main cause of spam

                                This is something I already do currently when possible, but this is also not the issue here because that spam would be going to me then in that case as it's my contacts on the WHOIS. In my case here, it's my clients being sent the spam. It's basically an attack as far as I can tell on my mail server given the domains that point to my same IP so they just fire away at my server hoping to get some. In some cases, it's definitely a targeted recipient, but it's pretty generic and nothing to do with WHOIS spam in this case.

                                The issue I'm raising is basically two-fold...

                                1. that there's currently no way to prevent messages marked as spam by SpamAssassin to mailing list recipients, and

                                2. that there's no way to deny message processing either with custom lists. Right now Spamhaus Zen is the only list that mail gets outright denied from, despite there being plenty of other lists that marked these mail servers as spammy.

                                I believe an administrator should have the ability to manually add in new checks though to have it denied just like it currently does for Spamhaus Zen. I don't see any reason why that shouldn't be enabled.

                                I'm not wanting Cloudron to make new blocklists mandatory in code where a bunch are used to deny connections, that's agreeably bad, but we should at least have the option. In fact, while I’d certainly enable Spamhaus to deny messages outright, some mail admins may not want to be denying mail at all and it’s a little weird to me that even that is forced in code, honestly. Running a run a mail server is unique to everyone, and that’s why there needs to be customization abilities. I ran a mail server for a few years before moving to Cloudron, and before Cloudron when I was running my own mail server spam was practically never an issue because I took a lot of time to set it well to suit my clients needs and stayed on top of it. Unfortunately much of what I did on my mail server before Cloudron can’t be done in Cloudron.

                                • As seen above, my server is frequently getting "spam assaulted" from a few mail servers in particular that rotate every so often. Spamhaus Zen gets some of them but not many. The ability to add in SORBS for example would be a huge help on my server to outright deny processing of the message. Currently the only workaround I have is to manually add in the IPs of the mail servers to the Network IP Blocklist function, but this is an ever-rotating list I'd basically have to update every few days manually.

                                • Also, since currently half my clients just have mailing lists setup to forward to their own personal email address, and Cloudron doesn'tr currently prevent messages identified as Spam from just the SpamAssassin headers to be denied forwarding, it continues onwards to the Gmail.com or iCloud.com address for example which is not good and easily avoidable if we had the ability to prevent forwarding spam messages, or the ability to add in custom DNSBL checks to outright deny the message to begin with.

                                Ultimately, my problems would be solved with better customization for spam filtering. SpamAssassin rules alone are insufficient due to the inability to prevent messages identified as spam onwards to mailing lists.

                                --
                                Dustin Dauncey
                                www.d19.ca

                                1 Reply Last reply Reply Quote 0
                                • girish
                                  girish Staff last edited by

                                  @d19dotca For SORBS (which seems to be reliable), can you try this:

                                  • docker exec -ti mail /bin/bash
                                  • edit /run/haraka/config/dnsbl.ini
                                  • Change zones to "zen.spamhaus.org;dnsbl.sorbs.net"
                                  • supervisorctl restart haraka

                                  This won't survive restarts, but it will give us a good idea of how effective this is. I put this in our cloudron.io mail server as well (but then again, we don't really get that much spam).

                                  d19dotca 2 Replies Last reply Reply Quote 2
                                  • d19dotca
                                    d19dotca @girish last edited by

                                    @girish Certainly, I’ll try that today and see how it goes and will report back. πŸ™‚ Thanks for being open minded about this.

                                    Side note: I know you wrote above earlier that you want to control this list in code, but that part I actually would respectfully disagree with. Example: some mail server admins never want to deny anything and simply classify as spam and deliver - but that’s also not possible currently given it’s hard coded. Conversely, others may want it just a tad more aggressive at denying connections (such as me especially during β€œspam attacks” like the one that started a few weeks ago on mine) and I can’t add any new lists. I’d hope that if it’s as easy as running those commands to change the behaviour that we’d be able to expose that in the UI. πŸ™‚

                                    --
                                    Dustin Dauncey
                                    www.d19.ca

                                    1 Reply Last reply Reply Quote 1
                                    • d19dotca
                                      d19dotca @girish last edited by d19dotca

                                      @girish Quick update: I changed from SORBS to SpamCop and am trying again, as I already found a false-positive when on SORBS. I checked the IP and it was basically only on SORBS and Backscatter, not SpamCop which means it would have passed as expected. I think this jives with what I thought earlier too but couldn't remember which one, I recall one of them was a bit too aggressive in years past as it often would block even Gmail and Hotmail mail servers which is just not feasible to do since so much legit email comes from them too.

                                      I think it was both SpamAssassin and SpamCop I used on my mail server before Cloudron, so I've set it accordingly now. It now reads zen.spamhaus.org;bl.spamcop.net for the zone.

                                      --
                                      Dustin Dauncey
                                      www.d19.ca

                                      1 Reply Last reply Reply Quote 1
                                      • d19dotca
                                        d19dotca last edited by

                                        Relevant but slightly off-topic, but wanted to share: https://www.intra2net.com/en/support/antispam/index.php

                                        That list essentially monitors many DNSBLs for effectiveness and inaccuracies too (false-positives) using their own network for running the tests on. I find it quite interesting and stumbled into it today again, and I remember seeing it many years ago too. It's always up-to-date data which is interesting.

                                        --
                                        Dustin Dauncey
                                        www.d19.ca

                                        MooCloud_Matt 1 Reply Last reply Reply Quote 0
                                        • MooCloud_Matt
                                          MooCloud_Matt @d19dotca last edited by

                                          @d19dotca
                                          ok,
                                          that's the main reason that pushes us to use a centralized mail gateway, having control over incoming and outgoing traffic is fundamental for provider, and learning+settings are easier to do.
                                          Cloudron with Haraka can't be a replacement of good email proxy or antispam, if you think all the service to prevent spam been send or received are some kind of proxy, for example, rspamd is build to have a demon on the mail server but all the elaboration is done in an external server.

                                          I'm sure that with better setup of DNSBL, URIBL,DCC, and SURBL will be better, but will not resolve the issue and make the setup harder for newbies.

                                          For the fwd issue use a sieve forward from imbox this should prevent email marked as spam to been sent out.

                                          Sorry if miss some point, I'm in paternity leaving so sleep is not a thing. (not for a baby. for now just an adorable husky that doesn't understand that he can sleep at night)

                                          Matteo. R.
                                          Founder and Tech-Support Manager.
                                          MooCloud MSP
                                          Swiss Managed Service Provider

                                          d19dotca 1 Reply Last reply Reply Quote 0
                                          • d19dotca
                                            d19dotca @MooCloud_Matt last edited by d19dotca

                                            @moocloud_matt Congratulations on the new baby, that's awesome news! πŸ‘Ά

                                            will not resolve the issue and make the setup harder for newbies

                                            I don't think I agree with that. Nothing will "resolve the issue" of spam itself (if that's what you meant by "the issue"), spam will never be 100% blocked and there will always be false-positives too. The goal is simply to reduce the level of spam and reducing the level of false-positives (or at least keeping it at an acceptable level), and that's where the extra customization comes into play.

                                            I also don't think it will make it harder for "newbies" at all, because the out-of-the-box Cloudron setup would not change (at least I don't envision it would). Having the ability to set extra DNSBL checks for denying messages before they get processed shouldn't make anything harder for anyone - I should be able to setup a new Cloudron instance just as easily as I can today. Nothing should change there. Only the option to add new DNSBL checks to deny messages for example would be added as a completely optional feature to enable - it'd basically only be touched by "power users" and actual mail administrators who are comfortable making those tweaks and already looking to make such changes in the first place.

                                            I agree though that there's plenty of different ways to improve spam filtering and this is just one of many possible ways that I hope to see (and many others from the community too judging by how many mail improvements / feature requests exist in the Cloudron forum).

                                            For the fwd issue use a sieve forward from imbox this should prevent email marked as spam to been sent out

                                            That is an interesting approach, and I'll consider it. My first thought though is... while it may technically be a valid workaround, in my case I don't think this option is feasible though as I have too many accounts to do this for. I have roughly 20+ recipients on my server who only are setup for mailing lists to forward to their own personal email accounts on common domains (no mailboxes on Cloudron). This workaround means I'd need to setup about 20+ mailboxes, not only that but also set them all up consistently and accurately. This leaves a lot of room for human error in my case and a lot of overhead if I ever wanted to make a quick tweak and keep it consistent across them all. If I only had a few, that'd be no problem, but I think I have too many for that to be feasible in my case, unfortunately. I appreciate the thought there though, I hadn't really considered that as a possible workaround before.

                                            --
                                            Dustin Dauncey
                                            www.d19.ca

                                            MooCloud_Matt 2 Replies Last reply Reply Quote 0
                                            • robi
                                              robi last edited by

                                              it sounds like the mailing list feature just needs to take into consideration the spam score and avoid processing that mail.

                                              usually forwards happen before a sieve filter, so unless you can only fwd things from a specific folder like inbox, it's going to send everything.

                                              Life of Advanced Technology

                                              d19dotca 1 Reply Last reply Reply Quote 0
                                              • d19dotca
                                                d19dotca @robi last edited by

                                                @robi Correct, yes. And I filed a feature request for improving the mail lists when it detects a spam message a few days ago too for anyone else coming across this that wants to vote it up πŸ˜‰ (I see you already did which is awesome)

                                                --
                                                Dustin Dauncey
                                                www.d19.ca

                                                1 Reply Last reply Reply Quote 1
                                                • d19dotca
                                                  d19dotca last edited by d19dotca

                                                  I've filed a formal feature request for the ability to add further DNSBLs at https://forum.cloudron.io/topic/4694/add-dnsbls-to-deny-incoming-spam-messages -- Please upvote if you wish to see this functionality.

                                                  A further update to my testing of manually adding in the DNBSLs:

                                                  • I have settled for now on the following zone, which so far has worked perfectly with no false-positives today with these three DNSBLs set in Haraka (but I'm very carefully monitoring this and may make further tweaks if needed): zen.spamhaus.org;bl.mailspike.net;bl.0spam.org
                                                  • Two previous tests were run with zen.spamhaus.org;dnsbl.sorbs.net but the SORBS quickly caught a false-positive within minutes, so I removed SORBS in favour of zen.spamhaus.org;bl.spamcop.net and while that was much better it still got one false-positive after a few hours so since this is all being tested I opted to try the above and current list of zen.spamhaus.org;bl.mailspike.net;bl.0spam.org which so far has worked perfectly with no false-positives, but am still testing and watching the "denied" mail server logs carefully for any false-positives.

                                                  --
                                                  Dustin Dauncey
                                                  www.d19.ca

                                                  1 Reply Last reply Reply Quote 0
                                                  • d19dotca
                                                    d19dotca last edited by d19dotca

                                                    Latest update:

                                                    • The blacklists still are working perfectly with no false-positives since I changed it to the one above. I also added one late last night (because I saw a few getting through that were clear spam again) the noptr.spamrats.com which according to blacklist checks the spammy servers were listed on it. So now it reads as follows: zen.spamhaus.org;bl.mailspike.net;noptr.spamrats.com;bl.0spam.org - So on the side of denying more spam connections while not having any false-positives, this seems like a huge win. πŸ™‚ Notice the many emails blocked from sources other than Spamhaus Zen, and I've confirmed none of them are false-positives.

                                                    618c7a5a-18b8-470c-9463-5e00375e96ef-image.png

                                                    • On the other side of things... I noticed a clearly-spam message getting past still though and reported it in the other thread.

                                                    --
                                                    Dustin Dauncey
                                                    www.d19.ca

                                                    girish 1 Reply Last reply Reply Quote 2
                                                    • girish
                                                      girish Staff @d19dotca last edited by

                                                      @d19dotca The mailspike is a great find. It seems very professionally done - https://www.mailspike.net/usage.html . Spamrats is also updated - https://spamrats.com/lists.php

                                                      d19dotca 1 Reply Last reply Reply Quote 1
                                                      • d19dotca
                                                        d19dotca @girish last edited by d19dotca

                                                        @girish Yeah Mailspike is a great one. I completely forgot about it until all of this recent testing, haha. I am about 95% sure after visiting it's website that I used that on my email server before Cloudron too and it served really well. It was so many years ago now that I basically forgot which lists I used before, I think I'm basically going through all the same tests as I did about 4 years ago, lol.

                                                        For anyone who doesn't want Mailspike actually denying connections though, it should also work great in SpamAssassin as they already gave all the needed headers for it too so at least it'll help in identifying spam better too, using the text below:

                                                        header RCVD_IN_MSPIKE_BL eval:check_rbl('mspike-lastexternal', 'bl.mailspike.net.')
                                                        tflags RCVD_IN_MSPIKE_BL net
                                                        score RCVD_IN_MSPIKE_BL 3.5
                                                        header RCVD_IN_MSPIKE_WL eval:check_rbl('mspike-lastexternal', 'wl.mailspike.net.')
                                                        tflags RCVD_IN_MSPIKE_WL net
                                                        score RCVD_IN_MSPIKE_WL -2.1
                                                        

                                                        --
                                                        Dustin Dauncey
                                                        www.d19.ca

                                                        1 Reply Last reply Reply Quote 2
                                                        • robi
                                                          robi last edited by

                                                          Woohoo, so much mail goodness, when do we get these as new defaults in Cloudron? πŸ˜‰

                                                          Life of Advanced Technology

                                                          1 Reply Last reply Reply Quote 1
                                                          • MooCloud_Matt
                                                            MooCloud_Matt @d19dotca last edited by

                                                            @d19dotca said in Is there a way to add in more DNSBL / RBL sources?:

                                                            spam will never be 100% blocked and there will always be false-positives too

                                                            Yes, I totally agree with that.

                                                            By big question here is how much is useful to add feature and complexity to cloudron, when there are solutions that are specifically built for that.

                                                            About mailspike and Spamhaus, they both have SpamAssassin module, and if cloudron provides the possibility to disable SA, I think is a grate idea to have them.
                                                            But not as simple DNSBL, but as SA module.

                                                            For spamrats idk, having to many DNSBL or SpamAssassin module will slow down the server a lot, remember for every incoming email, you need to call unbound that check his cache and if doesn't have the record call the DNSBL.
                                                            This happens for every email, and every external check you have, and maybe they are slow at that moment and the request take more time than usual, ... and so on
                                                            Resources usage need to be taken into count.

                                                            Matteo. R.
                                                            Founder and Tech-Support Manager.
                                                            MooCloud MSP
                                                            Swiss Managed Service Provider

                                                            1 Reply Last reply Reply Quote 0
                                                            • MooCloud_Matt
                                                              MooCloud_Matt @d19dotca last edited by MooCloud_Matt

                                                              @d19dotca said in Is there a way to add in more DNSBL / RBL sources?:

                                                              That is an interesting approach, and I'll consider it.

                                                              I know that is not the best solution out there.
                                                              But you can automate it, with API and Sieve is an open protocol.

                                                              But is a start for now, and yes having a mail filter also for mail fwd is a good option, and if you don't need it you should just be able to disable SpamAssassin from the server.

                                                              I really don't like to w8st resources

                                                              Matteo. R.
                                                              Founder and Tech-Support Manager.
                                                              MooCloud MSP
                                                              Swiss Managed Service Provider

                                                              d19dotca 1 Reply Last reply Reply Quote 0
                                                              • d19dotca
                                                                d19dotca @MooCloud_Matt last edited by d19dotca

                                                                @moocloud_matt I think it's safe to say from our conversations in this post and several others related to email improvements that you and I have different views overall on how to run our own mail servers. And that's totally okay! πŸ™‚ That's actually the point... that there is no one-size-fits-all approach to running a mail server. Thus, you should be able to run your mail server how you best see fit for your clients, and how you run your mail server may not be the best way for me to run mine for my clients. Options are the key here, and that's what I'm trying to make sure is understood. It should all be optional and there should be no changes to the defaults on a new install, IMO. In fact I'd go to far to say that even Spamhaus shouldn't be used by default to block at the MTA level.

                                                                In my case as an example, I've had a lot of email denied now at the MTA level by adding in the extra DNSBLs, and they've worked perfectly with zero false-positives so far in nearly 48 hours of running it on a very actively used mail server. So I see that as a big win for me and my clients. And of course I can always reconsider if I start seeing too many false-positives (or heck, even one false-positives for that matter - which I've yet to see so far in my testing), and that's kind of another example of the point I'm making here too... that we need options so we can enable and disable at will. In other words, none of what I'm asking for or proposing should be enabled by default, and none of it should be hard-coded as it currently is - we should be exposing this to users so they can make the decisions that best suits their own needs. πŸ™‚

                                                                I know you seem worried about "complexity to cloudron" which is a fair concern always on how to keep Cloudron user-friendly, however I don't quite share that one this time because the data is already in Cloudron (it already uses Spamhaus Zen for blocking at MTA level and as we've seen is easily configurable already - it just doesn't survive restarts the way we can manually do it now), so the existing functionality just needs to be exposed in the UI. It's really no different than how they had recently exposed the SpamAssassin configuration too. It just needs a new little line on the Mail page with the other settings for message sized and such, a toggle and a box to list new DNSBLs. Of course it's more complicated than that in the backend, but from a user perspective this should not add any additional complexities. And I'd assume even in the backend once it's done it's done and shouldn't really need to be maintained at all.

                                                                All to say... we should have options available to us to benefit all of us uniquely as running a mail server is a complex task where it is not feasible to use a one-size-fits-all approach, IMO. πŸ™‚

                                                                --
                                                                Dustin Dauncey
                                                                www.d19.ca

                                                                1 Reply Last reply Reply Quote 2
                                                                • d19dotca
                                                                  d19dotca last edited by d19dotca

                                                                  Latest update. I've been trialling a few different DNSBLs for use here at the MTA level for denying connections. So far I am very happy to say that I've had zero false-positives in over 3 days now on a very active mail server. This makes me very comfortable that this is a very safe configuration, but at the same itme would not necessarily recommend we make these default at all (in fact I don't even know if Spamhaus should be enabled by default to be totally honest- I really think that should be up to the mail admin).

                                                                  The zone I've settled on and most recently been using...

                                                                  black.junkemailfilter.com;bl.mailspike.net;all.spamrats.com;zen.spamhaus.org

                                                                  To give a bit of context to each of them...

                                                                  • black.junkemailfilter.com was one I added the other day because of a clearly-spam message getting through and I had seen it in the "just now" timeframe so quickly checked mxtoolbox.com to see which DNSBL had it listed, and four of them did. This was one of them. The other was Barracuda which requires registration that I didn't want to try yet, and the other two were UCEPROTECT-2 and UCEPROTECT-3 which I didn't want to use (see side note at bottom). It’s caught a fair bit of spam the others didn’t catch earlier. This is known as "JMF-Black" on the Intra2Net list and has zero false-positives.

                                                                  • bl.mailspike.net is one that seems very useful and is very accurate, has helped block a lot that Spamhaus Zen didn't catch earlier (before I had the zen.spamhaus.org earlier in the list meaning it should be checked first so we'd know if anything else blocked it that Spamhaus Zen didn't have yet). This blocked a good amount of spam. Zero false-positives.

                                                                  • all.spamrats.com is also an excellent one and I'd say blocked as many as Spamhaus Zen did, it was catching spam frequently. Earlier I was using the noptr.spamrats.com which worked very well too but later learned of the broader all.spamrats.com and started using that yesterday with continued success and zero false-positives still.

                                                                  • zen.spamhaus.org needs no introduction, it's probably the most popular DNSBL ever created. Much like the others, it's highly accurate and hasn't been seen to have any false-positives.

                                                                  For all of the above and more, I'd recommend checking out the Intra2Net service which monitors the accuracy of the various DNSBLs. My recommendation is to stick to ones if blocking from the MTA level that has a 0% or at least no higher than a 0.05% chance of false-positives as they'd be considered safe. All the other DNSBLs that are more aggressive should really just go a step down to the SpamAssassin level for scoring metrics there (which I later did too for the URIBLs instead and they've been great so far too). I did have two false-positives in very early tests with the dnsbl.sorbs.net one and also with bl.spamcop.net one. I'd suggest avoiding those for denying connections but can be used in SpamAssassin instead.

                                                                  I hope the above is a good test report for people, and others will hopefully find this helpful. Certainly there is no one-size-fits-all approach here, and I'd argue that none of these should even be enabled by default, however I believe them all to be "safe" based on my own experience and was glad to see the spam cut down further than it was prior to adding in the additional DNSBLs. What works for me may not work for you of course, depends likely on a lot of different factors, so "your mileage may vary" as they say. I've been watching the logs like a hawk all week and been checking every single "denied" entry and happy to report no false-positives in my testing so far over the last few days using the four listed above.

                                                                  Here’s a quick screenshot:

                                                                  C2A66D4D-8E31-491C-B1B0-2AC804F66093.jpeg


                                                                  Side note regarding UCEPROTECT DNSBLs: I strongly discourage use of the UCEPROTECT-* lists except possibly UCEPROTECT-1, because the level 2 and 3 seem to just blacklist large IP ranges that affect entire providers such as DigitalOcean, OVH, and more and basically demand fees for "express delisting" which doesn't even guarantee anything as it can be re-listed the next day. I question the ethics of that particular DNSBL provider as they seem to "extort" money from large network providers, and there is also this article I found that pretty much strips them apart line by line and explains why they may not be good to trust or use. My advice is to stay away from the UCEPROTECT DNSBLs based on the above plus they'd surely have a fairly high false-positive rate (you can see from the link above that the UCEPROTECT-3 has a whopping 17% inaccuracy rate.


                                                                  @girish - Hopefully the above report will be useful for you and @nebulon when discussing some of the mail changes that may be coming in 6.3 there. πŸ™‚


                                                                  Update - March 26, 2021: Out of thousands of emails over the last week, I've only found two false-positives (thankfully non-critical emails, one was a Snapchat newsletter for example). That is a very impressive result to me and my users and I'm pleased with that as that seems to be within the reasonable threshold when weighing the pros and cons.

                                                                  With that said, I have started a second test which involves removing one of the DNSBLs which made the false-positive result, and then added instead to the SpamAssassin side of things to at least help with identifying spam better to avoid the inbox. While this has led to more spam processing on my server, it seems to still be working well to achieving the ultimate goal of keeping spam messages out of my users inboxes. Here is my current DNSBL list zones in effect: zen.spamhaus.org;bl.mailspike.net;noptr.spamrats.com (notice I removed the black.junkemailfilter.com and changed from all.spamrats.com back to noptr.spamrats.com)

                                                                  So far the results are good. This however means unfortunately some of my clients who have mailing lists on the server that forward to their personal accounts elsewhere are receiving a bit more spam again until the new feature request is implemented to prevent external spam messages from being sent.

                                                                  Depending on the results of the above tests, I may either stick to the current implementation or go back to how it was last week.

                                                                  --
                                                                  Dustin Dauncey
                                                                  www.d19.ca

                                                                  necrevistonnezr JOduMonT 2 Replies Last reply Reply Quote 4
                                                                  • necrevistonnezr
                                                                    necrevistonnezr @d19dotca last edited by

                                                                    @d19dotca
                                                                    Many thanks for these detailed tests and documentation!

                                                                    1 Reply Last reply Reply Quote 2
                                                                    • d19dotca
                                                                      d19dotca last edited by d19dotca

                                                                      Wanted to write a quick update: Anyone wanting to enable an RBL can now do so very easily in the new 7.x Cloudron version! Big thanks to the Cloudron team for implementing that feature! πŸ™‚

                                                                      Since many visit this thread (it's even linked in the documentation now too!) for the list of the various RBLs and experience with them, reviewing them, etc... I wanted to add one more to the list which I've been testing out for a little bit and so far seems great, blocking spam from bad IPs which even hours later still isn't on some of the other popular blacklists when I've been checking manually to verify things.

                                                                      Abusix is a premium service, however they do have a free tier which offers a rather large 5,000 queries per day - and I suspect most of us are not close to that amount in a single day, many likely not even over the course of a week - effectively meaning we can get premium-level spam filtering for free. They have several different lists they manage, but the recommended one to use is their combined.mail.abusix.zone zone which checks three separate lists of theirs out of the several. It is their "recommended" one for production servers offering a good balance of more checks and performance using one single lookup zone without being too overbearing as to include false-positives, this way it greatly limits any false-positives (of which I've seen zero so far!). πŸ™‚

                                                                      The only downside is a very minor cosmetic issue in Cloudron with it as the Abusix list is something like <UUID>.combined.mail.abusix.zone since it's premium so it's a unique URL to every user, and as such it's a very long URL due to the UUID which means some of the log entries in Cloudron's UI for denied messages get pretty long looking. I may file a feature request later for us to perhaps try naming our zones how owe want them to so we can avoid really long named ones in the logs, but overall it's just a cosmetic issue and nothing else.

                                                                      So just to summarize, the ones I'm using with great success so far are the following:

                                                                      <UUID>.combined.mail.abusix.zone
                                                                      zen.spamhaus.org
                                                                      bl.mailspike.net
                                                                      

                                                                      --
                                                                      Dustin Dauncey
                                                                      www.d19.ca

                                                                      necrevistonnezr 2 Replies Last reply Reply Quote 7
                                                                      • necrevistonnezr
                                                                        necrevistonnezr @d19dotca last edited by

                                                                        In case you’re wondering: https://docs.cloudron.io/email/#dnsbl

                                                                        Thanks to @d19dotca and the Cloudron Team!

                                                                        d19dotca 1 Reply Last reply Reply Quote 3
                                                                        • d19dotca
                                                                          d19dotca @necrevistonnezr last edited by

                                                                          @necrevistonnezr Ah thank you! I should have included that. Edited my earlier comment to include that link too now.

                                                                          --
                                                                          Dustin Dauncey
                                                                          www.d19.ca

                                                                          1 Reply Last reply Reply Quote 0
                                                                          • JOduMonT
                                                                            JOduMonT @d19dotca last edited by

                                                                            @d19dotca said in Is there a way to add in more DNSBL / RBL sources?:

                                                                            black.junkemailfilter.com;bl.mailspike.net;all.spamrats.com;zen.spamhaus.org

                                                                            interesting, I never eared about mailspike before πŸ˜‰
                                                                            mainly I use [dbl and zen]spamhaus + barracudacentral.org

                                                                            Also on wikipedia they have a great listing with nice description:
                                                                            https://en.wikipedia.org/wiki/Comparison_of_DNS_blacklists

                                                                            1 Reply Last reply Reply Quote 2
                                                                            • necrevistonnezr
                                                                              necrevistonnezr @d19dotca last edited by

                                                                              @d19dotca I found zen.spamhaus.org way too agressive, blocking several non-spam messages from even reaching the server.

                                                                              @girish BTW is there a way to manually release "blocked" messages or let them through?

                                                                              girish d19dotca 2 Replies Last reply Reply Quote 3
                                                                              • girish
                                                                                girish Staff @necrevistonnezr last edited by

                                                                                @necrevistonnezr The DNSBL IP check is done as the very first thing as soon as the sender connects. So, the mail is not even received and thus not stored anywhere.

                                                                                1 Reply Last reply Reply Quote 3
                                                                                • d19dotca
                                                                                  d19dotca @necrevistonnezr last edited by

                                                                                  @necrevistonnezr I found Abusix to be superior to Spamhaus Zen, and use that as my only DNSBL at the moment with everything else going through the SpamAssassin rules.

                                                                                  --
                                                                                  Dustin Dauncey
                                                                                  www.d19.ca

                                                                                  MooCloud_Matt 1 Reply Last reply Reply Quote 1
                                                                                  • MooCloud_Matt
                                                                                    MooCloud_Matt @d19dotca last edited by MooCloud_Matt

                                                                                    @d19dotca
                                                                                    We use abusix with rspamd, and after the implementation on the new signature base list, the quality improvement.

                                                                                    But abusix with spamassasin, is good but still got a lot of spam passing, because SA don't support signature base abusix list.

                                                                                    Matteo. R.
                                                                                    Founder and Tech-Support Manager.
                                                                                    MooCloud MSP
                                                                                    Swiss Managed Service Provider

                                                                                    1 Reply Last reply Reply Quote 0
                                                                                    • First post
                                                                                      Last post
                                                                                    Powered by NodeBB