Cloudron makes it easy to run web apps like WordPress, Nextcloud, GitLab on your server. Find out more or install now.


Skip to content
  • Categories
  • Recent
  • Tags
  • Popular
  • Bookmarks
  • Search
Skins
  • Light
  • Cerulean
  • Cosmo
  • Flatly
  • Journal
  • Litera
  • Lumen
  • Lux
  • Materia
  • Minty
  • Morph
  • Pulse
  • Sandstone
  • Simplex
  • Sketchy
  • Spacelab
  • United
  • Yeti
  • Zephyr
  • Dark
  • Cyborg
  • Darkly
  • Quartz
  • Slate
  • Solar
  • Superhero
  • Vapor

  • Default (No Skin)
  • No Skin
Collapse
Brand Logo

Cloudron Forum

Apps | Demo | Docs | Install
  1. Cloudron Forum
  2. Feature Requests
  3. Cloudron email: feature improvements/ideas

Cloudron email: feature improvements/ideas

Scheduled Pinned Locked Moved Feature Requests
emailfeature-request
37 Posts 11 Posters 3.4k Views 12 Watching
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • jimcavoliJ Offline
      jimcavoliJ Offline
      jimcavoli
      App Dev
      wrote on last edited by girish
      #1

      I recently stumbled across the MIT-licensed SPFToolbox project and it got me thinking a lot more about more comprehensive mail server features and management tools that would be good to have in Cloudron. SPFToolbox has some nice functionality for running checks for all sorts of things, but its blacklist checker is what started to get my mind running about that feature and potentially others that would be awesome additions to the Cloudron email system to help administrators manage deliverability and so on.

      This probably starts to creep into a bigger re-vamp of the email management area of the product, but IMO since mail servers objectively suck to maintain, the ease of use of the Cloudron mail system is a huge selling point. While it's probably not realistic to target everything that services like MXToolbox do, but there could be some more built-in and automated to make life even easier.

      Some thoughts out of the gates:

      • Blacklist monitoring
      • Rspamd - Haraka is ready (https://github.com/haraka/haraka-plugin-rspamd) - with some of its handy modules:
        • Possibly include web ui to shortcut the configuration management; definitely an "advanced" option though
        • Antivirus - https://rspamd.com/doc/modules/antivirus.html
        • Phishing - https://rspamd.com/doc/modules/phishing.html
        • Greylisting - https://rspamd.com/doc/modules/greylisting.html
        • Metric Exporter - https://rspamd.com/doc/modules/metric_exporter.html
          • would let rspamd feed metrics into graphite for graphing in the box ui
        • I'd favor probably pushing some of what are currently haraka plugins' responsibilities around the SPF/DKIM/DMARC space down to rspamd as well given its much higher configurability/visibility if well-implemented
      • Perhaps getting into automatic cryptographic signing/encryption with something like CipherMail in the mix - definitely needs some more thought
        • This might be best done by rethinking the way haraka fits in the MTA chain - maybe making other MTA-interfaced apps available via the app store like this one would be a good thing, then in the mail server admin, you could choose which one(s) of those installed are inserted for each domain?

      I'm just sort of thinking out loud here, and would like to see what others think about those ideas or any other ideas about the mail server here.

      1 Reply Last reply
      6
      • girishG Offline
        girishG Offline
        girish
        Staff
        wrote on last edited by
        #2

        Thanks for the ideas!

        • Blacklist monitoring - would be great if we can find some service/API endpoint we can call to check the domain/IP against. We can integrate this into the Status tab.
        • Do you get a lot of spam through in the inbox? If so, a first step would be to understand why SA doesn't work before/if we switch to rspamd. So far the main complaint on support was visibility into the delivery/activity of mail server and this is why we added the event log in 5.0.
        • Security: on the chat, one user brought up https://www.hardenize.com/blog/mta-sts . Have to look into this more.
        d19dotcaD necrevistonnezrN 2 Replies Last reply
        0
        • jimcavoliJ Offline
          jimcavoliJ Offline
          jimcavoli
          App Dev
          wrote on last edited by jimcavoli
          #3

          The code from SPFToolbox just kicks back JSON - maybe the author would be okay with putting the hosted version up to service those API requests? Alternately, it could be deployed centrally for support or as an internal tool for checking DNS stuff generally within box.

          I don't think that SpamAssassin doesn't do a decent job already anyway for most of the accounts I manage; it just has to be trained. I think the value of rspamd is much more on its transparency and ease of management over a wider breadth of features. I postulate that giving it more delegated responsibility from haraka would pass those benefits on to the mail server solution overall.

          MTA-STS would be a great add as well! Hadn't even thought of that

          1 Reply Last reply
          1
          • girishG girish

            Thanks for the ideas!

            • Blacklist monitoring - would be great if we can find some service/API endpoint we can call to check the domain/IP against. We can integrate this into the Status tab.
            • Do you get a lot of spam through in the inbox? If so, a first step would be to understand why SA doesn't work before/if we switch to rspamd. So far the main complaint on support was visibility into the delivery/activity of mail server and this is why we added the event log in 5.0.
            • Security: on the chat, one user brought up https://www.hardenize.com/blog/mta-sts . Have to look into this more.
            d19dotcaD Offline
            d19dotcaD Offline
            d19dotca
            wrote on last edited by d19dotca
            #4

            @girish I may be off-topic but I’d like to add that that the current issue with spam control is there’s almost no configuration available, meaning I can’t do any of the following (or if I am able to then I’m not currently aware of how to do them currently):

            1. Set the aggressiveness of the filtering
            2. Make my own blacklist of domains or email addresses
            3. Make my own whitelist of domains or email addresses
            4. Enable or disable different rule sets such as changing how many “points” a certain rule will add to the spam results for example

            On top of that, might just be me, but I find the learning takes longer than desired too. It seems like it needs a lot of the same type of message before it starts to filter that type of message out of my inbox.

            Don’t get me wrong, it works decently well out of box, I think it could just be greatly improved (from various different angles) as well, with the list above being a nice addition to giving more control over spam handling in the Cloudron server.

            For what it’s worth, rspamd also seems to offer graphs so an admin can quickly see at a glance how much is identified as spam, how much is greylisted (does greylisting even occur in Cloudron right now?), and how much is non-spam. While this isn’t required info it can be nice to have some statistics so people can get a better idea if it’s improving or not, especially if they’re wanting to make changes and run some comparisons, etc.

            --
            Dustin Dauncey
            www.d19.ca

            1 Reply Last reply
            0
            • girishG girish

              Thanks for the ideas!

              • Blacklist monitoring - would be great if we can find some service/API endpoint we can call to check the domain/IP against. We can integrate this into the Status tab.
              • Do you get a lot of spam through in the inbox? If so, a first step would be to understand why SA doesn't work before/if we switch to rspamd. So far the main complaint on support was visibility into the delivery/activity of mail server and this is why we added the event log in 5.0.
              • Security: on the chat, one user brought up https://www.hardenize.com/blog/mta-sts . Have to look into this more.
              necrevistonnezrN Offline
              necrevistonnezrN Offline
              necrevistonnezr
              wrote on last edited by
              #5

              @girish said in Cloudron email: feature improvements/ideas:

              • Security: on the chat, one user brought up https://www.hardenize.com/blog/mta-sts . Have to look into this more.

              See this article by Tutanota as a starting point: https://tutanota.com/blog/posts/tutanota-letsencryt-launch-mta-sts-tls/

              1 Reply Last reply
              0
              • MallewaxM Offline
                MallewaxM Offline
                Mallewax
                wrote on last edited by
                #6

                What about full text search for Cloudron e-mail boxes? That is probably the only "one real feature" I am missing.
                Thanks to Cloudron I have successfully "de-googled". It is just that I came to appreciate FTS while I was still on google mail.
                Is this in the cards?

                jdaviescoatesJ d19dotcaD 2 Replies Last reply
                3
                • MallewaxM Mallewax

                  What about full text search for Cloudron e-mail boxes? That is probably the only "one real feature" I am missing.
                  Thanks to Cloudron I have successfully "de-googled". It is just that I came to appreciate FTS while I was still on google mail.
                  Is this in the cards?

                  jdaviescoatesJ Offline
                  jdaviescoatesJ Offline
                  jdaviescoates
                  wrote on last edited by
                  #7

                  @alotz said in Cloudron email: feature improvements/ideas:

                  Is this in the cards?

                  Yes, as you've spotted it was mentioned in what's coming in 4.5 announcement thread.

                  I use Cloudron with Gandi & Hetzner

                  1 Reply Last reply
                  0
                  • MallewaxM Mallewax

                    What about full text search for Cloudron e-mail boxes? That is probably the only "one real feature" I am missing.
                    Thanks to Cloudron I have successfully "de-googled". It is just that I came to appreciate FTS while I was still on google mail.
                    Is this in the cards?

                    d19dotcaD Offline
                    d19dotcaD Offline
                    d19dotca
                    wrote on last edited by
                    #8

                    @Mallewax Do you mind providing an example or explaining what FTS is exactly? I ask because I always thought this feature was more client-oriented. For example, I can do complete text searches via the Apple Mail client and find any message I’m looking for. So what does adding FTS on the Cloudron side do in this case? I’ve never had a problem finding a message. Maybe I’m thinking of something totally different though? Just want to make sure I’m on the same page on this full text search thing.

                    --
                    Dustin Dauncey
                    www.d19.ca

                    mehdiM 1 Reply Last reply
                    0
                    • d19dotcaD d19dotca

                      @Mallewax Do you mind providing an example or explaining what FTS is exactly? I ask because I always thought this feature was more client-oriented. For example, I can do complete text searches via the Apple Mail client and find any message I’m looking for. So what does adding FTS on the Cloudron side do in this case? I’ve never had a problem finding a message. Maybe I’m thinking of something totally different though? Just want to make sure I’m on the same page on this full text search thing.

                      mehdiM Offline
                      mehdiM Offline
                      mehdi
                      App Dev
                      wrote on last edited by
                      #9

                      @d19dotca I guess it's not the same thing when you are using a webmail client instead of a desktop app 🤷

                      d19dotcaD 1 Reply Last reply
                      0
                      • mehdiM mehdi

                        @d19dotca I guess it's not the same thing when you are using a webmail client instead of a desktop app 🤷

                        d19dotcaD Offline
                        d19dotcaD Offline
                        d19dotca
                        wrote on last edited by
                        #10

                        @mehdi But even in Roundcube for example, I’ve never struggled to find a message I search for. That’s why I’m confused on what the advantage to FTS really is. Maybe I’ve just had good luck? lol. I mean if it improves search on webmail providers than cool, but I just wonder if it really would because the webmail apps are also just clients and I always thought this was a client-side feature, not server-side.

                        --
                        Dustin Dauncey
                        www.d19.ca

                        1 Reply Last reply
                        0
                        • N Offline
                          N Offline
                          NCKNE
                          wrote on last edited by
                          #11

                          The IMAP protocol (also used by webmail) can offer a server side search for the clients to offload the search for the client to the server. Using full blown clients like Apple Mail do not really need this as Spotlight on Mac OS already indexes all mails, but especially for webmail/groupware clients this is very useful.

                          d19dotcaD 1 Reply Last reply
                          0
                          • N NCKNE

                            The IMAP protocol (also used by webmail) can offer a server side search for the clients to offload the search for the client to the server. Using full blown clients like Apple Mail do not really need this as Spotlight on Mac OS already indexes all mails, but especially for webmail/groupware clients this is very useful.

                            d19dotcaD Offline
                            d19dotcaD Offline
                            d19dotca
                            wrote on last edited by
                            #12

                            @NCKNE Sure, but as I mentioned earlier, I have never personally had an issue with the webmail clients either. I've always been able to find the message/email I was looking for (I was using Roundcube specifically more than Rainloop if that matters).

                            So I guess I have to ask... what problem are we actually trying to solve here with FTS? I'm just not convinced there's a problem to begin with. I worry that we're asking for something just because we think it should be available but without really grasping the full picture. Is there an example I could try with in webmail where I cannot find a message based on a particular set of data? Maybe I just need to experience this myself before I recognize the benefit of FTS.

                            --
                            Dustin Dauncey
                            www.d19.ca

                            N 1 Reply Last reply
                            0
                            • d19dotcaD d19dotca

                              @NCKNE Sure, but as I mentioned earlier, I have never personally had an issue with the webmail clients either. I've always been able to find the message/email I was looking for (I was using Roundcube specifically more than Rainloop if that matters).

                              So I guess I have to ask... what problem are we actually trying to solve here with FTS? I'm just not convinced there's a problem to begin with. I worry that we're asking for something just because we think it should be available but without really grasping the full picture. Is there an example I could try with in webmail where I cannot find a message based on a particular set of data? Maybe I just need to experience this myself before I recognize the benefit of FTS.

                              N Offline
                              N Offline
                              NCKNE
                              wrote on last edited by
                              #13

                              @d19dotca For small mailboxes, the client side webmail search might be good enough. Implementing a server side search can speed up the fulltext (eg. text within the mail body) search by a lot though. I am personally more keen on the implementation itself (solr or elastic search) as this core service could be offered to other apps such as Nextcloud for a fulltext search in documents as well.

                              MallewaxM 1 Reply Last reply
                              1
                              • N NCKNE

                                @d19dotca For small mailboxes, the client side webmail search might be good enough. Implementing a server side search can speed up the fulltext (eg. text within the mail body) search by a lot though. I am personally more keen on the implementation itself (solr or elastic search) as this core service could be offered to other apps such as Nextcloud for a fulltext search in documents as well.

                                MallewaxM Offline
                                MallewaxM Offline
                                Mallewax
                                wrote on last edited by
                                #14

                                @NCKNE Thank you, this is exactly my point, too. Sorry for not having been clearer. It is mostly a performance issue. IMAP Server-side search works sequential, without an Index.
                                For smaller mailboxes not so much an issue, but for larger ones very clearly noticeable.

                                Here some more background:
                                https://doc.dovecot.org/configuration_manual/fts/

                                Also, before I switched to Cloudron for my e-mail needs I got my feet wet and installed an e-mail server from scratch by following

                                https://workaround.org/ispmail/buster/

                                This was mostly for educating myself. I then migrated my ~ 7 GB Gmail over to this server, and this is when I realized that there is a huge perforamnce difference in search speed. The set up at "workaround" is just a "vanilla" Dovecot IMAP server.

                                I then managed to install Apache/Solr on this setup and it is then when I realized how much IMAP search improves.
                                From minutes to seconds...

                                Hope this helps to shed more light on my desire...:-)

                                Cloudron rocks and I am super happy with what I have now....even without FTS (yet)...

                                d19dotcaD 1 Reply Last reply
                                1
                                • MallewaxM Mallewax

                                  @NCKNE Thank you, this is exactly my point, too. Sorry for not having been clearer. It is mostly a performance issue. IMAP Server-side search works sequential, without an Index.
                                  For smaller mailboxes not so much an issue, but for larger ones very clearly noticeable.

                                  Here some more background:
                                  https://doc.dovecot.org/configuration_manual/fts/

                                  Also, before I switched to Cloudron for my e-mail needs I got my feet wet and installed an e-mail server from scratch by following

                                  https://workaround.org/ispmail/buster/

                                  This was mostly for educating myself. I then migrated my ~ 7 GB Gmail over to this server, and this is when I realized that there is a huge perforamnce difference in search speed. The set up at "workaround" is just a "vanilla" Dovecot IMAP server.

                                  I then managed to install Apache/Solr on this setup and it is then when I realized how much IMAP search improves.
                                  From minutes to seconds...

                                  Hope this helps to shed more light on my desire...:-)

                                  Cloudron rocks and I am super happy with what I have now....even without FTS (yet)...

                                  d19dotcaD Offline
                                  d19dotcaD Offline
                                  d19dotca
                                  wrote on last edited by
                                  #15

                                  @Mallewax what would you consider a small mailbox? Mine is over 6 GB right now and again - no noticeable issue at all. Everything I can find very quickly in both Apple Mail and webmail on Cloudron. I just can’t see the benefit yet, still.

                                  --
                                  Dustin Dauncey
                                  www.d19.ca

                                  MallewaxM 1 Reply Last reply
                                  0
                                  • d19dotcaD d19dotca

                                    @Mallewax what would you consider a small mailbox? Mine is over 6 GB right now and again - no noticeable issue at all. Everything I can find very quickly in both Apple Mail and webmail on Cloudron. I just can’t see the benefit yet, still.

                                    MallewaxM Offline
                                    MallewaxM Offline
                                    Mallewax
                                    wrote on last edited by
                                    #16

                                    @d19dotca That's quite interesting. 6 GB is certainly a big mailbox. And that you find everything fast on Apple Mail is not surprising, as Spotlight Index works on your mailbox. I am just surprised that the same seems to work for you on Cloudron (Roundcube or Rainloop, I suppose).
                                    At any rate, I can only refer to the Dovecot reference again, where the developers themselves explain how indexing solutions improve the search speed.

                                    1 Reply Last reply
                                    1
                                    • marcusquinnM Offline
                                      marcusquinnM Offline
                                      marcusquinn
                                      wrote on last edited by marcusquinn
                                      #17

                                      Great thread!

                                      I kinda like these guys attitude to Spam filtering: https://posteo.de/en/site/faq

                                      (they run on Roundcube for webmail too, so decent endorsement)

                                      "Is there a spam folder?
                                      No, with Posteo there is no spam folder. This was a conscious decision we made. It has advantages for our users. Messages that are not spam can end up in spam folders, where they remain unattended to. The sender thinks that the emails were delivered and read, which often leads to problems.
                                      With Posteo, there are only delivered or rejected emails.
                                      Our spam filter conducts checks against various criteria in the process of receiving an email. If an email is classified as spam, it will not even arrive. The sending server can not deliver the email and informs the sender of this.
                                      The sender can either ignore the reason for classification as spam or try to contact you via a different method. In any case, they receive a notification to say that the email could not be delivered. This process ensures that no email goes missing."

                                      Web Design https://www.evergreen.je
                                      Development https://brandlight.org
                                      Life https://marcusquinn.com

                                      d19dotcaD 1 Reply Last reply
                                      2
                                      • marcusquinnM marcusquinn

                                        Great thread!

                                        I kinda like these guys attitude to Spam filtering: https://posteo.de/en/site/faq

                                        (they run on Roundcube for webmail too, so decent endorsement)

                                        "Is there a spam folder?
                                        No, with Posteo there is no spam folder. This was a conscious decision we made. It has advantages for our users. Messages that are not spam can end up in spam folders, where they remain unattended to. The sender thinks that the emails were delivered and read, which often leads to problems.
                                        With Posteo, there are only delivered or rejected emails.
                                        Our spam filter conducts checks against various criteria in the process of receiving an email. If an email is classified as spam, it will not even arrive. The sending server can not deliver the email and informs the sender of this.
                                        The sender can either ignore the reason for classification as spam or try to contact you via a different method. In any case, they receive a notification to say that the email could not be delivered. This process ensures that no email goes missing."

                                        d19dotcaD Offline
                                        d19dotcaD Offline
                                        d19dotca
                                        wrote on last edited by d19dotca
                                        #18

                                        @marcusquinn If that is the method we want to go with, I'd suggest that before it's added as a feature that admins have the ability to fully control/dictate what we want to see rejected at the SMTP level. In general, it's considered "bad practice" to block at the SMTP level although it's also a common practice to still use at least one DNBSL like SpamHaus or SORBS, etc.

                                        @girish, any way we can possibly get better spam control added to 6.0 or 6.1? It'd be great to see the spam controls open up and be more customizable to admins of Cloudron servers. I know this particular thread was started just a few months ago but myself and others have requested improvements back in 2019 too: here & here and I'm sure there's a few I missed too.

                                        --
                                        Dustin Dauncey
                                        www.d19.ca

                                        marcusquinnM 1 Reply Last reply
                                        2
                                        • d19dotcaD d19dotca

                                          @marcusquinn If that is the method we want to go with, I'd suggest that before it's added as a feature that admins have the ability to fully control/dictate what we want to see rejected at the SMTP level. In general, it's considered "bad practice" to block at the SMTP level although it's also a common practice to still use at least one DNBSL like SpamHaus or SORBS, etc.

                                          @girish, any way we can possibly get better spam control added to 6.0 or 6.1? It'd be great to see the spam controls open up and be more customizable to admins of Cloudron servers. I know this particular thread was started just a few months ago but myself and others have requested improvements back in 2019 too: here & here and I'm sure there's a few I missed too.

                                          marcusquinnM Offline
                                          marcusquinnM Offline
                                          marcusquinn
                                          wrote on last edited by
                                          #19

                                          @d19dotca Yup, anything potentially destructive should always be opt-in. There will always be preferences but having the options helps please more of the people more of the time eh 🙂

                                          Web Design https://www.evergreen.je
                                          Development https://brandlight.org
                                          Life https://marcusquinn.com

                                          1 Reply Last reply
                                          1
                                          • girishG Offline
                                            girishG Offline
                                            girish
                                            Staff
                                            wrote on last edited by
                                            #20

                                            This idea comes from @MooCloud_Matt but he did suggest that for advanced users we should allow the spam filtering to be "outsourced" to a service. For example, apparently Proxmox Mail Gateway can do a lot of the greylisting etc. Would you guys be more interested in building this into Cloudron or integrating with an external service? An angle here is that this external service can learn across all your clients and not just single Cloudron instance like it is right now. To be clear, we won't be providing or building this external service. We will just be integrating with it (like different DNS providers, storage providers, this is just a Spam filtering provider)

                                            d19dotcaD 1 Reply Last reply
                                            1
                                            Reply
                                            • Reply as topic
                                            Log in to reply
                                            • Oldest to Newest
                                            • Newest to Oldest
                                            • Most Votes


                                              • Login

                                              • Don't have an account? Register

                                              • Login or register to search.
                                              • First post
                                                Last post
                                              0
                                              • Categories
                                              • Recent
                                              • Tags
                                              • Popular
                                              • Bookmarks
                                              • Search