How does spam work with special mail folders?

girish

@d19dotca Thanks for the details! Let us get back by testing it out on a Mac.

imc67

Hi @girish whats the status of this investigation because today I noticed the same issue?

Kind regards,

Marcel

d19dotca

Hey @girish - it's almost a year since this topic was posted. Last it left off was you running some tests. It's a minor issue but would be very helpful if we could choose our own special spam folder so that it works better with iOS and macOS (iOS in particular as on iOS you can't actually pick the spam folder). But if Cloudron won't let us choose the spam folder that the spam learning will learn from, then there's no point and we'd need to wait for that feature to be introduced. Any insight into this one?

girish

@d19dotca @imc67 I don't have a Mac or iOS to test but the IMAP configuration is just:

    mailbox Spam {
        auto = subscribe
        special_use = \Junk
    }

So, the directory is called Spam and special use flag is set per RFC 6154. Is that not enough for the Mail client?

On Thunderbird, I need to configure it like this:

d19dotca

@girish That works on macOS the same way as it does for you on Thunderbird where one can set the junk folder. Unfortunately on iOS, it insists on creating its own junk folder if it doesn’t find one when using the junk button, and so it can’t be set to use the special spam folder that was auto-generated in Cloudron. This means on iOS I have to take a few extra steps/taps to file the message away in the spam folder, because if I hit the designated junk button it will create its own folder since one does not exists that it recognizes.

This issue presents a user experience conflict.

One way I would typically work this in any other email service solution is to create the folder iOS expects and use that for junk. However if I were to do this in Cloudron, I’m worried (and no answer has been provided to confirm this yet) that the actual spam learning that’s automatically done won’t actually take place, as I suspect since the spam folder can not be deleted that it is the only folder spamassassin is set to learn from.

I think a solution here would be to allow a Cloudron user (or maybe just admins / domain admins) to create or choose the special folder that spamassassin will learn from for spam or ham. Basically, giving users more insight and control into spam services. This was also addressed in a few other threads on spam and improving its usability for users by exposing some of spamassassin features to admins or users and allowing them to set their own spam preferences.

girish

Ah, I see. I happened to get hold of a Mac today and can confirm that I can select the Spam folder and it works correctly as per https://support.apple.com/en-us/HT202325

For iOS, it seems there is no solution. For example, someone has similar problem with outlook.com addresses as well - https://discussions.apple.com/thread/7805226. And per https://clients.websavers.ca/whmcs/knowledgebase/231/Using-iOSandsharp039s-Junk-mail-folder-alongside-Spam.html there is no way to configure Junk mailbox name in iOS...

Currently, Cloudron will only learn (and unlearn) from "Spam" folder. It's easy to add Junk folder as well for training purposes but have to then figure out which folder incoming spam should end up.

girish

@d19dotca How does this work with gmail accounts on iOS btw? Or is it that they have "pre-configured" provide listing when you create an account and thus hard coded spam folder name accordingly (because gmail uses Spam and not Junk afaik).

girish

OK, here's a workaround for iOS.

SSH into server and create symlink from Junk to Spam

  # cd /home/yellowtent/boxdata/mail/vmail/{mailboxname}/mail
  # ln -s .Spam .Junk`
  # docker restart mail

You have to do above for all mailboxes. This will make both Spam and Junk folder identical and they will both appear in mail clients (unfortunate...). But atleast, it will make iOS client work.

We are looking into migrating the folder names for a future Cloudron release (there is a bug that we used "." as the separator in folder names. this makes ACLs and shared mailboxes unviable). Maybe at that time, we can look into making these folder names more customizable.

d19dotca

@girish That's an interesting workaround idea. I assume the only downside to that workaround though is that anytime the Cloudron server restarts, those changes go away and need to be re-entered? Or do they stick around?

girish

The symlinks will stick around. They are not part of the backups though. So you have to re-create the symlinks if you restore cloudron to another server.

I got the symlink idea from here https://wiki.dovecot.org/SharedMailboxes/Symlinks . I tried it on my server and it does work. Works quite nicely on thunderbird because it doesn't auto-subscribe to Spam folder by default even (so I don't have to adjust my spam settings for new accounts).

imc67

found this, is this something to configure ourselfves in Haraka config?

https://apple.stackexchange.com/questions/105145/what-are-the-default-special-folder-names-for-imap-accounts-in-mail-app-like-dr#105146

d19dotca

@imc67 That SO post is great, I had seen that many years ago when running a mail server before and couldn't find it to raise in this forum. haha. Thanks for finding that!

@girish Per the RFC mentioned in that first response to the OP in SO, it indicates that the folder really should be named "Junk", not "Spam", and I think this is partly the root cause of why the default Mail app in macOS and iOS do not see it by default - because it's following RFC and expecting the folder to be named "Junk".

girish

The folder name is different from the special_use flags. The RFC is suggesting the use of the "\junk" flag. This is what Cloudron also does. The mailbox name is not suggested in the RFC AFAICT. Can you tell me which section you are referring to?

d19dotca

@girish Ah, I believe you are correct there. I think I just typically see the names the same so I mixed them up. I still think though that it should be named "Junk" instead of "Spam" as that is what I have used in the past and it works well. Even your Thunderbird screenshot indicates they expect it to be named "Junk" too as they don't use the word "Spam". Of course that's just my interpretation and past experience, but I believe you're correct that this isn't a defined requirement in the RFC.

I really think admins should have more control over this, such as setting up their own default folder list, for example, or the ability to mark multiple folders as '\Junk' and the ability to remove the Spam folder in favour of naming it Junk.

girish

@d19dotca said in How does spam work with special mail folders?:

I really think admins should have more control over this, such as setting up their own default folder list, for example, or the ability to mark multiple folders as '\Junk' and the ability to remove the Spam folder in favour of naming it Junk.

I can get behind this. But I am quite practical about these things. There's nothing we can do to fix iOS and mac, so I am OK to even rename the folder to Junk if it increases compatibility. In the coming release (5.3), I have put some "state" information that will help us migrate to the release after. Existing installations can be on Spam and new installations can use Junk. I am doing the same for the mailbox separator. (moving things from . to / to make mailbox sharing work).

necrevistonnezr

@girish said in How does spam work with special mail folders?:

OK, here's a workaround for iOS.

SSH into server and create symlink from Junk to Spam
  # cd /home/yellowtent/boxdata/mail/vmail/{mailboxname}/mail
  # ln -s .Spam .Junk`
  # docker restart mail
You have to do above for all mailboxes. This will make both Spam and Junk folder identical and they will both appear in mail clients (unfortunate...). But atleast, it will make iOS client work.

We are looking into migrating the folder names for a future Cloudron release (there is a bug that we used "." as the separator in folder names. this makes ACLs and shared mailboxes unviable). Maybe at that time, we can look into making these folder names more customizable.

BTW there's similar "problem" for "Sent" / "Sent Messages" and "Trash" / "Deleted Messages"...

d19dotca

@necrevistonnezr True. The difference though is at least it can be worked around on iOS and macOS as you can set the folder roles. Can’t do that with spam/junk folder unfortunately.

necrevistonnezr

@girish said in How does spam work with special mail folders?:

OK, here's a workaround for iOS.

SSH into server and create symlink from Junk to Spam
  # cd /home/yellowtent/boxdata/mail/vmail/{mailboxname}/mail
  # ln -s .Spam .Junk`
  # docker restart mail

@girish Wouldn't it make sense to create hardlinks instead of symlinks for these folders so that you don't have duplicate folders? Ar am I overlooking something?

girish

@necrevistonnezr said in How does spam work with special mail folders?:

@girish Wouldn't it make sense to create hardlinks instead of symlinks for these folders so that you don't have duplicate folders? Ar am I overlooking something?

The cloudron backup logic does not understand hardlinks. It would then end up taking backup of your junk folder twice. This is unlike symlinks which is ignored by the backup code. Neither the symlink nor the contents of a symlink are backed up.

This does bring up an important point that if you move your mail server to a different server, you have to remember to create this symlink by hand again!

Cloudron makes it easy to run web apps like WordPress, Nextcloud, GitLab on your server. Find out more or install now.

Cloudron Forum

How does spam work with special mail folders?