Searching emails by email address doesn't work :(

[jdaviescoates]

If I search for e.g.

someone@somedomain.* it works

But someone@somedomain.com does not work - no search results are found.

This is the same across SnappyMail, Roundcube and Sogo: as soon as there is anything after the . it doesn't work. I wonder if it is something to do with:

Cloudron uses dovecot as the IMAP server. The default namespace separator in dovecot is .. However, this separator conflicts with the mailbox sharing feature This is because Cloudron Email server users the user's email as the username and email addresses have a . in the domain portion.

The solution is to migrate to using / as the namespace separator. New installations use this separator by default. Existing installations must be migrated manually to use this separator.

As per https://docs.cloudron.io/guides/mailbox-sharing/#background ?

But whatever the cause, it is really very frustrating so I hope a solution can be found!

BTW FYI I have the "All Mail" Folder and Solr both enabled:

[jdaviescoates]

@james I can't presently reproduce the issue I had previously so I guess this is solved.

[girish]

I did a quick test but it's searching fine. I don't have all the options you have enabled though. My suspicion is solr and you are hitting the search word length. We use EdgeNGramFilterFactory with a max gram size of 20 . Are you able to search shorter addresses? Also, I assume you are searching the body and not to/from. solr is not used for to/from matching .

[jdaviescoates]

@girish said in Searching emails by email address doesn't work :

We use EdgeNGramFilterFactory with a max gram size of 20

Why so low? I've no idea if that equates to 20 characters or not(?), but loads of email addresses are longer than that! Please could that figure be significantly increased?

@girish said in Searching emails by email address doesn't work :

Also, I assume you are searching the body and not to/from

I've tried both just whacking the email address in the search box, or using from:emailaddress - neither work

But yeah I just tried a short info@123.co.uk length one and that worked, so it seems it the silly low limit that is making search not work. Please fix if possible.

@girish said in Searching emails by email address doesn't work :

solr is not used for to/from matching

So I wonder why from:longer@emailaddressdomain.com doesn't work either then?!?

[girish]

@jdaviescoates right, it's set for indexing words of max of 20 letters. Didn't occur to me email addresses will be single words... Let me see if this can be bumped and what the implications are.

[jdaviescoates]

@girish said in Searching emails by email address doesn't work :

@jdaviescoates right, it's set for indexing words of max of 20 letters. Didn't occur to me email addresses will be single words... Let me see if this can be bumped and what the implications are.

Just having a look at https://solr.apache.org/guide/8_6/filter-descriptions.html#edge-n-gram-filter and it looks to me like you could perhaps just add preserveOriginal="true":

But of course I've no actual idea if that is a better way to do it or not!

[girish]

@jdaviescoates yes, that's how it is

      <filter class="solr.EdgeNGramFilterFactory" minGramSize="3" maxGramSize="20" preserveOriginal="true"/>

[jdaviescoates]

@girish strange that it doesn't work then

[girish]

@jdaviescoates strange.. So, I can't reproduce this. Tried with some really long words. The EdgeNGramFilterFactory max setting is for the size of the prefixes and not for search length. So, that size is actually fine.

[girish]

Note that if you had just enabled solr before you searched, it can take a while for solr to complete indexing all your existing emails.

[necrevistonnezr]

@girish said in Searching emails by email address doesn't work :

@jdaviescoates right, it's set for indexing words of max of 20 letters.

Does that mean words with over 20 characters are not found? If so, I did not know that and relied on the index - i.e. assumed I didn’t have an email if the (index based) search does not yield results! This would be very bad.

Please remove such limits (if possible) as e.g. the German language has many words over 20 characters.
If you keep limits, we need to communicate them clearly in the docs AND in the UI, I think.

[jdaviescoates]

@girish said in Searching emails by email address doesn't work :

Note that if you had just enabled solr before you searched, it can take a while for solr to complete indexing all your existing emails.

I've had it enabled since we've had Solr on Cloudron, so that's not the problem.

[jdaviescoates]

@girish said in Searching emails by email address doesn't work :

Tried with some really long words.

I thought "OK, but you're searching for body text in an email, I'm search for the senders email address".

But I just tested sending myself an email with admin@sustianablebridport.com in the body and mine fails to find that too. It still only finds it if I search either for:

text=admin@sustainablebridport.

Again, if I have anything after the . it doesn't find it

But

text=josef@open.com

Works

[jdaviescoates]

@jdaviescoates said in Searching emails by email address doesn't work :

text=admin@sustainablebridport.

Again, if I have anything after the . it doesn't find it

This works on a newer install of Cloudron, so perhaps I need to migrate my older install as per https://docs.cloudron.io/guides/mailbox-sharing/#background ?

[jdaviescoates]

@jdaviescoates said in Searching emails by email address doesn't work :

@jdaviescoates said in Searching emails by email address doesn't work :

text=admin@sustainablebridport.

Again, if I have anything after the . it doesn't find it

This works on a newer install of Cloudron, so perhaps I need to migrate my older install as per https://docs.cloudron.io/guides/mailbox-sharing/#background ?

Nope, it's not that.

I just checked my /home/yellowtent/boxdata/mail/dovecot/config.ini and I've already got:

namespace_separator=/

[jdaviescoates]

I thought perhaps the difference with a newer server where it works might be that I don't have Solr enabled on that... and so I tried disabling Solr... but a few moments later it re-enables itself!

[girish]

@jdaviescoates I think the setting issue is fixed with https://git.cloudron.io/cloudron/box/-/commit/1afa2e87ec93881ebcd1f79d83d4477467b40770 . After changing the setting, manually restart the mail container (from services view) as a workaround. Fix is in the next release.

[jdaviescoates]

@girish said in Searching emails by email address doesn't work :

Fix is in the next release.

Presumably this is the fix to Solr re-starting itself?

Cool, thanks. I've try again once the updated has been rolled out.

Still unsure/ unclear if/ why Solr would effect this simple matter of searching by email address though?

[girish]

The fix is for solr showing the incorrect status. Let's re-test once the update is out.

[james]

Hello @jdaviescoates
Did you ever come back to this issue?

[jdaviescoates]

@james I can't presently reproduce the issue I had previously so I guess this is solved.