Possible to make "exact phrase" email searches work?

[jdaviescoates]

At present searching for an exact phrase (e.g. wrapping words in quotes in Roundcube or SnappyMail) doesn't actually return only messages where those words appear together and in order. Instead it seems to return any message containing all the individual words, regardless of order or adjacency.

Digging into this with Claude, it seems this comes down to how fts_solr builds its query. IMAP's base SEARCH command requires strict substring matching for TEXT/BODY searches, i.e. word order and adjacency should matter - this is specified in RFC 9051.

There's a separate FUZZY extension (RFC 6203) specifically for inexact, stemmed, token-based matching, meant to be opt-in. RFC 7377 makes the same point explicitly:

The base IMAP SEARCH command (Section 6.4.4. of [RFC3501]) requires strict substring matching in text searches. Many servers, however, use search engines that match strings in different ways... This is covered by the "Fuzzy Search" extension to IMAP [RFC6203]...

It looks like Dovecot's fts_solr currently searches the same way regardless of whether FUZZY was requested, essentially always doing the fuzzy/token-AND behaviour. There's an old Dovecot mailing list thread from a few years back where this exact same point was raised, with a proposed fts_fuzzy_default setting so non-fuzzy search would actually behave as the default, non-fuzzy per the spec, with FUZZY being needed to get the current AND-of-tokens behaviour.

I appreciate this isn't a trivial one-line fix, especially since Cloudron's Solr schema uses edge n-grams for partial-word matching (which came up in the previous thread), and phrase queries don't play nicely with n-gram-tokenized fields. It'd probably need a second, position-aware field for exact/phrase queries alongside the existing n-gram field. But given @girish is one of Dovecot's top contributors, I wondered whether this might be something worth raising upstream, since it'd presumably benefit every fts_solr user, not just Cloudron's setup.

Happy to help test if you want to try anything.

Thanks, Josef

[girish]

@jdaviescoates said:

But given @girish is one of Dovecot's top contributors

Not sure where you learnt that but I haven't contributed a single change to Dovecot. May be you mistook my contributions to Haraka.

I will have to investigate the exact issue though. Unfortunately, both roundcube and (now dead) snappymail, are both not great at searching .

[necrevistonnezr]

@girish said:

I will have to investigate the exact issue though. Unfortunately, both roundcube and (now dead) snappymail, are both not great at searching .

Is any self hosted mail client?!

[timconsidine]

@girish said:

(now dead) snappymail,

Snappymail still in AppStore and updated 4 months ago (FYI)

[girish]

@timconsidine there's a big discussion here - https://github.com/the-djmaze/snappymail/issues/1911

[jdaviescoates]

@necrevistonnezr said:

Is any self hosted mail client?!

Same question.

TBH they aren't too bad, but I needed to find something the other day and really wanted the "exact phrase" to work.

But it sounds like it's actually nothing to do with the clients themselves (and nothing they could really do about it) but actually how IMAP and dovecot work (which yes, I misremembered for Haraka)

[necrevistonnezr]

@timconsidine said:

@girish said:

(now dead) snappymail,

Snappymail still in AppStore and updated 4 months ago (FYI)

Snappymail itself - not the Cloudron package - hasn't been updated for almost two years. For a security-sensitive app like a mail client (still in many ways the "key to the kingdom"), I would strongly advise against using it.

@girish Maybe it`s even time to delist it...

[necrevistonnezr]

I just looked at the discussion linked on Github, and it seems there's a working fork with updates: https://github.com/kimusan/Tachyon