What's coming in 6.0 (take 2)

[girish]

An update on this. The i18n is going along well, I think @nebulon is ~80% done. Once we are almost there, we will let you guys know in the translations thread. There is a large number of strings to be translated. We plan to ship Cloudron 6 with the code being translatable while the actual translations will be made available whenever they get ready.

Email container now has solr integration! While search is fast and I can search body (and maybe in future release attachments as well), there is something strange about how IMAP searches work. It seems it will search only under INBOX without the mail client having support for ESEARCH IMAP extension. I am also trying to figure how various mail clients behave here. Hopefully, should get sorted out this week.

Other features are done.

[jdaviescoates]

@girish said in What's coming in 6.0 (take 2):

An update on this. The i18n is going along well, I think @nebulon is ~80% done.

Excellent and important work!

Once we are almost there, we will let you guys know in the translations thread. There is a large number of strings to be translated. We plan to ship Cloudron 6 with the code being translatable while the actual translations will be made available whenever they get ready.

Good plan

Email container now has solr integration!

Amazing!

While search is fast and I can search body (and maybe in future release attachments as well),

Attachments too would be excellent!

there is something strange about how IMAP searches work. It seems it will search only under INBOX without the mail client having support for ESEARCH IMAP extension. I am also trying to figure how various mail clients behave here. Hopefully, should get sorted out this week.

I could be way off mark here, but I've noticed in the past that lots of IMAP providers seem to put Sent and other folders as effectively subfolders of the inbox. Perhaps something to do with that?

Other features are done.

You're underselling yourself (tbf, that's always a good look).

Personally I am very exciting about these "other features"!

• Volumes!
• Mailbox sharing!
• Faster Nextcloud and WordPress!
• Ubuntu 20.04 support! (Edit: tbh not particularly excited by this, because I'm more excited about having BBB on Cloudron and that isnt' even compatible with 18.04 yet!)

These are all fantastic new features!

[MooCloud_Matt]

@jdaviescoates said in What's coming in 6.0 (take 2):

Faster Nextcloud and WordPress!

We have some test image with Nginx + WP, that i have already shared with @girish, in the next few day we will work with him on some special feature to make the manifest compatible with OLS base image that we have pretty much ready, and later we will work on WP and NextCloud performance boost.

Probably next week, we will post some performance benchmarks of the new WP image, for both Nginx and OLS base image.

[robi]

@moocloud_matt oh you beat me to it.. was going to do Nginx based stack then add other webservers to the mix.

Can't wait for your testing.. godspeed.

[avatar1024]

@jdaviescoates said in What's coming in 6.0 (take 2):

Edit: tbh not particularly excited by this, because I'm more excited about having BBB on Cloudron and that isnt' even compatible with 18.04 yet!

And the worst is, they're explicitly putting all of the work in making BBB compatible with 18.04 first and they have no plans for 20.04.

[jdaviescoates]

@avatar1024 I'm not too bothered by that but yeah it's a bit mad they are still on 16.04

[ei8fdb]

@girish Absolutely. I wasn't suggesting to implement it "now-now". More the case of suggesting to gather data earlier (could be done at any time really), so that when it came to thinking about implementation, you had some data to base decisions on.

To get enough data I'd suggest running the research for ~ 2-3 weeks. Analysis would be much quicker ~days. I'll happy to help with both.

[girish]

Another brief update:

• We added auth wall functionality that apps without any notion of login can use
• i18n is mostly done, just lots of testing before we let translators do their work. @nebulon can give some updates on how the language is selected etc.
• We added httpPaths functionality so that @MooCloud_Matt can work on his OLS app. Also, see https://forum.cloudron.io/topic/3755/apache-ols-and-nginx-custom-benchmarks/5
• Reworked a lot of the internal dns stuff, so that we can have apps as dns servers (ad blocker apps)
• solr fts search was merged today.

Looks like we are set for a release next week!

[doodlemania2]

@girish Did the TURN server API auth stuff make it in for Nextcloud high performance backend'ness?

[nebulon]

@doodlemania2 unfortunately not, in fact I forgot about it

I have put it now into milestone for 6.1 so this does not happen again. Sorry about that.

[jdaviescoates]

@girish said in What's coming in 6.0 (take 2):

solr fts search was merged today.

This is great... but I'm intrigued to how exactly we'll be able to take advantage of it e.g. to search email/ files?

Will Cloudron Roundcube/ Rainloop/ Sogo, Nextcloud Mail somehow automagically take advantage of solr? or what?

I guess the same applies to Nextcloud files too - will search files on there somehow benefit from the power of solr?

Thanks!

[girish]

@jdaviescoates said in What's coming in 6.0 (take 2):

Will Cloudron Roundcube/ Rainloop/ Sogo, Nextcloud Mail somehow automagically take advantage of solr? or what?

That's the idea, yes. To keep excitement in control, from my testing, the fts has real benefits only if you lots of email (like say 2GB and more). You also need a lot of RAM to run solr (like 3GB). The feature is turned off by default for this reason.

I guess the same applies to Nextcloud files too - will search files on there somehow benefit from the power of solr?

Unfortunately, no. I think there was this nextcloud app called nextant back in the day which used solr. That app was however deprecated and they moved to this fulltextsearch app. The fulltextsearch app can have multiple backends but afaik there is only an elastic search backend (someone has to write a solr backend).

[jdaviescoates]

@girish said in What's coming in 6.0 (take 2):

That's the idea, yes. To keep excitement in control, from my testing, the fts has real benefits only if you lots of email (like say 2GB and more). You also need a lot of RAM to run solr (like 3GB). The feature is turned off by default for this reason.

OK, thanks.

I guess part of what I'm asking is does having solr enabled some allow for using some Gmail-style search operators like from: to: in:sent has:attachment etc?

I'm guessing perhaps not and that such things would need to be built in/ added to the webmail clients themselves? But I'm unclear

[girish]

@jdaviescoates said in What's coming in 6.0 (take 2):

I'm guessing perhaps not and that such things would need to be built in/ added to the webmail clients themselves? But I'm unclear

Right, those features like operators have to come from mail clients themselves. Note that solr is actually not adding anything new to what is already there. It's just making search faster internally by having an index.

[jdaviescoates]

@girish said in What's coming in 6.0 (take 2):

Unfortunately, no. I think there was this nextcloud app called nextant back in the day which used solr. That app was however deprecated and they moved to this fulltextsearch app. The fulltextsearch app can have multiple backends but afaik there is only an elastic search backend (someone has to write a solr backend).

Looks like someone started to make one, but there hasn't been any updates since Jun 2019

https://github.com/rearden-logic-inc/fulltextsearch_solr

[jdaviescoates]

@girish said in What's coming in 6.0 (take 2):

Right, those features like operators have to come from mail clients themselves. Note that solr is actually not adding anything new to what is already there. It's just making search faster internally by having an index.

Thanks. I so wish there were better open webmail clients. Mailpile does most of what I want but development has seemingly largely stalled

Roundcube at least has these:

from:
to:
cc:
bcc:
subject:
body:

[marcusquinn]

@jdaviescoates for Roundcube, the others didn't pass my short attention span for acceptable daily-drivers

[marcusquinn]

@marcusquinn Nothing to do with 6.0 but since webmail distracted, I think EspoCRM is worth a mention as a webmail client, which I'm intending to give a whirl soon. Don't know how performant the search will be in that but will share feedback as I do.

[jdaviescoates]

@marcusquinn said in What's coming in 6.0 (take 2):

@jdaviescoates for Roundcube, the others didn't pass my short attention span for acceptable daily-drivers

It does seem the best of bad bunch (it's the second most popular after Mailpile on https://selfhosted.libhunt.com/categories/1665-webmail-clients ), but frustrating that e.g. it seemingly can't be used to set-up filters and so need to use Rainloop for that. The state of open source webmail really is bad

[girish]

@jdaviescoates I think roundcube is the best of the lot. But also, the desktop and mobile clients are quite great though.