Nextcloud Fulltext Search

[necrevistonnezr]

@spectrely said in Nextcloud Fulltext Search:

@necrevistonnezr oooooooh, I assume with that one, you'd need your own external server for that feature.. Let me know what you find!

Well, in theory you needed an external server for the Nextcloud Collabora Office app but the cloudron guys made it available, anyway...

[girish]

We do have plans to provide elasticsearch as an addon (just like other databases) at some point. For the moment, it looks like you have to run elasticsearch on a different server and connect the plugin to it.

[necrevistonnezr]

It seems there's a Docker image for the fulltext search backend engine on Nextcloud, now: https://github.com/nextcloud/fulltextsearch/wiki/Basic-Installation

[NCKNE]

Getting elasticsearch into Cloudron would be great! Fulltext search for Nextcloud is a must and running a separate instance side by side to cloudron is very inefficient...

[jdaviescoates]

Cloudron 6.0 now has Solr integration!

But even though https://github.com/nextcloud/fulltextsearch says:

Full text search

Full text search is the core app of a full-text search framework for your Nextcloud.
To have it operate, and get content indexed, some other apps are needed:

• Some Providers Apps to extract content from your Nextcloud.
• A Platform App that communicate with a search platform (ie. Elastic Search, Solr, …) in order to index the content provided by the Providers.
  Note: There is no limit to the number of platform-apps that can be installed, however only one can be selected from the admin interface

(I bolded Solr)

Currently Solr is not yet supported:

https://github.com/nextcloud/fulltextsearch/issues/334

Someone made a start a while back, but it doesn't appear they've done anything on it for the last couple of years now

https://github.com/rearden-logic-inc/fulltextsearch_solr

I wonder any @staff or @appdev's might be up for taking this on? It'd be awesome to be able to use Cloudron's new Solr addon within Nextcloud to do full text searches!

[girish]

@jdaviescoates I think only the elastic one is maintained upstream. It's unlikey we (cloudron) will write the solr fts integration for nextcloud. That's really something we don't have any expertise for.

[jdaviescoates]

@girish it was a longshot, but fair enough!

[vladimir.d]

We've created an ElasticSearch app for Cloudron.

https://git.cloudron.io/vladimir.d/elasticsearch

Comments are welcome!

[jdaviescoates]

@vladimir-d said in Nextcloud Fulltext Search:

We've created an ElasticSearch app for Cloudron.

https://git.cloudron.io/vladimir.d/elasticsearch

Comments are welcome!

Woah, cool!

This sounds worthy of a @staff and @appdev mention!

[fbartels]

I guess https://forum.cloudron.io/topic/5297/supervisord-to-manage-running-processes-in-nextcloud-app is related to this

[marcusquinn]

Yup, we have this working now. It's OKish but Nextcloud's search interface is kinda sh*t, so next task is going to be fixing that to make it actually useful and hopefully work similarly to Google Drive with an actual sortable results page.

Elasticsearch as an App from @vladimir-d is a significant win for other uses too.

[marcusquinn]

Anecdotally, this fulltext search (using elastic search) seems to be as fast as the filename search, or even makes both of them faster.

The search results interface as a modal is still cramped and limited, but we will try to develop a full results page to show on hitting enter or clicking the search icon.

[marcusquinn]

An interesting search string for anyone managing what multiple users do with these things is to search for password and see how many place plaintext passwords are stored in user's documents & spreadsheets!

Similar for things like passport number, driving licence, social security, national insurance. It's a handy way to do a review of personal data being stored in files and who has access to what.

[rmdes]

@vladimir-d @staff could this be part of the store in the near future ?

[girish]

@rmdes I think as a first step for next Cloudron release we will look into a way to run long running / periodic processes so we can support things like nextcloud indexer. Elasticsearch has to be installed as a separate package. From there, it's straightforward.

[rmdes]

@girish Wonderful, I have tons of PDF's that I want to parse, OCRise, make searchable, the ability to do this in a self-hosted environment is mandatory in my case because it's related to Justice documents that I don't want to upload anywhere, ever, so I have been searching ways to go around that without having VPS's running just for that, I hope it's going to be doable in the future (and I trust it will)

[marcusquinn]

@rmdes we're looking into the OCR stuff too, will get to that part very soon as well.

[necrevistonnezr]

@marcusquinn @vladimir-d @staff
What's the status of the Elastic Search App - according to the repo, there hasn't been an update for the last 9 months. Is this still functional?

[jdaviescoates]

@necrevistonnezr said in Nextcloud Fulltext Search:

What's the status of the Elastic Search App - according to the repo, there hasn't been an update for the last 9 months. Is this still functional?

Same question.

It'd be great to actually get this in the App Store if it is functional.

[marcusquinn]

@jdaviescoates Sorry, we're tied up trying to get Univention working as a master AD/LDAP server to multiple Cloudron, Unify and other systems, because sadly, the world still uses Active Directory.

Will try to get shared shortly...

Also need to compare to Filerun and Pydio for speed.