Unsolved indexing of office documents?

[fbartels]

@doodlemania2 Thanks for packaging it. Does your container also include the indexing of office documents? There was another software required as far as I remember.

[doodlemania2]

@fbartels not sure - I am using it at the moment for about a bazillion PDFs. Haven't tried Office Files. Let me know, I'm sure we can tweak if it requires a bit more software.

[nebulon]

@doodlemania2 maybe we can discuss further package shortcomings in the paperless-ng forum section then

[girish]

I moved this to a new topic.

[fbartels]

@doodlemania2 I was looking for some further information on this.

Paperless-ng uses Tika (to extract data from other file types) and Gotenberg (for pdf conversion) for this.

More at https://paperless-ng.readthedocs.io/en/latest/configuration.html#tika-settings

[ChristopherMag]

This is something I need as well, working on rolling out PaperlessNGX to replace an existing storage system and need to be able to store office documents as well so that all documents related to an entity are together.

It looks like Tika and Gotenberg both have docker containers available.

Do we need to have separate Tika and Gotenberg apps like we have OnlyOffice and Collabra Online as a separate apps though they are not usable on their own and are used by NextCloud for handling office documents?

[ChristopherMag]

For anyone wanting to get this up and running quickly if you have docker running on another system you can run the following:

docker run -d --restart unless-stopped -p 3000:3000 gotenberg/gotenberg
docker run -d --restart unless-stopped -p 9998:9998 apache/tika

And then add the following to your paperless.conf:

# Tika
PAPERLESS_TIKA_ENABLED=true
PAPERLESS_TIKA_ENDPOINT=http://<DockerHostnameOrIPGoesHere>:9998
PAPERLESS_TIKA_GOTENBERG_ENDPOINT=http://<DockerHostnameOrIPGoesHere>:3000

After this you can upload xlsx, docx, etc. to paperless-ngx.

In my testing if the Docker host running the Tika and Gotenberg containers goes down paperless-ngx keeps working fine but you won't be able to upload additional xlsx/docx/etc. documents until you restart the containers which works out fine as the reliability of paperless-ngx being accessible is way more important than this one feature working for us.

[jdaviescoates]

@ChristopherMag thanks for sharing!

@staff can we get this into the package somehow?

[girish]

@jdaviescoates yes, I will put this as a point to investigate for next release. I think we have to investigate what nextcloud needs for FTS as well as apps like these and design accordingly. (Maybe they have to become addons or alternately we can just put them in the app itself).

[jdaviescoates]

@girish said in indexing of office documents?:

I think we have to investigate what nextcloud needs for FTS as well

On that, I spotted recently that they do now at least mention Solr under Platform Apps over on https://github.com/nextcloud/fulltextsearch but as far as I can tell from the linked wiki https://github.com/nextcloud/fulltextsearch/wiki to date there is still only an Elastic Search Platform App.

[timconsidine]

@ChristopherMag do you or anyone else have experience of using this kind of setup for Mac OS document formats like Pages and Numbers ?

[ChristopherMag]

@timconsidine It looks like Apache Tika supports the document formats from the iWork suite like pages.

I tried to upload a .pages file to paperless-ngx with Tika and gotenberg configured and paperless popped up a failure message with the the error File type application/zip not supported.

I believe this signals.py file in the paperless-ngx project would need to add support for the various iWork software suite formats to resolve this error and get this working assuming you already have Tika and gotenberg setup and working with paperless-ngx.

You could probably open a github issue in the paperless-ngx repository on Github and see if they can assist with adding support fort his.