Gotenberg Permission Issue and large log volume

[nebulon]

latest package release has unoconv fixed for the .eml use-case

[lukasgabriel]

@luckow I'm looking at a ~2GB log file across a span of 5 days. But I turned off my Paperless instance for most of that time, so it's more like 2 days of uptime at most.
I don't think the number of documents matters much, unless you're reprocessing them, but I also have around the same number as you.

So I'd say I am solidly out of the normal

[lukasgabriel]

@nebulon I think HTML conversion is supported gotenberg: https://gotenberg.dev/docs/6.x/html - so for sanity-checking if gotenberg can run, it should work.
But Paperless doesn't support it: https://github.com/paperless-ngx/paperless-ngx/discussions/6700
So you can disregard that point of my post

[luckow]

I see a difference between the Cloudron log file https://my.example.org/logs.html?appId=123456-1234-1234-1234-123455 and the logs in Paperless. (constantly repeated same errors and a lot of visual distraction in the Cloudron logs compared to a quiet log from the app https://paperless.example.org/logs). But at the moment it is not possible for me to investigate further.

[lukasgabriel]

@nebulon The latest patch related to unoconv has resolved the Gotenberg/Tika issues for me! I can now upload EMLs again, and I also tested successfully with XLSX and some others. Great work!

The log spam still persists though. @luckow: paperless.log is normal for me as well, the problem is the celery warnings/errors from the container log, which you get by pressing the "Logs" button in the Cloudron panel. PAPERLESS_DEBUG is false.

[nebulon]

seems like the loggin situation has to be taken to paperless upstream development, maybe you can create an issue there about this.

[lukasgabriel]

I managed to solve the problem by deleting /app/data/data/celerybeat-schedule.db.db
It should have been named /app/data/data/celerybeat-schedule.db - not sure where it got the extra suffix from, but now the insane logs and CPU usage is fixed and everything runs smoothly again.

[lukasgabriel]

@nebulon The problem with the .db.db happened again not even 24h later.
This GitHub Issue suggests it might be a Cloudron problem after all: https://github.com/paperless-ngx/paperless-ngx/discussions/7440#discussioncomment-10324614

Any tips? I thought about setting up a cron job to check if the file has the duplicate file extension and rename/remove if it does, but an app package upgrade might remove that job, right? Should I set up the job inside the container, or on the Cloudron host?

[nebulon]

as the upstream author indicates, the .db.db is something python related (?) but sadly since we didn't find a way to validate the .db file on startup ourselves, there is little we can do for now to clear that if needed. At least my comments there to even be able to reproduce didn't move it forward. I guess in the end we would have to debug this and hopefully come up with a MR to fix it, but we aren't python developers with little insight into paperless code itself to be honest.

[robi]

@nebulon perhaps pasting the code into one of the AIs can help spot the problem