Cloudron Forum

I haven't symlinked any of these directories.
However, I'm running Cloudron on a luks-encrypted partition. Maybe that has to do with the problem?

@nebulon The problem with the .db.db happened again not even 24h later.
This GitHub Issue suggests it might be a Cloudron problem after all: https://github.com/paperless-ngx/paperless-ngx/discussions/7440#discussioncomment-10324614

Any tips? I thought about setting up a cron job to check if the file has the duplicate file extension and rename/remove if it does, but an app package upgrade might remove that job, right? Should I set up the job inside the container, or on the Cloudron host?

Same problem still happening...

@joseph v8.3.1 (Ubuntu 24.04.2 LTS)

/home/yellowtent/platformdata/logs/tasks/8216.log:

2025-03-28T23:22:58.129Z box:taskworker Starting task 8216. Logs are at /home/yellowtent/platformdata/logs/tasks/8216.log
2025-03-28T23:22:58.156Z box:taskworker Running task of type updateDiskUsage
2025-03-28T23:22:58.158Z box:shell df: df -B1 --output=source,fstype,size,used,avail,pcent,target
2025-03-28T23:22:58.176Z box:shell df: df -B1 --output=source,fstype,size,used,avail,pcent,target /home/yellowtent/platformdata
2025-03-28T23:22:58.184Z box:taskworker Task took 0.111 seconds
2025-03-28T23:22:58.184Z box:tasks setCompleted - 8216: {"result":null,"error":{"stack":"TypeError: Cannot read properties of undefined (reading 'contents')\n    at getFilesystems (/home/yellowtent/box/src/system.js:126:42)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n    at async updateDiskUsage (/home/yellowtent/box/src/system.js:231:25)","message":"Cannot read properties of undefined (reading 'contents')"}}
2025-03-28T23:22:58.184Z box:tasks update 8216: {"percent":100,"result":null,"error":{"stack":"TypeError: Cannot read properties of undefined (reading 'contents')\n    at getFilesystems (/home/yellowtent/box/src/system.js:126:42)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n    at async updateDiskUsage (/home/yellowtent/box/src/system.js:231:25)","message":"Cannot read properties of undefined (reading 'contents')"}}
TypeError: Cannot read properties of undefined (reading 'contents')
    at getFilesystems (/home/yellowtent/box/src/system.js:126:42)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async updateDiskUsage (/home/yellowtent/box/src/system.js:231:25)

I managed to solve the problem by deleting /app/data/data/celerybeat-schedule.db.db
It should have been named /app/data/data/celerybeat-schedule.db - not sure where it got the extra suffix from, but now the insane logs and CPU usage is fixed and everything runs smoothly again.

When running via System Info > Disk usage > Analyze Disk:

Mar 29 00:22:57 box:tasks startTask - starting task 8216 with options {}. logs at /home/yellowtent/platformdata/logs/tasks/8216.log
Mar 29 00:22:57 box:shell tasks /usr/bin/sudo -S -E /home/yellowtent/box/src/scripts/starttask.sh 8216 /home/yellowtent/platformdata/logs/tasks/8216.log 0 400 0
Mar 29 00:22:57 box:shell Running as unit: box-task-8216.service
Mar 29 00:22:57
Mar 29 00:22:58 box:shell Finished with result: exit-code
Mar 29 00:22:58 Main processes terminated with: code=exited/status=50
Mar 29 00:22:58 Service runtime: 737ms
Mar 29 00:22:58 CPU time consumed: 1.170s
Mar 29 00:22:58 Memory peak: 6.3M
Mar 29 00:22:58 Memory swap peak: 0B
Mar 29 00:22:58
Mar 29 00:22:58 box:shell Service box-task-8216 failed to run
Mar 29 00:22:58
Mar 29 00:22:58 box:shell Service box-task-8216 finished with exit code 1
Mar 29 00:22:58
Mar 29 00:22:58 box:shell tasks: /usr/bin/sudo -S -E /home/yellowtent/box/src/scripts/starttask.sh 8216 /home/yellowtent/platformdata/logs/tasks/8216.log 0 400 0 errored BoxError: tasks exited with code 1 signal null
Mar 29 00:22:58 at ChildProcess.<anonymous> (/home/yellowtent/box/src/shell.js:137:19)
Mar 29 00:22:58 at ChildProcess.emit (node:events:519:28)
Mar 29 00:22:58 at ChildProcess.emit (node:domain:488:12)
Mar 29 00:22:58 at ChildProcess._handle.onexit (node:internal/child_process:294:12) {
Mar 29 00:22:58 reason: 'Shell Error',
Mar 29 00:22:58 details: {},
Mar 29 00:22:58 code: 1,
Mar 29 00:22:58 signal: null
Mar 29 00:22:58 }
Mar 29 00:22:58 box:tasks startTask: 8216 completed with code 1
Mar 29 00:22:58 box:tasks startTask: 8216 done. error: { stack: "TypeError: Cannot read properties of undefined (reading 'contents')\n" + ' at getFilesystems (/home/yellowtent/box/src/system.js:126:42)\n' + ' at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n' + ' at async updateDiskUsage (/home/yellowtent/box/src/system.js:231:25)', message: "Cannot read properties of undefined (reading 'contents')" }

The UI shows nothing, just the button again.

@nebulon The latest patch related to unoconv has resolved the Gotenberg/Tika issues for me! I can now upload EMLs again, and I also tested successfully with XLSX and some others. Great work!

The log spam still persists though. @luckow: paperless.log is normal for me as well, the problem is the celery warnings/errors from the container log, which you get by pressing the "Logs" button in the Cloudron panel. PAPERLESS_DEBUG is false.

@nebulon I think HTML conversion is supported gotenberg: https://gotenberg.dev/docs/6.x/html - so for sanity-checking if gotenberg can run, it should work.
But Paperless doesn't support it: https://github.com/paperless-ngx/paperless-ngx/discussions/6700
So you can disregard that point of my post

@luckow I'm looking at a ~2GB log file across a span of 5 days. But I turned off my Paperless instance for most of that time, so it's more like 2 days of uptime at most.
I don't think the number of documents matters much, unless you're reprocessing them, but I also have around the same number as you.

So I'd say I am solidly out of the normal

Hi,

I am having two major issues with the Paperless-ngx app.

The first one deals with Gotenberg. Processing of - in my example - .eml and .html files fails with 500 Internal Server Error.
On closer inspection of the logs, I can see this error relating to permissions:

2025-03-26T03:15:17Z [2025-03-26 04:15:17,718] [ERROR] [paperless.consumer] Error occurred while consuming document Example.eml: Error while converting email to PDF: Server error '500 Internal Server Error' for url 'http://localhost:3000/forms/chromium/convert/html'

2025-03-26T03:15:17Z [2025-03-26 04:15:17,728] [ERROR] [paperless.tasks] ConsumeTaskPlugin failed: Example.eml: Error occurred while consuming document Example.eml: Error while converting email to PDF: Server error '500 Internal Server Error' for url 'http://localhost:3000/forms/chromium/convert/html'

2025-03-26T03:15:17Z [2025-03-26 04:15:17,824] [ERROR] [celery.app.trace] Task documents.tasks.consume_file[1368eaec-b063-4883-8ed2-1a11d2a46474] raised unexpected: ConsumerError("Example.eml: Error occurred while consuming document Example.eml: Error while converting email to PDF: Server error '500 Internal Server Error' for url 'http://localhost:3000/forms/chromium/convert/html'\nFor more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/500")

2025-03-26T03:15:17Z documents.consumer.ConsumerError: Example.eml: Error occurred while consuming document Example.eml: Error while converting email to PDF: Server error '500 Internal Server Error' for url 'http://localhost:3000/forms/chromium/convert/html'

2025-03-26T03:15:17Z documents.consumer.ConsumerError: Example.eml: Error occurred while consuming document Example.eml: Error while converting email to PDF: Server error '500 Internal Server Error' for url 'http://localhost:3000/forms/chromium/convert/html'

2025-03-26T03:15:17Z {"level":"error","ts":1742958917.707243,"logger":"api","msg":"convert HTML to PDF: convert to PDF: process first start: start process: run exec allocator: chrome failed to start:\nmkdir: cannot create directory ‘/root’: Permission denied\ntouch: cannot touch '/root/.local/share/applications/mimeapps.list': Permission denied\nchrome_crashpad_handler: --database is required\nTry 'chrome_crashpad_handler --help' for more information.\n[2868:2868:0326/031517.474572:ERROR:socket.cc(120)] recvmsg: Connection reset by peer (104)\n","log_type":"access","trace":"8ba08943-1dec-432b-8fdc-8b375407d48f","remote_ip":"::1","host":"localhost:3000","uri":"/forms/chromium/convert/html","method":"POST","path":"/forms/chromium/convert/html","referer":"","user_agent":"gotenberg-client/0.9.0","status":500,"latency":380718662,"latency_human":"380.718662ms","bytes_in":14690,"bytes_out":21}

It seems the chromium binary wants to write to /root/.local/share/applications where it has no permissions?

The second issue comes from the fact that the Paperless-ngx app is generating an enormous amount of logs, even under normal conditions.
I am pretty sure that this is why my Paperless-ngx instance is causing extreme CPU load (completely maxed) as well.

I'm not able to interpret the cause of the logs, but it looks like something related to Celery Scheduling or Redis is out of whack.

Here are things I have already tried:

• Using the Cloudron Panel, move Paperless-ngx data, from a volume I used before, to the default location in container (no effect)
• Delete /app/code and let container recreate it (no effect)
• Use cURL to send a simple test HTML file to Gotenberg to see if it can process it (same 500 Internal Server Error with accompanying logs)

I should note that I recently performed a "backup and restore" as described in your documentation, which has been uneventful with the exception of the above-stated issues with Paperless.
However, I don't know if they are directly related.

I have linked the relevant part of the application log, where you can see both the issues with Gotenberg as well as the issue of very high log volume:
https://gist.githubusercontent.com/lukasgabriel/0053a7c0e41a4c32980eb99acf248da9/raw/f288542530c321f925555ef3e5397dd21def33b1/aa8f2c69-31fc-4102-a375-7d175580f6b4.log

@Kubernetes Thanks for the heads up! I was on Ubuntu 20.04
The update did indeed fix the SPF issue.
I'm still struggling with getting the mail delivered - paradoxically not even automatic forwarding works, but manual forwarding does... - but I don't think that is related to this issue.

@Kubernetes Alright, thanks! Going from the last update entry in my Notifications tab, I think I am on version 7.4.3 - not sure how to definitely check that. But that would explain the problem!
Found the version in the settings tab & will update!

Some more info: When I add my home IP to my SPF record, it does no longer SoftFail, but it still doesn't arrive in my ProtonMail inbox. I'll keep trying

Forked from https://forum.cloudron.io/topic/9030/cloudron-spf-record-does-not-permit-ip/36

Just wanted to weigh in and mention that I'm experiencing the same issue - my scanner tries to send mail via the Cloudron mail server, it shows up with my home ip in the logs and mentions a SoftFail, and gets entirely rejected by ProtonMail.
When I send the same mail to a mailbox on the Cloudron instance, it arrives. And it also arrives when sending "manually" from the Roundcube webmail.

Any help would be greatly appreciated as I'm a bit lost here. I can also provide logs and headers.

Would LOVE to have this!