High cpu utilization since update 1.25.1

[nebulon]

This very much reads like task tries to work a job which then fails due to a database conflict. Since that job never finishes it just retries in a loop. I think it is worth if you report this upstream as it kinda seems like a regression in the app itself.

[humptydumpty]

Upstream ticket opened here: https://github.com/paperless-ngx/paperless-ngx/issues/7439

[humptydumpty]

@nebulon upstream closed the ticket and said the following:

Perhaps a better question for cloudron not a bug report here. Memory yes, 100% CPU, no.

I appreciate that cloud hosting companies tend to push users to the project itself but since the only people having this problem are those hosting on cloudron (there have been no similar reports lately and I think there would be since it sounds like the app is effectively broken) I think the more logical conclusion is that this is a cloudron issue not a paperless one.

Could you please get in touch with them and sort this out?

[niquon]

Googling this suggests trying to remove a corrupted file eg data/celerybeat.db

https://stackoverflow.com/questions/12362045/django-celery-error-when-starting-celerybeat-cannot-add-item-to-database

[humptydumpty]

Shoutout to stumpylog (paperless maintainer) for the fix. Using the file manager, browse to /app/data/data and delete celerybeat-schedule.db.db and restart the server app. CPU usage went down for me. Hopefully, it stays that way.

[imc67]

@humptydumpty said in High cpu utilization since update 1.25.1:

Shoutout to stumpylog (paperless maintainer) for the fix. Using the file manager, browse to /app/data/data and delete celerybeat-schedule.db.db and restart the server app. CPU usage went down for me. Hopefully, it stays that way.

This solution solved the issue for me! Thanks!!

[necrevistonnezr]

@humptydumpty said in High cpu utilization since update 1.25.1:

Shoutout to stumpylog (paperless maintainer) for the fix. Using the file manager, browse to /app/data/data and delete celerybeat-schedule.db.db and restart the server app. CPU usage went down for me. Hopefully, it stays that way.

So it was not a Cloudron issue in the end?

[humptydumpty]

@necrevistonnezr we still don’t know the root cause of the file corruption. It could happen again anytime. Upstream closed the discussion and moved on. It looks like only Cloudron users are affected on existing installs and not on a fresh app. App update snafu?

Btw, upstream said the +500mb ram on idle is normal. I don’t get why it should be that high.

[humptydumpty]

@nebulon I uploaded the corrupted db file upstream as requested.. edit: just realized I can copy the github link https://github.com/user-attachments/files/16591433/celerybeat-schedule.db.zip

I'll keep the cloned app for the time being. Please let me know if you need more info.

[humptydumpty]

CPU is high again on Paperless-ngx 2.11.6. Deleting the celery file and restarting still works as a temp fix. I did turn off the server today and updated the BIOS. Idk if that has anything to do with it.

[wheez]

Do we know by now what is causing this problem? I have had it for the second time today. Last time about a Month ago. The fix worked in both cases but I guess it would still be interesting to know why this is happening.

[humptydumpty]

It's going nuts again after it updated to 1.27 then 1.28 today.

edit: This time I had to delete the celery file and restart the app twice for the fix to work.

[humptydumpty]

Just an FYI, it's still recurring on v.1.3.2.

[flwd]

Same issue for me. I ran cloudron as a VM on unraid and on proxmox, on 2 different hosts and with and without a CIFS mount as consume folder. The issue recurred in every constellation.

[joseph]

@flwd said in High cpu utilization since update 1.25.1:

The issue recurred in every constellation

What does this mean?

[jdaviescoates]

@joseph installation typo would be my guess

[flwd]

@joseph it occured in every installation variant i tried, independant of the CIFS mount and config.

[joseph]

@flwd thanks for the clarification. I think the original issue atleast was narrowed down to celery database going corrupt and it's not related to volume type. Have you tried deleting that database pointed out earlier in the thread?

[flwd]

@joseph yes, deleting the database fixes the problem temporarily.

[lukasgabriel]

Same problem still happening...