<![CDATA[High cpu utilization since update 1.25.1]]>Hello,
First time publishing here πŸ™‚

The app is waiting for production, for now there is 0 user.

Since update 1.25.1 the CPU utilization is constantly at 62% (caused by the errors i got every 2 seconds i guess).
Capture d'Γ©cran 2024-07-31 122119.png

Here's the logs i have :

2024-07-31T07:12:30.000Z Traceback (most recent call last):
2024-07-31T07:12:30.000Z [2024-07-31 07:12:30,565] [INFO] [celery.beat] beat: Starting...
2024-07-31T07:12:30.000Z [2024-07-31 07:12:30,574] [CRITICAL] [celery.beat] beat raised exception <class '_dbm.error'>: error('cannot add item to database')
2024-07-31T07:12:30.000Z [2024-07-31 07:12:30,576] [WARNING] [celery.redirected]
2024-07-31T07:12:30.000Z [2024-07-31 07:12:30,576] [WARNING] [celery.redirected]   File "/usr/lib/python3.10/shelve.py", line 111, in __getitem__
2024-07-31T07:12:30.000Z [2024-07-31 07:12:30,576] [WARNING] [celery.redirected] KeyError
2024-07-31T07:12:30.000Z [2024-07-31 07:12:30,576] [WARNING] [celery.redirected] Traceback (most recent call last):
2024-07-31T07:12:30.000Z [2024-07-31 07:12:30,576] [WARNING] [celery.redirected] value = self.cache[key]
----
...
----
2024-07-31T07:12:30.000Z [2024-07-31 07:12:30,686] [WARNING] [celery.redirected] self[key] = entry
2024-07-31T07:12:30.000Z [2024-07-31 07:12:30,687] [WARNING] [celery.redirected] .
2024-07-31T07:12:30.000Z [2024-07-31 07:12:30,687] [WARNING] [celery.redirected] :
2024-07-31T07:12:30.000Z [2024-07-31 07:12:30,687] [WARNING] [celery.redirected] _dbm
2024-07-31T07:12:30.000Z [2024-07-31 07:12:30,687] [WARNING] [celery.redirected] cannot add item to database
2024-07-31T07:12:30.000Z [2024-07-31 07:12:30,687] [WARNING] [celery.redirected] error
2024-07-31T07:12:30.000Z [2024-07-31 07:12:30,687] [WARNING] [celery.redirected] self.dict[key.encode(self.keyencoding)] = f.getvalue()
2024-07-31T07:12:30.000Z __    -    ... __   -        _
2024-07-31T07:12:30.000Z _dbm.error: cannot add item to database
2024-07-31T07:12:30.000Z celery beat v5.4.0 (opalescent) is starting.
2024-07-31T07:12:30.000Z f = BytesIO(self.dict[key.encode(self.keyencoding)])
2024-07-31T07:12:30.000Z humanize_seconds(self.scheduler.max_interval))
2024-07-31T07:12:30.000Z return self.get_scheduler()
2024-07-31T07:12:30.000Z return super().__get__(instance, owner)
2024-07-31T07:12:30.000Z return symbol_by_name(self.scheduler_cls, aliases=aliases)(
2024-07-31T07:12:30.000Z self._create_schedule()
2024-07-31T07:12:30.000Z self._store['entries']
2024-07-31T07:12:30.000Z self._store['entries'] = {}
2024-07-31T07:12:30.000Z self.dict[key.encode(self.keyencoding)] = f.getvalue()
2024-07-31T07:12:30.000Z self.setup_schedule()
2024-07-31T07:12:30.000Z service.start()
2024-07-31T07:12:30.000Z super().__init__(*args, **kwargs)
2024-07-31T07:12:30.000Z val = self.func(instance)
2024-07-31T07:12:30.000Z value = self.cache[key]

Tell me if the whole log file is necessary.

I tried to restart the app, but didn't resolved my problem 😞
If someone experienced the same problem can help me !

Thanks

]]>https://forum.cloudron.io/topic/12195/high-cpu-utilization-since-update-1-25-1RSS for NodeFri, 17 Jul 2026 14:24:31 GMTWed, 31 Jul 2024 08:38:09 GMT60<![CDATA[Reply to High cpu utilization since update 1.25.1 on Mon, 31 Mar 2025 11:37:17 GMT]]>Also just run into this issue. Deleted the .db.db file which fixed it (for now) com.paperlessng.cloudronapp@1.35.0

]]>https://forum.cloudron.io/post/104793https://forum.cloudron.io/post/104793Mon, 31 Mar 2025 11:37:17 GMT<![CDATA[Reply to High cpu utilization since update 1.25.1 on Thu, 03 Apr 2025 14:10:24 GMT]]>Yup, its up again. Took 1 day. Maybe cron/automating the deletion of the .db.db file is something that I will try.

]]>https://forum.cloudron.io/post/105100https://forum.cloudron.io/post/105100Thu, 03 Apr 2025 14:10:24 GMT<![CDATA[Reply to High cpu utilization since update 1.25.1 on Mon, 31 Mar 2025 14:25:13 GMT]]>It was going crazy over the weekend for me too and of course, it happened the only time when I didn't check it (been monitoring it daily nowadays). I'll stick to using the Quickscan app on iOS which has OCR and will have it upload to my Nextcloud directly. Adios papersh*t.

]]>https://forum.cloudron.io/post/104819https://forum.cloudron.io/post/104819Mon, 31 Mar 2025 14:25:13 GMT<![CDATA[Reply to High cpu utilization since update 1.25.1 on Mon, 31 Mar 2025 11:37:17 GMT]]>Also just run into this issue. Deleted the .db.db file which fixed it (for now) com.paperlessng.cloudronapp@1.35.0

]]>https://forum.cloudron.io/post/104793https://forum.cloudron.io/post/104793Mon, 31 Mar 2025 11:37:17 GMT<![CDATA[Reply to High cpu utilization since update 1.25.1 on Mon, 31 Mar 2025 00:26:08 GMT]]>@lukasgabriel at this point consider it a feature rather than a bug

]]>https://forum.cloudron.io/post/104732https://forum.cloudron.io/post/104732Mon, 31 Mar 2025 00:26:08 GMT<![CDATA[Reply to High cpu utilization since update 1.25.1 on Sun, 30 Mar 2025 22:07:34 GMT]]>Same problem still happening...

]]>https://forum.cloudron.io/post/104725https://forum.cloudron.io/post/104725Sun, 30 Mar 2025 22:07:34 GMT<![CDATA[Reply to High cpu utilization since update 1.25.1 on Thu, 13 Feb 2025 06:02:56 GMT]]>@joseph yes, deleting the database fixes the problem temporarily.

]]>https://forum.cloudron.io/post/101677https://forum.cloudron.io/post/101677Thu, 13 Feb 2025 06:02:56 GMT<![CDATA[Reply to High cpu utilization since update 1.25.1 on Wed, 12 Feb 2025 06:22:44 GMT]]>@flwd thanks for the clarification. I think the original issue atleast was narrowed down to celery database going corrupt and it's not related to volume type. Have you tried deleting that database pointed out earlier in the thread?

]]>https://forum.cloudron.io/post/101619https://forum.cloudron.io/post/101619Wed, 12 Feb 2025 06:22:44 GMT<![CDATA[Reply to High cpu utilization since update 1.25.1 on Tue, 11 Feb 2025 19:21:32 GMT]]>@joseph it occured in every installation variant i tried, independant of the CIFS mount and config.

]]>https://forum.cloudron.io/post/101600https://forum.cloudron.io/post/101600Tue, 11 Feb 2025 19:21:32 GMT<![CDATA[Reply to High cpu utilization since update 1.25.1 on Tue, 11 Feb 2025 18:07:23 GMT]]>@joseph installation typo would be my guess πŸ™‚

]]>https://forum.cloudron.io/post/101593https://forum.cloudron.io/post/101593Tue, 11 Feb 2025 18:07:23 GMT<![CDATA[Reply to High cpu utilization since update 1.25.1 on Tue, 11 Feb 2025 16:06:43 GMT]]>@flwd said in High cpu utilization since update 1.25.1:

The issue recurred in every constellation

What does this mean?

]]>https://forum.cloudron.io/post/101589https://forum.cloudron.io/post/101589Tue, 11 Feb 2025 16:06:43 GMT<![CDATA[Reply to High cpu utilization since update 1.25.1 on Tue, 11 Feb 2025 15:52:54 GMT]]>Same issue for me. I ran cloudron as a VM on unraid and on proxmox, on 2 different hosts and with and without a CIFS mount as consume folder. The issue recurred in every constellation.

]]>https://forum.cloudron.io/post/101588https://forum.cloudron.io/post/101588Tue, 11 Feb 2025 15:52:54 GMT<![CDATA[Reply to High cpu utilization since update 1.25.1 on Mon, 10 Feb 2025 15:28:51 GMT]]>Just an FYI, it's still recurring on v.1.3.2.

]]>https://forum.cloudron.io/post/101539https://forum.cloudron.io/post/101539Mon, 10 Feb 2025 15:28:51 GMT<![CDATA[Reply to High cpu utilization since update 1.25.1 on Thu, 31 Oct 2024 14:05:38 GMT]]>It's going nuts again after it updated to 1.27 then 1.28 today.

image.png

edit: This time I had to delete the celery file and restart the app twice for the fix to work.

]]>https://forum.cloudron.io/post/96278https://forum.cloudron.io/post/96278Thu, 31 Oct 2024 14:05:38 GMT<![CDATA[Reply to High cpu utilization since update 1.25.1 on Fri, 25 Oct 2024 16:55:03 GMT]]>Do we know by now what is causing this problem? I have had it for the second time today. Last time about a Month ago. The fix worked in both cases but I guess it would still be interesting to know why this is happening.

]]>https://forum.cloudron.io/post/96073https://forum.cloudron.io/post/96073Fri, 25 Oct 2024 16:55:03 GMT<![CDATA[Reply to High cpu utilization since update 1.25.1 on Sat, 31 Aug 2024 14:21:23 GMT]]>CPU is high again on Paperless-ngx 2.11.6. Deleting the celery file and restarting still works as a temp fix. I did turn off the server today and updated the BIOS. Idk if that has anything to do with it.

]]>https://forum.cloudron.io/post/93589https://forum.cloudron.io/post/93589Sat, 31 Aug 2024 14:21:23 GMT<![CDATA[Reply to High cpu utilization since update 1.25.1 on Mon, 12 Aug 2024 20:17:34 GMT]]>@nebulon I uploaded the corrupted db file upstream as requested.. edit: just realized I can copy the github link https://github.com/user-attachments/files/16591433/celerybeat-schedule.db.zip

I'll keep the cloned app for the time being. Please let me know if you need more info.

]]>https://forum.cloudron.io/post/92660https://forum.cloudron.io/post/92660Mon, 12 Aug 2024 20:17:34 GMT<![CDATA[Reply to High cpu utilization since update 1.25.1 on Sun, 11 Aug 2024 11:47:36 GMT]]>@necrevistonnezr we still don’t know the root cause of the file corruption. It could happen again anytime. Upstream closed the discussion and moved on. It looks like only Cloudron users are affected on existing installs and not on a fresh app. App update snafu?

Btw, upstream said the +500mb ram on idle is normal. I don’t get why it should be that high.

]]>https://forum.cloudron.io/post/92577https://forum.cloudron.io/post/92577Sun, 11 Aug 2024 11:47:36 GMT<![CDATA[Reply to High cpu utilization since update 1.25.1 on Sun, 11 Aug 2024 11:11:05 GMT]]>@humptydumpty said in High cpu utilization since update 1.25.1:

Shoutout to stumpylog (paperless maintainer) for the fix. Using the file manager, browse to /app/data/data and delete celerybeat-schedule.db.db and restart the server app. CPU usage went down for me. Hopefully, it stays that way.

So it was not a Cloudron issue in the end?

]]>https://forum.cloudron.io/post/92575https://forum.cloudron.io/post/92575Sun, 11 Aug 2024 11:11:05 GMT<![CDATA[Reply to High cpu utilization since update 1.25.1 on Sun, 11 Aug 2024 07:37:32 GMT]]>@humptydumpty said in High cpu utilization since update 1.25.1:

Shoutout to stumpylog (paperless maintainer) for the fix. Using the file manager, browse to /app/data/data and delete celerybeat-schedule.db.db and restart the server app. CPU usage went down for me. Hopefully, it stays that way.

image.png

image.png

This solution solved the issue for me! Thanks!!πŸ™

]]>https://forum.cloudron.io/post/92574https://forum.cloudron.io/post/92574Sun, 11 Aug 2024 07:37:32 GMT<![CDATA[Reply to High cpu utilization since update 1.25.1 on Sat, 10 Aug 2024 21:44:47 GMT]]>Shoutout to stumpylog (paperless maintainer) for the fix. Using the file manager, browse to /app/data/data and delete celerybeat-schedule.db.db and restart the server app. CPU usage went down for me. Hopefully, it stays that way.

image.png

image.png

]]>https://forum.cloudron.io/post/92570https://forum.cloudron.io/post/92570Sat, 10 Aug 2024 21:44:47 GMT<![CDATA[Reply to High cpu utilization since update 1.25.1 on Sat, 10 Aug 2024 18:28:39 GMT]]>Googling this suggests trying to remove a corrupted file eg data/celerybeat.db

https://stackoverflow.com/questions/12362045/django-celery-error-when-starting-celerybeat-cannot-add-item-to-database

]]>https://forum.cloudron.io/post/92603https://forum.cloudron.io/post/92603Sat, 10 Aug 2024 18:28:39 GMT<![CDATA[Reply to High cpu utilization since update 1.25.1 on Sat, 10 Aug 2024 16:33:39 GMT]]>@nebulon upstream closed the ticket and said the following:

Perhaps a better question for cloudron not a bug report here. Memory yes, 100% CPU, no.

I appreciate that cloud hosting companies tend to push users to the project itself but since the only people having this problem are those hosting on cloudron (there have been no similar reports lately and I think there would be since it sounds like the app is effectively broken) I think the more logical conclusion is that this is a cloudron issue not a paperless one.

Could you please get in touch with them and sort this out?

]]>https://forum.cloudron.io/post/92567https://forum.cloudron.io/post/92567Sat, 10 Aug 2024 16:33:39 GMT<![CDATA[Reply to High cpu utilization since update 1.25.1 on Sat, 10 Aug 2024 15:51:16 GMT]]>Upstream ticket opened here: https://github.com/paperless-ngx/paperless-ngx/issues/7439

]]>https://forum.cloudron.io/post/92564https://forum.cloudron.io/post/92564Sat, 10 Aug 2024 15:51:16 GMT<![CDATA[Reply to High cpu utilization since update 1.25.1 on Sat, 10 Aug 2024 14:45:53 GMT]]>This very much reads like task tries to work a job which then fails due to a database conflict. Since that job never finishes it just retries in a loop. I think it is worth if you report this upstream as it kinda seems like a regression in the app itself.

]]>https://forum.cloudron.io/post/92559https://forum.cloudron.io/post/92559Sat, 10 Aug 2024 14:45:53 GMT