Hey everyone,
I have had paperless running on Cloudron for several months without a Problem.
Now I realized that the E-Mail importer is stuck. When I tried to find out why I saw that celery is stuck in a restart loop.
I restarted the app and also the server.
When I do a clean reinstallation on my server the error does not show up. So I could recover from a backup. But if possible I would be able to solve the problem under the hood.
Anyone an Idea what might be going on?
Thank you for your help!
2024-09-02T11:09:52.000Z __ - ... __ - _
2024-09-02T11:09:52.000Z _dbm.error: cannot add item to database
2024-09-02T11:09:52.000Z celery beat v5.4.0 (opalescent) is starting.
2024-09-02T11:09:52.000Z f = BytesIO(self.dict[key.encode(self.keyencoding)])
2024-09-02T11:09:52.000Z humanize_seconds(self.scheduler.max_interval))
2024-09-02T11:09:52.000Z return self.get_scheduler()
2024-09-02T11:09:52.000Z return super().__get__(instance, owner)
2024-09-02T11:09:52.000Z return symbol_by_name(self.scheduler_cls, aliases=aliases)(
2024-09-02T11:09:52.000Z self._create_schedule()
2024-09-02T11:09:52.000Z self._store['entries']
2024-09-02T11:09:52.000Z self._store['entries'] = {}
2024-09-02T11:09:52.000Z self.dict[key.encode(self.keyencoding)] = f.getvalue()
2024-09-02T11:09:52.000Z self.setup_schedule()
2024-09-02T11:09:52.000Z service.start()
2024-09-02T11:09:52.000Z super().__init__(*args, **kwargs)
2024-09-02T11:09:52.000Z val = self.func(instance)
2024-09-02T11:09:52.000Z value = self.cache[key]
2024-09-02T11:09:53.000Z 2024-09-02 11:09:53,996 INFO exited: paperless-scheduler.service (exit status 1; not expected)
2024-09-02T11:09:55.000Z 2024-09-02 11:09:54,999 INFO spawned: 'paperless-scheduler.service' with pid 620
2024-09-02T11:09:56.000Z 2024-09-02 11:09:56,001 INFO success: paperless-scheduler.service entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-09-02T11:09:57.000Z
2024-09-02T11:09:57.000Z
2024-09-02T11:09:57.000Z
2024-09-02T11:09:57.000Z
2024-09-02T11:09:57.000Z . broker -> redis://redis-8c6815a8-fe2f-4f22-bcdf-fd9c324f5b7c:6379//
2024-09-02T11:09:57.000Z . db -> /app/data/data/celerybeat-schedule.db
2024-09-02T11:09:57.000Z . loader -> celery.loaders.app.AppLoader
2024-09-02T11:09:57.000Z . logfile -> [stderr]@%INFO
2024-09-02T11:09:57.000Z . maxinterval -> 5.00 minutes (300s)
2024-09-02T11:09:57.000Z . scheduler -> celery.beat.PersistentScheduler
2024-09-02T11:09:57.000Z Configuration ->
2024-09-02T11:09:57.000Z During handling of the above exception, another exception occurred:
2024-09-02T11:09:57.000Z During handling of the above exception, another exception occurred:
2024-09-02T11:09:57.000Z During handling of the above exception, another exception occurred:
2024-09-02T11:09:57.000Z During handling of the above exception, another exception occurred:
2024-09-02T11:09:57.000Z File "/usr/lib/python3.10/functools.py", line 981, in __get__
2024-09-02T11:09:57.000Z File "/usr/lib/python3.10/shelve.py", line 111, in __getitem__
2024-09-02T11:09:57.000Z File "/usr/lib/python3.10/shelve.py", line 113, in __getitem__
2024-09-02T11:09:57.000Z File "/usr/lib/python3.10/shelve.py", line 125, in __setitem__
2024-09-02T11:09:57.000Z File "/usr/local/lib/python3.10/dist-packages/celery/apps/beat.py", line 113, in start_scheduler
2024-09-02T11:09:57.000Z File "/usr/local/lib/python3.10/dist-packages/celery/beat.py", line 264, in __init__
2024-09-02T11:09:57.000Z File "/usr/local/lib/python3.10/dist-packages/celery/beat.py", line 513, in __init__
2024-09-02T11:09:57.000Z File "/usr/local/lib/python3.10/dist-packages/celery/beat.py", line 541, in setup_schedule
2024-09-02T11:09:57.000Z File "/usr/local/lib/python3.10/dist-packages/celery/beat.py", line 570, in _create_schedule
2024-09-02T11:09:57.000Z File "/usr/local/lib/python3.10/dist-packages/celery/beat.py", line 574, in _create_schedule
2024-09-02T11:09:57.000Z File "/usr/local/lib/python3.10/dist-packages/celery/beat.py", line 634, in start
2024-09-02T11:09:57.000Z File "/usr/local/lib/python3.10/dist-packages/celery/beat.py", line 668, in get_scheduler
2024-09-02T11:09:57.000Z File "/usr/local/lib/python3.10/dist-packages/celery/beat.py", line 677, in scheduler
2024-09-02T11:09:57.000Z File "/usr/local/lib/python3.10/dist-packages/kombu/utils/objects.py", line 40, in __get__
2024-09-02T11:09:57.000Z KeyError: 'entries'
2024-09-02T11:09:57.000Z KeyError: b'entries'
2024-09-02T11:09:57.000Z LocalTime -> 2024-09-02 11:09:57
2024-09-02T11:09:57.000Z Traceback (most recent call last):
2024-09-02T11:09:57.000Z Traceback (most recent call last):
2024-09-02T11:09:57.000Z Traceback (most recent call last):
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,474] [INFO] [celery.beat] beat: Starting...
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,485] [CRITICAL] [celery.beat] beat raised exception <class '_dbm.error'>: error('cannot add item to database')
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,486] [WARNING] [celery.redirected] Traceback (most recent call last):
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,487] [WARNING] [celery.redirected]
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,487] [WARNING] [celery.redirected] File "/usr/lib/python3.10/shelve.py", line 111, in __getitem__
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,487] [WARNING] [celery.redirected] :
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,487] [WARNING] [celery.redirected] KeyError
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,487] [WARNING] [celery.redirected] value = self.cache[key]
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,488] [WARNING] [celery.redirected]
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,488] [WARNING] [celery.redirected]
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,488] [WARNING] [celery.redirected] File "/usr/local/lib/python3.10/dist-packages/celery/beat.py", line 570, in _create_schedule
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,488] [WARNING] [celery.redirected] 'entries'
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,488] [WARNING] [celery.redirected] Traceback (most recent call last):
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,488] [WARNING] [celery.redirected] self._store['entries']
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,489] [WARNING] [celery.redirected]
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,489] [WARNING] [celery.redirected] File "/usr/lib/python3.10/shelve.py", line 113, in __getitem__
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,489] [WARNING] [celery.redirected] KeyError
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,489] [WARNING] [celery.redirected] f = BytesIO(self.dict[key.encode(self.keyencoding)])
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,490] [WARNING] [celery.redirected]
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,490] [WARNING] [celery.redirected]
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,490] [WARNING] [celery.redirected] File "/usr/local/bin/celery", line 8, in <module>
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,490] [WARNING] [celery.redirected] :
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,490] [WARNING] [celery.redirected] Traceback (most recent call last):
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,490] [WARNING] [celery.redirected] b'entries'
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,491] [WARNING] [celery.redirected]
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,491] [WARNING] [celery.redirected] File "/usr/local/lib/python3.10/dist-packages/celery/__main__.py", line 15, in main
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,491] [WARNING] [celery.redirected] File "/usr/local/lib/python3.10/dist-packages/celery/bin/celery.py", line 236, in main
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,491] [WARNING] [celery.redirected] sys.exit(_main())
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,491] [WARNING] [celery.redirected] sys.exit(main())
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,492] [WARNING] [celery.redirected]
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,492] [WARNING] [celery.redirected]
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,492] [WARNING] [celery.redirected] File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1157, in __call__
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,492] [WARNING] [celery.redirected] return celery(auto_envvar_prefix="CELERY")
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,492] [WARNING] [celery.redirected] return self.main(*args, **kwargs)
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,493] [WARNING] [celery.redirected]
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,493] [WARNING] [celery.redirected] File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1078, in main
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,493] [WARNING] [celery.redirected] rv = self.invoke(ctx)
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,494] [WARNING] [celery.redirected]
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,494] [WARNING] [celery.redirected] File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1434, in invoke
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,494] [WARNING] [celery.redirected] File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1688, in invoke
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,494] [WARNING] [celery.redirected] return _process_result(sub_ctx.command.invoke(sub_ctx))
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,495] [WARNING] [celery.redirected]
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,495] [WARNING] [celery.redirected]
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,495] [WARNING] [celery.redirected] File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 783, in invoke
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,495] [WARNING] [celery.redirected] File "/usr/local/lib/python3.10/dist-packages/click/decorators.py", line 33, in new_func
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,495] [WARNING] [celery.redirected] return __callback(*args, **kwargs)
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,495] [WARNING] [celery.redirected] return ctx.invoke(self.callback, **ctx.params)
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,496] [WARNING] [celery.redirected]
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,496] [WARNING] [celery.redirected]
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,496] [WARNING] [celery.redirected] File "/usr/local/lib/python3.10/dist-packages/celery/bin/base.py", line 135, in caller
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,496] [WARNING] [celery.redirected] return f(ctx, *args, **kwargs)
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,496] [WARNING] [celery.redirected] return f(get_current_context(), *args, **kwargs)
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,497] [WARNING] [celery.redirected]
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,497] [WARNING] [celery.redirected]
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,497] [WARNING] [celery.redirected] File "/usr/local/lib/python3.10/dist-packages/celery/apps/beat.py", line 84, in run
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,497] [WARNING] [celery.redirected] File "/usr/local/lib/python3.10/dist-packages/celery/bin/beat.py", line 72, in beat
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,497] [WARNING] [celery.redirected] return beat().run()
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,497] [WARNING] [celery.redirected] self.start_scheduler()
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,498] [WARNING] [celery.redirected]
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,498] [WARNING] [celery.redirected]
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,498] [WARNING] [celery.redirected] File "/usr/local/lib/python3.10/dist-packages/celery/apps/beat.py", line 113, in start_scheduler
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,498] [WARNING] [celery.redirected] File "/usr/local/lib/python3.10/dist-packages/celery/beat.py", line 634, in start
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,498] [WARNING] [celery.redirected] service.start()
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,499] [WARNING] [celery.redirected]
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,499] [WARNING] [celery.redirected] File "/usr/local/lib/python3.10/dist-packages/kombu/utils/objects.py", line 40, in __get__
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,499] [WARNING] [celery.redirected] humanize_seconds(self.scheduler.max_interval))
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,500] [WARNING] [celery.redirected]
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,500] [WARNING] [celery.redirected] File "/usr/lib/python3.10/functools.py", line 981, in __get__
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,500] [WARNING] [celery.redirected] return super().__get__(instance, owner)
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,500] [WARNING] [celery.redirected] val = self.func(instance)
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,501] [WARNING] [celery.redirected]
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,501] [WARNING] [celery.redirected] File "/usr/local/lib/python3.10/dist-packages/celery/beat.py", line 668, in get_scheduler
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,501] [WARNING] [celery.redirected] File "/usr/local/lib/python3.10/dist-packages/celery/beat.py", line 677, in scheduler
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,501] [WARNING] [celery.redirected] return self.get_scheduler()
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,502] [WARNING] [celery.redirected]
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,502] [WARNING] [celery.redirected]
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,502] [WARNING] [celery.redirected] File "/usr/local/lib/python3.10/dist-packages/celery/beat.py", line 513, in __init__
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,502] [WARNING] [celery.redirected] return symbol_by_name(self.scheduler_cls, aliases=aliases)(
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,502] [WARNING] [celery.redirected] super().__init__(*args, **kwargs)
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,503] [WARNING] [celery.redirected]
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,503] [WARNING] [celery.redirected]
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,503] [WARNING] [celery.redirected] File "/usr/local/lib/python3.10/dist-packages/celery/beat.py", line 264, in __init__
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,503] [WARNING] [celery.redirected] File "/usr/local/lib/python3.10/dist-packages/celery/beat.py", line 541, in setup_schedule
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,503] [WARNING] [celery.redirected] self._create_schedule()
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,503] [WARNING] [celery.redirected] self.setup_schedule()
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,504] [WARNING] [celery.redirected]
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,504] [WARNING] [celery.redirected]
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,504] [WARNING] [celery.redirected] File "/usr/lib/python3.10/shelve.py", line 125, in __setitem__
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,504] [WARNING] [celery.redirected] File "/usr/local/lib/python3.10/dist-packages/celery/beat.py", line 574, in _create_schedule
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,504] [WARNING] [celery.redirected] self._store['entries'] = {}
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,504] [WARNING] [celery.redirected] self.dict[key.encode(self.keyencoding)] = f.getvalue()
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,505] [WARNING] [celery.redirected] .
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,505] [WARNING] [celery.redirected] :
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,505] [WARNING] [celery.redirected] _dbm
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,505] [WARNING] [celery.redirected] cannot add item to database
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,505] [WARNING] [celery.redirected] error
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,684] [WARNING] [celery.redirected] Exception ignored in:
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,685] [WARNING] [celery.redirected]
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,685] [WARNING] [celery.redirected] File "/usr/lib/python3.10/shelve.py", line 162, in __del__
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,685] [WARNING] [celery.redirected] <function Shelf.__del__ at 0x7f94faea8f70>
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,685] [WARNING] [celery.redirected] Traceback (most recent call last):
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,686] [WARNING] [celery.redirected]
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,686] [WARNING] [celery.redirected]
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,686] [WARNING] [celery.redirected] File "/usr/lib/python3.10/shelve.py", line 144, in close
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,686] [WARNING] [celery.redirected] File "/usr/lib/python3.10/shelve.py", line 168, in sync
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,686] [WARNING] [celery.redirected] self.close()
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,686] [WARNING] [celery.redirected] self.sync()
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,687] [WARNING] [celery.redirected]
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,687] [WARNING] [celery.redirected] File "/usr/lib/python3.10/shelve.py", line 125, in __setitem__
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,687] [WARNING] [celery.redirected] _dbm
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,687] [WARNING] [celery.redirected] self.dict[key.encode(self.keyencoding)] = f.getvalue()
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,687] [WARNING] [celery.redirected] self[key] = entry
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,688] [WARNING] [celery.redirected] .
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,688] [WARNING] [celery.redirected] :
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,688] [WARNING] [celery.redirected] cannot add item to database
2024-09-02T11:09:57.000Z [2024-09-02 11:09:57,688] [WARNING] [celery.redirected] error