Issue indexer queue not getting processed, 8000+ unfinished jobs
This instance is a couple of years old, so it's been through some updates, and one server migration.
I tried deleting
/app/data/appdata/indexersin order to try and rebuild the indexing but since found that workers do not process that queue anyway. The issue indexer work queue is building up without getting worked on. Adding workers manually doesn’t help.
I found a note for a recommendation from the 1.15 release, stating:
It is recommended to ensure that queues are empty before upgrading using
gitea manager flush-queuesor on the admin pages.
Should I do that?
I went ahead and tried to flush the queue:
sudo -u git /home/git/gitea/gitea -c /run/gitea/app.ini manager flush-queues --timeout 20m
but even with that generous timeout value it timed out:
flush timeout with non-empty queues: issue_indexer
I've little to no idea of what I'm doing here btw.
I'm using Gitea for quite some years as well and this got me curious. The current queue can be seen in the webinterface of Gitea at
https://your-domain/admin/monitor. My queues however are at
If you have not done yet I would check the logs of gitea if there are any abnormalities in regards to queue handling. Error messages at startup?
The same website also allows to see stacktraces for the running processes.
In the end it may be a bug within Gitea that would need reporting and fixing upstream.
@girish Must've got it wrong somehow. Anyway,
--debugonly adds one line of little insight:
sudo -u git /home/git/gitea/gitea -c /run/gitea/app.ini manager flush-queues --debug 2023/01/16 09:45:29 ...dules/git/command.go:213:Run() [D] /usr/bin/git version flush timeout with non-empty queues: issue_indexer
@yusf I guess we have to ask upstream for ideas...
But I found https://github.com/go-gitea/gitea/issues/17563 . It suggests changing the indexer to db instead of the default bleve. Don't know if that will help your case.
I think another idea is to figure where these jobs are stored and clear that. There is a
tasktable, probably there?
app.inisolved the issue immediately. Thanks alot!
Now I'm curious to know what
dbmeans in this context (but the urgency is gone ). Should the Cloudron package have db as ISSUE_INDEXER_TYPE out of the box, for example?
Some Cloudron app config data got alot of legacy files from earlier versions, so sometimes it's hard to know what the true good defaults are.