Weblate Celery Workers Stalled/Inconsistent on One Instance (TM Indexing Fails, but Stats Update Works)

[sfund]

Hello,

I'm looking for advice on a strange issue with one of my Weblate instances on Cloudron. I have two separate Weblate apps running on the same server. One works perfectly, but the other has an issue with its background tasks.

My setup is as follows:

• Two Weblate apps on the same Cloudron VM.
• Both are the same Weblate version and have identical memory allocations (4 GB).
• The goal is to use a manually uploaded TMX file as a Translation Memory.

The Problem on the Non-Working Instance:

• A manually imported Translation Memory (TMX file) provides no suggestions.
• Log analysis confirms the TM index is effectively empty, even after a successful file upload.
• Initially, component statistics (string/word counts) were not updating and remained at 0 after adding files.

Troubleshooting Steps & Key Findings:

• Confirmed all project and component settings are identical to the working instance.
• Confirmed memory limits are identical and the resource graphs show no signs of memory exhaustion.
• Multiple full app restarts from the Cloudron dashboard did not fix the issue.
• Restarting all celery-* services from the web terminal did not fix the issue.

Key Finding 1: The project-level Rebuild button for the TM erases the index of the manually uploaded file. The logs confirm it only re-indexes from version-controlled components, not manual uploads.

Key Finding 2: After I manually deleted and re-added a language to a component, the statistics for that component updated correctly.

I think that this final point proves that the background worker service (Celery) is not completely dead, but it is in an inconsistent state. It is processing some tasks (like the statistics update after a language change) but is clearly not processing others (like the Translation Memory import/indexing).

My conclusion is that the instance is in a persistently unstable state that requires a full reinstall.

Before I do that, my question is: Has anyone encountered this specific behavior where Celery workers are running but seem to ignore certain task queues (like TM indexing) while still processing others? Is there any known state that could cause this, or is a full reinstall the only reliable path forward?

Thanks for any insights.

[nebulon]

I have not encountered this issues and it seems very specific and hard to reproduce. However to debug this maybe further, you can run the app's jobs manually also within the webterminal into the app.

Not clear to me which celery job is required to to this but as an example if you run upervisorctl restart celery-memory then it will restart and execute https://git.cloudron.io/packages/weblate-app/-/blob/master/supervisor/celery-memory.conf?ref_type=heads#L3

Also maybe check in that instance if all services run via supervisorctl status

[nebulon]

@sfund did you get a chance to reproduce this when manually running the worker and maybe see some errors in the logs about that job?

[sfund]

Hi @nabulon,

Thanks for your help with this. I've figured out what was happening, and it turns out this wasn't an issue with the Celery workers. Both of my Weblate instances were behaving as designed.

The problem was that the entries from my uploaded TMX file were being set to "pending" status. According to the documentation (https://docs.weblate.org/en/latest/admin/memory.html), pending entries receive a quality penalty.

This penalty was significant enough that even my exact matches were rated too low to appear in automatic suggestions or be used for automatic translation.

I confirmed this when I manually translated one of the strings—after that, the automatic suggestion from the TM finally appeared for identical strings.

Thanks again for pointing me in the right direction!