Baserow 2.3.1 spawns 29 celery workers → daily OOM (worker env var ignored)

[sclemens]

Hi,

since updating to package 1.33.x (Baserow 2.3.1) my instance restarts several times a day due to OOM, even after raising the memory limit from 4 GB to 6 GB.

Diagnosis

ps aux --sort=-rss inside the container shows 29 celery processes: three queue masters (celery, export, and the new automation_workflow queue introduced with 2.3) each forking ~8 children (= CPU core count) at ~225 MB RSS each:

cloudron  32  ... celery -A baserow worker -l INFO -Q automation_workflow
cloudron  36  ... celery -A baserow worker -l INFO -Q export
cloudron  39  ... celery -A baserow worker -l INFO -Q celery
(+ ~8 forked children per queue)

Note the workers are started without any --concurrency parameter, so celery defaults to one fork per CPU core, per queue.

I have export BASEROW_AMOUNT_OF_WORKERS=1 set in /app/data/env.sh (sourced correctly at line 15 of start.sh), but it has no effect:

grep -rn "AMOUNT_OF_WORKERS\|concurrency" /app/pkg/start.sh
# (no output)

This variable is only consumed by the upstream Baserow image's entrypoint (which translates it into --concurrency). The Cloudron package's start.sh launches the workers directly via supervisord without that translation, so the variable is silently ignored. With three full fork pools this makes 4 GB (previously fine) insufficient, and the new third queue explains why it surfaced with the 2.3 update.

Suggested fix

Pass --concurrency ${BASEROW_AMOUNT_OF_WORKERS:-1} (or a sensible default) to the celery worker commands in the supervisor config / start.sh, like the upstream image does.

Workaround for anyone affected

Since celery reads its config from Django settings (namespace="CELERY"), you can force the concurrency via a sitecustomize.py. Create /app/data/sitecustomize.py:

import os
os.environ["DJANGO_SETTINGS_MODULE"] = "sitecustomize"

from baserow.config.settings.base import *  # noqa
CELERY_WORKER_CONCURRENCY = 1

and add to /app/data/env.sh:

export PYTHONPATH="/app/data:$PYTHONPATH"

After a restart the process count drops from 29 to 8 and memory usage is back to normal. (Remember to remove this again once the package is fixed.)

Happy to provide more logs if needed. Thanks!