Old images filling the disk

[james]

Hello @caspear
Please run the following commands and send me the output from your server:

docker ps -a --format '{{.ID}} {{.Image}} {{.Names}} {{.Status}} {{.Label "appId"}} {{.Label "isCloudronManaged"}}' | grep -i uptimekuma
docker images --digests | grep -i uptimekuma | head
grep -h "deleteImage: removing cloudron/louislam" /home/yellowtent/platformdata/logs/tasks/*.log | tail

[caspear]

# docker ps -a --format '{{.ID}} {{.Image}} {{.Names}} {{.Status}} {{.Label "appId"}} {{.Label "isCloudronManaged"}}' | grep -i uptimekuma
docker images --digests | grep -i uptimekuma | head
grep -h "deleteImage: removing cloudron/louislam" /home/yellowtent/platformdata/logs/tasks/*.log | tail
167109c4b757 cloudron/louislam.uptimekuma.app:202611011424070000 95a0b41d-0525-4b60-b62e-3bb8b11edaf2 Up 2 days 95a0b41d-0525-4b60-b62e-3bb8b11edaf2 true
cloudron/louislam.uptimekuma.app               202611011424070000          <none>                                                                    81ce16f02a7e   4 days ago      2.49GB
cloudron/louislam.uptimekuma.app               202610311425350000          sha256:585554c282e88cfa79efd6f6ef0e651700f394de3d2e5f7ae8f48bb612b160d8   555d251b4542   5 days ago      2.49GB
cloudron/louislam.uptimekuma.app               202610281425490000          sha256:15ade59f35a8aa256706bb8c7930b9a6a56270c2d2b3bb6b39d472c0100ac464   a8fee11920ee   8 days ago      2.49GB
cloudron/louislam.uptimekuma.app               202610231334320000          sha256:0c00116c5fb25f8c58ebd40505b849c26d0252220883403586d3ec64cc1ccdf7   5db34eb4f263   13 days ago     2.49GB
cloudron/louislam.uptimekuma.app               202610221334590000          sha256:7da13fbe7c4a0e14f0de5f000ffc773b68a573de349c5fd318deba7a79098c06   a8885b2c1967   2 weeks ago     2.49GB
cloudron/louislam.uptimekuma.app               202610201324590000          sha256:1f02d509787ccea7f2e016dc75ae5a32d36a2c0d8a394ad21d3b1fed943e0861   1fc29da1e576   2 weeks ago     2.49GB
cloudron/louislam.uptimekuma.app               202610101326010000          sha256:4b3938f95fa9b0dd525feacae43b8f9c2471e855177e0bcc8aebf41acf8be1c2   0ed29fbb1a3e   3 weeks ago     2.48GB
cloudron/louislam.uptimekuma.app               202610061328140000          sha256:47bdd8aa81cd64880741735676c6abac34835c142f96c0182c86fdb4895646a4   41ee44f20c5b   4 weeks ago     2.48GB
cloudron/louislam.uptimekuma.app               202609251853530000          sha256:d00039a73a9b8e6e88c1ad9040ba99a4f5f8998970538c0813c35171596b9f81   b784b501be0e   5 weeks ago     2.48GB
cloudron/louislam.uptimekuma.app               202609230702240000          sha256:91938385453ce51292f868822ccda85d4d562429a9da7b8099b1de31e8681d27   9c9a62d0439a   6 weeks ago     2.48GB

[james]

Hello @caspear
It looks like you did not correctly execute the commands or did some of them really return nothing?
There is a # at the start of the first command which would make it a comment in bash and execute nothing.

[caspear]

I pasted them together and so they executed sequentially. The # at the beginning is the root prompt.

I will execute them again one at a time so that it is more clear.

# docker ps -a --format '{{.ID}} {{.Image}} {{.Names}} {{.Status}} {{.Label "appId"}} {{.Label "isCloudronManaged"}}' | grep -i uptimekuma
167109c4b757 cloudron/louislam.uptimekuma.app:202611011424070000 95a0b41d-0525-4b60-b62e-3bb8b11edaf2 Up 2 days 95a0b41d-0525-4b60-b62e-3bb8b11edaf2 true

# docker images --digests | grep -i uptimekuma | head
cloudron/louislam.uptimekuma.app               202611011424070000          <none>                                                                    81ce16f02a7e   5 days ago      2.49GB
cloudron/louislam.uptimekuma.app               202610311425350000          sha256:585554c282e88cfa79efd6f6ef0e651700f394de3d2e5f7ae8f48bb612b160d8   555d251b4542   5 days ago      2.49GB
cloudron/louislam.uptimekuma.app               202610281425490000          sha256:15ade59f35a8aa256706bb8c7930b9a6a56270c2d2b3bb6b39d472c0100ac464   a8fee11920ee   8 days ago      2.49GB
cloudron/louislam.uptimekuma.app               202610231334320000          sha256:0c00116c5fb25f8c58ebd40505b849c26d0252220883403586d3ec64cc1ccdf7   5db34eb4f263   2 weeks ago     2.49GB
cloudron/louislam.uptimekuma.app               202610221334590000          sha256:7da13fbe7c4a0e14f0de5f000ffc773b68a573de349c5fd318deba7a79098c06   a8885b2c1967   2 weeks ago     2.49GB
cloudron/louislam.uptimekuma.app               202610201324590000          sha256:1f02d509787ccea7f2e016dc75ae5a32d36a2c0d8a394ad21d3b1fed943e0861   1fc29da1e576   2 weeks ago     2.49GB
cloudron/louislam.uptimekuma.app               202610101326010000          sha256:4b3938f95fa9b0dd525feacae43b8f9c2471e855177e0bcc8aebf41acf8be1c2   0ed29fbb1a3e   3 weeks ago     2.48GB
cloudron/louislam.uptimekuma.app               202610061328140000          sha256:47bdd8aa81cd64880741735676c6abac34835c142f96c0182c86fdb4895646a4   41ee44f20c5b   4 weeks ago     2.48GB
cloudron/louislam.uptimekuma.app               202609251853530000          sha256:d00039a73a9b8e6e88c1ad9040ba99a4f5f8998970538c0813c35171596b9f81   b784b501be0e   5 weeks ago     2.48GB
cloudron/louislam.uptimekuma.app               202609230702240000          sha256:91938385453ce51292f868822ccda85d4d562429a9da7b8099b1de31e8681d27   9c9a62d0439a   6 weeks ago     2.48GB

# grep -h "deleteImage: removing cloudron/louislam" /home/yellowtent/platformdata/logs/tasks/*.log | tail

Last command returns nothing - uptime kuma images never get deleted, that is indeed my bug report!

[james]

Hello @caspear

Thanks for the output. It rules out a leftover container holding the old images: only the current Kuma container exists.

Your current image has no digest (<none>), but every old image has a sha256: digest. Cloudron downloads app images under a registry name and then retags them, and an image downloaded that way has no digest.
It looks like your old images are being pulled a second time after an app update has deleted them.

My suspicion is a race between the nightly full backup and the app auto-update. By default both start at 23:00.

Please run these two commands:

docker image inspect --format '{{index .RepoTags 0}} {{.Metadata.LastTagTime}}' cloudron/louislam.uptimekuma.app:202610311425350000 cloudron/louislam.uptimekuma.app:202611011424070000

grep -H "uptimekuma.app:202610311425350000" /home/yellowtent/platformdata/logs/tasks/*.log /home/yellowtent/platformdata/logs/95a0b41d-0525-4b60-b62e-3bb8b11edaf2/apptask.log* | cut -c1-300

If the first command shows a time after the update on 1 November for the older tag, or a backup task log mentions the old tag after the update, that could confirm the race.

Until there is a fix, you can safely remove the old images with docker rmi <image>:<tag> for every tag except the one in use.

[james]

Hello @caspear

You might have found a "good" issue there.
Thanks for reporting! We are investigating this more detailed.

[caspear]

@james said:

grep -H "uptimekuma.app:202610311425350000" /home/yellowtent/platformdata/logs/tasks/.log /home/yellowtent/platformdata/logs/95a0b41d-0525-4b60-b62e-3bb8b11edaf2/apptask.log | cut -c1-300

# docker image inspect --format '{{index .RepoTags 0}} {{.Metadata.LastTagTime}}' cloudron/louislam.uptimekuma.app:202610311425350000 cloudron/louislam.uptimekuma.app:202611011424070000
cloudron/louislam.uptimekuma.app:202610311425350000 0001-01-01 00:00:00 +0000 UTC
cloudron/louislam.uptimekuma.app:202611011424070000 2026-11-01 23:00:41.030181353 +0000 UTC

# grep -H "uptimekuma.app:202610311425350000" /home/yellowtent/platformdata/logs/tasks/*.log /home/yellowtent/platformdata/logs/95a0b41d-0525-4b60-b62e-3bb8b11edaf2/apptask.log* | cut -c1-300
/home/yellowtent/platformdata/logs/tasks/12500.log:2026-11-01T23:32:35.295Z shell: services: /bin/bash -c docker run --rm --name=sqlite-95a0b41d-0525-4b60-b62e-3bb8b11edaf2             --net cloudron             --log-driver=none             -v /home/yellowtent/appsdata/95a0b41d-0525-4b60-b62e-3bb8b
/home/yellowtent/platformdata/logs/tasks/12500.log:2026-11-01T23:33:02.040Z shell: services: /bin/bash -c docker run --rm --name=sqlite-95a0b41d-0525-4b60-b62e-3bb8b11edaf2             --net cloudron             --log-driver=none             -v /home/yellowtent/appsdata/95a0b41d-0525-4b60-b62e-3bb8b
/home/yellowtent/platformdata/logs/95a0b41d-0525-4b60-b62e-3bb8b11edaf2/apptask.log:2026-11-01T01:00:12.842Z docker: downloadImage: cloudron/louislam.uptimekuma.app:202610311425350000
/home/yellowtent/platformdata/logs/95a0b41d-0525-4b60-b62e-3bb8b11edaf2/apptask.log:2026-11-01T01:00:12.847Z docker: pullImage: will pull registry.docker.com/cloudron/louislam.uptimekuma.app:202610311425350000. auth: no
/home/yellowtent/platformdata/logs/95a0b41d-0525-4b60-b62e-3bb8b11edaf2/apptask.log:2026-11-01T01:00:36.738Z docker: pullImage: {"status":"Status: Downloaded newer image for registry.docker.com/cloudron/louislam.uptimekuma.app:202610311425350000"}
/home/yellowtent/platformdata/logs/95a0b41d-0525-4b60-b62e-3bb8b11edaf2/apptask.log:2026-11-01T01:00:36.741Z docker: downloaded image registry.docker.com/cloudron/louislam.uptimekuma.app:202610311425350000 . error: false
/home/yellowtent/platformdata/logs/95a0b41d-0525-4b60-b62e-3bb8b11edaf2/apptask.log:2026-11-01T01:00:36.741Z docker: downloadImage: tagging registry.docker.com/cloudron/louislam.uptimekuma.app:202610311425350000 as cloudron/louislam.uptimekuma.app:202610311425350000
/home/yellowtent/platformdata/logs/95a0b41d-0525-4b60-b62e-3bb8b11edaf2/apptask.log:2026-11-01T01:00:36.748Z docker: downloadImage: untagging registry.docker.com/cloudron/louislam.uptimekuma.app:202610311425350000
/home/yellowtent/platformdata/logs/95a0b41d-0525-4b60-b62e-3bb8b11edaf2/apptask.log:2026-11-01T01:00:36.748Z docker: deleteImage: removing registry.docker.com/cloudron/louislam.uptimekuma.app:202610311425350000
/home/yellowtent/platformdata/logs/95a0b41d-0525-4b60-b62e-3bb8b11edaf2/apptask.log:2026-11-01T23:00:02.520Z shell: services: /bin/bash -c docker run --rm --name=sqlite-95a0b41d-0525-4b60-b62e-3bb8b11edaf2             --net cloudron             --log-driver=none             -v /home/yellowtent/appsd
/home/yellowtent/platformdata/logs/95a0b41d-0525-4b60-b62e-3bb8b11edaf2/apptask.log:2026-11-01T23:00:06.820Z shell: services: /bin/bash -c docker run --rm --name=sqlite-95a0b41d-0525-4b60-b62e-3bb8b11edaf2             --net cloudron             --log-driver=none             -v /home/yellowtent/appsd
/home/yellowtent/platformdata/logs/95a0b41d-0525-4b60-b62e-3bb8b11edaf2/apptask.log:2026-11-01T23:00:42.444Z docker: deleteImage: removing cloudron/louislam.uptimekuma.app:202610311425350000

[caspear]

Oh no!

To me this does not immediately feel like a race condition: These images are very reliably retained, and other types of images get cleaned up without any issues. So, why this one?

I am happy to add a cron job that deletes all but the 3-4 most recent ones, just to see if the issue is still recurring.

[james]

Hello @caspear
We already have narrowed it down to a race condition happening when the SQLite addon is used.
Uptimekuma uses the SQLite addon.

[caspear]

Fantastic!