Hello all. We have been doing a deep recce on what GPU support would take in 2026, and we think the landscape has shifted enough since it was last assessed to be worth a fresh summary. Posting the salient parts with links, in case it helps the team and anyone experimenting. The "patched docker" requirement appears to be gone. The last assessment (here) noted that NVIDIA seemed to need a patched Docker while Cloudron relies on the official Ubuntu packages. That was true for years, the old nvidia-docker stack did require a custom runtime, but the Container Device Interface (CDI) has since removed the requirement: Docker Engine supports CDI natively since 25.0 and enables it by default in the 28.x series (the CNCF CDI docs say from 28.2.0, the Docker docs say 28.3.0). The official Ubuntu 24.04 docker.io package now carries 28.2.2 in noble-updates and 27.5.1 in the security pocket (Launchpad). So the stock Ubuntu packages already have CDI: default-on at 28.2.2, or a one-line daemon.json feature flag on 27.5.1. With CDI there is no custom runtime and no default-runtime change. The host needs the NVIDIA driver plus nvidia-container-toolkit, which provides nvidia-ctk cdi generate and now ships an nvidia-cdi-refresh service that regenerates the spec automatically when the driver changes (NVIDIA docs, releases, currently 1.19.1). Containers then receive GPUs via --device nvidia.com/gpu=all, which maps to an ordinary device request in the Engine API. Quick check on any host: docker version --format '{{.Server.Version}}' docker info | grep -iA2 cdi # on a GPU host with a generated spec: docker run --rm --device nvidia.com/gpu=all ubuntu:24.04 nvidia-smi -L The platform change may be smaller than expected. The manifest already has the vaapi capability passing /dev/dri. An equivalent capability, or a per-app admin toggle, that appends a CDI device request at container create looks like most of the change, and it can no-op silently when no CDI spec exists, so GPU-less servers are unaffected. AMD compute (ROCm) would only additionally need /dev/kfd; Intel and AMD media is arguably covered by vaapi already. Two caveats from the research: The reliability risk has moved from Docker to the driver lifecycle. The recent report of a server left unbootable after a driver update is the failure class to design against: the -server LTS driver branch, apt holds, and a deliberate upgrade procedure rather than unattended driver updates. The container toolkit had several critical escape CVEs through 2024 and 2025, mostly in the legacy hook path. CDI mode plus a current toolkit is the safer default, and GPU servers are best treated as trusted-workload machines. On the testing-hardware constraint mentioned previously: hardware offers have been made in this thread before, and we are happy to run validation on real NVIDIA hardware and share results.

This is a regression of @openwebui version 0.10.1 and is already fixed by version 0.10.2. With the upcoming release of 0.10.2 this is fixed.

@dsp76 are you using OpenRouter? Perhaps you've just run out of credit or something? My updated instance of Open WebUI still seems to be working fine.

new package is out

The new package update is already in our release pipeline yes

@charlesnw I am running open terminal but in another container, i think it's best to maintain Open WebUI with original package

@charlesnw [image: 1773858777076-screenshot-2026-03-18-at-18.32.36-resized.png] you should invite some friends !

@nebulon it works...thanks cloudron team for fast update

Newest package that ships Open WebUI 0.7.2 resolved this

@omnichannel I’m using openwebui and ollama apps installed in the same Cloudron instance. In openwebui Admin settings, under Connections, I am using similar to what you have. Server connection is verified.

the new package now disables the max body check for the nginx within the app. I couldn't quite reproduce the issue in my setup, not sure how you ended up with such big queries, but hopefully it fixes your case.

Hello @harryz There is your problem. The model requires 8.7 GB and your system only has 7.9 GB of free memory left.

Installation of additional packages would be great! Would a quick fix be to make the directory "/app/code/venv/lib/python3.12/site-packages/" writable in the cloudron app (eg. symlink to a /app/data/site-packages directory)?

@girish ya, i think this is a good approach because in real use case, I think most people prefer to run inference models not in the same machine with openWebUI instance. OpenWebUI is good enough bundled with chromadb , sql lite or postgressql database and redis as one package and the inference models (including ollama) running in different servers.

Looks like this was released on Sep 23

@micmc all credit to the author of the repository

Thanks everyone for your assistance

Hello @firmansi Thanks for the feedback. I was only able to confirm it was working with a debug startup. So it is very good to read from you that it is in fact working.

It depend on the PDF itself I guess. I've uploaded one, and got a new error message. [image: 1753521763160-c7c88ba2-416b-499e-825b-64ffcb747c7c-image.png]