We package several AI servers for the community store (text embeddings, reranking, speech, LLM inference). Two platform limits come up in almost every one, and both look cheap to lift. They sit alongside the gpu capability (topic 15756) and /dev/shm size (topic 15844) you have already agreed to, and are not a bump of either.
1. A start period for the health check. Several model servers only bind their HTTP port after warming up the model, which takes tens of seconds on a CPU. During that window healthCheckPath gets connection-refused rather than a 503, and the app restart-loops before it is ever ready. A local docker run never shows this, because Docker does not health-check during warm-up, so it only appears on a real box. Our workaround is an nginx shim inside the image that answers /health at once and proxies everything else. It works, but every package has to rediscover it. Suggested shape: an optional healthCheckStartPeriod in seconds, like Docker's own HEALTHCHECK --start-period, during which failures are not counted.
2. A per-app proxy read timeout. The reverse proxy cuts a request at about 60 seconds, and it cannot be set per app. A cold-model inference call, a long summary or a large synchronous job hits it, while the app's own server (gunicorn at 600 seconds, for example) would have waited. Streaming responses help where the app supports them, but many endpoints do not stream. Suggested shape: an optional proxyReadTimeout in seconds, with today's value as the default and a sensible maximum.
Both are optional, both default to today's behaviour, and older boxes can ignore them, like other unknown fields.
