LiteLLM removed from OpenWebUI, requires own separate container

[jagan]

It seems LiteLLM has been removed from OpenWebUI (Github thread) as an internal tool.

Now, the advice is to run LiteLLM separately in its own container.

Does anybody know the implications of this please?

Can we still use OpenAI compatible APIs?
What about OpenRouter API?

A how to on migrating the LiteLLM to external LLM has been posted in the documentation.

[cpa]

Any update on this front? LiteLLM is very useful in order to use different cloud backends with OpenWebUI (mistral, anthropic, gemini…). OpenWebUI without LiteLLM only supports local LLMs or OpenAI.

[girish]

Is the idea here that we have to run LiteLLM as a separate process ?

[girish]

Apparently, there is also this pipeline thing - https://github.com/open-webui/pipelines/tree/main/examples/pipelines/providers . https://github.com/open-webui/open-webui/issues/3288 has a tip to integrate with Anthropic using "functions"

[cpa]

LiteLLM is getting fairly mainstream imo (although things are moving very fast) and does more stuff than just accessing other models (load balancing, logging, spending checks…) so have a LiteLLM process started besides OpenWebUI would be useful.
In the meantime, I've followed this specific comment (https://github.com/open-webui/open-webui/issues/3288#issuecomment-2219524566) from the thread and I was able to get the anthropic/claude models with just configuration of openwebui, so that's already a bit of a win.

[jagan]

I have added LiteLLM to the App Wishlist, please upvote!
https://forum.cloudron.io/topic/13141/litellm-openrouter-self-hosted-alternative-proxy-provides-access-to-openai-bedrock-anthropic-gemini-etc?_=1737171723819

Paging @cpa

[robw]

This would certainly be a very valuable addition to Cloudron. I have upvoted. That said, I guess that the configuration process could be challenging since there doesn't appear to be any point-and-click UI for that.