LiteLLM removed from OpenWebUI, requires own separate container
-
wrote on Jun 22, 2024, 2:33 AM last edited by
It seems LiteLLM has been removed from OpenWebUI (Github thread) as an internal tool.
Now, the advice is to run LiteLLM separately in its own container.
Does anybody know the implications of this please?
Can we still use OpenAI compatible APIs?
What about OpenRouter API?A how to on migrating the LiteLLM to external LLM has been posted in the documentation.
-
-
wrote on Aug 21, 2024, 3:05 PM last edited by
Any update on this front? LiteLLM is very useful in order to use different cloud backends with OpenWebUI (mistral, anthropic, gemini…). OpenWebUI without LiteLLM only supports local LLMs or OpenAI.
-
Apparently, there is also this pipeline thing - https://github.com/open-webui/pipelines/tree/main/examples/pipelines/providers . https://github.com/open-webui/open-webui/issues/3288 has a tip to integrate with Anthropic using "functions"
-
wrote on Aug 21, 2024, 4:23 PM last edited by
LiteLLM is getting fairly mainstream imo (although things are moving very fast) and does more stuff than just accessing other models (load balancing, logging, spending checks…) so have a LiteLLM process started besides OpenWebUI would be useful.
In the meantime, I've followed this specific comment (https://github.com/open-webui/open-webui/issues/3288#issuecomment-2219524566) from the thread and I was able to get the anthropic/claude models with just configuration of openwebui, so that's already a bit of a win. -
-
wrote on Jan 18, 2025, 4:17 AM last edited by jagan Jan 18, 2025, 4:17 AM
I have added LiteLLM to the App Wishlist, please upvote!
https://forum.cloudron.io/topic/13141/litellm-openrouter-self-hosted-alternative-proxy-provides-access-to-openai-bedrock-anthropic-gemini-etc?_=1737171723819Paging @cpa
-
-
wrote on Feb 2, 2025, 1:27 PM last edited by
This would certainly be a very valuable addition to Cloudron. I have upvoted. That said, I guess that the configuration process could be challenging since there doesn't appear to be any point-and-click UI for that.