@nebulon I followed this advice, so I think is not a single command to run but a series of.

Thanks a lot

@eddowding said in OpenWebUI + Anthropic / Claude:

You don't need to!

Go to Workspace > Functions > Add

Then paste in the raw code from:
https://github.com/open-webui/pipelines/blob/main/examples/pipelines/providers/anthropic_manifold_pipeline.py

Then click settings and enter the API key from Anthropic.

@robw

If we installed the NVIDIA Ubuntu GPU drivers in the Cloudron OS and started the OpenWebUI container with the GPU switch, would it just work?

Not necessarily, it depends on the GPU. While it may seem like a good idea, results will be very random. Also, nouveau (or whatever they're called now) drivers are the worst available out there. I've only had good results with nvidia official drivers.

Does installing the GPU drivers interfere with Cloudron's upgrade processes? (In our case, we wouldn't mind having to manage the GPU drivers separately, we don't expect Cloudron to manage non-native additions.)

Yes. Nvidia drivers are a pain to manage and often need debugging.

thanks I was able to reproduce and fix it. It will be part of the next package update

Awesome, now everything works just fine...salute for cloudron team

We would have to adjust the package to pre-configure the app to use optionally redis. However this is only relevant for multi-instance setups which is anyways not the case for Cloudron and only relevant for instances with many concurrent users, which is likely not the main focus for this app on Cloudron. So so far this optional redis is out of scope.

From the Changelog: Enhanced load balancing capabilities for multiple instance setups

The Open Web UI project initially started out as a Ollama frontend. To run local models, one needs GPU support and Cloudron currently does not have GPU support. Over time, Open Web UI has shifed to be a frontend for OpenAI API compatible services . It is also now possible to configure Ollama in another computer which has a GPU and then point Open Web UI to use that Ollama.

I guess we can mark this as stable since it's still useful while we figure out GPU support in Cloudron.

@vladimir-d In the future, I believe we should let the app handle most of it's configuration, environment-wise.

Variables such as RAG_EMBEDDING_MODEL, etc. are all set via the application UI and saved in the config.json file.

Now, for the initial issue of this topic, I've solved it by resetting the embedding model engine.

@firmansi not needed. apps are packaged as containers that have the right dependencies

LiteLLM is getting fairly mainstream imo (although things are moving very fast) and does more stuff than just accessing other models (load balancing, logging, spending checks…) so have a LiteLLM process started besides OpenWebUI would be useful.
In the meantime, I've followed this specific comment (https://github.com/open-webui/open-webui/issues/3288#issuecomment-2219524566) from the thread and I was able to get the anthropic/claude models with just configuration of openwebui, so that's already a bit of a win.

@Jack-613 this is a Cloudron related forum, so if you are not using OpenWebUI on Cloudron, then you should probably ask in the upstream project instead.

😖

Ah ok. I was under the impression that ollama was listening just on the localhost inside this app. If its not, this would work then. I'll test.

@eddowding said in Which models know most about Doughnut Economics?:

Does it matter much if you can add docs to them and have them answer intelligently?

I tried that yesterday. It didn't help in the slightest.

@taowang maybe worth reporting upstream? I think config.py is source code and not a configurable file. Even in their docker it's not stashed.

@shrey said in Using Mistral API seems broken on cloudron:

@nebulon Thanks!
Indeed, that was it.

RAM was set at 2GB. Whereas, it seems to require a min. of 5.5GB to function.

That's right, and if you can provide it with at least 8-16GB of RAM you will see a huge difference. These things are meant to consume large resources for now, but it's getting better.

I already have a volume, but I will create one more and mount it!