AI on Cloudron

[robi]

@LoudLemur are these on Alibaba?

[LoudLemur]

@robi I haven't seen them there. A Chinese ID like a wechat or phone might be needed. I'd be wary of knock offs and consider what people who have bought Chinese steel have had to say.

[LoudLemur]

MCP (Model Context Protocol)
https://modelcontextprotocol.io/introduction

According to the CEO of Anthropic, this is what is going to be writing 90% of code in 6 months, and almost all of it by the end of the year.

You can see some uses here:
https://github.com/punkpeye/awesome-mcp-servers

The brilliant "Fireship" has a fast introduction to MCP:
https://odysee.com/@fireship:6/i-gave-claude-root-access-to-my:b

It would require somebody with something like @robi or @Kubernetes skill levels to be able to use it or make worth of it on Cloudron.

[robi]

Aww, thanks @LoudLemur

Actually it's already part of the update for OpenWebUI which can use it.

There are also many MCP directory websites out there who are keeping lists of many as well as some better ones who are letting you run them remotely. One such is Smithery.ai

[jdaviescoates]

@robi said in AI on Cloudron:

One such is Smithery.com

I think you mean https://smithery.ai/

[robi]

@jdaviescoates Thanks again for the correction.

[necrevistonnezr]

@LoudLemur said in AI on Cloudron:

MCP (Model Context Protocol)
https://modelcontextprotocol.io/introduction

According to the CEO of Anthropic, this is what is going to be writing 90% of code in 6 months, and almost all of it by the end of the year.

MCP has been adopted by OpenAI and will hence be the de-facto standard https://techcrunch.com/2025/03/26/openai-adopts-rival-anthropics-standard-for-connecting-ai-models-to-data/

In a post on X on Wednesday, OpenAI CEO Sam Altman said that OpenAI will add support for Anthropic’s Model Context Protocol, or MCP, across its products, including the desktop app for ChatGPT. MCP is an open source standard that helps AI models produce better, more relevant responses to certain queries.

[robi]

It would be great to have additional MCP support in OWUI.
@girish or @nebulon can you take a look?

It requires a few things added to the package it seems.

https://github.com/open-webui/openapi-servers

Also, the import functions feature seems to be broken, although one can use the +
Add function manually and copy/paste the code from https://openwebui.com/functions

[robi]

For those that want to experience how simple it is to build an agent and imagine implementing it on Cloudron.. read on: https://github.com/voxmenthe/coding-agent/blob/main/tutorial.md

[LoudLemur]

RAG with documents in a Docker
https://github.com/SciPhi-AI/R2R

[LoudLemur]

StabilityMatrix

If you use a lot of inferencing software, one thing this can do is have the models kept in one folder, which saves storage and bandwidth.

https://github.com/LykosAI/StabilityMatrix/releases/tag/v2.14.0

[LoudLemur]

AI Search with Lepton
This helps you have an ai language model which makes internet searches.

An interesting feature that Cloudron might like is its one-click deployment tool.

https://github.com/leptonai/search_with_lepton?tab=readme-ov-file

[jdaviescoates]

Apparently this is one of the best open source models yet:

https://qwenlm.github.io/blog/qwen3/

[LoudLemur]

LLM VRAM Calculator

If you would like to know what models your VRAM can handle, you can check with this online tool:

https://apxml.com/tools/vram-calculator

[LoudLemur]

@jdaviescoates I just tried it.
Qwen 3 is very good!

[LoudLemur]

Live Search is available on Grok's API without charge for a limited time for beta testing:

https://docs.x.ai/docs/guides/live-search

search through realtime data from 𝕏, the internet, trending news, RSS feeds,

[necrevistonnezr]

@LoudLemur said in AI on Cloudron:

Live Search is available on Grok's API without charge for a limited time for beta testing:

https://docs.x.ai/docs/guides/live-search

search through realtime data from 𝕏, the internet, trending news, RSS feeds,

In the US, Musk is simply walking into the authorities with a presidential power of attorney and copying everything he can. Once the files are copied, they end up in the system of Musk's X AI. Or they are poured into Palantir software with the aim of creating deportation lists.
So please boycott this PoS and his PoS endeavors.

[marcusquinn]

@necrevistonnezr Interesting take. Companies are already more powerful that governments in many ways.

[necrevistonnezr]

@marcusquinn it’s from an interview with one of the best-known network political activists in Germany Markus Beckedahl. The trained IT merchant created the online medium netzpolitik.org and is one of the founders of the re:publica conference. He is also co-founder of the Society for Freedom Rights, who bring lawsuits dealing with constitutional rigths to court on behalf of citizens. He blogs at digitalpolitik.de.
See https://www.heise.de/news/Interview-mit-Markus-Beckedahl-Musk-putscht-mit-Software-statt-Panzern-10379742.html (German)

[marcusquinn]

@necrevistonnezr Yeah, seen some not-good reading on demographic targeting by those capturing system admin permissions for cultural preference/detriment/targeting objectives.

Doesn't matter what these company leaders and politicians say or do, we all know the power admin permissions can give for the nefarious, and the power that vast amounts of personal data can give for tracking and algo tuning for mind-bending targets.