AI on Cloudron

[LoudLemur]

RAG with documents in a Docker
https://github.com/SciPhi-AI/R2R

[LoudLemur]

StabilityMatrix

If you use a lot of inferencing software, one thing this can do is have the models kept in one folder, which saves storage and bandwidth.

https://github.com/LykosAI/StabilityMatrix/releases/tag/v2.14.0

[LoudLemur]

AI Search with Lepton
This helps you have an ai language model which makes internet searches.

An interesting feature that Cloudron might like is its one-click deployment tool.

https://github.com/leptonai/search_with_lepton?tab=readme-ov-file

[jdaviescoates]

Apparently this is one of the best open source models yet:

https://qwenlm.github.io/blog/qwen3/

[LoudLemur]

LLM VRAM Calculator

If you would like to know what models your VRAM can handle, you can check with this online tool:

https://apxml.com/tools/vram-calculator

[LoudLemur]

@jdaviescoates I just tried it.
Qwen 3 is very good!

[LoudLemur]

Live Search is available on Grok's API without charge for a limited time for beta testing:

https://docs.x.ai/docs/guides/live-search

search through realtime data from 𝕏, the internet, trending news, RSS feeds,

[necrevistonnezr]

@LoudLemur said in AI on Cloudron:

Live Search is available on Grok's API without charge for a limited time for beta testing:

https://docs.x.ai/docs/guides/live-search

search through realtime data from 𝕏, the internet, trending news, RSS feeds,

In the US, Musk is simply walking into the authorities with a presidential power of attorney and copying everything he can. Once the files are copied, they end up in the system of Musk's X AI. Or they are poured into Palantir software with the aim of creating deportation lists.
So please boycott this PoS and his PoS endeavors.

[marcusquinn]

@necrevistonnezr Interesting take. Companies are already more powerful that governments in many ways.

[necrevistonnezr]

@marcusquinn it’s from an interview with one of the best-known network political activists in Germany Markus Beckedahl. The trained IT merchant created the online medium netzpolitik.org and is one of the founders of the re:publica conference. He is also co-founder of the Society for Freedom Rights, who bring lawsuits dealing with constitutional rigths to court on behalf of citizens. He blogs at digitalpolitik.de.
See https://www.heise.de/news/Interview-mit-Markus-Beckedahl-Musk-putscht-mit-Software-statt-Panzern-10379742.html (German)

[marcusquinn]

@necrevistonnezr Yeah, seen some not-good reading on demographic targeting by those capturing system admin permissions for cultural preference/detriment/targeting objectives.

Doesn't matter what these company leaders and politicians say or do, we all know the power admin permissions can give for the nefarious, and the power that vast amounts of personal data can give for tracking and algo tuning for mind-bending targets.

[DNA PC]

https://localai.io/basics/getting_started/

Would be super interesting to get this on Cloudron, they have a Docker port.

[LoudLemur]

US$4000 mini super computer:
https://wccftech.com/nvidia-mini-supercomputer-the-dgx-spark-launches-this-month/