AI on Cloudron

LoudLemur

This ASR (Automatic Speech Recognition) software is impressive. It can work in real-time and perform transcriptions. I don't know whether it would be suitable for hosting on Cloudron.

https://huggingface.co/openai/whisper-large-v3

robi

@LoudLemur not really as it needs GPUs to be fast enough.

There is an even faster version with better quality output: https://huggingface.co/spaces/sanchit-gandhi/whisper-jax

..and you can run them for free on HF.

micmc

@LoudLemur

Demo: http://imagebind-llm.opengvlab.com/

Interesting vision model, but still a bit confused
I might try self-hosted version, have you?

micmc

@LoudLemur said in AI on Cloudron:

plugin.surf
Thousands of free as in beer ChatGPTs
https://plugin.surf/

Have chosen to not sleep anymore?

Nice find! Thanks for sharing.

robi

Someone sent me a message with a streamlined image generation interface for Krita https://github.com/Acly/krita-ai-diffusion

jagan

Check out Mistral's new version, Mixtral 8x7B, whose performance has now reached or exceeded OpenAI's GPT 3.5.

https://mistral.ai/news/mixtral-of-experts/

LoudLemur

@jagan LMstudio now supports this mixtral model.

It also supports vision models!

#### New
- **Vision models** support! 🎉 (BakLLaVA, LLaVA, Obsidian 3B)
  - Vision supported via chat UI and server API (same format as OpenAI API)
- **Mixtral 8x7B** support! 🎉  (Experimental)
- **Rich Error Messages** for model loading errors (includes system diagnostics info)
- **Model Inspector**: peek into the model file metadata to see trained n_ctx, and more details.
- [Local Server] New client code examples for chat + vision (incl. updated OpenAI SDK example)
- Auto detection for \*.preset.json in Hugging Face repositories
- Support for StableLM Zephyr, Deepseek Coder, Persimmon, Qwen (thanks to llama.cpp improvements)

#### Bug Fixes
- `prompt_tokens` are now correctly counted in API responses
- Fix for multi-part model file downloads```


Andrew Torba, CEO of https://gab.com, says that 2024 will have a totally based AI model for us all to use.

LoudLemur

Chat with a (self-hosted) AI that took 40 seconds to learn a Github repo and find out how it works. This might be useful for people who want to package software for Cloudron:

https://app.getonboardai.com/

LoudLemur

Enable search (via DuckDuckGo ) on your local llama:
https://github.com/mamei16/LLM_Web_search

Actually, this has some support now for using SearXNG as the search engine, so it would work with a Cloudron package.

https://github.com/mamei16/LLM_Web_search/commit/68a85f0eda681a5e890c0de36fcf5f14f88abb46

AmbroiseUnly

Is there any plan on adding AI-based projects to Cloudron?

I'm asking because I wanted to build something with OpenAI, and got stuck because the OpenAI API doesn't allow browsing the internet (unlike ChatGPT), which came as a surprise, and I'm looking to use a self-hosted solution instead.

LoudLemur

@AmbroiseUnly said in AI on Cloudron:

Is there any plan on adding AI-based projects to Cloudron?

I'm asking because I wanted to build something with OpenAI, and got stuck because the OpenAI API doesn't allow browsing the internet (unlike ChatGPT), which came as a surprise, and I'm looking to use a self-hosted solution instead.

It was said that this area of AI is developing so rapidly and so many changes are happening that a "wait and see" approach would be preferred. After a while (which in AI is not very long at all) it is hoped that some of the many applications will emerge as a leader and I think Cloudron will then think about supporting one of those.

Some people in this thread have already posted some very juicy candidates though!

Which applications or functions do people here think will most merit support on Cloudron?

robi

@AmbroiseUnly So far we have a few self-install how-to posts using the LAMP app.

However you can always find/hire/sponsor a developer to create a custom package for what you want and submit it to the community and Cloudron team which can then more easily add it to the App Store.