AI on Cloudron

[LoudLemur]

@jagan LMstudio now supports this mixtral model.

It also supports vision models!

#### New
- **Vision models** support! 🎉 (BakLLaVA, LLaVA, Obsidian 3B)
  - Vision supported via chat UI and server API (same format as OpenAI API)
- **Mixtral 8x7B** support! 🎉  (Experimental)
- **Rich Error Messages** for model loading errors (includes system diagnostics info)
- **Model Inspector**: peek into the model file metadata to see trained n_ctx, and more details.
- [Local Server] New client code examples for chat + vision (incl. updated OpenAI SDK example)
- Auto detection for \*.preset.json in Hugging Face repositories
- Support for StableLM Zephyr, Deepseek Coder, Persimmon, Qwen (thanks to llama.cpp improvements)

#### Bug Fixes
- `prompt_tokens` are now correctly counted in API responses
- Fix for multi-part model file downloads```


Andrew Torba, CEO of https://gab.com, says that 2024 will have a totally based AI model for us all to use.

[LoudLemur]

Chat with a (self-hosted) AI that took 40 seconds to learn a Github repo and find out how it works. This might be useful for people who want to package software for Cloudron:

https://app.getonboardai.com/

[LoudLemur]

Enable search (via DuckDuckGo ) on your local llama:
https://github.com/mamei16/LLM_Web_search

Actually, this has some support now for using SearXNG as the search engine, so it would work with a Cloudron package.

https://github.com/mamei16/LLM_Web_search/commit/68a85f0eda681a5e890c0de36fcf5f14f88abb46

[AmbroiseUnly]

Is there any plan on adding AI-based projects to Cloudron?

I'm asking because I wanted to build something with OpenAI, and got stuck because the OpenAI API doesn't allow browsing the internet (unlike ChatGPT), which came as a surprise, and I'm looking to use a self-hosted solution instead.

[LoudLemur]

@AmbroiseUnly said in AI on Cloudron:

Is there any plan on adding AI-based projects to Cloudron?

I'm asking because I wanted to build something with OpenAI, and got stuck because the OpenAI API doesn't allow browsing the internet (unlike ChatGPT), which came as a surprise, and I'm looking to use a self-hosted solution instead.

It was said that this area of AI is developing so rapidly and so many changes are happening that a "wait and see" approach would be preferred. After a while (which in AI is not very long at all) it is hoped that some of the many applications will emerge as a leader and I think Cloudron will then think about supporting one of those.

Some people in this thread have already posted some very juicy candidates though!

Which applications or functions do people here think will most merit support on Cloudron?

[robi]

@AmbroiseUnly So far we have a few self-install how-to posts using the LAMP app.

However you can always find/hire/sponsor a developer to create a custom package for what you want and submit it to the community and Cloudron team which can then more easily add it to the App Store.

[marcusquinn]

Looks like an interesting next-step in AI... LAMs:

• https://www.rabbit.tech/research

[LoudLemur]

New device - Rabbit R1:
https://iv.ggtyler.dev/watch?v=DlnJlG1SOZo&quality=dash&t=3

Dedicated AI assistant
https://www.rabbit.tech/

subscription business model by the look of it...

"...there aren't going to be any apps, and then who is going to build them...?"

Haha! I just noticed @marcusquinn just posted before me!

[marcusquinn]

@LoudLemur Great minds! I do predict this is how AI is going to get into most people's lives. Moreso than ChatGPT.

[LoudLemur]

Stable Video Diffusion 1.1 Image-to-Video Model Card
generate short video from an image

https://huggingface.co/stabilityai/stable-video-diffusion-img2vid-xt-1-1

[robi]

Mac + Linux, Windows coming up - https://ollama.ai/

[marcusquinn]

@robi Get shares in RAM manufacturers!

[jdaviescoates]

@marcusquinn said in AI on Cloudron:

@robi Get shares in RAM manufacturers!

https://github.com/ollama/ollama says:

Note: You should have at least 8 GB of RAM available to run the 7B models, 16 GB to run the 13B models, and 32 GB to run the 33B models.

Not as much as I expected.

My laptop has 64GB RAM, I may give this a spin at some point...

[robi]

@marcusquinn I'll have you one better: https://openinterpreter.com/

it's insanely cool as a meetup group in Seattle has been working on an OSS alternative to the Rabbit R1
https://github.com/KillianLucas/01

https://twitter.com/hellokillian/status/1745875973583896950

Can be used with https://jan.ai/ and ollama.ai https://nitro.jan.ai/

[robi]

https://github.com/adamcohenhillel/ADeus
Use your own Supabase

ADeus: Open-Source AI Wearable Device

In the upcoming world of AI devices like Tab, Pin, Rewind, that are with us all the time, literally listening to everything we say, and know much about us as our closest friends - it is crucial to be able to own this setup, to own our data, to have this completely open source, managed by the user itself.

This is Adeus, the Open Source AI Wearble device - and in this repo, you will be guided on how to set up your own! From buying the hardware (~$100, and will be cheaper once we finish the Raspberry PI Zero version) to setting up the backend, the software, and start using your wearable!

• Introduction
• Setup
  • Prerequisite
  • Supabase
  • Web / Mobile App
  • Hardware (Coral AI)
  • Hardware (Rasberry Pi Zero)
  • Run with Ollama
• Areas to Contribute
• CTA for the Community

[timconsidine]

Oooh, very interesting and raspberry pi zero support coming !
Been looking for a project for mine.
I've tried to package supabase to work as docker-compose installation with traefik, but failed (had to deploy on its own VPS.
Would be great to have Parse and/or Supabase on Cloudron.

[marcusquinn]

@robi Sometimes YouTube taunts me with insult ads when I complain about Google anything.

[robi]

@marcusquinn why put up with any ads?

[LoudLemur]

@timconsidine said in AI on Cloudron:

Oooh, very interesting and raspberry pi zero support coming !

People who like raspberrypi and similar kit, might enjoy esp32, espressif being just one provider:

https://www.espressif.com/en/products/socs/esp32

[LoudLemur]

@marcusquinn said in AI on Cloudron:

Sometimes YouTube taunts me

Invidious is on Cloudron. It is lovely. Thank you Cloudron!