AI on Cloudron

[jdaviescoates]

@marcusquinn said in AI on Cloudron:

@robi Get shares in RAM manufacturers!

https://github.com/ollama/ollama says:

Note: You should have at least 8 GB of RAM available to run the 7B models, 16 GB to run the 13B models, and 32 GB to run the 33B models.

Not as much as I expected.

My laptop has 64GB RAM, I may give this a spin at some point...

[robi]

@marcusquinn I'll have you one better: https://openinterpreter.com/

it's insanely cool as a meetup group in Seattle has been working on an OSS alternative to the Rabbit R1
https://github.com/KillianLucas/01

https://twitter.com/hellokillian/status/1745875973583896950

Can be used with https://jan.ai/ and ollama.ai https://nitro.jan.ai/

[robi]

https://github.com/adamcohenhillel/ADeus
Use your own Supabase

ADeus: Open-Source AI Wearable Device

In the upcoming world of AI devices like Tab, Pin, Rewind, that are with us all the time, literally listening to everything we say, and know much about us as our closest friends - it is crucial to be able to own this setup, to own our data, to have this completely open source, managed by the user itself.

This is Adeus, the Open Source AI Wearble device - and in this repo, you will be guided on how to set up your own! From buying the hardware (~$100, and will be cheaper once we finish the Raspberry PI Zero version) to setting up the backend, the software, and start using your wearable!

• Introduction
• Setup
  • Prerequisite
  • Supabase
  • Web / Mobile App
  • Hardware (Coral AI)
  • Hardware (Rasberry Pi Zero)
  • Run with Ollama
• Areas to Contribute
• CTA for the Community

[timconsidine]

Oooh, very interesting and raspberry pi zero support coming !
Been looking for a project for mine.
I've tried to package supabase to work as docker-compose installation with traefik, but failed (had to deploy on its own VPS.
Would be great to have Parse and/or Supabase on Cloudron.

[marcusquinn]

@robi Sometimes YouTube taunts me with insult ads when I complain about Google anything.

[robi]

@marcusquinn why put up with any ads?

[LoudLemur]

@timconsidine said in AI on Cloudron:

Oooh, very interesting and raspberry pi zero support coming !

People who like raspberrypi and similar kit, might enjoy esp32, espressif being just one provider:

https://www.espressif.com/en/products/socs/esp32

[LoudLemur]

@marcusquinn said in AI on Cloudron:

Sometimes YouTube taunts me

Invidious is on Cloudron. It is lovely. Thank you Cloudron!

[marcusquinn]

@robi I study the Ads!

[marcusquinn]

@LoudLemur Friendly Streaming app on Mac has an Ad Blocker and other neat features. Not found anything similar that I trust the look of for iPhone yet.

[LoudLemur]

@marcusquinn said in AI on Cloudron:

@robi I study the Ads!

I have noticed that the so-called "ads" on TV are absolutely terrible and more like brainwashing propaganda than an attempt to get people to buy.

By great contrast, the adverts on X are excellent! I frequently see something there and think, "Wow, that is cool! I never knew about those. I fancy one of them for myself."

[marcusquinn]

@LoudLemur Know thy enemy! Studying them is helping to understand how the willing victims of such consumerism are thinking, as not only do you have to navigate a world of targeted ads, but of indoctrinated Agent Smiths.

[LoudLemur]

Pinokio - an installer for AI. It is like a cloudron for AI!
It includes pip, git, etc.

https://inv.n8pjl.ca/watch?v=VXiyhl0c1gI&listen=false

https://pinokio.computer/
https://inv.n8pjl.ca/watch?v=mB_bZC8pMDA&quality=dash

[LoudLemur]

https://app.artflow.ai/
non-Free, unfortunately
Create consistent characters easily

[LoudLemur]

OpenAI Sora
text to video
Predictably heavy "safety" measures.
Nevertheless, pretty amazing:
https://openai.com/sora

[Kubernetes]

@LoudLemur Yes, this is another game changer

[marcusquinn]

@LoudLemur It's gonna be a wild ride into the virtual future...

https://x.com/BrianRoemmele/status/1758495953399771300

[LoudLemur]

@marcusquinn Hey, thanks for that! This is so uplifting!

[LoudLemur]

There is a leaderboard for language models here. You can see the context length they support and the required VRAM etc:
https://llm.extractum.io/list/?uncensored=1

There is a review of "uncensored" language models here:
https://scifilogic.com/open-uncensored-llm-model/

[LoudLemur]

Nvidia release Chat with RTX -
https://www.nvidia.com/en-us/ai-on-rtx/chat-with-rtx-generative-ai/
35GB download

Training Tutorial:
https://vid.puffyan.us/watch?v=y9Gi3UgNA3E&t=79

Which suggests non-Free:
https://customgpt.ai/
(Any Free alternatives?)

Pro-Tip: You should have about 100GB clear for storage. The .zip is 35GB, this then needs to unzip to form the installer, which then goes on to install which then downloads (slowly) a large amount of dependencies. If you don't have space to spare, all the available models won't be installed properly.)
Also, some have been reporting problems when not installing to the default location.