AI on Cloudron

LoudLemur

Chroma https://www.trychroma.com/

"the AI-native open-source embedding database"

What do you do with it? For example, throw in your medical books and query it with an AI.

Apache 2.0
Has Docker
They hope to have hosting soon.

I hope somebody who likes database tech might have a look at this.

https://github.com/chroma-core/chroma
https://www.trychroma.com/

LoudLemur

Ollama.ai
Download and run language models locally:
https://www.ollama.ai/

You can see a video here where somebody integrated an AI into their editor using Ollama:
https://yt.artemislena.eu/watch?v=FIZt7MinpMY&si=fLRMYyhss0hOO1ZQ&quality=dash

LoudLemur

Axolotl
A tool to help you fine-tune models.
Apache 2.0
https://github.com/OpenAccess-AI-Collective/axolotl

Axolotl was used to help create the following, for example:
Minotaur 13b
Manticore 13b
Hippogriff 30b

marcusquinn

@LoudLemur Chroma post here: https://forum.cloudron.io/topic/9078/chroma-open-source-vector-database-for-ai-long-term-memory-and-local-data-alternative-to-pinecone

robi

@LoudLemur et'all

I've got ollama working in the LAMP app on the Demo server (if it's still there)

Use the terminal to run llama2.sh or llama2uncesored.sh and see how it runs.

There may be a web/desktop client for it too.

LoudLemur

@robi said in AI on Cloudron:

llama2uncesored.sh

Thank you very much for doing that, @robi. From where do we run the command? The ollama application doesn't seem to have a terminal in Cloudron. I tried from the LAMP application's terminal and that didn't work either.

robi

@LoudLemur it's linked in my post.. it's installed in the LAMP app labeled php-8.2. It cannot be installed in the proxy app named ollama.

ankitpr89

Check: https://n8n.io/langchain/

marcusquinn

@ankitpr89 V cool!

LoudLemur

GPUs for AI comparison spreadsheet:
https://www.edlabs.it/gpus4ai

micmc

Amazon (AWS) recently made a big announcement: a service Bedrock was finally released.

If you missed the news:
Amazon decided to enter massively into the generative AI field and invested in the company Anthropic for $4 billion so they can get into the race and compete with popular AI services like ChatGPT, Dall-E, MidJourney, etc.

The first proposal of Amazon is Bedrock, a fully managed AI service that offers multiple foundational models such as Claude, Jurassic, Titan, and Stable Diffusion XL, among others.

I've started to test Bedrock myself, what's interesting is being able to test and use different FMs from different vendors from the same web UI from inside your AWS account BUT, from a unique API too!

That means, AWS is creating a single API access that can be used to access all the actual avaialable FMs through its Bedrock service, and any new ones they will eventually add. Interesting approach.

Some has already made a video here,

, if you want to take a peak.

LoudLemur

@micmc said in AI on Cloudron:

what's interesting is being able to test and use different FMs

FMs?

Claude, which is the AI owned by Anthropic, has beautiful prose.

LoudLemur

https://lambdalabs.com/papers

$1.99/hour for Nvidia H100 80GB GPU

Enjoy the amazing graphics in this paper!
https://invidious.lunar.icu/watch?v=ffarLQDQmC4

jdaviescoates

Nothing really to do with AI on Cloudron, but just tapping into the knowledge of the people watching this space who are contributing to this thread:

I had an idea I like which AI could probably help with/ be a good fit for:

give an AI access to an ebook library
ask an AI write summaries of those books
ask an AI to turn those summaries into scripts for 2 minute animated video summaries
ask an AI to create those animations based on those scripts.
ask an AI to publish those videos to a PeerTube/ TikTok/ YouTube/ Twitter/ Facebook Shorts/ Instagram channel etc etc

I'm presuming that this might already be perfectly possible with already existing tools, but I've no idea how one might go about it. Can anyone here enlighten me/ give me a few pointers.

I had this idea because one of my favourite books is "The Resilience Imperative: Cooperative Transitions to a Steady-state Economy" by Michael Lewis and Pat Conaty, and I've often thought that a short animated video version would be really great.

And as I'm about to start some work with Michael Lewis (helping to promote the next iteration of Synergia Institute's MOOC "Toward Co-operative Commonwealth: Transition in a Perilous Century") I reminded myself of that idea and then thought: AI could probably do it for me! And then figured if it could do it for one book why not All The Books!

LoudLemur

@jdaviescoates It is a lovely idea, and sounds doable in principle. In practice, somebody talented like e.g. robi would need to do it at the moment, because the tech required is not quite there, yet. A lot of fixing would need to be accomplished.

eBook library - you can train an ai on a local repository of texts using e.g. gpt4all. The AIs I have managed to get working with this aren't impressive, but that is more me than the AI! It would help if you had practiced it a few times, and also had a big spec computer to do it properly after pilot studies.
AI Summaries - Yes, that is possible, but the summaries are too short. The number of tokens you can create are about 2000, which isn't long for a summary. You might have to summarize chapter by chapter and then stitch these together later. The chunking could be useful for video.
AI Scripts - The amount of output you can generate on a local llama is not gigantic and it is a bit hit or miss what sort of script format you would get. Trying to keep consistency between chapters might be troublesome. Proprietary AI would make it a bit easier, but then you might have refusals / alignment problems.
AI Animations - The ones I have seen are about 20 seconds or so. They are becoming better. Trying to make your first 20 second animation have a resemblance to a subsequent animation would be tricky.
Automating the flow - robi would be good at that, but then he has great expertise.

I think if you wait for a while, a lot of the friction will be reduced and soon "everybody will be doing it".

robi

@LoudLemur Wow, thanks for the gracious mention(s)

While I used to be in that industry and can see how it might be done, it's a deep multi stage pipeline of things that would need lots of creative solutions.

So my recommendation would be to reach out and partner with a startup/dev/community in the space who is interested in making a splash with such a solution and get more traction behind it.

Years ago I was pitched to do a startup with similar goals but more around TV series and Movies.

In any case, great idea @jdaviescoates

LoudLemur

@robi said in AI on Cloudron:

a startup with similar goals but more around TV series and Movies

Even for me, it would be interesting to imagine what you would now be doing if you had ended up following that route. For you, you must wonder sometimes. I am just glad you are here.

FaceChain

Rapid training and face generation based on a few of your uploaded images:
https://huggingface.co/spaces/modelscope/FaceChain

https://github.com/modelscope/facechain

(I haven't had success using this on Huggingface. Maybe it is overloaded...)