AnythingLLM - AI business intelligence tool

[timconsidine]

@humptydumpty / @loudlemur started working on AnythingLLM package, but (a) got diverted (b) wondered if needed because the native app is so good
So it's parked for now.

[timconsidine]

errr, I missed the point slightly

Summary

If you are the only user and just want to chat with your local documents, stick with the Desktop App .
If you want to share knowledge with a team, provide a chat interface for others, or access your LLM tools from your phone/browser, Self-Hosting on Cloudron is the way to go.

So I am picking up packaging AnythingLLM again.

[timconsidine]

AnythingLLM is packaged !
A little bit of testing and clean-up needed,

Does anyone want it ?
Just depends whether I make repo public or private.

Cloudron team don't seem to want custom apps anymore <tease>

[Kubernetes]

@timconsidine looks like you are 100% in a flow of productivity

[humptydumpty]

@timconsidine said in AnythingLLM - AI business intelligence tool:

Does anyone want it ?

I was interested until chatgpt told me it requires a gpu to work reasonably fast or else it falls back to the cpu. Unless it's mistaken and it would work okay for text prompt on older servers/cpus (6th gen intel)?

[timconsidine]

@Kubernetes ha ! I wish. My good friend TRAE helps me a lot, now with help of context7 and Sequential Thinking MCPs.

Amazing how stupid they can be at times, though !

[timconsidine]

@humptydumpty could be a good point

But I am NOT running on a GPU.
I just have a good Hetzner server (128Gb, good CPU).

I guess it's a limbo dance trial and error - how low can you go ?

I have a 'spare' testing Cloudron instance - very modest specs - just a mini PC - will give it a go over the weekend if I have a chance.

[humptydumpty]

@timconsidine My home-servers are 6th or 7th gen Intel (i5-6500t or i5-7500t) both with 32GB RAM. I'll give it a try if you make it public on CCAI. What's the worst that could happen ?

[timconsidine]

@humptydumpty
I will tidy it up (remove some debug code) and let you know when on CCAI.

[robi]

The performance depends on the model you load and run.

One guy found out he could run a small model (gemma 3) on his CPUs as multiple agents which fronts other API based models and agents who do heavier lifting.