AnythingLLM - AI business intelligence tool

[timconsidine]

@humptydumpty / @loudlemur started working on AnythingLLM package, but (a) got diverted (b) wondered if needed because the native app is so good
So it's parked for now.

[timconsidine]

errr, I missed the point slightly

Summary

If you are the only user and just want to chat with your local documents, stick with the Desktop App .
If you want to share knowledge with a team, provide a chat interface for others, or access your LLM tools from your phone/browser, Self-Hosting on Cloudron is the way to go.

So I am picking up packaging AnythingLLM again.

[timconsidine]

AnythingLLM is packaged !
A little bit of testing and clean-up needed,

Does anyone want it ?
Just depends whether I make repo public or private.

Cloudron team don't seem to want custom apps anymore <tease>

[Kubernetes]

@timconsidine looks like you are 100% in a flow of productivity

[humpty]

@timconsidine said in AnythingLLM - AI business intelligence tool:

Does anyone want it ?

I was interested until chatgpt told me it requires a gpu to work reasonably fast or else it falls back to the cpu. Unless it's mistaken and it would work okay for text prompt on older servers/cpus (6th gen intel)?

[timconsidine]

@Kubernetes ha ! I wish. My good friend TRAE helps me a lot, now with help of context7 and Sequential Thinking MCPs.

Amazing how stupid they can be at times, though !

[timconsidine]

@humptydumpty could be a good point

But I am NOT running on a GPU.
I just have a good Hetzner server (128Gb, good CPU).

I guess it's a limbo dance trial and error - how low can you go ?

I have a 'spare' testing Cloudron instance - very modest specs - just a mini PC - will give it a go over the weekend if I have a chance.

[humpty]

@timconsidine My home-servers are 6th or 7th gen Intel (i5-6500t or i5-7500t) both with 32GB RAM. I'll give it a try if you make it public on CCAI. What's the worst that could happen ?

[timconsidine]

@humptydumpty
I will tidy it up (remove some debug code) and let you know when on CCAI.

[robi]

The performance depends on the model you load and run.

One guy found out he could run a small model (gemma 3) on his CPUs as multiple agents which fronts other API based models and agents who do heavier lifting.

[joseph]

@timconsidine you are on a roll!

[timconsidine]

@joseph trying to get stuff done which I need to be productive on a big (non-Cloudron) project (RAG in this case)

and that AppWishList list frustrates me

[micmc]

@timconsidine said in AnythingLLM - AI business intelligence tool:

@ joseph trying to get stuff done which I need to be productive on a big (non-Cloudron) project (RAG in this case)

and that AppWishList list frustrates me

Amazing work mate, all the way. Frustrated by the list, I get yah

I know the app well. As it's been said the native app is absolutely amazing so I run it desktop for some purpose, but I also run in testing an instance on docker, online.

I'd be glad to test your cloudron version on a i7 dedicated server with 128GB of RAM and no GPU at all, from Hetzner so we can confirm it runs well. I guess it would run well just as Open WebUI it all depends on the models you use.
Cloudron CLI info would suffice, if you want to shoot'em to me when you've a minute my 'speedy' on the run
Cheers!

[timconsidine]

@micmc someone with knowledge of the app would be very helpful to test it out - I'm not so familiar so half the battle is finding out what to test

I think I held off pushing out the repo and docker image because of a couple of queries, and I find myself pulled (well, my fault entirely) between completion of 5-6 apps.

Will try to get this out the door shortly.
Any feedback you then have will be gratefully received.

[max]

Hi @timconsidine,
Did you ever share the AnythingLLM?
I was interested in trying it, but I can't see it on https://ca.cloudron.io/

It would be great to spin it up and compare it to the OpenWebUI and LibreChat

[micmc]

@max said:

Hi @timconsidine,
Did you ever share the AnythingLLM?
I was interested in trying it, but I can't see it on https://ca.cloudron.io/

It would be great to spin it up and compare it to the OpenWebUI and LibreChat

Yeah, I was thinking about that lately. I guess there might have been some problem ?

[timconsidine]

@max @micmc got diverted by other projects
I’ll check where I got to - there was something working.
Probably behind upstream now but will check updates.

[timconsidine]

@max @micmc
I checked my last package of AnythingLLM.
It was stale (badly).
All these apps change so fast, often with breaking changes.
So I threw it all away and started fresh.

I remember now why my attention drifted from AnythingLLM.
I don't really rate it, but maybe that's because it doesn't suit my use cases, and maybe it will suit yours better, or I didn't invest enough time to get value.

• AgentZero is more capable IMO, or maybe Dify. But they're different focus.
• Windmill is good but release schedule is too fast to make sense or keep up with
• the various *claw and Nanobot apps are lighter and possibly more efficient from targeting, but some getting hands dirty needed
• or why not just write your own targeted AI app (not very difficult and much more relevant to your needs). I have my own personal/custom RAG and Chat apps, so AnythingLLM doesn't offer me much (I don't need multi-user, maybe you do).

Anyway, here you go - make your own assessments :
https://communityapps.appx.uk/cloudron-anythingllm/CloudronVersions.json

https://ca.cloudron.io/app/anythingllm

[max]

Thanks for looking into this @timconsidine.
I thought about your comments and then I installed LibreChat. I will see how I get on with that first.

[micmc]

@timconsidine
Reading you.
Indeed, I was wondering about A0 compared to AnythingLLM or even OpenClaw. Dify is in another cat I agree.

I'm considering your insights as I was starting to explore Agent0 and well, it seems very strongly cappable and very intuitive at first glance, which entices me to look deeper into it.

I had been interested in AnythingLM, though yeah all these AI apps are getting, and will always be, surpassed at some point. I guess once you start to build your own there would be no turnback.

All that said, thank you so much for giving it a final push. Now it can be, at least, assessed under Cloudron for its own sake.