AnythingLLM - AI business intelligence tool
-
AnythingLLM has had enormous improvements since it was first requested by @TheMoodBoardz
In particular, support for agents and RAG has been advanced to try and keep up with the huge demand in this area.Installing AnythingLLM on GNU+Linux now automatically installs Ollama, too.
Here is a summary of the main advances. If you want to make RAG and ai agents easier, AnythingLLM would be a great application to support.
Comparison between v1.0Based.0 and v1.9.0 of AnythingLLM, here are the most significant changes categorized into key areas:
Agent System Overhaul
- Complete redesign of the agent experience with real-time streaming tool calls
- Agents can now download and ingest web files (PDF, Excel, CSV) during conversations
- All providers and models support agentic streaming capabilities
Local LLM Integration
- Added Microsoft Foundry Local integration for Windows/MacOS (free local LLM solution)
- Linux now includes Ollama (0.11.4) built-in for immediate local LLM support
- ARM64 support for both Linux and Windows platforms
Platform & Infrastructure
- Expanded Linux support with automatic apparmor rules and .desktop file creation
- NVIDIA NIM being phased out in favor of newer solutions
- Upgraded core Electron version for improved stability
UI/UX Improvements
- Model swap capability during chats (Ctrl/Cmd+Shift+L)
- Workspace and thread searching in sidebar
- System prompt version tracking and restoration
- Enhanced mobile support with Android beta app
Developer Features
- MCP (Model Context Protocol) compatibility
- No-code AI agent builder
- Enhanced API endpoints and developer tools
- Multi-modal support for both open and closed-source LLMs
Internationalization
- Added Portuguese and Estonian translations
- Improved multi-language OCR support
- Expanded localization coverage
The update represents a massive leap from the initial version, transforming from a basic document chat system to a comprehensive AI agent platform with robust local operation capabilities and extensive customization options.
-
AnythingLLM has had enormous improvements since it was first requested by @TheMoodBoardz
In particular, support for agents and RAG has been advanced to try and keep up with the huge demand in this area.Installing AnythingLLM on GNU+Linux now automatically installs Ollama, too.
Here is a summary of the main advances. If you want to make RAG and ai agents easier, AnythingLLM would be a great application to support.
Comparison between v1.0Based.0 and v1.9.0 of AnythingLLM, here are the most significant changes categorized into key areas:
Agent System Overhaul
- Complete redesign of the agent experience with real-time streaming tool calls
- Agents can now download and ingest web files (PDF, Excel, CSV) during conversations
- All providers and models support agentic streaming capabilities
Local LLM Integration
- Added Microsoft Foundry Local integration for Windows/MacOS (free local LLM solution)
- Linux now includes Ollama (0.11.4) built-in for immediate local LLM support
- ARM64 support for both Linux and Windows platforms
Platform & Infrastructure
- Expanded Linux support with automatic apparmor rules and .desktop file creation
- NVIDIA NIM being phased out in favor of newer solutions
- Upgraded core Electron version for improved stability
UI/UX Improvements
- Model swap capability during chats (Ctrl/Cmd+Shift+L)
- Workspace and thread searching in sidebar
- System prompt version tracking and restoration
- Enhanced mobile support with Android beta app
Developer Features
- MCP (Model Context Protocol) compatibility
- No-code AI agent builder
- Enhanced API endpoints and developer tools
- Multi-modal support for both open and closed-source LLMs
Internationalization
- Added Portuguese and Estonian translations
- Improved multi-language OCR support
- Expanded localization coverage
The update represents a massive leap from the initial version, transforming from a basic document chat system to a comprehensive AI agent platform with robust local operation capabilities and extensive customization options.
@LoudLemur Yeah AnythingLLM is amazing, I believe much cleaner and intuitive than Open WebUI.
One can install it online as well via Coolify or EasyPanel and it's quite impressive out of the box too.
Thanks for sharing this summary.
-
-
@humptydumpty / @loudlemur started working on AnythingLLM package, but (a) got diverted (b) wondered if needed because the native app is so good
So it's parked for now. -
@humptydumpty / @loudlemur started working on AnythingLLM package, but (a) got diverted (b) wondered if needed because the native app is so good
So it's parked for now.errr, I missed the point slightly
Summary
If you are the only user and just want to chat with your local documents, stick with the Desktop App .
If you want to share knowledge with a team, provide a chat interface for others, or access your LLM tools from your phone/browser, Self-Hosting on Cloudron is the way to go.So I am picking up packaging AnythingLLM again.
-
AnythingLLM is packaged !
A little bit of testing and clean-up needed,Does anyone want it ?
Just depends whether I make repo public or private.Cloudron team don't seem to want custom apps anymore <tease>
-
AnythingLLM is packaged !
A little bit of testing and clean-up needed,Does anyone want it ?
Just depends whether I make repo public or private.Cloudron team don't seem to want custom apps anymore <tease>
@timconsidine looks like you are 100% in a flow of productivity
-
AnythingLLM is packaged !
A little bit of testing and clean-up needed,Does anyone want it ?
Just depends whether I make repo public or private.Cloudron team don't seem to want custom apps anymore <tease>
@timconsidine said in AnythingLLM - AI business intelligence tool:
Does anyone want it ?
I was interested until chatgpt told me it requires a gpu to work reasonably fast or else it falls back to the cpu. Unless it's mistaken and it would work okay for text prompt on older servers/cpus (6th gen intel)?
-
@timconsidine looks like you are 100% in a flow of productivity
@Kubernetes ha ! I wish. My good friend TRAE helps me a lot, now with help of context7 and Sequential Thinking MCPs.
Amazing how stupid they can be at times, though !
-
@timconsidine said in AnythingLLM - AI business intelligence tool:
Does anyone want it ?
I was interested until chatgpt told me it requires a gpu to work reasonably fast or else it falls back to the cpu. Unless it's mistaken and it would work okay for text prompt on older servers/cpus (6th gen intel)?
@humptydumpty could be a good point
But I am NOT running on a GPU.
I just have a good Hetzner server (128Gb, good CPU).I guess it's a limbo dance trial and error - how low can you go ?
I have a 'spare' testing Cloudron instance - very modest specs - just a mini PC - will give it a go over the weekend if I have a chance.
-
@humptydumpty could be a good point
But I am NOT running on a GPU.
I just have a good Hetzner server (128Gb, good CPU).I guess it's a limbo dance trial and error - how low can you go ?
I have a 'spare' testing Cloudron instance - very modest specs - just a mini PC - will give it a go over the weekend if I have a chance.
@timconsidine My home-servers are 6th or 7th gen Intel (i5-6500t or i5-7500t) both with 32GB RAM. I'll give it a try if you make it public on CCAI. What's the worst that could happen
? -
@timconsidine My home-servers are 6th or 7th gen Intel (i5-6500t or i5-7500t) both with 32GB RAM. I'll give it a try if you make it public on CCAI. What's the worst that could happen
?@humptydumpty
I will tidy it up (remove some debug code) and let you know when on CCAI.
