Weaviate - open-source Vector Database for AI long-term memory and local data — alternative to Pinecone

[marcusquinn]

• https://weaviate.io
• https://github.com/weaviate/weaviate
• https://weaviate.io/developers/weaviate/quickstart
• https://weaviate.io/developers/weaviate/installation/docker-compose
• https://newsletter.weaviate.io
• https://twitter.com/weaviate_io
• https://weaviate.io/blog/why-is-vector-search-so-fast
• https://console.weaviate.io/console/query#weaviate_uri=https://demo.dataset.playground.semi.technology&graphql_query={ Get { Publication( nearText%3A { concepts%3A ["fashion"] } limit%3A 1 ) { name _additional { certainty distance vector } } } }

Weaviate is an open-source vector database.
It allows you to store data objects and vector embeddings from your favourite ML-models,
and scale seamlessly into billions of data objects.

With Weaviate, you can turn your text, images and more into a searchable vector database using state-of-the-art ML models.

Vector Search
Whether you bring your own vectors or use one of the vectorisation modules, you can index billions of data objects to search through.

Hybrid Search
Combine multiple search techniques, such as keyword-based and vector search, to provide state-of-the-art search experiences.

Generative Search
Improve your search results by piping them through LLM models like GPT-3 to create next-gen search experiences.

IMHO this is an essential app for AI application building. Pinecone has the mindshare at the moment, but this does the same thing and self-hosed open-source.

You'd use it with any GPT/LLM and LangChain to built AI apps with long-term memory and interrogate local documents and data that stay local — which is how you build things that can build and self-improve beyond the current 8k token limits of GPT-4.

I can also see this becoming an option to "chat to your Nextcloud data" etc.

[jordanurbs]

+1 would love a vector store app on cloudron!

[robi]

@jordanurbs there is one already hidden in Postgres. pgvector is installed and enabled by default.

Just use it.

[firmansi]

@robi How to use it in Cloudron?

[robi]

@firmansi You can install the LAMP app to experiment and just push the DB button at the top of the terminal to paste the connection command and you're in the db.

For anything else specific look at the pgvector docs from PG.

[jordanurbs]

@robi Fire, thanks for the tip!