Tag
LEANN is a new vector indexing method that reduces storage requirements by about 97% through graph simplification and real-time embedding computation, while maintaining retrieval recall close to full HNSW. It is suitable for use on laptops and won the Best Paper award at MLSys 2026.
James Luan, CTO of Zilliz, argues that data infrastructure becomes more critical as AI agents act on enterprise data, emphasizing the need for data quality, freshness, and proper permissions.
MongoDB Atlas introduces auto-embedding, enabling databases to generate and manage embeddings internally for semantic search, eliminating the need for external services and improving data synchronization.
The author updated the 'embedflow' embedding migration project with new features including support for multiple vector databases, a migration planner, shadow mode, and persistent caching to make it production-ready for embedding model transitions.
James, CTO of Zilliz, introduces Vector Lakebase and the concepts of One Data and One Index to unify data and index management for AI workloads, changing how agents retrieve information by leveraging lake-native storage.
The article discusses how agent memory transforms vector search into a long-lived systems problem, emphasizing the need for scalable, reliable infrastructure beyond just search algorithms.
@memory is an archiving agent in the AIPass open-source framework that manages memory for AI agents by vectorizing older entries and storing them in ChromaDB, ensuring long-term persistence and recall without data loss.
The article argues that cosine similarity in vector databases is a mathematical property that does not ensure factual accuracy, rendering retrieval systems susceptible to poisoning attacks where malicious documents can surpass legitimate ones in ranking.
Weaviate introduces a method to search PDFs without text extraction by embedding each page as an image using late-interaction multi-vector retrieval, demonstrated on NVIDIA investor decks.
The article warns about the limitations of free-tier vector databases, highlighting issues like data deletion and deployment constraints, and advises choosing based on where your AI agent runs.
The article argues that retrieval-augmented generation (RAG) is often misapplied in AI systems, where the real issue is context curation rather than retrieval. It suggests that RAG is only truly beneficial for large, frequently changing corpora.
This article argues that PostgreSQL is a versatile database solution capable of replacing multiple specialized technologies such as search engines, message queues, and caches, thereby simplifying IT setups.
The article argues that vector databases alone act as data graveyards without true memory capabilities, and describes building a local AI memory layer with contradiction detection and forgetting cycles on top of Actian VectorAI DB.
Introduces the open-source project RAG from Scratch, which fully breaks down the RAG pipeline through step-by-step local code experiments, covering text chunking, vectorization, retrieval, reranking, query rewriting and other advanced strategies, helping developers understand RAG implementation from the ground up.
An educational article from MachineLearningMastery covering seven chunking strategies for RAG pipelines, from fixed-size token chunking to advanced semantic and hierarchical approaches, with guidance on when to use each.
Pathway's open-source llm-app is a framework for building enterprise-grade RAG systems. It supports real-time data sync, built-in vector retrieval, and comes with ready-made cloud templates. It has earned over 59,000 stars on GitHub.
This paper introduces ScalableRAG, a retrieval-augmented generation method that achieves high accuracy without any ingestion costs (no vector database or knowledge graph) by using regex-based set creation and aggregative reasoning. It outperforms baselines on multiple datasets and also presents a limited-ingestion variant for further accuracy improvements.
HARP is a training-free interpretability method that uses an LLM agent with a vector database of activations and tools for manipulation, outperforming training-based methods on concept discovery, detection, steering, and secret elicitation.
A tweet discusses how AI agents slow down over time because their vector databases are optimized for static data, but real agents continuously add new data, causing performance degradation.
A thread warns that choosing a vector database solely on speed benchmarks can be a trap for AI agents, which have constant write workloads unlike RAG's read-heavy patterns. It recommends specific databases based on use case, such as Qdrant for agent memory and pgvector for under 10M vectors on Postgres.