@shivam74689: Day 62 — Becoming AI Engineer Today I started building an Enterprise Knowledge Graph Agentic RAG system and realized th…

Day 62 — Becoming AI Engineer Today I started building an Enterprise Knowledge Graph Agentic RAG system and realized that retrieval is far more than searching embeddings. Before today, I thought RAG was mostly about chunking documents, creating embeddings, storing vectors, and retrieving the most similar chunks. After building multiple retrieval pipelines from scratch, I realized that production AI systems retrieve knowledge in multiple ways, because no single retrieval method is good at everything. The biggest lesson: Enterprise AI is about combining different forms of knowledge, not relying on a single retrieval strategy. The system I'm building is designed to answer questions over an organization's internal knowledge—contracts, policies, support tickets, PDFs, HTML pages, markdown files, CSVs, and more. Instead of depending on vector search alone, it combines three different retrieval approaches: • Semantic retrieval using vector embeddings. • Lexical retrieval using BM25. • Knowledge graph traversal for relationship reasoning. Eventually, an intelligent agent will decide which retrieval strategy—or combination of strategies—is best for every query. The first thing I built was the document ingestion pipeline. Rather than relying on framework-provided document loaders, I implemented the entire pipeline from scratch. I created a common Document model to standardize every file entering the system, regardless of format. Then I built loaders for: • PDF • Markdown • HTML • CSV A Loader Factory automatically selects the correct loader, while a Directory Loader recursively scans an entire knowledge base and converts everything into a unified collection of documents. One lesson became obvious very quickly: Metadata is just as important as the document itself. Every document now carries information like source file, page number, and row index. That metadata will later power citations, graph construction, and explainable retrieval. Next came chunking. At first, chunking looked like a simple preprocessing step. It turned out to be one of the most important parts of the entire pipeline. Bad chunking leads to bad retrieval. Bad retrieval leads to bad answers. I built configurable chunk sizes with overlapping context and unique chunk IDs so every chunk maintains its connection back to the original document. Those chunks now become the fundamental unit used throughout the rest of the system. I then implemented semantic retrieval. Using the BAAI/bge-small-en-v1.5 embedding model together with a custom Qdrant vector store, I created a reusable pipeline that embeds every chunk and performs similarity search over the vector database. Separating the embedding model from the vector database taught me an important software engineering principle: Single responsibility makes every component replaceable. If I ever change embedding models or migrate to another vector database, the rest of the system stays untouched. After semantic search, I built keyword retrieval using BM25. Unlike embeddings, BM25 has no understanding of meaning. It only understands words. Initially that sounded like a limitation. Instead, it became one of its biggest strengths. BM25 excels at retrieving things embeddings often miss: • Error codes • Contract IDs • Product names • Ticket numbers • Technical identifiers These exact matches are incredibly common in enterprise knowledge bases. The next step was combining both retrieval systems. I implemented Reciprocal Rank Fusion (RRF), which merges rankings instead of trying to compare completely different scoring systems. Rather than asking which score is larger, RRF simply rewards documents that consistently appear near the top of multiple retrieval methods. This became my Hybrid Retriever, combining semantic understanding with exact keyword matching. Then came the part that completely changed how I think about retrieval. Knowledge graphs. Unlike vector stores, graph stores don't organize information by similarity. They organize it by relationships. I built structured models for: • Entities • Relationships • Graph Documents To validate the storage pipeline before introducing an LLM extractor, I created a temporary Dummy Extractor that identifies capitalized words as entities and generates placeholder relationships. Although simple, it allowed me to verify the entire graph ingestion process. I then implemented a Neo4j graph store that creates: • Chunk nodes • Entity nodes • MENTIONS relationships • Entity-to-entity relationships using Neo4j's MERGE operation to prevent duplicate nodes. I couldn't fully test the graph because my Neo4j server wasn't running. But the application failed with a connection error—not a code error—which confirmed that the implementation itself was working correctly. One concept completely changed my understanding of enterprise retrieval. Vector stores and graph stores are not competitors. They solve entirely different problems. Vector stores answer: "What information looks similar?" Graph stores answer: "How are these pieces of information connected?" That difference makes graph traversal especially valuable for multi-hop reasoning across people, companies, products, contracts, suppliers, or organizational structures. Another important realization was the role of databases inside modern AI systems. Advanced AI agents rarely rely on a single database. Instead, they often combine: • Vector databases for semantic knowledge. • Graph databases for relationship reasoning. • SQL databases for structured business data. Each contributes a different type of understanding. Together they create far more capable AI systems than any one database could alone. The biggest takeaway from today: Building production RAG systems is no longer just about retrieving similar documents. It's about combining semantic understanding, exact keyword matching, and structured relationship reasoning into a retrieval system that can gather evidence from multiple perspectives before generating an answer. Today's work laid the foundation. Next, I'll replace the Dummy Extractor with an LLM-powered entity and relationship extraction pipeline, build graph traversal retrieval, and finally combine vector search, BM25, and knowledge graph reasoning into a true Agentic RAG system capable of multi-hop reasoning with fully cited answers. Every day, AI engineering feels a little less like building models and a little more like designing intelligent systems. #AIEngineering #AgenticAI #RAG #KnowledgeGraph #Neo4j #BM25 #VectorSearch #EnterpriseAI #LLM #GenerativeAI #SoftwareEngineering #BuildingInPublic