Retrieval agents need to handle multiple data types because real-world knowledge is rarely just text

Most retrieval assistants assume that context can be converted into text chunks and placed in a vector database. Real-world data is messier: documentation may live in Markdown or PDFs, operational data in SQL tables, dependencies in graphs, procedures in SOPs, and user-specific decisions in persistent memory. Flattening all of this into one retrieval format creates predictable problems: Tables lose their schema and become difficult to filter or aggregate. Relationships become isolated text fragments instead of traversable paths. Procedures are mixed with factual knowledge. User preferences and project decisions lose their scope. Updates can leave multiple representations out of sync. A better approach is to expose several typed data surfaces: KB for documents, notes, and hybrid semantic/keyword retrieval. DB for structured records, filters, joins, and aggregations. Graph for entities, dependencies, and bounded traversal. Skills for reusable procedures and operational instructions. Memory for scoped facts, preferences, and decisions. On the retrieval side, an agent should select the smallest sufficient set of surfaces for each request. Document questions go to the KB, numerical questions to the DB, relationship questions to the Graph, while compound tasks can merge evidence from several surfaces. The storage path should remain separate. A store agent can route each input according to its structure and meaning, preserve source metadata, and return receipts containing the affected surface, revision, and write result. One source may be projected into several representations, such as searchable document chunks in the KB and extracted relationships in the Graph, without treating every surface as interchangeable. This is currently implemented in OpenDCAI/DataMind. Not sure whether there is real demand for this kind of architecture.