The article explains how agent memory systems can leak data due to post-filter tenant scoping in vector search and recommends scoping at write time to prevent cross-tenant data exposure.
We started with a shared index and a system prompt saying only use customer X's memories but that doesn't work because retrieval is where it leaks before the model even touches anything. Where the leak happens The prompt is the last stage so if out of scope records reached the context window they were already read and queried against and cost you tokens. A prompt rule is the only thing you have and it barely does anything. There are three failure paths and all are in retrieval. 1. Post filtering in vector search Single index with tenant_id in metadata and scoping as post filter. ANN search runs the whole index, returns top k, then discards rows that don't match the tenant. You get fewer results and recall drops for small tenants and the search still went through other tenants' vectors. Pre filtering helps but some ANN implementations degrade hard when the filter masks most of the graph. Single index with metadata is fragile and it only comes up at realistic tenant counts not in dev We went partitioned, partition key required on every write. 2. Your graph doesn't respect tenant boundaries If your store has a graph then scoping the seed node is not enough, you have to check scope on every edge traversal. Two tenants share a vendor, those resolve to one global node, a two hop traversal from tenant A lands on that node and walks into tenant B's subgraph. Every record has the right tenant id but the edges don't know about the boundary. We scope the subgraph not the node now, entities are tenant local even for the same real world thing. Yeah it's duplicated storage but whatever. 3. Cache keys missing the tenant If your cache is keyed on query text and embedding hash then tenant A's warm result serves tenant B verbatim. Same bug in embedding caches and dedupe tables and rerank caches. Grep for this today because it's a one line bug and it can go very wrong very fast. How we model it Scope attaches at write time and if it's missing the write just fails, never a default bucket because a default bucket is a slow leak you won't catch until someone offboards. The hierarchy is user then customer then client stored as a materialised path. Retrieval takes the resolved scope path as an argument not a filter and there is no code path that accepts a query without one. Out of scope records are not retrievable not just not returned so prompt injection doesn't matter because the data was never fetched into the process. Entity resolution has to be scoped too If your dedupe similarity search isn't scoped you will merge entities across tenants. Worse than a read leak because it stays and the merged record is wrong in the store and no access control catches it. Resolution runs strictly inside scope with strategies ordered by how confident the match is and anything below threshold goes to a review queue instead of auto merge. We initially tuned for fewer duplicates which was stupid because an unmerged duplicate is an annoyance and a wrong merge is an incident. Deletion is the main test If tenant id is metadata on a shared index then depending on the engine vectors stay behind tombstones until compaction so you could still hold data you said you deleted. If you partitioned then offboarding is dropping a partition and it's just gone. If the answer differs per store then you don't have one isolation model you have four. What I'd check Two tenants with near identical records, same names, same phrasing, closest embeddings you can make, and check that cross tenant retrieval gives you zero rows at every k. Most implementations passing a normal test suite fail this because normal fixtures use distinct tenants and vector search separates them for free which tells you nothing about your scoping. Then run it with one tenant a thousand times larger because that's where post filter recall dropping shows up. Are people enforcing at the store or the query layer? We went with the store so the guarantee survives a buggy query but you lose flexibility because cross tenant analytics needs a separate path with its own auth. (PS: I work on agent memory at Maximem (Synap) and this is how we do it, this applies regardless whether you use us or build it yourself)
The article warns that using shared vector databases with only logical isolation (metadata filters) for multi-tenant AI agents can silently cause data breaches, and advocates for physical isolation per user to guarantee zero data bleed.
Verity is a new open-source, permission-aware memory layer for multi-tenant AI agents, preventing cross-tenant data leaks by enforcing access controls directly in retrieval queries rather than relying on model behavior.
The article discusses how agent memory transforms vector search into a long-lived systems problem, emphasizing the need for scalable, reliable infrastructure beyond just search algorithms.
The article reveals that frequent debugging issues in AI agents are often due to memory scoping problems where agents act on stale or incorrectly scoped context, and proposes tagging memory operations to efficiently resolve such issues.
MosaicLeaks introduces a new benchmark for measuring privacy leakage in deep-research AI agents, showing that agents often leak private information through external queries and proposing a training method (PA-DR) to reduce leakage while improving task performance.