Who decides what an AI agent is allowed to know?

Can I jump into the discussion about the downsides of AI? The thing that worries me most is not really the AI itself, but sensitive data access. With a traditional search engine, you ask for information and get links. With an AI agent, the intermediary can potentially query huge amounts of data. So who decides what it is allowed to access? It's like having a gigantic library where AI can instantly find the exact chapter you're looking for. The interesting question isn't just how good the search is, it's who decides which books are in the library, and who is allowed to read them. "Use an enterprise account" doesn't seem like an architectural answer to me. The provider still has to define how data access, permissions, auditing and accountability work. I actually made a small repo, if doesn't violate any rule I can add the link, while thinking about this problem. I'm not looking for a specific product or solution. I'm wondering: is there already a generally accepted architecture/pattern for governing what an AI agent can access and do with data? Or are we still figuring this out?