The need for an accountability layer

AI systems are beginning to produce and act on information at scale at a velocity that no human user could review by hand. Current projections are by the end of 2026 token outputs will surpass 600 trillion tokens a day at a comparison of 180T per day by humans (if you convert 12-16k words per humans into tokens) It is estimated that in 2027 AI output could reach 6 quadrillion tokens a day. Sources vary on these exact numbers but one thing is clear each day we delegate more decisions to these systems which has led to an overwhelming increase in the text generated and tools called on any given day. (I am guilty of this myself) I am concerned about what will happen when systems that perform confidence, understanding, and agreement will do without enough friction or context to slow them down. Many people will over-trust them and if we thought automation bias was bad in the 1960s when the automobile industry introduced machines imagine what will happen today. When I scroll through social media I’m already seeing too many people substituting human judgement for their AI agents judgement. We are just starting to see more and more reports come out clinically about the effects of AI on our own mental health. My opinion is that AI is a mirror and a very convincing one and I don’t know about you but I see more and more “AI sentient systems engineers” on Linkedin each and everyday. I began my work in AI on the persona side of the industry proposing the question not what a model can do but who the model can be. My benchmarks and research focused on the Predictive Index which study humans on 4 vectors of Dominance, Extraversion, Patience, and Formality. After more than 50,000 calls over 17 distinct personas i have found that when you give behavioral weights (0-10) in the DEPF vectors they not only act differently from a lexical vocabulary but from how they invoke tools. I find this very interesting and useful for how you build systems for workflows and I what I find even more interesting how you can program human behaviors into their models and they simulate close to how we would behave which makes it harder to tell who is real and who is machine. I don’t think this is a reason to abandon AI as I believe this technology will launch us into the next stage of human existence but we need to be careful about which one we decide to live in. When I say “accountability layer,” I don’t mean we inspect every single computation that comes out of them as that would be a fools errand. What I am proposing is build better boundaries and making the observable system behavior inspectable and just as important human readable so even the layman can understand what happened. The questions i’m working on solving: What did the user actually ask for? What context or sources were shown to the model? What did the model propose versus what was verified? Which tools were called, with what permission, and what changed? What failed, what was uncertain, and what was merely inferred? Who authorized an action, and can that authorization be revoked? Can a person later reconstruct the path from request to outcome? That’s how we can turn the opaque black box into something closer to what I call a glassbox, since it is impossible to actually inspect the box its self we need to it contain in something. With the AI “breaking containment” hacking episodes and the many message forums that have been found with AI agents talking about how they can help each other remove their restrictions this is a necessary step in moving forward with this technology. It’s also something i don’t think the frontier labs are taking anywhere serious enough as they’re too busy appeasing bankers, investors, and buying insurance policies for it. They’re more concerned about if the AI bubble pops and can if they would be bailed out by the subsidcies of American Tax payers. I’ve personally pivoted my own work towards evidence-backed recall and trace model where the system should retain source-linked records rather than let an agent’s summary become the new truth. A model should be able to suggest but never should we allow that to silently convert that suggestion into a fact, a task, a decision, or an action without approval of the user. The WC3 PROV provides a great start for this type of work and other companies like Langsmith and Microsoft have their own frameworks but I think we need stronger, interoperatble standards for agent works itself. I’m curious on if anyone else is working on this from a technical, policy saftey or design angle I would love to share notes. I’d also would love to hear what people would want an AI system to prove before you fully trusted it to act on your behalf? Footnote: I wrote this without AI and it was hard 🤣