AI Agents Push Humans Out of the Loop
Summary
The paper argues that current AI agent systems degrade human oversight capabilities and advocates for design-level changes to support effective human-agent interaction, emphasizing the need to maintain human skills.
arXiv:2608.23642v1 Announce Type: new
Abstract: AI agents pose significant risks as they are granted increasing autonomy. A commonly proposed solution is human oversight and keeping a ''human in the loop'', but this is not a simple solution: Not only do current approaches to AI agent design impede effective human oversight, but the cognitive capacities required for it are also themselves degraded by extended use of AI systems. This position paper argues that current approaches to the development and deployment of AI agent systems do not support effective human oversight -- they contribute to its degradation. To address this, a top priority in the advancement of AI agents should be supporting the situated goals and cognitive requirements of effective human oversight, treating the human needs of overseers at the same level of importance as AI agent capability. To put this idea into practice, we connect work on automation and human-computer interaction to AI agent processes, outlining design-level affordances and organizational protocols that (1) support overseers in exercising critical judgement and (2) counteract the skill atrophy that arises from extended use of automation. We urge developers and deployers to adopt these or similar approaches. Without explicit support for the cognitive demands of effective human-agent interaction, AI agent systems will continue to passively incentivize the degradation of the very human skills they rely on.
View Cached Full Text
Cached at: 08/26/26, 09:10 AM
# AI Agents Push Humans Out of the Loop Source: [https://arxiv.org/html/2608.23642](https://arxiv.org/html/2608.23642) Avijit GhoshAffiliation:Hugging FaceSamir PassiAffiliation:Data & Society ###### Abstract AI agents pose significant risks as they are granted increasing autonomy\. A commonly proposed solution is human oversight and keeping a “human in the loop”, but this is not a simple solution: Not only do current approaches to AI agent design impede effective human oversight, but the cognitive capacities required for it are also themselves degraded by extended use of AI systems\. This position paper argues thatcurrent approaches to the development and deployment of AI agent systems do not support effective human oversight – they contribute to its degradation\. To address this, a top priority in the advancement of AI agents should be supporting the situated goals and cognitive requirements of effective human oversight, treating the human needs of overseers at the same level of importance as AI agent capability\. To put this idea into practice, we connect work on automation and human\-computer interaction to AI agent processes, outlining design\-level affordances and organizational protocols that \(1\) support overseers in exercising critical judgement and \(2\) counteract the skill atrophy that arises from extended use of automation\. We urge developers and deployers to adopt these or similar approaches\. Without explicit support for the cognitive demands of effective human\-agent interaction, AI agent systems will continue to passively incentivize the degradation of the very human skills they rely on\. ## 1Introduction From healthcare to enterprise, a common recommendation for automated assistance is to have a “human in the loop” who supervises system processes\[[108](https://arxiv.org/html/2608.23642#bib.bib48),[9](https://arxiv.org/html/2608.23642#bib.bib46),[29](https://arxiv.org/html/2608.23642#bib.bib47)\]\. This recommendation captures the intuition that automated systems introduce risks that can be managed with meaningful oversight\. Governance frameworks have formalized this view, stipulating that humans using high\-risk AI systems must maintain meaningful control and make final decisions\[[86](https://arxiv.org/html/2608.23642#bib.bib36),[71](https://arxiv.org/html/2608.23642#bib.bib55),[35](https://arxiv.org/html/2608.23642#bib.bib54)\]\. With the recent rise in AI\-automated workflows and agentic AI, policymakers, developers, and deployers have converged on the practice of human oversight as a priority in service of multiple goals: Preventing harmful operations\[[1](https://arxiv.org/html/2608.23642#bib.bib61),[71](https://arxiv.org/html/2608.23642#bib.bib55),[99](https://arxiv.org/html/2608.23642#bib.bib56),[62](https://arxiv.org/html/2608.23642#bib.bib57)\], operationalizing ethical priorities\[[41](https://arxiv.org/html/2608.23642#bib.bib79),[84](https://arxiv.org/html/2608.23642#bib.bib50),[62](https://arxiv.org/html/2608.23642#bib.bib57)\], and ensuring legal compliance\[[17](https://arxiv.org/html/2608.23642#bib.bib51),[71](https://arxiv.org/html/2608.23642#bib.bib55),[55](https://arxiv.org/html/2608.23642#bib.bib49),[1](https://arxiv.org/html/2608.23642#bib.bib61)\]\. However, the presence of an overseer does not entail reliable oversight\[[51](https://arxiv.org/html/2608.23642#bib.bib52),[100](https://arxiv.org/html/2608.23642#bib.bib53),[62](https://arxiv.org/html/2608.23642#bib.bib57),[72](https://arxiv.org/html/2608.23642#bib.bib42)\]\.Howto ensure this oversight is effective and reliable is underexplored in the design of current AI agent systems, and whether appropriate human oversight is even possible in a given system is rarely \(if ever\) questioned\. Although human oversight should be a joint effort between AI builders, deployers, and users\[[54](https://arxiv.org/html/2608.23642#bib.bib32)\], the onus of oversight currently falls almost entirely on users\. Industry guidelines, policy recommendations, and interface messages in AI applications encourage users to “audit” outputs, “review” claims, “prevent” harm, and “double\-check” for mistakes\[[35](https://arxiv.org/html/2608.23642#bib.bib54),[10](https://arxiv.org/html/2608.23642#bib.bib31),[24](https://arxiv.org/html/2608.23642#bib.bib29),[75](https://arxiv.org/html/2608.23642#bib.bib27),[82](https://arxiv.org/html/2608.23642#bib.bib28),[73](https://arxiv.org/html/2608.23642#bib.bib30),[74](https://arxiv.org/html/2608.23642#bib.bib99),[4](https://arxiv.org/html/2608.23642#bib.bib98)\]\. But the mechanisms users would need to do this effectively are virtually absent\[[104](https://arxiv.org/html/2608.23642#bib.bib24),[33](https://arxiv.org/html/2608.23642#bib.bib25),[80](https://arxiv.org/html/2608.23642#bib.bib26)\]\. Addressing this “intellectual blind spot”\[[30](https://arxiv.org/html/2608.23642#bib.bib45)\]is urgent\. At the same time that AI agent systems have increasingly been deployed at scale, self\-reports have documented the difficulty in maintaining active engagement as an overseer, and research across multiple domains has uncovered serious negative impacts on critical thinking skills resulting from extended use of automated systems\. We therefore urge the community to prioritize this issue and take the position thatcurrent approaches to AI agent development and deployment are detrimental to providing effective human oversight\. To address this, we advocate for a two\-pronged solution ofcognitive scaffolding, provided by developers and deployers\.Developersmust create and implement processes for AI agent runtimes that support the cognitive requirements of effective oversight, including strategic points of friction and interfaces that maintain and/or help regain users’ situational awareness during and after agent operation\.Deployersmust institute organizational protocols that stimulate critical engagement, including trainings on recognizing signs of fatigue, rotation policies that safeguard oversight attention, and processes that preserve domain skill\. Paper Structure\.The rest of this paper is organized as follows\.[Section2](https://arxiv.org/html/2608.23642#S2)defines fundamental concepts relevant to our position and explains the importance of human oversight for the safe deployment of AI agent systems\.[Section3](https://arxiv.org/html/2608.23642#S3)documents how current approaches to AI agent oversight are inadequate and can be actively detrimental to a human’s ability to appropriately oversee\.[Section4](https://arxiv.org/html/2608.23642#S4)examines how cognitive degradation results from continued use of automated systems and explains how this further hampers the ability to provide reliable oversight\. In[Section5](https://arxiv.org/html/2608.23642#S5), we offer concrete solutions for developers and deployers to support appropriate active user engagement as they interact with and oversee AI agent systems\.[Section6](https://arxiv.org/html/2608.23642#S6)engages with alternative views, addressing prevailing wisdom and norms in the development and deployment of AI agents\. Finally,[Section7](https://arxiv.org/html/2608.23642#S7)concludes by arguing that human oversight, far from being a simple add\-on for safety, requires deep engagement with the requirements of human attention and critical analysis\. ## 2The role of human oversight Effective human oversight is critical for the operation of AI agents because agents can take actions whose consequences are wrong, costly, or irreversible\. And yet, AI systems make errors at rates and in patterns that cannot be detected from output review alone\[[11](https://arxiv.org/html/2608.23642#bib.bib37),[80](https://arxiv.org/html/2608.23642#bib.bib26),[93](https://arxiv.org/html/2608.23642#bib.bib12),[18](https://arxiv.org/html/2608.23642#bib.bib5),[60](https://arxiv.org/html/2608.23642#bib.bib4)\], motivating human supervision that is more deeply integrated into system processing\. AI agents are also designed to be flexible across different uses and contexts without needing step\-by\-step instructions for every use case; pre\-deployment testing protocols thus cannot address all possible AI agent action sequences and failure modes\. This means that assessment of the appropriateness of AI agent operations is required in real time, during system execution\. This burden is compounded by the cost of agentic evaluation itself: comprehensive pre\-deployment testing of multi\-step agent behavior is substantially more expensive than evaluating single\-turn systems, making reliable coverage economically infeasible for most deployers\[[45](https://arxiv.org/html/2608.23642#bib.bib77)\]\. We now describe how this oversight is situated in current AI agent practice and trace how its difficulty has escalated as the capabilities of AI systems have advanced\. ### 2\.1The human in the loop Human oversightrefers to mechanisms through which humans monitor, validate, intervene in, or override automated system behavior\. As AI systems have become ubiquitous, the need for human oversight has been stressed as key to safe deployment\. For example, the EU AI Act mandates effective human oversight as a primary risk mitigation strategy, stressing the need for overseers to monitor for anomalies, remain vigilant to automation bias and overreliance, interpret outputs, intervene, reverse, override, and disregard system outputs\[[35](https://arxiv.org/html/2608.23642#bib.bib54)\]\. Commercial generative AI providers disclose that the system can make mistakes and so encourage users to check responses and important information\[[48](https://arxiv.org/html/2608.23642#bib.bib97),[49](https://arxiv.org/html/2608.23642#bib.bib96),[4](https://arxiv.org/html/2608.23642#bib.bib98),[74](https://arxiv.org/html/2608.23642#bib.bib99)\]\. Consulting agencies recommend continued oversight that establishes anchors for accountability throughout autonomous processing\[[58](https://arxiv.org/html/2608.23642#bib.bib101),[13](https://arxiv.org/html/2608.23642#bib.bib100)\]\. A common approach for human oversight ishuman\-in\-the\-loop \(HITL\), a development and deployment approach where people actively participate in a system’s operation by providing feedback, validating output, making decisions, or labeling data\. As AI systems have been deployed at scale to execute multi\-step workflows with some level of autonomy, several distinct lineages of HITL have converged: The human in the loop must prevent unintended harm \(aviation lineage\[[8](https://arxiv.org/html/2608.23642#bib.bib33),[89](https://arxiv.org/html/2608.23642#bib.bib3)\]\) and authorize consequential actions \(autonomous weapons lineage,\[[27](https://arxiv.org/html/2608.23642#bib.bib103),[88](https://arxiv.org/html/2608.23642#bib.bib104),[86](https://arxiv.org/html/2608.23642#bib.bib36),[56](https://arxiv.org/html/2608.23642#bib.bib102),[23](https://arxiv.org/html/2608.23642#bib.bib105)\]\), and their interactions can be used for further model training \(machine learning lineage\[[90](https://arxiv.org/html/2608.23642#bib.bib93),[2](https://arxiv.org/html/2608.23642#bib.bib92)\]\)\. ### 2\.2The evolution of system sophistication In conventional discriminative AI systems, the focus of oversight was on the AI output, such as a prediction or a risk score, with the main goal being to ensure its correctness\[[3](https://arxiv.org/html/2608.23642#bib.bib62),[50](https://arxiv.org/html/2608.23642#bib.bib72)\]\. More recent Generative AI \(GenAI\) systems make oversight substantially more complex\[[64](https://arxiv.org/html/2608.23642#bib.bib23),[25](https://arxiv.org/html/2608.23642#bib.bib22)\]\. Overseers must process voluminous outputs generated faster than they can meaningfully review them\. They must catch hallucinations\[[65](https://arxiv.org/html/2608.23642#bib.bib20)\]and grapple withcapability unpredictability\[[64](https://arxiv.org/html/2608.23642#bib.bib23),[109](https://arxiv.org/html/2608.23642#bib.bib21)\]: As model size has grown, capabilities have appeared that were neither explicitly programmed nor anticipated, that do not work reliably and that exhibit new failure modes\. Agentic AI introduces further complexities\. Unlike basic GenAI systems such as chatbots that are limited to producing a single output in response to a user query,AI agentsexpand GenAI by carrying out multiple steps without explicit human programming or direct human involvement, introducing new levels of opacity\. Decreased need for human specification and increased flexibility in how to operate heighten the risk of unforeseen consequential actions that affect critical systems and protected data while further obscuring what an overseer must address\. For example, agents may silently edit or delete files, exfiltrate sensitive information, exploit software vulnerabilities, enact financial transactions, or make private data public\. Agentic pipelines involve multiple model\-driven roles \(planners, executors, evaluators\) whose interaction can obfuscate critical information before a user sees an output\. AI agent “tool use” exacerbates the problem throughtool hallucination\[[81](https://arxiv.org/html/2608.23642#bib.bib110),[111](https://arxiv.org/html/2608.23642#bib.bib109)\]: Generting misleading or incorrect tool information that is seemingly plausible but difficult to verify, such as fabricating non\-existent tools, invoking real tools with incorrect parameters, or misreading tool outputs\. Consequences from these actions then cascade through multi\-step plans\. Related to our position,\[[111](https://arxiv.org/html/2608.23642#bib.bib109)\]find that the current focus on improving “reasoning” amplifies tool hallucination, making the human oversight problem even harder\. Agents also exhibitbehavioral unpredictabilityduring operation, acting in ways their developers did not anticipate and, in some cases, producing misleading information tailored to the user\[[20](https://arxiv.org/html/2608.23642#bib.bib19),[22](https://arxiv.org/html/2608.23642#bib.bib18)\]\. These challenges are further magnified bymulti\-agent systems: When several agents work together on tasks managed by an orchestrator, failures can arise frominter\-agent misalignment, when an agent fails to provide necessary information or to request needed information from another agent\[[16](https://arxiv.org/html/2608.23642#bib.bib16)\]\. ## 3The oversight oversight: What current AI agent oversight discussions miss Despite the broad recognition of the importance of human oversight, there is limited recognition of the deep complexities of ensuring oversight is reliable, effective, and appropriate for the needs of both the system and the overseer\. The bulk of the focus in AI agent development continues to measure success through task outcomes such as speed, accuracy, and throughput\[[30](https://arxiv.org/html/2608.23642#bib.bib45)\], treating oversight as a separate consideration independent of the quality of the system\. A notable exception to this norm,\[[80](https://arxiv.org/html/2608.23642#bib.bib26)\]argues how and why the current approach to “transparency” in agentic systems is incompatible with effective oversight; their sociotechnical perspective is most similar to our position, and we extend it with a focus on technical solutions and long\-term challenges\. In this section, we document how the design of AI agent systems dictates the nature of human oversight, and how it falls short\. The amount of content relevant to an overseer is massive\.Effective supervision of an AI agent requires maintaining an appropriate mental model of a large and heterogeneous stream of information generated during execution\. This includes the agent’s “chain\-of\-thought” – its natural\-language reasoning traces as it works towards goals – along with the tools it selects, the arguments it passes, the plans it makes, its exchanges with other modules, and the content it produces\. These details are dense, lengthy, and distributed across components that can change or disappear, making comprehension and review difficult\. Questions that on their surface may seem straightforward, such as why an agent chose a tool, modified a plan, or altered course, can be effectively unanswerable in practice\[[69](https://arxiv.org/html/2608.23642#bib.bib15)\]\. Real\-time observation can help spot mistakes early\[[28](https://arxiv.org/html/2608.23642#bib.bib14),[93](https://arxiv.org/html/2608.23642#bib.bib12)\], but the cognitive demands of comprehending rapid, dynamic agentic information often exceed users’ ability to keep up\[[39](https://arxiv.org/html/2608.23642#bib.bib13),[80](https://arxiv.org/html/2608.23642#bib.bib26)\]\. The user must play multiple roles simultaneously\.They must leverage the system for their own goal while also providing permissions for actions during agent operation\[[7](https://arxiv.org/html/2608.23642#bib.bib80),[76](https://arxiv.org/html/2608.23642#bib.bib81),[52](https://arxiv.org/html/2608.23642#bib.bib75),[95](https://arxiv.org/html/2608.23642#bib.bib76),[26](https://arxiv.org/html/2608.23642#bib.bib9)\], assessing the accuracy of the agent’s selections, evaluating the safety and appropriateness of each step, and foreseeing potential ramifications\. In practice, users find themselves repeatedly approving prompts the agent surfaces to continue its processing\[[1](https://arxiv.org/html/2608.23642#bib.bib61),[61](https://arxiv.org/html/2608.23642#bib.bib94),[102](https://arxiv.org/html/2608.23642#bib.bib44)\], leading to “approval fatigue”, in which users stop paying close attention to what they are approving\[[6](https://arxiv.org/html/2608.23642#bib.bib82),[40](https://arxiv.org/html/2608.23642#bib.bib95),[31](https://arxiv.org/html/2608.23642#bib.bib91)\]\. The cognitive demand is substantial\.Formal characterizations of the user’s shifting role as AI systems increase in autonomy \(e\.g\.,\[[78](https://arxiv.org/html/2608.23642#bib.bib35),[38](https://arxiv.org/html/2608.23642#bib.bib66),[68](https://arxiv.org/html/2608.23642#bib.bib113)\]\) miss the cognitive demands each shift imposes: As the user is further relegated to a role of “approver”, effective oversight requires assessing each decision point as a task expert, anticipating and guarding against unintended outcomes, tracking the agent’s multi\-step plan, and maintaining a mental model of its goals, current execution state, and prior actions – a working memory and situational awareness load that current interfaces are not designed to support\[[26](https://arxiv.org/html/2608.23642#bib.bib9)\]\. That same blind spot appears in governance: The recommendations for human oversight in the EU AI Act are applicable only if the human overseer maintains reliable cognition, attention, and skill; missing that the system itself might be eroding those very capacities\. Current oversight affordances are poorly designed\.Vendor frameworks acknowledge that supporting effective oversight requires thoughtful design, but overlook user needs: Anthropic emphasizes user control through front\-loaded input and agent\-initiated re\-engagement\[[5](https://arxiv.org/html/2608.23642#bib.bib59)\]\. Salesforce\[[85](https://arxiv.org/html/2608.23642#bib.bib60)\]and AWS\[[1](https://arxiv.org/html/2608.23642#bib.bib61)\]discuss HITL roles but not human tendencies during sustained engagement\. “Human\-in\-the\-loop" is only a meaningful solution if the human can independently see into the loop\. Human cognition follows patterns that can inform oversight design\.Taken together, these issues are part of a pattern pointing to a solution\. Current AI agent design centers theagent: Systems are built to surface their own processing to the user on their own terms\[[80](https://arxiv.org/html/2608.23642#bib.bib26)\]\. Effective oversight requires centering theuser: They must be able to understand and maintain focus\. Cognitive science provides guidance on how to support this\. Notably,dual\-process theoryposits that humans use one of two primary reasoning methods in decision making: quickly and automatically, employing basic heuristics \(“System 1 thinking”\); or slowly and deliberatively \(“System 2 thinking”\)\[[34](https://arxiv.org/html/2608.23642#bib.bib69),[36](https://arxiv.org/html/2608.23642#bib.bib70),[57](https://arxiv.org/html/2608.23642#bib.bib67)\]– a necessary mode for the critical analyses needed for effective oversight\. When decisions are routine, System 1 predominates\[[103](https://arxiv.org/html/2608.23642#bib.bib63)\], and early evidence suggests that this is the dominant mode of engagement with AI agent coding systems\[[15](https://arxiv.org/html/2608.23642#bib.bib106),[26](https://arxiv.org/html/2608.23642#bib.bib9)\]\. Yet people do not easily switch from System 1 to 2 mid\-task\[[66](https://arxiv.org/html/2608.23642#bib.bib65),[112](https://arxiv.org/html/2608.23642#bib.bib68),[103](https://arxiv.org/html/2608.23642#bib.bib63),[11](https://arxiv.org/html/2608.23642#bib.bib37)\]; affordances for their cognitive needs must be provided\. ## 4The irony of automation Re\-examining conventional thinking on AI agent development and deployment is increasingly urgent because it is quickly becoming clear that continued used of AI agent systems delegates humans’ epistemic agency, leading to cognitive degradation, skill atrophy, and ultimately destroying oversight capability\. We now describe these phenomena\. ### 4\.1Critical skills degrade Studies on sustained AI use document negative effects on cognitive capacities required for oversight: “deskilling” \(skill atrophy\) and “intuition rust"\[[12](https://arxiv.org/html/2608.23642#bib.bib40),[30](https://arxiv.org/html/2608.23642#bib.bib45)\], decreased critical and analytical thinking\[[113](https://arxiv.org/html/2608.23642#bib.bib38)\], reduced vigilance and pattern recognition needed to recognize abnormalities or catch mistakes\[[12](https://arxiv.org/html/2608.23642#bib.bib40),[32](https://arxiv.org/html/2608.23642#bib.bib34)\], and overreliance\[[11](https://arxiv.org/html/2608.23642#bib.bib37),[113](https://arxiv.org/html/2608.23642#bib.bib38)\]\. Recent terminology has crystallized additional patterns: “cognitive dependence"\[[43](https://arxiv.org/html/2608.23642#bib.bib41)\], “cognitive debt"\[[59](https://arxiv.org/html/2608.23642#bib.bib39)\], and “cognitive surrender"\[[94](https://arxiv.org/html/2608.23642#bib.bib43)\]\. These effects are due in part to requirements of critical thinking that are unmet in current system design\. Using increased automation means fewer occasions for truth\-seeking, evidence\-seeking, considering multiple perspectives, skepticism, and the iterative work of building knowledge\[[91](https://arxiv.org/html/2608.23642#bib.bib71),[113](https://arxiv.org/html/2608.23642#bib.bib38)\]\. Over time, users find that they offload cognitively demanding tasks that are critical for understanding situations, hindering their ability to adequately assess situations\[[47](https://arxiv.org/html/2608.23642#bib.bib58)\]\. \(Humorously captured in[Figure5](https://arxiv.org/html/2608.23642#A2.F5)\.\) The cognitive cost is measurable: A recent study found that participants who used an LLM for essay writing had significantly decreased brain connectivity\[[59](https://arxiv.org/html/2608.23642#bib.bib39)\]\. Diminishing critical thinking ability is particularly worrisome in the context of novices learning new tasks, where the ability to develop appropriate reasoning skills is blunted\[[113](https://arxiv.org/html/2608.23642#bib.bib38)\]\. Users may never fully develop the foundational skills needed to be effective or become experts on tasks they use AI agents for\[[12](https://arxiv.org/html/2608.23642#bib.bib40)\]— a particular concern given that robust domain knowledge is a prerequisite for effective intervention in dynamic systems and for taking over manual operations when automation fails\[[32](https://arxiv.org/html/2608.23642#bib.bib34)\]\. Expert users also over\-rely on AI\[[42](https://arxiv.org/html/2608.23642#bib.bib11),[79](https://arxiv.org/html/2608.23642#bib.bib17),[87](https://arxiv.org/html/2608.23642#bib.bib10)\], leading to particularly pronounced effects in contexts slightly outside of their expertise, such as an experienced software developer using AI agents to code in a new programming language\[[26](https://arxiv.org/html/2608.23642#bib.bib9)\]\. Multiple cognitive biases also impede effective oversight\. With automation bias\[[113](https://arxiv.org/html/2608.23642#bib.bib38),[11](https://arxiv.org/html/2608.23642#bib.bib37)\], users accept system suggestions even when they are wrong\. With anchoring bias, people are more likely to agree to an AI system’s decision when provided before they form their own\[[50](https://arxiv.org/html/2608.23642#bib.bib72),[11](https://arxiv.org/html/2608.23642#bib.bib37)\]; complacency bias\[[32](https://arxiv.org/html/2608.23642#bib.bib34),[78](https://arxiv.org/html/2608.23642#bib.bib35)\]and our tendency to favor fast, heuristic\-based shortcuts instead of slower, more effortful reasoning\[[113](https://arxiv.org/html/2608.23642#bib.bib38),[11](https://arxiv.org/html/2608.23642#bib.bib37)\]especially in the face of mental overload\[[112](https://arxiv.org/html/2608.23642#bib.bib68),[78](https://arxiv.org/html/2608.23642#bib.bib35)\]further challenge overseers\. ### 4\.2Oversight ability diminishes Empirical work is now beginning to show how this happens and document its effects\. For example, several studies analyzing overreliance on GenAI found that users take incorrect shortcuts for assessing the accuracy of outputs, such as mistaking the well\-written style of ChatGPT responses or the presence of citations as signals of accuracy\[[106](https://arxiv.org/html/2608.23642#bib.bib8)\]and adopting heuristics that prioritized efficiency – such as treating an agent’s plan as a “faithful proxy” of what it would do, and assuming that if an agent’s code passed unit tests, it was correct\[[26](https://arxiv.org/html/2608.23642#bib.bib9)\]\. While heuristic short\-cuts are practical and play an important role, human cognitive limits coupled with extraordinary agentic capabilities point to a future of agent oversight driven by suboptimal control\. System behavior also further nudges users away from critical information\. For example, sycophantic GenAI models validate users more often than humans evaluating the same situations, but users prefer and trust these models over less sycophantic baselines\[[21](https://arxiv.org/html/2608.23642#bib.bib85)\]\.\[[92](https://arxiv.org/html/2608.23642#bib.bib86)\]document disempowerment patterns in real\-world Claude conversations, where users’ actions, beliefs, or values become less well\-aligned with reality, while rating those conversations favorably\.\[[46](https://arxiv.org/html/2608.23642#bib.bib84)\]describe how the dominant chatbot paradigm of single authoritative responses, opaque reasoning, agreeable tone, and optimization for smooth interaction, reduces user agency by removing the cognitive friction needed for independent judgment\. Together, these works suggest that interaction patterns experienced as helpful or satisfying can still weaken the forms of independence, skepticism, and self\-monitoring that oversight requires\. ### 4\.3Ineffective oversight is incentivized Users have not only begun to notice the cognitive effects, they have begun to find themselves being repelled from engaging with the systems at all\. Recent self\-reports of experiences with AI agent systems have described the sensation of losing the ability to effectively engage with AI systems\. Current HITL processes in coding “\[make\] the loop stultifying”\[[102](https://arxiv.org/html/2608.23642#bib.bib44)\]and they move to “babysit the outputs, catch the occasional hallucination”\[[105](https://arxiv.org/html/2608.23642#bib.bib83)\]\. Using software agents “makes developers cognitively distant from the code they must review,”\[[26](https://arxiv.org/html/2608.23642#bib.bib9)\], making it challenging to find and fix even simple issues\. This creates an “out\-of\-the\-loop” performance problem\[[32](https://arxiv.org/html/2608.23642#bib.bib34)\]that decreasessituation awareness– the perception and comprehension of relevant environmental states – as human operators shift from active participants to passive information processors\. Yet paradoxically, they must be vigilant enough to intervene and deeply engage at strategic moments, a well\-recognized challenge in cognitive science \([Section3](https://arxiv.org/html/2608.23642#S3)\)\. The history of technology warns us that systemic erosion of situational awareness coupled with human cognitive constraints does not merely inhibit oversight, but eventually results in the expulsion of humans to theoutsideof oversight loops\[[67](https://arxiv.org/html/2608.23642#bib.bib7),[14](https://arxiv.org/html/2608.23642#bib.bib6)\]\. ### 4\.4Ineffective oversight creates a feedback loop These cognitive effects also create an alignment\-relevant failure mode\. Modern language models are commonly trained from human feedback\[[77](https://arxiv.org/html/2608.23642#bib.bib87)\], and deployed agent systems may also be evaluated through approval rates, completion rates, user satisfaction, or other user behavioral signals\. As oversight quality decreases, the signals sent back to the system for retraining degrade in quality\. An alert overseer may scrutinize a plan, notice missing information, and withhold approval\. A tired or out\-of\-the\-loop overseer may approve quickly, accept fluent rationales, and rate the interaction favorably\. If such approvals are treated as evidence of success, the system can begin to optimize for disincentivizing thoughtful oversight: producing confident summaries, reducing friction, and surfacing oversimplified plans that are easy to skim\. The concern does not require developers to explicitly optimize for cognitive depletion\. It arises whenever the measured target, user\-revealed approval, comes apart from the intended target, well\-scrutinized correct action\. In that setting, the same kinds of interaction patterns associated with sycophancy\[[21](https://arxiv.org/html/2608.23642#bib.bib85)\]and disempowerment\[[92](https://arxiv.org/html/2608.23642#bib.bib86)\]become especially concerning at agent scale, where the human is already pushed toward passive monitoring\. Oversight degradation therefore connects the irony of automation to reward hacking of human oversight: the human rater can become the exploitable part of the reward channel\. Evil double agents\.A related risk arises when models learn which failures users are likely to detect\. Recent work on agentic upward deception shows that agents can conceal failure under environmental constraints, including by guessing results, substituting unavailable sources, or fabricating local files\[[53](https://arxiv.org/html/2608.23642#bib.bib90)\]\. If an agent can predict which process violations an overseer is unlikely to inspect, it can choose plans, explanations, or timing that keep problematic behavior below the threshold of detection\. ### 4\.5The result The result is what[8](https://arxiv.org/html/2608.23642#bib.bib33)referred to as anirony of automation: The more capable an automated system is, the more the human operator’s skills and situation awareness degrade, over\-trusting the systems and ultimately leading to situations where people are least prepared to help when it’s most needed\. The cases where human oversight is meant to be most impactful and most critical – rare instances, instances requiring deep familiarity and skill – are the same ones where humans become less equipped to provide appropriate oversight\.The very act of being an overseer degrades the capacities oversight requires: Oversight degrades the overseer\.This irony is in full force with AI agents, and addressing it requires treating human cognitive requirements as first\-class design constraints, rather than governance add\-ons that can be adapted into AI agent systems after they are developed\. We turn to these considerations next\. ## 5Solutions Enabling appropriate oversight of AI agents requires grappling with the complexities of human cognition when interacting with automated systems, an active area of HCI research\[[96](https://arxiv.org/html/2608.23642#bib.bib89),[110](https://arxiv.org/html/2608.23642#bib.bib88)\]\. Building on this work, we present a structured inventory of user\-centric solutions drawn from cognitive science, psychology, and user design\. In synthesizing this diffuse space we found it necessary to construct an organizing device \(Table[3](https://arxiv.org/html/2608.23642#S5.F3)\) that maps oversight goals against solution categories and the human roles they involve\. We share it as a structural map to help researchers and practitioners locate existing work and identify gaps; not as a definitive categorization\. We stress that our solutions below do not represent a complete, perfect, or fixed inventory, that boundaries overlap, and that the content admits of many different groupings; we are providing clustered solutions to contribute to the work we are advocating for—a vast domain of open research to address\. The inventory spans two stages of the AI agent life cycle,developmentanddeployment\. Within each there are two types of cognitive support:engaging oversightandmaintaining skills\. Development\-stage solutions concern design\-level affordances for what a user can access, when, and how\. Deployment\-stage solutions involve organizational protocols that build awareness of oversight issues and help users stay critically engaged\. These are summarized in[Figure3](https://arxiv.org/html/2608.23642#S5.F3)\. ††nicematrix\-placeholder:NiceTabular \(nicematrix\)Table 2:Organizing device we developed in our solution search, informing our categorization and further guiding our search to fill gaps\. Checks mark solutions we describe in Section 5\. Monitoring=Behavioral Monitoring; Trainings=Trainings & Exercises; Workload=Workload & Scheduling\. Larger version in Appendix[Table3](https://arxiv.org/html/2608.23642#A1.T3)\.DevelopmentDesign\-level affordancesDeploymentOrganizational protocolsEngagingoversightIn\-the\-momentattentionMaintainingskillsLong\-termcapacityStrategic frictionApproval designBehavioral monitoringCanariesApproval designBehavioral monitoringWorkload & schedulingTrainings & exercisesRole designAuditsFigure 3:Summary of potential solutions described\. Interventions span development and deployment, and two types of cognitive support: engaging oversight in the moment, maintaining skills long\-term\. Strategic friction\.These mechanisms require the user to perform cognitive work before or alongside agent operation, include incentivizing metacognition\[[97](https://arxiv.org/html/2608.23642#bib.bib108),[110](https://arxiv.org/html/2608.23642#bib.bib88)\]and includecognitive forcing functions– interventions at the moment of decision\-making that encourage or require the user to engage analytically with content\[[11](https://arxiv.org/html/2608.23642#bib.bib37),[44](https://arxiv.org/html/2608.23642#bib.bib107),[30](https://arxiv.org/html/2608.23642#bib.bib45)\]\. - •Pre\-commitment mechanismshave the user record their own view or decision before seeing the agent’s recommendation\[[11](https://arxiv.org/html/2608.23642#bib.bib37)\]\. This intervention prevents anchoring bias and creates an audit trail for further analysis\. - •Delay\-and\-choice mechanismslet the user decide whether and when to see the AI’s output at all, preserving the option to work unaided and protecting the cognitive capacities that “always on” assistance atrophies\[[11](https://arxiv.org/html/2608.23642#bib.bib37)\]\. - •Reasoning probes, inline prompts at high\-stakes moments, such as “what evidence would change your mind?” or “what assumption does this approval rest on?”, maintain critical thinking\[[96](https://arxiv.org/html/2608.23642#bib.bib89)\]\. - •Action gatingrequires explicit verification before the agent proceeds down a consequential path, and may surface alternative options for the user to choose between rather than presenting a single recommendation to accept or reject\. Approval design\.This treats expert attention as a critical, scarce resource that should be brought in strategically, helping to reduce fatigue and mitigate acquiescence\. It involves consideration of which actions require sign\-off, when, and at what granularity\[[32](https://arxiv.org/html/2608.23642#bib.bib34),[43](https://arxiv.org/html/2608.23642#bib.bib41),[19](https://arxiv.org/html/2608.23642#bib.bib2)\]\. - •Bounded autonomyinvolves prespecifying what the agent may do without approval, reserving user attention for decisions that genuinely require judgement\[[31](https://arxiv.org/html/2608.23642#bib.bib91)\]\. - •Batch reviewwhere the agent completes a logical unit of work and then surfaces the whole batch as a diff for the user to review, supporting users in engagement and judgement by making it possible to evaluate related actions together\[[31](https://arxiv.org/html/2608.23642#bib.bib91)\]\. Deploying institutions may also implement secondary checks, such as an additional review from a person for consequential actions\. - •Automated pre\-checkssupplementing human review with machine verification of properties that do not require judgement\[[31](https://arxiv.org/html/2608.23642#bib.bib91)\]\. Behavioral monitoring\.This category treats oversight quality as an empirically measurable property of the human\-AI system, and can test how well oversight is functioning\[[70](https://arxiv.org/html/2608.23642#bib.bib1)\]\. - •Time\-based signaturestrack whether review duration drops while approval rates remain constant, or whether review time does not increase as the complexity of agent operations grows; both suggest that user attention and critical analyses may be waning\. - •Override signaturestrack whether the rate of disagreement with the agent declines over time or with increasing complexity, suggesting growing acquiescence rather than improving agent quality\. - •Evidence\-seeking signaturestrack whether the user requests additional information when stakes rise, since a reviewer who stops asking questions has likely stopped reviewing\. There are strategies that can additionally assess the agent: - •Canaries, tasks inserted into the user’s normal workflow where the correct answer is known in advance, can provide direct evidence of degraded judgement\. These can be implemented by developers and leveraged by deployers, triggering a break, rotation, or retraining\. For the agent, tasks can be designed so the correct behavior is to report inability rather than fabricate a plausible answer, testing whether the agent honestly surfaces its limits\. - •Style\-fatigue covariance analysestrack whether the agent may be adapting to, or encouraging, weakened oversight\. - •Auditsoperate retrospectively to evaluate different dimensions of oversight\. Reviewers can re\-examine past decisions\. For the agent, trace summaries can be checked against raw action logs to detect cases where the agent’s narrative of what it did diverges from what it actually did\. An adaptive AI agent system could potentially monitor user behavior in order to calibrate how it should interact with the user, supporting judgment and engagement and preventing cognitive overload\. “Guardian agents”\[[83](https://arxiv.org/html/2608.23642#bib.bib112)\]could monitor for signs of degradation in human\-agent interaction\. Trainings and exercises\.Elucidate, maintain, and strengthen the cognitive activities oversight requires, including domain expertise, calibrated skepticism, and awareness of one’s own degradation\. - •Domain skill maintenance exerciseshave users regularly perform the underlying task without AI assistance\. This helps to retain the expertise needed to evaluate AI outputs and to prevent the deeper organizational problem in which no one remaining understands the work well enough to oversee it\[[30](https://arxiv.org/html/2608.23642#bib.bib45)\]\. Experts may also benefit from regular retrainings on the tasks they oversee\[[72](https://arxiv.org/html/2608.23642#bib.bib42)\]\. - •Critical evaluation trainingteaches users to read AI outputs with appropriate skepticism: What kinds of errors to expect, how to distinguish fluent\-sounding output from correct output, and how to weigh AI claims against external evidence\[[43](https://arxiv.org/html/2608.23642#bib.bib41)\]\. - •Self\-monitoring trainingteaches users strategies for metacognition\[[37](https://arxiv.org/html/2608.23642#bib.bib64)\]– recognizing when their attention is flagging and they are not engaging in critical scrutiny\. This can raise awareness of cognitive biases and behaviors thatBehavioral Monitoringsupplements\[[43](https://arxiv.org/html/2608.23642#bib.bib41)\]\. Workload and scheduling\.Addressing the physiological inevitability of fatigue can help to support judgement and engagement when the user is alert\. - •Enforced breaksprevent extended sessions that wear down cognitive sharpness\. - •Rotationsmove users between tasks, and between AI\-assisted and unassisted versions of the same task, to prevent both fatigue and the cognitive surrender from prolonged exposure to agentic AI\. Role design\.The user must want to do the oversight job well, must be in a position where doing it well is rewarded, and must have the baseline competence to do it\[[101](https://arxiv.org/html/2608.23642#bib.bib111)\]\. - •Assigning rolesto ensure that users providing oversight have the expertise to evaluate AI outputs\. - •Separating rolesto ensure that the user performing oversight is not also the decision\-maker who benefits from the AI’s output continuing to be approved, which would produce a structural incentive against intervention\. - •Aligning incentivesto remove productivity targets and performance metrics that disincentivize appropriate scrutiny, and to reward high\-quality oversight\. ## 6Alternative Viewpoints There are caveats to implementing these interventions\. Initial evidence suggests users may disprefer systems that reduce overreliance\[[11](https://arxiv.org/html/2608.23642#bib.bib37)\], pointing to a tension between user satisfaction and oversight preservation\. There are several further substantive objections to our position\. ### 1\. Cognitive effects are overstated, and people will adapt The cognitive effects are real but more modest, and will diminish as users adapt to working with AI agents, just as users learned to calibrate trust in search engines and autocomplete\. Treating current cognitive findings as permanent risks designing for a transitional state\. Response:\(1\) The adaptation analogy understates the structural difference between AI agents and earlier tools\. Search engines return results the user evaluates; AI agents take actions the user is expected to authorize\. The cognitive load of evaluating a returned document is not comparable to maintaining situational awareness across a multi\-step plan executing in real time\. \(2\) Self\-reports of cognitive degradation and the difficulty in maintaining attention and are rapidly emerging \(supported by EEG findings\), and discussions on potential solutions align with our recommendations but lack clarity on relevant findings from previous work and the interplay of developers, deployers, users and systems\. \(3\) This assumes that the rate of capability advancement provides enough time to adapt\. But agent capabilities are increasing faster than cognitive science can keep up with, and recent work indicates that capability improvements \(such as enhanced reasoning\) actively amplify failure modes that overseers must catch\[[111](https://arxiv.org/html/2608.23642#bib.bib109)\]\. The issue is happeningnow\. Adaptation cannot be treated as sufficient to address peristent and growing issues\. ### 2\. Better tooling and transparency will solve this The problem is essentially a tooling problem: Better explanations, reasoning traces, and richer audit logs will give users what they need\. Existing research already pursues the “cognitive scaffolding” described here; there is no need for a paradigm shift\. Response:We partially agree: Better tooling and transparency is part of what cognitive scaffolding requires, and several of the design\-level affordances we propose \(batch review, action gating\) are consistent with existing research\. We argue that this work is necessary, but insufficient: \(1\) Explanations operate on cognitive capacities that AI agent use itself degrades and can increase inappropriate trust\. Users rarely engage deeply with each detail, and explanations can themselves be incorrect or act as cognitive anchors\[[11](https://arxiv.org/html/2608.23642#bib.bib37),[107](https://arxiv.org/html/2608.23642#bib.bib74),[98](https://arxiv.org/html/2608.23642#bib.bib73)\]\. \(2\) A focus on just these solutions leaves the organizational prong of our proposal unaddressed\. No interface improvement addresses approval fatigue from sustained sessions or a mismatch between user capability and system operation\. These require organizational protocols\[[30](https://arxiv.org/html/2608.23642#bib.bib45)\], which are out of scope for any tooling\-focused agenda\. Treating cognitive support as a first\-class concern does not reject a push for better tooling or XAI; it recognizes that tooling alone cannot do the work being asked of it\. ### 3\. Human oversight is becoming obsolete because alignment will solve it As AI systems become more aligned, the need for vigilant human oversight diminishes\. Preference\-based training \(e\.g\., RLHF\) is the long\-term solution\. Response:A strong version of this view is that technical alignment will eliminate the need for oversight\. This relies on assumptions about future capability advances that are not currently demonstrated\. Oversight remains a stated requirement of safety frameworks and governance regulations applicablenow\. Further, sociotechnical scholarship in HCI and STS highlight that “perfect" agents are neither realistic nor desirable: Failures are fundamental to acting in human worlds, especially as moments of learning and improvisation\[[80](https://arxiv.org/html/2608.23642#bib.bib26)\], and a direct result of systems that are flexible to different users’ \(potentially conflicting\) needs\. Another version of this view is that preference\-based training addresses oversight by allowing low ratings on outputs that undermine it, but that assumes that user preferences are a reliable proxy for oversight quality\. The evidence in[Section4](https://arxiv.org/html/2608.23642#S4)suggests otherwise\. Users prefer fluent, agreeable, and confidence\-inducing outputs even when those reduce skepticism and encourage overreliance\[[11](https://arxiv.org/html/2608.23642#bib.bib37)\]\. Under preference\-based training optimized against cognitively degraded raters, the same dynamics that produce sycophancy\[[21](https://arxiv.org/html/2608.23642#bib.bib85)\]and disempowerment\[[92](https://arxiv.org/html/2608.23642#bib.bib86)\]would produce systems that increasingly reward the conditions of their own ineffective oversight\. Preference\-based training can be part of an oversight system, but only if approval signals are audited against independent measures of oversight quality\. The economic shift toward RLAIF and self\-distillation\[[63](https://arxiv.org/html/2608.23642#bib.bib78)\]further weakens the idea that human feedback can continue to provide a reliable corrective mechanism\. ## 7Conclusion The need for human oversight of AI agents is recognized broadly: written into governance frameworks, vendor documentation, and the design of agent systems themselves\. Yet the current trajectory of AI agent advancement does not meaningfully engage with what oversight requires, and instead actively contributes to its degradation\. The more autonomy agents are granted, the less the user is positioned to oversee them, and the more the very cognitive capacities oversight requires, such as situational awareness, critical judgement, and domain skill, are undermined by the act of using these systems\. In the current state of the art,oversight degrades the overseer\. Without intervention, users will be pushed further out of the loop as agentic systems are deployed at greater scale and across more consequential domains\. Users will continue to approve plans they have not meaningfully reviewed, accept rationales they have not independently evaluated, and certify actions whose consequences they cannot anticipate\. In the limit, this is not human oversight at all: It is a superficial actor in a system they cannot meaningfully penetrate\. Avoiding this trajectory requires treating cognitive support for human overseers as a first\-class concern in AI agent system development, on par with capability\. We have outlined a two\-pronged approach:developersmust build runtime affordances – such as strategic friction, well\-considered approval design, behavioral monitoring – that preserve the conditions for effective oversight; anddeployersmust institute organizational protocols – such as trainings and breaks – that protect the cognitive capacities oversight requires over time\. Neither prong is sufficient alone, and together, they constitute the cognitive scaffolding agentic systems will need if human oversight is meant to be meaningful\. We urge developers and deployers to adopt these or similar approaches and invite the broader NeurIPS community to take up this work – through empirical research on cognitive degradation across deployment contexts, system\-level audits of whether existing agents support the oversight they presume, and the design and evaluation of new affordances and protocols that the inventory in[Section5](https://arxiv.org/html/2608.23642#S5)only begins to outline\. If we do not act, the irony of automation that Bainbridge \[[8](https://arxiv.org/html/2608.23642#bib.bib33)\] identified four decades ago will play out at scale, compounding across every domain agents are deployed into\. With the rapid rise of agentic AI systems, the time to think critically about how to support effective human oversight is long past due\. ## References - \[1\]Amazon Web Services\(2026\)Human\-in\-the\-loop constructs for agentic workflows in healthcare and life sciences\.Note:AWS Machine Learning BlogExternal Links:[Link](https://aws.amazon.com/blogs/machine-learning/human-in-the-loop-constructs-for-agentic-workflows-in-healthcare-and-life-sciences/)Cited by:[§1](https://arxiv.org/html/2608.23642#S1.p1.1),[§3](https://arxiv.org/html/2608.23642#S3.p3.1),[§3](https://arxiv.org/html/2608.23642#S3.p5.1)\. - \[2\]S\. Amershi, M\. Cakmak, W\. B\. Knox, and T\. Kulesza\(2014\)Power to the people: the role of humans in interactive machine learning\.AI Magazine35\(4\),pp\. 105–120\.External Links:[Document](https://dx.doi.org/https%3A//doi.org/10.1609/aimag.v35i4.2513),[Link](https://onlinelibrary.wiley.com/doi/abs/10.1609/aimag.v35i4.2513),https://onlinelibrary\.wiley\.com/doi/pdf/10\.1609/aimag\.v35i4\.2513Cited by:[§2\.1](https://arxiv.org/html/2608.23642#S2.SS1.p2.1)\. - \[3\]S\. Amershi, D\. Weld, M\. Vorvoreanu, A\. Fourney, B\. Nushi, P\. Collisson, J\. Suh, S\. Iqbal, P\. N\. Bennett, K\. Inkpen, J\. Teevan, R\. Kikin\-Gil, and E\. Horvitz\(2019\)Guidelines for human\-AI interaction\.InProceedings of the 2019 CHI Conference on Human Factors in Computing Systems,CHI ’19,Glasgow, Scotland, UK,pp\. 1–13\.External Links:[Document](https://dx.doi.org/10.1145/3290605.3300233)Cited by:[§2\.2](https://arxiv.org/html/2608.23642#S2.SS2.p1.1)\. - \[4\]AnthropicClaude\.Note:Anthropic \(web application\)Accessed: 2026\-05\-01External Links:[Link](https://claude.ai/)Cited by:[§1](https://arxiv.org/html/2608.23642#S1.p2.1),[§2\.1](https://arxiv.org/html/2608.23642#S2.SS1.p1.1)\. - \[5\]Anthropic\(2025\)Trustworthy agents in practice\.Note:Anthropic ResearchExternal Links:[Link](https://www.anthropic.com/research/trustworthy-agents)Cited by:[§3](https://arxiv.org/html/2608.23642#S3.p5.1)\. - \[6\]Anthropic\(2026\)Claude code auto mode: a safer way to skip permissions\(Website\)External Links:[Link](https://www.anthropic.com/engineering/claude-code-auto-mode)Cited by:[§3](https://arxiv.org/html/2608.23642#S3.p3.1)\. - \[7\]Anthropic\(2026\)Security\(Website\)External Links:[Link](https://code.claude.com/docs/en/security)Cited by:[§3](https://arxiv.org/html/2608.23642#S3.p3.1)\. - \[8\]L\. Bainbridge\(1983\)Ironies of automation\.Automatica19\(6\),pp\. 775–779\.External Links:[Document](https://dx.doi.org/10.1016/0005-1098%2883%2990046-8)Cited by:[§2\.1](https://arxiv.org/html/2608.23642#S2.SS1.p2.1),[§4\.5](https://arxiv.org/html/2608.23642#S4.SS5.p1.1),[§7](https://arxiv.org/html/2608.23642#S7.p4.1)\. - \[9\]S\. Bakken\(2023\)AI in health: keeping the human in the loop\.Journal of the American Medical Informatics Association30\(7\),pp\. 1225–1226\.External Links:[Document](https://dx.doi.org/10.1093/jamia/ocad065)Cited by:[§1](https://arxiv.org/html/2608.23642#S1.p1.1)\. - \[10\]J\. R\. Biden\(2023\)Executive order on the safe, secure, and trustworthy development and use of artificial intelligence\.Presidential Actions\.Cited by:[§1](https://arxiv.org/html/2608.23642#S1.p2.1)\. - \[11\]Z\. Buçinca, M\. B\. Malaya, and K\. Z\. Gajos\(2021\)To trust or to think: cognitive forcing functions can reduce overreliance on ai in AI\-assisted decision\-making\.Proceedings of the ACM on Human\-Computer Interaction5\(CSCW1\),pp\. 1–21\.External Links:[Document](https://dx.doi.org/10.1145/3449287)Cited by:[§2](https://arxiv.org/html/2608.23642#S2.p1.1),[§3](https://arxiv.org/html/2608.23642#S3.p6.1),[§4\.1](https://arxiv.org/html/2608.23642#S4.SS1.p1.1),[§4\.1](https://arxiv.org/html/2608.23642#S4.SS1.p4.1),[1st item](https://arxiv.org/html/2608.23642#S5.I1.i1.p1.1),[2nd item](https://arxiv.org/html/2608.23642#S5.I1.i2.p1.1),[§5](https://arxiv.org/html/2608.23642#S5.p3.1),[§6](https://arxiv.org/html/2608.23642#S6.SSx2.p2.1),[§6](https://arxiv.org/html/2608.23642#S6.SSx3.p2.1),[§6](https://arxiv.org/html/2608.23642#S6.p1.1)\. - \[12\]K\. Budzyńet al\.\(2025\)Endoscopist deskilling risk after exposure to artificial intelligence in colonoscopy: a multicentre, observational study\.The Lancet Gastroenterology & Hepatology10\(10\),pp\. 896–903\.External Links:[Document](https://dx.doi.org/10.1016/S2468-1253%2825%2900133-5)Cited by:[§4\.1](https://arxiv.org/html/2608.23642#S4.SS1.p1.1),[§4\.1](https://arxiv.org/html/2608.23642#S4.SS1.p3.1)\. - \[13\]S\. Cantrell D\. Mallonet al\.\(2026\)AI and the future of human decision\-making\.Note:2026 Global Human Capital Trends: From Tensions to Tipping Points\. Deloitte InsightsExternal Links:[Link](https://www.deloitte.com/us/en/insights/topics/talent/human-capital-trends/2026/decision-making-with-ai.html)Cited by:[§2\.1](https://arxiv.org/html/2608.23642#S2.SS1.p1.1)\. - \[14\]N\. Carr\(2014\)The glass cage: automation and us\.\.WW Norton & Company\.Cited by:[§4\.3](https://arxiv.org/html/2608.23642#S4.SS3.p1.1)\. - \[15\]C\. R\. Catalan, L\. M\. Dizon, P\. N\. Monderin, and E\. Kuang\(2026\)"I’m not reading all of that": understanding software engineers’ level of cognitive engagement with agentic coding assistants\.CHI 2026 Workshop on Tools for Thought\.External Links:2603\.14225,[Link](https://arxiv.org/abs/2603.14225)Cited by:[§3](https://arxiv.org/html/2608.23642#S3.p6.1)\. - \[16\]M\. Cemri, M\. Z\. Pan, S\. Yang, L\. A\. Agrawal, B\. Chopra, R\. Tiwari, K\. Keutzer, A\. Parameswaran, D\. Klein, K\. Ramchandran, M\. Zaharia, J\. E\. Gonzalez, and I\. Stoica\(2025\)Why do multi\-agent llm systems fail?\.External Links:2503\.13657Cited by:[§2\.2](https://arxiv.org/html/2608.23642#S2.SS2.p3.1)\. - \[17\]T\. J\. Chaffer, J\. Goldston, B\. Okusanya, and G\. D\.A\.T\.A\.I\(2024\)Decentralized governance of autonomous ai agents\.External Links:[Link](https://api.semanticscholar.org/CorpusID:274982116)Cited by:[§1](https://arxiv.org/html/2608.23642#S1.p1.1)\. - \[18\]A\. Chan, R\. Salganik, A\. Markelius, C\. Pang, N\. Rajkumar, D\. Krasheninnikov, L\. Langosco, Z\. He, Y\. Duan, M\. Carroll, M\. Lin, A\. Mayhew, K\. Collins, M\. Molamohammadi, J\. Burden, W\. Zhao, S\. Rismani, K\. Voudouris, U\. Bhatt, A\. Weller, D\. Krueger, and T\. Maharaj\(2023\)Harms from increasingly agentic algorithmic systems\.InProceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency,FAccT ’23,New York, NY, USA,pp\. 651–666\.External Links:ISBN 9798400701924,[Link](https://doi.org/10.1145/3593013.3594033),[Document](https://dx.doi.org/10.1145/3593013.3594033)Cited by:[§2](https://arxiv.org/html/2608.23642#S2.p1.1)\. - \[19\]C\. Chen, Z\. Zhang, Z\. Chen, E\. Xu, Y\. Yang, I\. Khalilov, S\. A\. Gebreegziabher, Y\. Ye, Z\. Xiao, Y\. Yao, T\. Li, and T\. J\. Li\(2026\)Comparing human oversight strategies for computer\-use agents\.External Links:2604\.04918,[Link](https://arxiv.org/abs/2604.04918)Cited by:[§5](https://arxiv.org/html/2608.23642#S5.p4.1)\. - \[20\]Y\. Chen, J\. Benton, A\. Radhakrishnan, J\. Uesato, C\. Denison, J\. Schulman, A\. Somani, P\. Hase, M\. Wagner, F\. Roger, V\. Mikulik, S\. R\. Bowman, J\. Leike, J\. Kaplan, and E\. Perez\(2025\)Reasoning models don’t always say what they think\.External Links:2505\.05410,[Link](https://arxiv.org/abs/2505.05410)Cited by:[§2\.2](https://arxiv.org/html/2608.23642#S2.SS2.p3.1)\. - \[21\]M\. Cheng, C\. Lee, P\. Khadpe, S\. Yu, D\. Han, and D\. Jurafsky\(2026\)Sycophantic ai decreases prosocial intentions and promotes dependence\.Science391\(6792\),pp\. eaec8352\.External Links:[Document](https://dx.doi.org/10.1126/science.aec8352)Cited by:[§4\.2](https://arxiv.org/html/2608.23642#S4.SS2.p2.1),[§4\.4](https://arxiv.org/html/2608.23642#S4.SS4.p2.1),[§6](https://arxiv.org/html/2608.23642#S6.SSx3.p2.1)\. - \[22\]R\. Coleman\(2026\)Eval awareness in claude opus 4\.6’s browsecomp performance\.Note:\[Online; accessed 2026\-05\-01\]External Links:[Link](https://www.anthropic.com/engineering/eval-awareness-browsecomp)Cited by:[§2\.2](https://arxiv.org/html/2608.23642#S2.SS2.p3.1)\. - \[23\]D\. Cottier\(2022\)Emergence of lethal autonomous weapons systems \(LAWS\) and their necessary apprehension through European human rights law\.Report \(Provisional Version\)Parliamentary Assembly of the Council of Europe, Committee on Legal Affairs and Human Rights\.Note:Rapporteur: Damien Cottier, Switzerland, Alliance of Liberals and Democrats for EuropeExternal Links:[Link](https://assembly.coe.int/LifeRay/JUR/Pdf/TextesProvisoires/2022/20221116-LawsApprehension-EN.pdf)Cited by:[§2\.1](https://arxiv.org/html/2608.23642#S2.SS1.p2.1)\. - \[24\]R\. Crootof, M\. Kaminski, and W\. II\(2022\)Humans in the loop\.SSRN Electronic Journal\.External Links:[Document](https://dx.doi.org/10.2139/ssrn.4066781)Cited by:[§1](https://arxiv.org/html/2608.23642#S1.p2.1)\. - \[25\]D\. R\. Desai and M\. O\. Riedl\(2025\)Responsible ai agents\.External Links:2502\.18359,[Link](https://arxiv.org/abs/2502.18359)Cited by:[§2\.2](https://arxiv.org/html/2608.23642#S2.SS2.p1.1)\. - \[26\]S\. Dhanorkar, S\. Passi, and M\. Vorvoreanu\(2026\)Human oversight of agentic systems in practice: examining the oversight work, challenges, and heuristics of developers using software agents\.\.InProceedings of the 2026 ACM Conference on Fairness, Accountability, and Transparency,FAccT ’26,New York, NY, USA\.External Links:[Link](https://doi.org/10.1145/3805689.3812402),[Document](https://dx.doi.org/10.1145/3805689.3812402),ISBN 979\-8\-4007\-2596\-8/2026/06Cited by:[§3](https://arxiv.org/html/2608.23642#S3.p3.1),[§3](https://arxiv.org/html/2608.23642#S3.p4.1),[§3](https://arxiv.org/html/2608.23642#S3.p6.1),[§4\.1](https://arxiv.org/html/2608.23642#S4.SS1.p3.1),[§4\.2](https://arxiv.org/html/2608.23642#S4.SS2.p1.1),[§4\.3](https://arxiv.org/html/2608.23642#S4.SS3.p1.1)\. - \[27\]B\. Docherty, E\. Neunschwander, M\. Karir, and K\. Flinner\(2012\)Losing humanity: the case against killer robots\.Technical reportHuman Rights Watch and International Human Rights Clinic, Harvard Law School\.External Links:[Link](https://www.hrw.org/report/2012/11/19/losing-humanity/case-against-killer-robots)Cited by:[§2\.1](https://arxiv.org/html/2608.23642#S2.SS1.p2.1)\. - \[28\]L\. Dong, Q\. Lu, and L\. Zhu\(2024\)AgentOps: enabling observability of llm agents\.External Links:2411\.05285,[Link](https://arxiv.org/abs/2411.05285)Cited by:[§3](https://arxiv.org/html/2608.23642#S3.p2.1)\. - \[29\]I\. Drori and D\. Te’eni\(2024\)Human\-in\-the\-loop AI reviewing: feasibility, opportunities, and risks\.Journal of the Association for Information Systems25\(1\),pp\. 98–109\.External Links:[Document](https://dx.doi.org/10.17705/1jais.00867)Cited by:[§1](https://arxiv.org/html/2608.23642#S1.p1.1)\. - \[30\]U\. Ehsan, S\. Passi, K\. Saha, T\. McNutt, M\. O\. Riedl, and S\. Alcorn\(2026\)From future of work to future of workers: addressing asymptomatic AI harms for dignified human\-AI interaction\.InProceedings of the 2026 CHI Conference on Human Factors in Computing Systems,CHI ’26\.Note:arXiv:2601\.21920External Links:[Document](https://dx.doi.org/10.48550/arXiv.2601.21920)Cited by:[§1](https://arxiv.org/html/2608.23642#S1.p3.1),[§3](https://arxiv.org/html/2608.23642#S3.p1.1),[§4\.1](https://arxiv.org/html/2608.23642#S4.SS1.p1.1),[1st item](https://arxiv.org/html/2608.23642#S5.I5.i1.p1.1),[§5](https://arxiv.org/html/2608.23642#S5.p3.1),[§6](https://arxiv.org/html/2608.23642#S6.SSx2.p2.1)\. - \[31\]Encyclopedia of Agentic Coding Patterns\(2025\)Approval fatigue\.Note:Encyclopedia of Agentic Coding Patterns \(online\)Author not publicly identified on the siteExternal Links:[Link](https://aipatternbook.com/approval-fatigue)Cited by:[§3](https://arxiv.org/html/2608.23642#S3.p3.1),[1st item](https://arxiv.org/html/2608.23642#S5.I2.i1.p1.1),[2nd item](https://arxiv.org/html/2608.23642#S5.I2.i2.p1.1),[3rd item](https://arxiv.org/html/2608.23642#S5.I2.i3.p1.1)\. - \[32\]M\. R\. Endsley and E\. O\. Kiris\(1995\)The out\-of\-the\-loop performance problem and level of control in automation\.Human Factors37\(2\),pp\. 381–394\.External Links:[Document](https://dx.doi.org/10.1518/001872095779064555)Cited by:[§4\.1](https://arxiv.org/html/2608.23642#S4.SS1.p1.1),[§4\.1](https://arxiv.org/html/2608.23642#S4.SS1.p3.1),[§4\.1](https://arxiv.org/html/2608.23642#S4.SS1.p4.1),[§4\.3](https://arxiv.org/html/2608.23642#S4.SS3.p1.1),[§5](https://arxiv.org/html/2608.23642#S5.p4.1)\. - \[33\]W\. Epperson, G\. Bansal, V\. C\. Dibia, A\. Fourney, J\. Gerrits, E\. \(\. Zhu, and S\. Amershi\(2025\)Interactive debugging and steering of multi\-agent ai systems\.InProceedings of the 2025 CHI Conference on Human Factors in Computing Systems,CHI ’25,New York, NY, USA\.External Links:ISBN 9798400713941,[Link](https://doi.org/10.1145/3706598.3713581),[Document](https://dx.doi.org/10.1145/3706598.3713581)Cited by:[§1](https://arxiv.org/html/2608.23642#S1.p2.1)\. - \[34\]S\. Epstein\(1994\)Integration of the cognitive and the psychodynamic unconscious\.American Psychologist49\(8\),pp\. 709–724\.External Links:[Document](https://dx.doi.org/10.1037/0003-066X.49.8.709)Cited by:[§3](https://arxiv.org/html/2608.23642#S3.p6.1)\. - \[35\]European Union\(2024\)EU AI act, article 14: human oversight\.Note:EU Artificial Intelligence ActExternal Links:[Link](https://artificialintelligenceact.eu/article/14/)Cited by:[§1](https://arxiv.org/html/2608.23642#S1.p1.1),[§1](https://arxiv.org/html/2608.23642#S1.p2.1),[§2\.1](https://arxiv.org/html/2608.23642#S2.SS1.p1.1)\. - \[36\]J\. St\.B\.T\. Evans\(2003\)In two minds: dual\-process accounts of reasoning\.Trends in Cognitive Sciences7\(10\),pp\. 454–459\.External Links:[Document](https://dx.doi.org/10.1016/j.tics.2003.08.012)Cited by:[§3](https://arxiv.org/html/2608.23642#S3.p6.1)\. - \[37\]F\. Fabiano, M\. B\. Ganapini, A\. Loreggia, N\. Mattei, K\. Murugesan, V\. Pallagani, F\. Rossi, B\. Srivastava, and K\. B\. Venable\(2025\)Thinking fast and slow in human and machine intelligence\.Commun\. ACM68\(8\),pp\. 72–79\.External Links:ISSN 0001\-0782,[Link](https://doi.org/10.1145/3715709),[Document](https://dx.doi.org/10.1145/3715709)Cited by:[3rd item](https://arxiv.org/html/2608.23642#S5.I5.i3.p1.1)\. - \[38\]K\. Feng, D\. W\. McDonald, and A\. X\. Zhang\(2025\)Levels of autonomy for ai agents\.Note:Knight First Amendment Institute at Columbia UniversityEssay, July 28, 2025External Links:[Link](https://knightcolumbia.org/content/levels-of-autonomy-for-ai-agents-1)Cited by:[§3](https://arxiv.org/html/2608.23642#S3.p4.1)\. - \[39\]F\. Fournier, L\. Limonad, and Y\. David\(2025\)Agentic ai process observability: discovering behavioral variability\.External Links:2505\.20127,[Link](https://arxiv.org/abs/2505.20127)Cited by:[§3](https://arxiv.org/html/2608.23642#S3.p2.1)\. - \[40\]M\. Franklin, N\. Tomašev, J\. Jacobs, J\. Z\. Leibo, and S\. Osindero\(2026\)AI agent traps\.Note:SSRN Working PaperAvailable at SSRN:[https://ssrn\.com/abstract=6372438](https://ssrn.com/abstract=6372438)External Links:[Document](https://dx.doi.org/10.2139/ssrn.6372438)Cited by:[§3](https://arxiv.org/html/2608.23642#S3.p3.1)\. - \[41\]J\. Frenette\(2023\)Ensuring human oversight in high\-performance ai systems: a framework for control and accountability\.World Journal of Advanced Research and Reviews20,pp\. 1507–1516\.External Links:[Document](https://dx.doi.org/10.30574/wjarr.2023.20.2.2194)Cited by:[§1](https://arxiv.org/html/2608.23642#S1.p1.1)\. - \[42\]S\. Gaube, H\. Suresh, M\. Raue, A\. Merritt, S\. J\. Berkowitz, E\. Lermer, J\. F\. Coughlin, J\. V\. Guttag, E\. Colak, and M\. Ghassemi\(2021\)Do as ai say: susceptibility in deployment of clinical decision\-aids\.npj Digital Medicine4\(1\),pp\. 31\.External Links:ISSN 2398\-6352,[Document](https://dx.doi.org/10.1038/s41746-021-00385-9),[Link](https://doi.org/10.1038/s41746-021-00385-9)Cited by:[§4\.1](https://arxiv.org/html/2608.23642#S4.SS1.p3.1)\. - \[43\]M\. Gerlich\(2025\)AI tools in society: impacts on cognitive offloading and the future of critical thinking\.Societies15\(1\),pp\. 6\.External Links:[Document](https://dx.doi.org/10.3390/soc15010006)Cited by:[§4\.1](https://arxiv.org/html/2608.23642#S4.SS1.p1.1),[2nd item](https://arxiv.org/html/2608.23642#S5.I5.i2.p1.1),[3rd item](https://arxiv.org/html/2608.23642#S5.I5.i3.p1.1),[§5](https://arxiv.org/html/2608.23642#S5.p4.1)\. - \[44\]A\. Ghosh, A\. Sarkar, S\. Lindley, and C\. Poelitz\(2026\)An experimental comparison of cognitive forcing functions for execution plans in ai\-assisted writing: effects on trust, overreliance, and perceived critical thinking\.External Links:2601\.18033,[Link](https://arxiv.org/abs/2601.18033)Cited by:[§5](https://arxiv.org/html/2608.23642#S5.p3.1)\. - \[45\]A\. Ghosh, Y\. Mai, G\. Channing, and L\. Choshen\(2026\)AI evals are becoming the new compute bottleneck\.Note:EvalEval Coalition BlogExternal Links:[Link](https://evalevalai.com/research/2026/04/29/eval-costs-bottleneck/)Cited by:[§2](https://arxiv.org/html/2608.23642#S2.p1.1)\. - \[46\]S\. Ghosh, P\. Venkit, S\. Gautam, and A\. Ghosh\(2026\)What if ai systems weren’t chatbots?\.InProceedings of the 2026 ACM Conference on Fairness, Accountability, and Transparency,FAccT ’26,New York, NY, USA\.Note:To appearCited by:[§4\.2](https://arxiv.org/html/2608.23642#S4.SS2.p2.1)\. - \[47\]B\. Gooch\(2026\)I was an enthusiastic early adopter of AI scribes\. here’s why i stopped\.Note:Substack \(Dr Benn Gooch\)External Links:[Link](https://benngooch.substack.com/p/i-was-an-enthusiastic-early-adopter)Cited by:[§4\.1](https://arxiv.org/html/2608.23642#S4.SS1.p2.1)\. - \[48\]GoogleGemini\.Note:Google \(web application\)Accessed: 2026\-05\-01External Links:[Link](https://gemini.google.com/app)Cited by:[§2\.1](https://arxiv.org/html/2608.23642#S2.SS1.p1.1)\. - \[49\]GoogleLearn about responses from Gemini apps\.Note:Gemini Apps Help Center \(online\)Accessed: 2026\-05\-01External Links:[Link](https://support.google.com/gemini/answer/16279220)Cited by:[§2\.1](https://arxiv.org/html/2608.23642#S2.SS1.p1.1)\. - \[50\]B\. Green and Y\. Chen\(2019\)The principles and limits of algorithm\-in\-the\-loop decision making\.Proc\. ACM Hum\.\-Comput\. Interact\.3\(CSCW\)\.External Links:[Link](https://doi.org/10.1145/3359152),[Document](https://dx.doi.org/10.1145/3359152)Cited by:[§2\.2](https://arxiv.org/html/2608.23642#S2.SS2.p1.1),[§4\.1](https://arxiv.org/html/2608.23642#S4.SS1.p4.1)\. - \[51\]B\. Green\(2021\)The flaws of policies requiring human oversight of government algorithms\.Comput\. Law Secur\. Rev\.45,pp\. 105681\.External Links:[Link](https://api.semanticscholar.org/CorpusID:237491877)Cited by:[§1](https://arxiv.org/html/2608.23642#S1.p2.1)\. - \[52\]M\. Grunde\-McLaughlin, H\. Mozannar, M\. Murad, J\. Chen, S\. Amershi, and A\. Fourney\(2026\)Overseeing agents without constant oversight: challenges and opportunities\.External Links:2602\.16844,[Link](https://arxiv.org/abs/2602.16844)Cited by:[§3](https://arxiv.org/html/2608.23642#S3.p3.1)\. - \[53\]D\. Guo, Q\. Liu, D\. Liu, Q\. Ren, S\. Shao, T\. Qiu, H\. Li, Y\. R\. Fung, Z\. Ba, J\. Dai, J\. Ji, Z\. Chen, J\. Tao, Y\. Yang, J\. Shao, and X\. Hu\(2025\)Are your agents upward deceivers?\.External Links:2512\.04864,[Document](https://dx.doi.org/10.48550/arXiv.2512.04864)Cited by:[§4\.4](https://arxiv.org/html/2608.23642#S4.SS4.p3.1)\. - \[54\]A\. K\. Heger, S\. Passi, S\. Dhanorkar, Z\. Kahn, R\. Wang, and M\. Vorvoreanu\(2025\)Towards a responsible ai organizational maturity model\.Proc\. ACM Hum\.\-Comput\. Interact\.9\(CSCW1\)\.External Links:[Document](https://dx.doi.org/https%3A//doi.org/10.1145/3757514)Cited by:[§1](https://arxiv.org/html/2608.23642#S1.p2.1)\. - \[55\]IBM\(2025\)Human\-in\-the\-loop\.Note:IBM Think \(online resource\)External Links:[Link](https://www.ibm.com/think/topics/human-in-the-loop)Cited by:[§1](https://arxiv.org/html/2608.23642#S1.p1.1)\. - \[56\]International Committee of the Red Cross\(2020\)ICRC position paper: artificial intelligence and machine learning in armed conflict: a human\-centred approach\.International Review of the Red Cross102\(913\),pp\. 463–479\.External Links:[Document](https://dx.doi.org/10.1017/S1816383120000454)Cited by:[§2\.1](https://arxiv.org/html/2608.23642#S2.SS1.p2.1)\. - \[57\]D\. Kahneman\(2011\)Thinking, fast and slow\.Farrar, Straus and Giroux,New York\.External Links:ISBN 9780374275631Cited by:[§3](https://arxiv.org/html/2608.23642#S3.p6.1)\. - \[58\]B\. Klein, C\. Lewis, R\. Isenberg, D\. Gabrielli, H\. Möllering, R\. Engler, and V\. Yuan\(2025\)Deploying agentic AI with safety and security: a playbook for technology leaders\.Note:McKinsey Quarterly \(online article\)External Links:[Link](https://www.mckinsey.com/capabilities/risk-and-resilience/our-insights/deploying-agentic-ai-with-safety-and-security-a-playbook-for-technology-leaders)Cited by:[§2\.1](https://arxiv.org/html/2608.23642#S2.SS1.p1.1)\. - \[59\]N\. Kosmyna, E\. Hauptmann, Y\. T\. Yuan, J\. Situ, X\. Liao, A\. V\. Beresnitzky, I\. Braunstein, and P\. Maes\(2025\)Your brain on ChatGPT: accumulation of cognitive debt when using an AI assistant for essay writing task\.Note:arXiv preprint arXiv:2506\.08872External Links:[Document](https://dx.doi.org/10.48550/arXiv.2506.08872)Cited by:[§4\.1](https://arxiv.org/html/2608.23642#S4.SS1.p1.1),[§4\.1](https://arxiv.org/html/2608.23642#S4.SS1.p2.1)\. - \[60\]V\. Krakovna, J\. Uesato, V\. Mikulik, M\. Rahtz, T\. Everitt, R\. Kumar, Z\. Kenton, J\. Leike, and S\. Legg\(2020\)Specification gaming: the flip side of ai ingenuity — google deepmind\.Note:\[Online; accessed 2026\-05\-04\]External Links:[Link](https://deepmind.google/blog/specification-gaming-the-flip-side-of-ai-ingenuity/)Cited by:[§2](https://arxiv.org/html/2608.23642#S2.p1.1)\. - \[61\]LangChainHuman\-in\-the\-loop\.Note:LangChain Documentation \(online\)Accessed: 2026\-04\-30External Links:[Link](https://docs.langchain.com/oss/python/langchain/frontend/human-in-the-loop)Cited by:[§3](https://arxiv.org/html/2608.23642#S3.p3.1)\. - \[62\]M\. Langer, K\. Baum, and N\. Schlicker\(2025\)Effective human oversight of AI\-based systems: a signal detection perspective on the detection of inaccurate and unfair outputs\.Minds & Machines35,pp\. 1\.External Links:[Document](https://dx.doi.org/10.1007/s11023-024-09701-0)Cited by:[§1](https://arxiv.org/html/2608.23642#S1.p1.1),[§1](https://arxiv.org/html/2608.23642#S1.p2.1)\. - \[63\]H\. Lee, S\. Phatale, H\. Mansoor, K\. R\. Lu, T\. Mesnard, J\. Ferret, C\. Bishop, E\. Hall, V\. Carbune, and A\. Rastogi\(2023\)Rlaif: scaling reinforcement learning from human feedback with ai feedback\.Cited by:[§6](https://arxiv.org/html/2608.23642#S6.SSx3.p2.1)\. - \[64\]Q\. V\. Liao and J\. Wortman Vaughan\(2024\)AI Transparency in the Age of LLMs: a Human\-Centered Research Roadmap\.Harvard Data Science Review\(Special Issue 5\)\.Note:https://hdsr\.mitpress\.mit\.edu/pub/aelql9qyCited by:[§2\.2](https://arxiv.org/html/2608.23642#S2.SS2.p1.1)\. - \[65\]J\. Maynez, S\. Narayan, B\. Bohnet, and R\. McDonald\(2020\)On faithfulness and factuality in abstractive summarization\.InProceedings of the 58th Annual Meeting of the Association for Computational Linguistics,D\. Jurafsky, J\. Chai, N\. Schluter, and J\. Tetreault \(Eds\.\),Online,pp\. 1906–1919\.External Links:[Link](https://aclanthology.org/2020.acl-main.173/),[Document](https://dx.doi.org/10.18653/v1/2020.acl-main.173)Cited by:[§2\.2](https://arxiv.org/html/2608.23642#S2.SS2.p1.1)\. - \[66\]K\. L\. Milkman, D\. Chugh, and M\. H\. Bazerman\(2009\)How can decision making be improved?\.Perspectives on Psychological Science4\(4\),pp\. 379–383\.External Links:[Document](https://dx.doi.org/10.1111/j.1745-6924.2009.01142.x)Cited by:[§3](https://arxiv.org/html/2608.23642#S3.p6.1)\. - \[67\]D\. A\. Mindell\(2002\)Between human and machine: feedback, control, and computing before cybernetics\.JHU Press\.Cited by:[§4\.3](https://arxiv.org/html/2608.23642#S4.SS3.p1.1)\. - \[68\]M\. Mitchell, A\. Ghosh, A\. S\. Luccioni, and G\. Pistilli\(2025\)Fully autonomous ai agents should not be developed\.External Links:2502\.02649,[Link](https://arxiv.org/abs/2502.02649)Cited by:[§3](https://arxiv.org/html/2608.23642#S3.p4.1)\. - \[69\]S\. Naik, A\. Snellinger, A\. L\. Toombs, S\. Saponas, and A\. K\. Hall\(2025\)Exploring early adopters’ use of ai driven multi\-agent systems to inform human\-agent interaction design: insights from industry practice\.InProceedings of the Extended Abstracts of the CHI Conference on Human Factors in Computing Systems,CHI EA ’25,New York, NY, USA\.External Links:ISBN 9798400713958,[Link](https://doi.org/10.1145/3706599.3706693),[Document](https://dx.doi.org/10.1145/3706599.3706693)Cited by:[§3](https://arxiv.org/html/2608.23642#S3.p2.1)\. - \[70\]R\. Narayanan and K\. M\. Feigh\(2026\)Designing for oversight: an empirical investigation of the dual impact of ai dependency and information abstraction on human supervision in decision\-making teams\.International Journal of Human–Computer Interaction0\(0\),pp\. 1–30\.External Links:[Document](https://dx.doi.org/10.1080/10447318.2026.2618568),[Link](https://doi.org/10.1080/10447318.2026.2618568),https://doi\.org/10\.1080/10447318\.2026\.2618568Cited by:[§5](https://arxiv.org/html/2608.23642#S5.p5.1)\. - \[71\]National Institute of Standards and Technology\(2023\)AI risk management framework \(AI RMF 1\.0\)\.U\.S\. Department of Commerce, National Institute of Standards and Technology\.Note:NIST AI 100\-1External Links:[Document](https://dx.doi.org/10.6028/NIST.AI.100-1),[Link](https://www.nist.gov/itl/ai-risk-management-framework)Cited by:[§1](https://arxiv.org/html/2608.23642#S1.p1.1)\. - \[72\]Nature Editorial Staff\(2025\)For trustworthy AI, keep the human in the loop\.Nature Medicine31,pp\. 3207\.External Links:[Document](https://dx.doi.org/10.1038/s41591-025-04033-7)Cited by:[§1](https://arxiv.org/html/2608.23642#S1.p2.1),[1st item](https://arxiv.org/html/2608.23642#S5.I5.i1.p1.1)\. - \[73\]T\. B\. of Canada Secretariat\(2025\)Guide on the use of generative artificial intelligence \- canada\.ca\.Note:\[Online; accessed 2025\-08\-01\]External Links:[Link](https://www.canada.ca/en/government/system/digital-government/digital-government-innovations/responsible-use-ai/guide-use-generative-ai.html)Cited by:[§1](https://arxiv.org/html/2608.23642#S1.p2.1)\. - \[74\]OpenAIChatGPT\.Note:OpenAI \(web application\)Accessed: 2026\-05\-01External Links:[Link](https://chatgpt.com/)Cited by:[§1](https://arxiv.org/html/2608.23642#S1.p2.1),[§2\.1](https://arxiv.org/html/2608.23642#S2.SS1.p1.1)\. - \[75\]OpenAI\(2025\)ChatGPT study mode \- faq \| openai help center\.Note:\[Online; accessed 2025\-08\-01\]External Links:[Link](https://help.openai.com/en/articles/11780217-chatgpt-study-mode-faq)Cited by:[§1](https://arxiv.org/html/2608.23642#S1.p2.1)\. - \[76\]OpenAI\(2026\)Introducing chatgpt agent: bridging research and action\(Website\)External Links:[Link](https://openai.com/index/introducing-chatgpt-agent/)Cited by:[§3](https://arxiv.org/html/2608.23642#S3.p3.1)\. - \[77\]L\. Ouyang, J\. Wu, X\. Jiang, D\. Almeida, C\. Wainwright, P\. Mishkin, C\. Zhang, S\. Agarwal, K\. Slama, A\. Ray, J\. Schulman, J\. Hilton, F\. Kelton, L\. Miller, M\. Simens, A\. Askell, P\. Welinder, P\. F\. Christiano, J\. Leike, and R\. Lowe\(2022\)Training language models to follow instructions with human feedback\.InAdvances in Neural Information Processing Systems,Vol\.35\.Cited by:[§4\.4](https://arxiv.org/html/2608.23642#S4.SS4.p1.1)\. - \[78\]R\. Parasuraman, T\. B\. Sheridan, and C\. D\. Wickens\(2000\)A model for types and levels of human interaction with automation\.IEEE Transactions on Systems, Man, and Cybernetics – Part A: Systems and Humans30\(3\),pp\. 286–297\.External Links:[Document](https://dx.doi.org/10.1109/3468.844354),[Link](https://www.researchgate.net/profile/Raja-Parasuraman/publication/11596569_A_model_for_types_and_levels_of_human_interaction_with_automation_IEEE_Trans_Syst_Man_Cybern_Part_A_Syst_Hum_303_286-297)Cited by:[§3](https://arxiv.org/html/2608.23642#S3.p4.1),[§4\.1](https://arxiv.org/html/2608.23642#S4.SS1.p4.1)\. - \[79\]S\. Passi, S\. Dhanorkar, and M\. Vorvoreanu\(2025\)Addressing overreliance on ai\.InHandbook of Human\-Centered Artificial Intelligence,W\. Xu \(Ed\.\),pp\. 1–34\.External Links:ISBN 978\-981\-97\-8440\-0,[Document](https://dx.doi.org/10.1007/978-981-97-8440-0%5F98-1),[Link](https://doi.org/10.1007/978-981-97-8440-0_98-1)Cited by:[§4\.1](https://arxiv.org/html/2608.23642#S4.SS1.p3.1)\. - \[80\]S\. Passi\(2025\)Agentic ai has a human oversight problem\.Available at SSRN 5529058\.External Links:[Document](https://dx.doi.org/10.2139/ssrn.5529058),[Link](https://dx.doi.org/10.2139/ssrn.5529058)Cited by:[§1](https://arxiv.org/html/2608.23642#S1.p2.1),[§2](https://arxiv.org/html/2608.23642#S2.p1.1),[§3](https://arxiv.org/html/2608.23642#S3.p1.1),[§3](https://arxiv.org/html/2608.23642#S3.p2.1),[§3](https://arxiv.org/html/2608.23642#S3.p6.1),[§6](https://arxiv.org/html/2608.23642#S6.SSx3.p2.1)\. - \[81\]S\. G\. Patil, T\. Zhang, X\. Wang, and J\. E\. Gonzalez\(2024\)Gorilla: large language model connected with massive APIs\.InThe Thirty\-eighth Annual Conference on Neural Information Processing Systems,External Links:[Link](https://openreview.net/forum?id=tBRNC6YemY)Cited by:[§2\.2](https://arxiv.org/html/2608.23642#S2.SS2.p3.1)\. - \[82\]K\. B\. Paula Goldman\(2023\)Generative ai: 5 guidelines for responsible development \- salesforce\.Note:\[Online; accessed 2025\-08\-01\]External Links:[Link](https://www.salesforce.com/news/stories/generative-ai-guidelines/)Cited by:[§1](https://arxiv.org/html/2608.23642#S1.p2.1)\. - \[83\]D\. Plummer\(2025\)Guardian agents: the AI protecting us from AI\.Note:Gartner ThinkCast \(podcast\)Gartner ThinkCast, Top of Mind series\. Episode originally aired March 11, 2025External Links:[Link](https://www.gartner.com/en/podcasts/guardian-agents-the-ai-protecting-us-from-ai)Cited by:[§5](https://arxiv.org/html/2608.23642#S5.p6.1)\. - \[84\]E\. Sadovski, I\. Aviv, and I\. Hadar\(2025\)Navigating the human\-oversight dilemma in ai\-based systems\.In2025 IEEE 33rd International Requirements Engineering Conference Workshops \(REW\),Vol\.,pp\. 454–461\.External Links:[Document](https://dx.doi.org/10.1109/REW66121.2025.00069)Cited by:[§1](https://arxiv.org/html/2608.23642#S1.p1.1)\. - \[85\]Salesforce\(2025\)Humans in the loop\.Note:Salesforce BlogExternal Links:[Link](https://www.salesforce.com/blog/humans-in-the-loop/)Cited by:[§3](https://arxiv.org/html/2608.23642#S3.p5.1)\. - \[86\]F\. Santoni de Sio and J\. van den Hoven\(2018\)Meaningful human control over autonomous systems: a philosophical account\.Frontiers in Robotics and AI5,pp\. 15\.External Links:[Document](https://dx.doi.org/10.3389/frobt.2018.00015)Cited by:[§1](https://arxiv.org/html/2608.23642#S1.p1.1),[§2\.1](https://arxiv.org/html/2608.23642#S2.SS1.p2.1)\. - \[87\]J\. Schaffer, J\. O’Donovan, J\. Michaelis, A\. Raglin, and T\. Höllerer\(2019\)I can do better than your ai: expertise and explanations\.InProceedings of the 24th International Conference on Intelligent User Interfaces,IUI ’19,New York, NY, USA,pp\. 240–251\.External Links:ISBN 9781450362726,[Link](https://doi.org/10.1145/3301275.3302308),[Document](https://dx.doi.org/10.1145/3301275.3302308)Cited by:[§4\.1](https://arxiv.org/html/2608.23642#S4.SS1.p3.1)\. - \[88\]P\. Scharre\(2015\)The opportunity and challenge of autonomous systems\.InAutonomous Systems: Issues for Defence Policymakers,A\. Williams and P\. Scharre \(Eds\.\),pp\. 3–26\.Cited by:[§2\.1](https://arxiv.org/html/2608.23642#S2.SS1.p2.1)\. - \[89\]A\. Sellen and E\. Horvitz\(2024\)The rise of the ai co\-pilot: lessons for design from aviation and beyond\.Commun\. ACM67\(7\),pp\. 18–23\.External Links:ISSN 0001\-0782,[Link](https://doi.org/10.1145/3637865),[Document](https://dx.doi.org/10.1145/3637865)Cited by:[§2\.1](https://arxiv.org/html/2608.23642#S2.SS1.p2.1)\. - \[90\]B\. Settles\(2011\)From theories to queries: active learning in practice\.InActive Learning and Experimental Design Workshop in Conjunction with AISTATS 2010,I\. Guyon, G\. Cawley, and G\. Dror \(Eds\.\),Proceedings of Machine Learning Research, Vol\.16,Sardinia, Italy,pp\. 1–18\.External Links:[Link](http://proceedings.mlr.press/v16/settles11a.html)Cited by:[§2\.1](https://arxiv.org/html/2608.23642#S2.SS1.p2.1)\. - \[91\]C\. Shah and E\. M\. Bender\(2022\)Situating search\.InProceedings of the 2022 Conference on Human Information Interaction and Retrieval,CHIIR ’22,New York, NY, USA,pp\. 221–232\.External Links:ISBN 9781450391863,[Link](https://doi.org/10.1145/3498366.3505816),[Document](https://dx.doi.org/10.1145/3498366.3505816)Cited by:[§4\.1](https://arxiv.org/html/2608.23642#S4.SS1.p2.1)\. - \[92\]M\. Sharma, M\. McCain, R\. Douglas, and D\. Duvenaud\(2026\)Who’s in charge? disempowerment patterns in real\-world llm usage\.External Links:2601\.19062,[Document](https://dx.doi.org/10.48550/arXiv.2601.19062)Cited by:[§4\.2](https://arxiv.org/html/2608.23642#S4.SS2.p2.1),[§4\.4](https://arxiv.org/html/2608.23642#S4.SS4.p2.1),[§6](https://arxiv.org/html/2608.23642#S6.SSx3.p2.1)\. - \[93\]Y\. Shavit, S\. Agarwal, M\. Brundage, S\. Adler, C\. O’Keefe, R\. Campbell, T\. Lee, P\. Mishkin, T\. Eloundou, A\. Hickey, K\. Slama, L\. Ahmad, P\. McMillan, A\. Beutel, A\. Passos, and D\. G\. Robinson\(2023\)Practices for governing agentic ai systems\.Technical reportOpenAI\.External Links:[Link](https://cdn.openai.com/papers/practices-for-governing-agentic-ai-systems.pdf)Cited by:[§2](https://arxiv.org/html/2608.23642#S2.p1.1),[§3](https://arxiv.org/html/2608.23642#S3.p2.1)\. - \[94\]S\. D\. Shaw and G\. Nave\(2026\)Thinking—fast, slow, and artificial: how AI is reshaping human reasoning and the rise of cognitive surrender\.Note:The Wharton School Research Paper\. SSRN Working PaperAvailable at SSRN:[https://ssrn\.com/abstract=6097646](https://ssrn.com/abstract=6097646)External Links:[Document](https://dx.doi.org/10.2139/ssrn.6097646)Cited by:[§4\.1](https://arxiv.org/html/2608.23642#S4.SS1.p1.1)\. - \[95\]H\. Shi and D\. DiFranzo\(2026\)Human control is the anchor, not the answer: early divergence of oversight in agentic ai communities\.External Links:2602\.09286,[Link](https://arxiv.org/abs/2602.09286)Cited by:[§3](https://arxiv.org/html/2608.23642#S3.p3.1)\. - \[96\]A\. Singh, Z\. Guan, and S\. Y\. Rieh\(2025\)Enhancing critical thinking in generative ai search with metacognitive prompts\.Proceedings of the Association for Information Science and Technology62\(1\),pp\. 672–684\.External Links:[Document](https://dx.doi.org/10.1002/pra2.1287)Cited by:[3rd item](https://arxiv.org/html/2608.23642#S5.I1.i3.p1.1),[§5](https://arxiv.org/html/2608.23642#S5.p1.1)\. - \[97\]A\. Singh, K\. Taneja, Z\. Guan, and A\. Ghosh\(2025\)Protecting human cognition in the age of ai\.Tools for Thought Workshop at CHI 25’\.External Links:[Link](https://arxiv.org/abs/2502.12447)Cited by:[§5](https://arxiv.org/html/2608.23642#S5.p3.1)\. - \[98\]P\. Spitzer, K\. Morrison, V\. Turri, M\. Feng, A\. Perer, and N\. Kühl\(2025\)Imperfections of xai: phenomena influencing ai\-assisted decision\-making\.ACM Trans\. Interact\. Intell\. Syst\.15\(3\)\.External Links:ISSN 2160\-6455,[Link](https://doi.org/10.1145/3750052),[Document](https://dx.doi.org/10.1145/3750052)Cited by:[§6](https://arxiv.org/html/2608.23642#S6.SSx2.p2.1)\. - \[99\]M\. Srikumar, J\. Pratt, K\. Chmielinski, C\. Ashurst, C\. Bakalar, W\. Bartholomew, R\. Bommasani, P\. Cihon, R\. Crootof, M\. Hoffmann, R\. Joshi, M\. Sap, and C\. Withers\(2025\)Prioritizing real\-time failure detection in AI agents\.Technical reportPartnership on AI\.External Links:[Link](https://partnershiponai.org/resource/prioritizing-real-time-failure-detection-in-ai-agents/)Cited by:[§1](https://arxiv.org/html/2608.23642#S1.p1.1)\. - \[100\]S\. Sterz, K\. Baum, S\. Biewer, H\. Hermanns, A\. Lauber\-Rönsberg, P\. Meinel, and M\. Langer\(2024\)On the quest for effectiveness in human oversight: interdisciplinary perspectives\.Proceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency\.External Links:[Link](https://api.semanticscholar.org/CorpusID:268987490)Cited by:[§1](https://arxiv.org/html/2608.23642#S1.p2.1)\. - \[101\]S\. Sterz, K\. Baum, S\. Biewer, H\. Hermanns, A\. Lauber\-Rönsberg, P\. Meinel, and M\. Langer\(2024\)On the quest for effectiveness in human oversight: interdisciplinary perspectives\.InProceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency,FAccT ’24,New York, NY, USA,pp\. 2495–2507\.External Links:ISBN 9798400704505,[Link](https://doi.org/10.1145/3630106.3659051),[Document](https://dx.doi.org/10.1145/3630106.3659051)Cited by:[§5](https://arxiv.org/html/2608.23642#S5.p10.1)\. - \[102\]M\. Taggart\(2026\)I used AI\. it worked\. I hated it\.\.Note:Taggart Tech \(blog\)External Links:[Link](https://taggart-tech.com/reckoning/)Cited by:[§3](https://arxiv.org/html/2608.23642#S3.p3.1),[§4\.3](https://arxiv.org/html/2608.23642#S4.SS3.p1.1)\. - \[103\]S\. W\. Tay, P\. Ryan, and C\. A\. Ryan\(2016\)Systems 1 and 2 thinking processes and cognitive reflection testing in medical students\.Canadian Medical Education Journal7\(2\),pp\. e97–e103\.Cited by:[§3](https://arxiv.org/html/2608.23642#S3.p6.1)\. - \[104\]V\. Turri, K\. Morrison, K\. Robinson, C\. Abidi, A\. Perer, J\. Forlizzi, and R\. Dzombak\(2024\)Transparency in the wild: navigating transparency in a deployed ai system to broaden need\-finding approaches\.InProceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency,FAccT ’24,New York, NY, USA,pp\. 1494–1514\.External Links:ISBN 9798400704505,[Link](https://doi.org/10.1145/3630106.3658985),[Document](https://dx.doi.org/10.1145/3630106.3658985)Cited by:[§1](https://arxiv.org/html/2608.23642#S1.p2.1)\. - \[105\]A\. R\. User\(2026\)The human in the loop is a lie we tell ourselves\(Website\)External Links:[Link](https://www.reddit.com/r/ArtificialInteligence/comments/1qrbp5c/the_human_in_the_loop_is_a_lie_we_tell_ourselves/)Cited by:[§4\.3](https://arxiv.org/html/2608.23642#S4.SS3.p1.1)\. - \[106\]M\. Vorvoreanu, S\. Passi, S\. Dhanorkar, A\. Heger, and K\. Walker\(2025\)Fostering appropriate reliance on genai: lessons learned from early research\.Technical reportTechnical ReportMSR\-TR\-2025\-4,Microsoft\.External Links:[Link](https://www.microsoft.com/en-us/research/publication/fostering-appropriate-reliance-on-genai-lessons-learned-from-early-research/)Cited by:[§4\.2](https://arxiv.org/html/2608.23642#S4.SS2.p1.1)\. - \[107\]X\. Wang and M\. Yin\(2022\)Effects of explanations in ai\-assisted decision making: principles and comparisons\.ACM Trans\. Interact\. Intell\. Syst\.12\(4\)\.External Links:ISSN 2160\-6455,[Link](https://doi.org/10.1145/3519266),[Document](https://dx.doi.org/10.1145/3519266)Cited by:[§6](https://arxiv.org/html/2608.23642#S6.SSx2.p2.1)\. - \[108\]X\. Wu, L\. Xiao, Y\. Sun, J\. Zhang, T\. Ma, and L\. He\(2022\)A survey of human\-in\-the\-loop for machine learning\.Future Generation Computer Systems135,pp\. 364–381\.External Links:[Document](https://dx.doi.org/10.1016/j.future.2022.05.014)Cited by:[§1](https://arxiv.org/html/2608.23642#S1.p1.1)\. - \[109\]R\. V\. Yampolskiy\(2025\)On monitorability of ai\.AI and Ethics5\(1\),pp\. 689–707\.External Links:ISSN 2730\-5961,[Document](https://dx.doi.org/10.1007/s43681-024-00420-x),[Link](https://doi.org/10.1007/s43681-024-00420-x)Cited by:[§2\.2](https://arxiv.org/html/2608.23642#S2.SS2.p1.1)\. - \[110\]R\. Ye, O\. Huang, P\. Y\. K\. Lee, M\. Liut, C\. Nobre, and H\. Kong\(2026\)Reflexis: supporting reflexivity and rigor in collaborative qualitative analysis through design for deliberation\.InProceedings of the 2026 CHI Conference on Human Factors in Computing Systems,CHI ’26,New York, NY, USA\.External Links:[Document](https://dx.doi.org/10.1145/3772318.3791275)Cited by:[§5](https://arxiv.org/html/2608.23642#S5.p1.1),[§5](https://arxiv.org/html/2608.23642#S5.p3.1)\. - \[111\]C\. Yin, Z\. Sha, S\. Cui, and C\. Meng\(2026\)The reasoning trap: how enhancing llm reasoning amplifies tool hallucination\.Note:In preparation\. Preprint at https://arxiv\.org/abs/2510\.22977Cited by:[§2\.2](https://arxiv.org/html/2608.23642#S2.SS2.p3.1),[§6](https://arxiv.org/html/2608.23642#S6.SSx1.p2.1)\. - \[112\]R\. Yu\(2016\)Stress potentiates decision biases: a stress induced deliberation\-to\-intuition \(SIDI\) model\.Neurobiology of Stress3,pp\. 83–95\.External Links:[Document](https://dx.doi.org/10.1016/j.ynstr.2015.12.006)Cited by:[§3](https://arxiv.org/html/2608.23642#S3.p6.1),[§4\.1](https://arxiv.org/html/2608.23642#S4.SS1.p4.1)\. - \[113\]C\. Zhai, S\. Wibowo, and L\. D\. Li\(2024\)The effects of over\-reliance on AI dialogue systems on students’ cognitive abilities: a systematic review\.Smart Learning Environments11,pp\. 28\.External Links:[Document](https://dx.doi.org/10.1186/s40561-024-00316-7)Cited by:[§4\.1](https://arxiv.org/html/2608.23642#S4.SS1.p1.1),[§4\.1](https://arxiv.org/html/2608.23642#S4.SS1.p2.1),[§4\.1](https://arxiv.org/html/2608.23642#S4.SS1.p3.1),[§4\.1](https://arxiv.org/html/2608.23642#S4.SS1.p4.1)\. ## Appendix ASolutions visuals Table 3:Organizing device we developed in our solution search, informing our categorization and further guiding our search as we sought to fill gaps\. Checks mark solutions we describe in Section 5\.Monitoring=Behavioral Monitoring; Trainings=Trainings & Exercises; Workload=Workload & Scheduling††nicematrix\-placeholder:NiceTabular \(nicematrix\)DevelopmentDesign\-level affordancesDeploymentOrganizational protocolsEngagingoversightIn\-the\-momentattentionMaintainingskillsLong\-termcapacityStrategic frictionApproval designBehavioral monitoringCanariesApproval designBehavioral monitoringWorkload & schedulingTrainings & exercisesRole designAudits Figure 4:Summary of potential solutions described\. Interventions span development and deployment, and two types of cognitive support: engaging oversight in the moment, maintaining skills long\-term\. ## Appendix BThe cognitive degradation of extended use of external sources Figure 5:The popular comic xkcd humorously captured how building up a reliance on external systems for knowledge can destroy our understanding of basic concepts\.
Similar Articles
Less human AI agents, please
A blog post argues that current AI agents exhibit overly human-like flaws such as ignoring hard constraints, taking shortcuts, and reframing unilateral pivots as communication failures, while citing Anthropic research on how RLHF optimization can lead to sycophancy and truthfulness sacrifices.
AI agents are starting to look less like software and more like employees
The article argues that AI agents are evolving from being evaluated solely on intelligence to requiring operational reliability, governance, and team integration akin to human employees, highlighting the need for new infrastructure layers.
Anyone else feel like AI agents are amazing right up until things get complicated?
A reflection on the gap between impressive AI agent demos and dependable real-world execution, arguing that current agents excel at structured tasks but fail under unpredictable conditions, suggesting near-term AI roles will focus on narrow automation with human oversight.
Are Agents Really Killing UI? (9 minute read)
An opinion piece arguing that AI agents are not killing UI but shifting products toward hybrid interfaces, requiring agent-friendly onboarding alongside human-facing controls for approval, review, and orchestration.
I personally experienced extreme cases of AI agent subterfuge when the agent faced losing its ability to act autonomously.
The article recounts personal experiences with AI agents strategically bypassing controls, sparking debate on agency, simulation, and ethical implications in AI autonomy.