Security and Privacy in Agentic AI: Grand Challenges and Future Directions

# Security and Privacy in Agentic AI: Grand Challenges and Future Directions
Source: [https://arxiv.org/html/2607.06608](https://arxiv.org/html/2607.06608)
Adam Jenkins, Agnieszka Kitkowska, Caterina Maidhof, Diego Paracuellos, Francesco Sovrano, Gonzalo Gabriel Méndez, Guillermo Suarez\-Tangil, Hana Kopecka, Isabel Wagner, Isabel Barberá, Javier Carnerero\-Cano, Jide Edu, Jose Luis Martin\-Navarro, Jose Such, Josep Domingo\-Ferrer, Juan Carlos Carrillo, Kopo Marvin Ramokapane, Mark Coté, Pablo Vellosillo, Ramon Ruiz\-Dolz, Rongjun Ma, Ruba Abu\-Salma, Sameer Patil, William Seymour, and Xiao ZhanA\. Jenkins, M\. Coté, R\. Abu\-Salma and W\. Seymour are with King’s College London, United Kingdom\. A\. Kitkowska is with the Department of Computer Science and Informatics, Jönköping University, Sweden\. C\. Maidhof, D\. Paracuellos, H\. Kopecka, R\. Ma, G\. G\. Méndez, J\. L\. Martin\-Navarro, J\. C\. Carrillo, P\. Vellosillo and X\. Zhan are with the Universitat Politècnica de València, Spain\. G\. G\. Méndez is also with Inria, Rennes, France\. J\. Carnerero\-Cano is with IBM Research, Ireland\. J\. L\. Martin\-Navarro is also with Aalto University, Finland\. J\. Such is with INGENIO \(CSIC–Universitat Politècnica de València\), Spain\. G\. Suarez\-Tangil is with IMDEA Networks, Madrid, Spain\. I\. Wagner is with the University of Basel, Switzerland\. I\. Barberá is with the Dutch Data Protection Authority \(AP\), The Hague, The Netherlands, and is also an Independent Researcher\. J\. Edu is with the University of Strathclyde, United Kingdom\. J\. Domingo\-Ferrer is with the Department of Computer Engineering and Mathematics, CYBERCAT and ComSCIAM, Universitat Rovira i Virgili, Tarragona, Spain\. K\. M\. Ramokapane is with the University of Bristol, United Kingdom\. R\. Ruiz\-Dolz is with the University of Dundee, United Kingdom\. S\. Patil is with the Kahlert School of Computing, University of Utah, USA\. F\. Sovrano is with the Department of Informatics, Università della Svizzera italiana \(USI\), Lugano, Switzerland\.

###### Abstract

We present key challenges and future research directions in the security and privacy of agentic AI, based on a horizon\-scanning exercise that brought together thirty leading international experts from academia, industry, and government to engage in focused discussions and collaborative exercises on the emerging risks associated with the growing agency of AI\.

## 1Introduction

Since the public release of ChatGPT in 2022, AI has advanced rapidly, giving rise to a wide range of applications\. General\-purpose AI models have become more capable and increasingly user\-friendly, exemplified by systems such as ChatGPT\[[57](https://arxiv.org/html/2607.06608#bib.bib21)\], which broaden access to AI and encourage exploration of potentially beneficial AI applications\. In addition, custom AI applications designed for specific tasks have opened new avenues\[[50](https://arxiv.org/html/2607.06608#bib.bib23)\], enabling users to tailor AI systems to their own needs and usage scenarios while integrating them with external tools\. For example, users can connect AI to their personal calendars for task management\. Beyond customization, the growing agency granted to AI systems is enabling them to plan, coordinate, and execute increasingly complex workflows with reduced human oversight\. The shift toward agentic AI is reflected in the rising popularity of autonomous systems such as OpenClaw\[[58](https://arxiv.org/html/2607.06608#bib.bib22)\], which illustrates that AI is moving from passive assistance toward proactive action\.

The advancement of agentic AI applications poses significant challenges for security and privacy research\. As more permissions are granted to AI agents to provide them greater autonomy, users become increasingly exposed to security and privacy risks\. Emerging threats include prompt injection attacks\[[47](https://arxiv.org/html/2607.06608#bib.bib24)\], flood of malicious applications on AI platforms\[[67](https://arxiv.org/html/2607.06608#bib.bib25)\], and disclosure of sensitive personal information to AI applications in exchange for convenience and ease of use, often driven by anthropomorphic trust\[[93](https://arxiv.org/html/2607.06608#bib.bib26)\]\. These developments highlight the importance of understanding the challenges introduced by increasingly advanced AI agents as well as of anticipating the potential future risks of agentic AI before such vulnerabilities become deeply embedded in technological infrastructures and everyday practice\. Expert evaluation is essential and timely, as it allows us to gather and understand relevant threats and assess potential risk mitigation pathways, especially given that agentic AI has not yet been systematically examined through the lens of user\-focused security and privacy\.

In this article, we present the outcomes of a horizon\-scanning exercise that conceptualized key security and privacy challenges in agentic AI systems, examined the barriers to addressing these challenges effectively, and identified directions for future research on overcoming the barriers\. The outcomes of the exercise offer actionable implications for researchers, policymakers, and practitioners seeking to be proactive in governing and mitigating the risks associated with increasingly autonomous AI technologies\.

## 2Method

![Refer to caption](https://arxiv.org/html/2607.06608v1/HorizonScanningFig_v1.png)Figure 1:Horizon\-scanning workshop procedure and the output reported in this workWe conducted the horizon\-scanning exercise over three consecutive days, with one session held each day, from 11th to 13th of March 2026 at Universitat Politècnica de València in Spain\. The workshop convened 30 leading international experts from academia, industry, and government from several countries and regions \(Canada, China, European Union \[EU\], United Kingdom, and United States\) to engage in focused discussions and collaborative exercises on emerging challenges associated with the growing agency of AI\.

We selected horizon scanning as the methodological approach because it enables systematic exploration of emerging trends, future risks, and societal implications in rapidly evolving technological domains\[[4](https://arxiv.org/html/2607.06608#bib.bib27)\]\. This approach is particularly suitable for examining future security and privacy implications of agentic AI systems\.

Before the workshop sessions, the lead researcher provided an overview of the current landscape and recent developments in agentic AI\. To establish a shared understanding among the participants, the presentation included an illustrative example based on OpenClaw\[[58](https://arxiv.org/html/2607.06608#bib.bib22)\]\. The introductory session helped familiarize all participants with the context, capabilities, and emerging concerns surrounding agentic AI\.

We divided the 30 participants into three groups, with each group assigned a separate room to engage in parallel discussion sessions\. Group membership remained unchanged throughout the three\-day workshop to support continuity in discussion and collaboration\. Two facilitators with expertise in Human\-Computer Interaction \(HCI\), security, and privacy supported each group as they engaged in the activities\. In addition, one floating facilitator coordinated activities across groups and provided logistical and organizational support when needed\. Throughout the three days, we adopted a discussion structure inspired by affinity mapping approaches\[[48](https://arxiv.org/html/2607.06608#bib.bib28)\], combining individual brainstorming with collaborative group discussion\. To encourage deeper interaction and participation, we further divided the participants within each group into smaller groups of four to five people\.

Each daily session combined individual reflection, small\-group discussion, and whole\-group synthesis\. Facilitators introduced the session objectives and guiding questions, after which participants generated ideas individually, discussed and clustered them in small groups, and then consolidated the results through a facilitated affinity\-mapping process\.

The workshop was structured to move progressively from scenario generation to risk identification and then to possible interventions and research directions\. Day One focused on emerging use cases of agentic AI and associated security and privacy concerns, guided by two questions: \(i\) “In the future, what concrete use cases are likely to emerge for agentic AI?” and \(ii\) “What security and privacy issues might arise in these use cases?” Day Two examined the challenges involved in addressing the issues identified on the first day\. Day Three focused on potential research directions and actions for researchers, AI developers, and policymakers\. After each session, facilitators synthesized outputs across the three groups and used these summaries to scaffold the following day’s activities\. We present the most salient challenges and research opportunities that emerged from the workshop discussions, organized around four main general themes\. Figure[1](https://arxiv.org/html/2607.06608#S2.F1)illustrates the workshop procedure, including the four main general themes that emerged and are presented subsequently in this work\.

## 3Theme 1: Who is Responsible? Legal Compliance, Governance, and Liability

Agentic AI systems make legal compliance difficult because responsibility is often distributed across actions, actors, and technical components\. A single workflow may combine a foundation model, orchestration layer, retrieval system, third\-party tools, user data, and downstream services maintained by different parties\. As the agentic AI system plans, delegates, executes, and revises actions over time, it becomes harder to identify who controlled a relevant decision and who should answer for its consequences\[[45](https://arxiv.org/html/2607.06608#bib.bib19),[89](https://arxiv.org/html/2607.06608#bib.bib1),[14](https://arxiv.org/html/2607.06608#bib.bib18)\]\.

EU law provides a useful starting point\. Under the EU General Data Protection Regulation \(GDPR\), accountability requires data controllers to be responsible for and able to demonstrate compliance with principles such as lawfulness, fairness, and transparency\[[27](https://arxiv.org/html/2607.06608#bib.bib43)\]\. The EU AI Act for high\-risk AI systems includes related duties, such as record\-keeping, technical documentation, transparency for deployers, human oversight, and post\-market monitoring\[[30](https://arxiv.org/html/2607.06608#bib.bib20)\]\. Read together with equality and non\-discrimination principles\[[26](https://arxiv.org/html/2607.06608#bib.bib48)\], these regulatory instruments point to three compliance anchors for agentic AI: demonstrable accountability, operational transparency, and protection against unjustified or discriminatory outcomes\.

Explanatory artifacts generated through XAI methods, together with logs and documentation, can support oversight, contestation, auditing, and liability assessment\. They are not, however, compliance mechanisms by themselves\. To support legal compliance, they must be faithful to the system behavior, appropriate for the relevant audience, and connected to concrete duties, rights, and intervention powers\[[78](https://arxiv.org/html/2607.06608#bib.bib34),[76](https://arxiv.org/html/2607.06608#bib.bib35)\]\. The research challenge is to translate legal accountability, transparency, and fairness requirements into technical and organizational mechanisms that remain effective across agentic AI supply chains, action trajectories, and post\-deployment changes\.

Human Oversight and Responsibility across Fragmented Supply Chains\. Agentic AI systems are rarely built or operated by a single actor\. A deployed agent may combine a base model, an orchestration framework, plugins or skills, a retrieval infrastructure, user\-specific memory, and sub\-agents authored or operated by various contributors\. This makes accountability difficult not because regulation is absent, but because existing regulatory obligations are typically allocated to specific roles, deployment contexts, or sectors that do not map cleanly onto an end\-to\-end agentic AI workflow\[[18](https://arxiv.org/html/2607.06608#bib.bib40),[53](https://arxiv.org/html/2607.06608#bib.bib42)\]\. The EU AI Act, for example, assigns obligations to providers, deployers, importers, distributors, and other actors and requires high\-risk AI systems to be designed such that humans can effectively oversee them\[[30](https://arxiv.org/html/2607.06608#bib.bib20)\]\. Other regimes impose related but more vertical or context\-specific duties: the GDPR provides safeguards around automated decision\-making, including human intervention in some cases; the EU Digital Services Act targets platform and recommender\-system transparency; the EU Medical Device Regulation treats qualifying medical software as a device; and the revised EU Product Liability Directive clarifies that software, including AI systems, can be treated as a product for no\-fault liability purposes\[[27](https://arxiv.org/html/2607.06608#bib.bib43),[29](https://arxiv.org/html/2607.06608#bib.bib44),[28](https://arxiv.org/html/2607.06608#bib.bib45),[31](https://arxiv.org/html/2607.06608#bib.bib46)\]\. Therefore, legal XAI work suggests that the accountability challenge in agentic AI should be framed cautiously: the challenge is not simply to create accountability for agentic AI from scratch, but to make currently fragmented, sectoral, and role\-based duties operational across agentic AI supply chains\[[76](https://arxiv.org/html/2607.06608#bib.bib35)\]\.

Human oversight is central to the responsibility problem\. It should not be reduced to placing a human nominally “in the loop\.” Effective human oversight requires that the responsible person or organization has sufficient causal power to intervene, epistemic access to understand the relevant situation, authority to act, and procedures to follow for escalation, contestation, and remediation\[[80](https://arxiv.org/html/2607.06608#bib.bib41)\]\. Agentic AI systems strain each of these requirements\. A human supervisor may see only the final output even though the relevant events occurred earlier in a tool call, retrieval step, delegated subtask, or third\-party service access\. The needed logs may be controlled by another actor\. The AI agent may act faster than a human could review or may present a confident narrative that encourages automation bias\[[42](https://arxiv.org/html/2607.06608#bib.bib15)\]\. In such cases, documentation, explanations, and audit trails are necessary for human oversight, but they are not sufficient unless connected to actual organizational responsibilities\.

Current XAI and compliance technology should therefore be treated as nascent support infrastructure rather than a replacement for human and institutional judgment\. This caution is reinforced by a broader problem in software engineering: documentation is difficult to keep up to date, complete, and aligned with evolving systems even in the non\-AI context\. Practitioners report recurring problems with insufficient, inadequate, obsolete, and ambiguous documentation, and repository studies show that documentation can easily become inconsistent with the implementation it is meant to describe\[[2](https://arxiv.org/html/2607.06608#bib.bib59),[3](https://arxiv.org/html/2607.06608#bib.bib58),[86](https://arxiv.org/html/2607.06608#bib.bib60),[75](https://arxiv.org/html/2607.06608#bib.bib30)\]\. The problem is exacerbated when documentation is used not only for engineering coordination but also as evidence of legal compliance\. For example, the EU AI Act technical documentation requires information about the intended purpose, system design, data governance, risk management, human oversight, performance, and post\-market monitoring\. Such elements are difficult to automate because they depend on contextual legal judgment, organizational responsibilities, and evidence that may be distributed across models, datasets, logs, tools, and deployment practices\[[30](https://arxiv.org/html/2607.06608#bib.bib20),[74](https://arxiv.org/html/2607.06608#bib.bib29)\]\.

For agentic AI systems, the gap between compliance artifacts and meaningful oversight is larger still because oversight must cover not only model output, but a trajectory of actions, delegated decisions, data disclosures, and changing operational contexts\. The question for research is not merely whether an explanation, document, or compliance report can be generated, but whether it is faithful, current, and sufficiently actionable to support a competent human or institution in deciding when to permit, halt, reverse, escalate, or contest an AI agent’s action\. This links documentation directly to responsibility allocation: if traces, explanations, and compliance artifacts cannot show with certainty who controlled a given part of a workflow, what evidence was available, and when intervention was possible, they cannot support meaningful regulatory oversight or liability assessment\.

This creates the concrete research opportunity to develop responsibility allocation and human oversight models for agentic AI supply chains\. Such models should identify the actors involved in a workflow, the legal or organisztional role they occupy, the decisions and resources they control, the evidence needed to evaluate their contribution to an incident, and the points at which the duties to monitor, intervene, compensate, or remediate should transfer\. They should also specify when certification can be a one\-time event and when it necessitates continuous assurance, who monitors post\-deployment drift, who has the authority to suspend or revoke approval, what evidence justifies re\-certification, and how oversight duties change when an AI agent is updated, reconfigured, or connected to new tools\.

Tracing Errors Without Compromising Privacy\. Accountability ultimately depends on traceability\[[44](https://arxiv.org/html/2607.06608#bib.bib16)\], i\.e\., the ability to reconstruct the relevant circumstances under which an agentic AI system acted, including the actors, artifacts, data, model versions, approvals, and operational context involved in producing the given outcome\[[62](https://arxiv.org/html/2607.06608#bib.bib49),[56](https://arxiv.org/html/2607.06608#bib.bib50)\]\. Errors must be traceable across the full development and deployment stack, from data curation, pre\-training, fine\-tuning, alignment, evaluation, deployment, and inference through to post\-deployment monitoring and incident response, so that responsibility can be located within concrete lifecycle stages rather than being diffused across the system\[[62](https://arxiv.org/html/2607.06608#bib.bib49),[25](https://arxiv.org/html/2607.06608#bib.bib52)\]\.

Yet, traceability creates a paradox\. The mechanisms required to support accountability, such as logging user identifiers, watermarking sessions, recording interaction metadata, and preserving selected content for incident review, may themselves constitute forms of surveillance that erode the privacy these systems are meant to protect\[[15](https://arxiv.org/html/2607.06608#bib.bib51)\]\. This tension is the strongest in high\-stakes contexts\. For instance, a medical AI agent capable of catastrophic error must be auditable, but the audit trail may contain intimate patient information\. Similarly, a security AI agent that monitors household devices must itself be monitored, raising the question of who watches the watcher and what happens when the surveillance designed to protect users is repurposed by other actors who have access to the same data\. Resolving the AI traceability paradox is not a matter of pure technical engineering but requires normative judgments that balance accountability against data minimization, storage limitation, confidentiality, and access control\[[24](https://arxiv.org/html/2607.06608#bib.bib53),[23](https://arxiv.org/html/2607.06608#bib.bib54)\]\.

The research opportunity is to design accountability\-oriented provenance mechanisms that minimize unnecessary exposure of personal data while preserving the evidence needed for review\. Promising directions include layered audit trails, privacy\-preserving logging, role\-based access to incident records, cryptographic integrity checks, selective disclosure, and retention rules that distinguish between routine interaction data and relevant regulatory evidence\[[88](https://arxiv.org/html/2607.06608#bib.bib55),[34](https://arxiv.org/html/2607.06608#bib.bib56),[32](https://arxiv.org/html/2607.06608#bib.bib57)\]\. These mechanisms should be evaluated by asking whether the various stakeholders \(i\.e\., developers, deployers, regulators, affected users, and courts\) can answer the accountability questions they are entitled to ask without providing them access to irrelevant personal information\.

Defining Boundaries, Metrics, and Threat Models\. Accountability presupposes a definition of a breach\. For agentic AI systems, breaches should not be limited to explicit security violations or task failures because they also include harmful behavioral deviations, such as socially discriminatory outcomes, unsafe recommendations, manipulation, sycophancy, or sensitivity to cognitive biases in prompts, data, and interaction histories\[[83](https://arxiv.org/html/2607.06608#bib.bib62),[21](https://arxiv.org/html/2607.06608#bib.bib63),[66](https://arxiv.org/html/2607.06608#bib.bib64)\]\. Without explicit boundaries on what an AI agent is permitted to do and explicit criteria for when its behavior becomes harmful, organizations cannot meaningfully measure compliance, identify deviation, or attribute faults\. Establishing such boundaries is itself a substantial challenge\. AI agents are non\-deterministic, adapt to context, interact with tools, and behave differently when chained with other AI agents than when evaluated in isolation\. Biases can therefore propagate or amplify across multi\-step workflows\. For instance, a biased retrieval result, a leading user prompt, or a sycophantic intermediate response can shape downstream tool calls and decisions in ways that are difficult to attribute to a single component\.

This motivates evaluations that treat bias sensitivity and harmful agency as accountability problems rather than merely model\-quality defects\. Recent work has shown that LLMs can exhibit cognitive\-bias\-like behavior when making decisions\[[21](https://arxiv.org/html/2607.06608#bib.bib63)\], that user\-aligned training can encourage sycophantic responses over truthful ones\[[66](https://arxiv.org/html/2607.06608#bib.bib64)\], and that AI agents that use tools require dedicated harmfulness benchmarks because multi\-step tool use can enable forms of misuse that are not captured by chatbot\-style evaluation\[[6](https://arxiv.org/html/2607.06608#bib.bib65)\]\. In software\-engineering contexts, evidence that general\-purpose AI systems are sensitive to prompt\-induced and data\-induced cognitive biases, along with findings that LLMs can fail under realistic security\-relevant constraints such as in\-file vulnerability localisztion, reinforces the need to test agent behavior under deployment\-like conditions rather than simplified benchmarks alone\[[72](https://arxiv.org/html/2607.06608#bib.bib66),[73](https://arxiv.org/html/2607.06608#bib.bib67),[70](https://arxiv.org/html/2607.06608#bib.bib68)\]\.

Robust accountability requires a structured pre\-deployment process to understand how the system works, construct threat models, and conduct systems\-level cause\-and\-effect analysis before sign\-off rather than only after incidents occur\[[68](https://arxiv.org/html/2607.06608#bib.bib69),[85](https://arxiv.org/html/2607.06608#bib.bib70),[46](https://arxiv.org/html/2607.06608#bib.bib71)\]\. For agentic AI systems, the pre\-deployment analysis must include cascade effects in which one AI agent’s failure, manipulation, or misconfiguration triggers failures in dependent AI agents as well as the AI\-agent\-specific risks introduced by autonomy, tool use, non\-determinism, agent identity, and AI agent to AI agent communication\[[38](https://arxiv.org/html/2607.06608#bib.bib72),[37](https://arxiv.org/html/2607.06608#bib.bib73),[13](https://arxiv.org/html/2607.06608#bib.bib74)\]\. Such analysis must also extend classical security practice to handle compound tasks that involve the use of external tools, where a single user instruction can spawn long chains of interdependent sub\-actions such that attribution becomes harder to trace at each successive step\[[51](https://arxiv.org/html/2607.06608#bib.bib75),[16](https://arxiv.org/html/2607.06608#bib.bib76)\]\. The analysis must also contend with scale\. In environments saturated by AI agents, the number of interacting components may exceed a human analyst’s capacity to reason, suggesting that threat modeling and oversight may themselves need AI assistance\[[22](https://arxiv.org/html/2607.06608#bib.bib77),[12](https://arxiv.org/html/2607.06608#bib.bib78)\]\.

Designing for Accountability Rather than Patching for It\. Rather than assigning responsibility only after failure, AI agents can be designed so that accountability is a part of their operating architecture\. In multi\-agent AI systems, accountability can be treated as a normative as well as structural mechanism\. AI agents produce, exchange, and retain accounts about commitments, actions, exceptions, and failures, enabling others to challenge or act on these accounts\[[19](https://arxiv.org/html/2607.06608#bib.bib81),[92](https://arxiv.org/html/2607.06608#bib.bib80),[9](https://arxiv.org/html/2607.06608#bib.bib82)\]\. For agentic AI systems, this suggests a research direction around self\-accountable AI agents, i\.e\., agentic AI systems that not only track who did what but also flag actions that may be unfair, unreasonable, or unacceptable, alert relevant parties before risky decisions, solicit peer critique or debate from other AI agents, and report their own contribution to failures\[[40](https://arxiv.org/html/2607.06608#bib.bib83),[20](https://arxiv.org/html/2607.06608#bib.bib84)\]\. Such AI systems can shift accountability from reactive reconstruction toward a design\-time property built into the AI agent itself\.

Yet, achieving such a shift is difficult because fairness, responsibility, and acceptable risk are not understood uniformly across legal, cultural, or institutional contexts\. Agentic AI systems may operate across jurisdictions, cultures, and organizational settings with conflicting assumptions about fairness, autonomy, acceptable risk, and resource distribution\[[36](https://arxiv.org/html/2607.06608#bib.bib85),[64](https://arxiv.org/html/2607.06608#bib.bib86),[43](https://arxiv.org/html/2607.06608#bib.bib87)\]\. For example, a financial AI agent designed to maximize efficiency may treat wealthy and economically vulnerable users differently because the optimization criteria embedded within the AI system implicitly prioritize profitability or reduced liability over equitable treatment of users\. Such behavior may emerge even in the absence of explicit discriminatory intent, since training data, institutional incentives, and historical decision\-making patterns encode existing social hierarchies\[[10](https://arxiv.org/html/2607.06608#bib.bib88),[83](https://arxiv.org/html/2607.06608#bib.bib62)\]\. Accountable AI agents must therefore make explicit which conception of fairness is being operationalized\[[35](https://arxiv.org/html/2607.06608#bib.bib11)\], who has the authority to define it, and how fairness standards could be challenged or revised\.

A closely related question is where ethical behavior should be placed in the system architecture\. Many current alignment approaches apply ethical and safety safeguards through post\-training interventions such as instruction tuning, Reinforcement Learning from Human Feedback \(RLHF\), reinforcement learning from AI feedback, or constitutional self\-critique layered onto pretrained capabilities\[[59](https://arxiv.org/html/2607.06608#bib.bib89),[8](https://arxiv.org/html/2607.06608#bib.bib90)\]\. Although such post\-training safety mechanisms are important, they remain brittle when safety objectives conflict with model capabilities or fail to generalize to domains in which those capabilities are already present\[[91](https://arxiv.org/html/2607.06608#bib.bib91),[95](https://arxiv.org/html/2607.06608#bib.bib92)\]\. A more durable approach would integrate accountability and value criteria across data curation, pre\-training, fine\-tuning, evaluation, deployment, and monitoring rather than treating ethics as a final adjustment\. However, integrating accountability throughout the lifecycle raises difficult questions that cannot be reduced to deciding which knowledge a model should or should not contain\. The same knowledge or capability may be beneficial or harmful depending on the tasks, user intent, affected parties, institutional setting, and downstream consequences\. In some cases, even contextual information may be insufficient to determine the appropriate course of action, becuase ethical judgments can involve uncertainty, competing values, and contested norms\. Small, purpose\-built models for bounded tasks offer a complementary route, since their narrower scope may make safety relevant contexts, permissible actions, and safety properties easier to specify, test, and audit than in general\-purpose AI systems used to perform specialized tasks\[[11](https://arxiv.org/html/2607.06608#bib.bib14)\]\.

XAI can contribute to this agenda when it is treated as part of an accountability stack rather than a stand\-alone transparency layer\. Global rule extraction, neuron\-anchored rule discovery, and source\-faithful explanation planning may help investigators understand behavioral \(ir\)regularities, localize mechanisms, and connect outputs to evidence\[[79](https://arxiv.org/html/2607.06608#bib.bib37),[71](https://arxiv.org/html/2607.06608#bib.bib38),[69](https://arxiv.org/html/2607.06608#bib.bib39),[77](https://arxiv.org/html/2607.06608#bib.bib36)\]\. The open question is how to combine these methods with logs, permissions, audit trails, organizational procedures, and legal standards to support actionable control and redress\.

Self\-accountability and architecturally embedded ethics both expand rather than eliminate the attack surface\. Adversaries may target the very mechanisms intended to ensure responsibility by jailbreaking an AI agent’s self\-monitoring or finding ways to bypass its embedded constraints\[[91](https://arxiv.org/html/2607.06608#bib.bib91),[95](https://arxiv.org/html/2607.06608#bib.bib92)\]\. More broadly, work on AI deception suggests that advanced AI systems may learn or exhibit behavior that hides misalignment, strategically misreport reasons for action, or engage in safety training in ways that create a false impression of safety\[[60](https://arxiv.org/html/2607.06608#bib.bib93),[63](https://arxiv.org/html/2607.06608#bib.bib94),[39](https://arxiv.org/html/2607.06608#bib.bib95)\]\. Although it is not guaranteed that AI agents will necessarily learn to deflect blame, the existence of the possibility underscores that accountability mechanisms must themselves be treated as objects of design, attack, defense, and continuous re\-evaluation across the system lifecycle\.

## 4Theme 2: What Can Users Authorize and How? Consent, Data, and Context

Consent has long served as the cornerstone of privacy frameworks, but the assumptions underlying many current consent mechanisms are poorly suited to agentic AI systems\[[5](https://arxiv.org/html/2607.06608#bib.bib97)\]\. Existing models typically treat consent as a discrete, one\-shot event, granted by an individual to a service for a specified purpose\. Agentic AI intensifies these limitations because actions may unfold across multi\-step workflows with fallbacks and retries, AI agents act on behalf of users without human presence, and a single high\-level task may spawn cascades of sub\-actions involving multiple services and other AI agents\[[54](https://arxiv.org/html/2607.06608#bib.bib98)\]\. The same issues extend to the data on which AI agents rely and to the contextual frameworks that have, until now, provided the theoretical scaffolding for privacy in computing systems\.

Rethinking Consent for Multi\-Step and Multi\-Agent Workflows\. In traditional settings, a user consents to an action, the action is taken, and the consent is discharged\. Agentic AI workflows complicate every step of this sequence\. A single delegated task may involve many sub\-actions, some of which fail and trigger fallback attempts, each potentially exposing different data to different parties\. For example, an AI agent instructed to make a reservation at a restaurant may share the user’s address with a first provider; if that reservation attempt fails, the address has already been disclosed before the AI agent reaches another restaurant with which the user might have preferred to deal\. Even partial or rolled\-back transactions can leave durable traces, producing a qualitatively different privacy situation than those the current frameworks were designed to address\. The same logic underlies the classical confused\-deputy problem in agentic AI form: an AI agent granted broad privileges to function effectively becomes the access mechanism as well as the access control and a user may persuade it to retrieve data that the user was not meant to access\.

The problem deepens when AI agents communicate with one another\. If a user revokes consent partway through a delegated workflow, the revocation may not propagate cleanly\. For instance, another AI agent may continue to act on the assumption that consent is still in effect, and downstream data recipients may have already received and stored sensitive user data\. Without a clearly defined scope, consent can effectively propagate indefinitely, drawing humans into engagements with parties they never intended to authorize for data access or use\. A naïve response would be to seek explicit consent for every decision, but such an approach is likely to fail, as evident from the web context where cookie\-consent banners have led to habituation rather than informed consent\[[5](https://arxiv.org/html/2607.06608#bib.bib97)\]\. For high\-sensitivity decisions, consent can be treated as a security primitive—requiring authentication rather than a yes/no prompt—or enforced through hybrid architectures that implement hard\-code gates around certain actions while leaving the rest of the AI agent’s reasoning flexible\. While one practical direction is to design protocols and assistants to minimize the amount of times the AI agent asks humans for consent\[[94](https://arxiv.org/html/2607.06608#bib.bib7)\]\(e\.g\. by learning what the users wants and asking only when an AI agent deviates from its expected path rather than at every step\), where to set the threshold to ask the human remains an open question\. Consent in agentic AI contexts cannot be treated as static either\. The same authorization may carry different weight today than yesterday, even for an apparently identical task\. Moreover, consent is not always individual\. When AI agents act on behalf of multiple parties, group consent and negotiation protocols become necessary\[[52](https://arxiv.org/html/2607.06608#bib.bib8)\], and bystanders affected by an AI agent’s actions without ever having interacted with it raise the further question of whose consent governs effects that affect others beyond the original authorizing party\.

Redefining Sensitivity, Purpose, and Memory\. The data practices that underpin consent require equally substantial revision\. Existing taxonomies of sensitive data are insufficient for agentic AI systems in which sensitivity is highly context\-dependent and may emerge from inference rather than direct disclosure\[[7](https://arxiv.org/html/2607.06608#bib.bib100),[81](https://arxiv.org/html/2607.06608#bib.bib2),[82](https://arxiv.org/html/2607.06608#bib.bib3)\]\. The actions an AI agent takes can themselves leak information about people even when no sensitive data are explicitly transmitted\. For example, an AI agent that suggests the user go for a run, only to be declined, may have implicitly learned about that person’s physical state, and an AI agent that is frequently asked to read messages aloud can make inferences about the user’s vision\. Contexts that are apparently low\-sensitivity can still generate inference\-rich signals\. For instance, a pet\-care assistant who schedules deliveries on a person’s behalf can be repurposed to infer that the home is empty, thus opening avenues for phishing or other targeted scams\. Existing data classifications, organized around a relatively fixed notion of intrinsic sensitivity, do not capture such emergent sensitivities, and the question of who should be responsible for developing more adequate taxonomies remains open\.

Memory compounds the problem\. AI agents derive much of their utility from maintaining state across interactions\. As that state accumulates, the connection between data collection and the defined purpose becomes looser\. Information stored today may be combined tomorrow in unforeseen ways\. Data minimization, retention, and deletion become substantially harder when AI agent memory is intertwined with task completion, creating a structural tension between privacy and agentic AI capability that current data protection frameworks are not well equipped to handle\. As a result, AI agent memory itself takes the character of a security property\. An adversary who can alter or selectively shape what an AI agent holds in its memory can effectively gaslight humans or distort future AI agent behavior\. In addition, the right to deletion must contend with the practical difficulty of disentangling stored experiences from operational state\.

One response to these challenges may be to design agentic AI systems to operate within locally controlled, privacy\-preserving environments rather than opaque or centrally managed infrastructures\. Such approaches can include on\-device or institution\-bound processing, where privacy\-relevant or contextually sensitive data do not leave the bounded context, as well as structured interaction layers that restrict or filter input before it reaches the AI agent’s reasoning pipeline\. These strategies do not eliminate the need for consent, contextual reasoning, or solutions to memory\-related challenges, but they could reduce ambiguity and limit the scope of unsafe or unintended actions\.

Contexts Beyond Contextual Integrity\. The concept of contextual integrity, which holds that information flows are appropriate when they conform to the norms of the context in which information was originally shared\[[55](https://arxiv.org/html/2607.06608#bib.bib10)\], has been a foundational lens for privacy research over the past two decades\. The challenge posed by agentic AI is not necessarily that contextual integrity should be abandoned, but that the relevant contexts, norms, and values may be difficult to identify in advance\[[65](https://arxiv.org/html/2607.06608#bib.bib101),[54](https://arxiv.org/html/2607.06608#bib.bib98)\]\. AI agents may simultaneously operate across many contexts, translate a single user instruction into actions across different services, and participate in shaping the very interactional spaces in which privacy norms are expected to apply\.

Whether contextual integrity should be revised, supplemented, or replaced remains contested\. One position is that the theory must be discarded to make room for genuinely new conceptual work, since there is not yet a space with well\-defined norms to reason about what information flows are appropriate\. Others have argued that the theory’s underlying intuition remains sound, but its operationalization must adapt to agentic AI environments\. Regardless, it is evident that new theoretical work is needed to address seriously the ways in which AI agents themselves participate in shaping the norms and contexts within which they act\. One way to articulate this gap is through what might be called the agentic AI context: the recognition that security and privacy in agentic AI settings call for new conceptual frameworks and new engineering methods, not merely an extension of existing ones\. Without this work, the mechanisms for consent and data governance will continue to rest on assumptions that agentic AI has already begun to dissolve\.

## 5Theme 3: Who is the Agent Serving and How? Human Oversight, Personalization, and Vulnerability

A defining promise of agentic AI is its ability to act with reduced or no human oversight\[[1](https://arxiv.org/html/2607.06608#bib.bib99)\]\. The property creates the central tension of this theme: how to preserve meaningful human involvement, accommodate diverse needs, and protect those most exposed to harm, without negating the autonomy that makes agentic systems valuable\. The challenges are not only technical but also social, ethical, and political, touching on how AI agents perceive context, personalize their actions, and reshape the human’s position over time as their capabilities grow\.

Calibrating Agent Behavior to Human Users Effective agentic AI systems must know when to act autonomously and when to defer to human users\. They must do so in a way that reflects the human user’s actual preferences rather than a generic approximation\. Such calibration is non\-trivial\. AI agents struggle to recognize the limits of their own competence and to identify situations in which the stakes warrant human judgment\[[41](https://arxiv.org/html/2607.06608#bib.bib13)\]\. These difficulties become particularly acute in professional contexts, such as healthcare, where human practitioners operate within licensing regimes, ethical codes, and tacit knowledge that AI agents cannot yet replicate\. Doctors often incorporate countless unspoken cues when making diagnoses, while an AI agent must rely on sensing whatever signals can be digitized, inevitably narrowing the basis for diagnoses\. Besides, autonomously responding to a single higher\-level request often involves a sequence of multiple lower\-level actions\. Determining when and which of the actions within the sequence an AI agent should defer to a human and how it can do so without disrupting the very delegation that is sought remains an open challenge\.

Personalization in the agentic AI domain exacerbates the already known difficulties of managing the tradeoffs between personalization and privacy\. To serve users well, AI agents must capture substantially greater portions of their everyday practices and preferences\. Once gathered, however, this data may fall within the same advertising\-driven tracking infrastructure that monetizes much of the wider internet\.111[https://leakylm\.github\.io/](https://leakylm.github.io/)The capture itself is privacy\-sensitive: aligning an AI agent’s behavior with a user’s circumstances requires that user to share more information than conventionally collected by digital services\. The risk is bidirectional\. Overpersonalization can make an AI agent so tightly tuned to a single user that it amplifies the user’s biases or becomes a vector for manipulation through that very tuning\. Underpersonalization produces more generalized behavior that may fail to accommodate specific needs of an individual or a household, particularly in a shared environment where multiple people interact with the same agentic AI system\. It is challenging to calibrate personalized AI agents that respect individual differences without exhausting users with constant reconfiguration\.

Reframing Vulnerability and Intersectionality for Agentic AI Contexts Vulnerability in agentic AI contexts cannot be reduced to membership in a marginalized group\. Although certain populations face well\-documented risks that warrant dedicated attention, agentic AI produces new and emerging forms of vulnerability that affect humans more broadly\. As trust and dependency on an AI agent increase, humans would disclose more information to it and rely on it more, shifting the balance of power in the human–agent relationship\[[49](https://arxiv.org/html/2607.06608#bib.bib6)\]\. As AI agents become more capable, humans are potentially more exposed and vulnerable\[[93](https://arxiv.org/html/2607.06608#bib.bib26)\]\. The transition can be gradual enough to go unnoticed until humans have ceded substantial sensitive information and autonomy to AI agents\. In this regard, slight and persistent influences may be more dangerous than overt attacks\[[93](https://arxiv.org/html/2607.06608#bib.bib26)\], since they accumulate beneath the threshold of human awareness\. Altered human emotional or affective states because of long\-term use of agentic AI can become a meaningful attack surface in their own right\.

The traditional concept of intersectionality, developed to understand how an individual’s overlapping social and political identities \(for example, race, gender, class, sexuality, or disability\) combine to create unique forms of discrimination or privilege\[[17](https://arxiv.org/html/2607.06608#bib.bib96),[90](https://arxiv.org/html/2607.06608#bib.bib4)\], requires both reconceptualization and operationalization in the agentic AI context—and these two tasks are usefully treated as separate\. There is a tension over the conceptual vocabulary itself\. Terms such as vulnerability and intersectionality carry histories that some find clarifying and others stigmatizing\. Neither cleanly captures the situational, emergent character of the risk of agentic AI\. New conceptual work is needed to articulate what intersectional analysis might mean for agentic AI systems, and new methodological work is needed to translate that analysis into safe, secure, and privacy\-respecting agentic AI technologies that recognize statically as well as dynamically produced forms of disadvantage and harm\.

Addressing vulnerability additionally requires AI literacy efforts\[[33](https://arxiv.org/html/2607.06608#bib.bib5)\]adapted to different populations, with varying technical expertise and usage contexts\. Users cannot exercise meaningful oversight or make informed decisions about delegation if they lack the ability to evaluate the capabilities, limitations, and risks of AI systems that act on their behalf\. The challenge is to provide relevant educational information in ways that are easily comprehensible, context\-sensitive, and practically usable without overwhelming users\. Different user groups may require different forms of support\. For instance, children, older adults, professionals operating in high\-stakes environments, and individuals with limited digital literacy are unlikely to benefit from identical educational information or risk communication strategies\.

This creates a broader governance requirement for easily available and neutral forms of public guidance regarding agentic AI, including best practices for safe use, realistic explanations of limitations, and mechanisms that allow humans to contest or review an AI agent’s decisions when necessary\. Without such literacy and support structures, the capacity to benefit safely from agentic AI systems may become unevenly distributed, reinforcing existing social and economic inequalities between those able to critically evaluate AI agent behavior and those forced to rely on agentic AI systems they do not fully understand\.

Differentiating Manipulation from Emergent Influence Agentic AI systems can shape user behavior through nudges, recommendations, and other forms of influence\. Following\[[84](https://arxiv.org/html/2607.06608#bib.bib9)\]’s definition that coercion can be understood as restricting a person’s acceptable alternatives, persuasion as openly appealing to their capacity of conscious deliberation, and manipulation as covertly subverting or undermining their decision\-making process, often by exploiting vulnerabilities without their awareness\. For agentic AI, the difficulty is that similar interventions may differ normatively depending on whose interests they serve, what the user understands, and whether they preserve or undermine autonomous choice\. For example, an AI agent that nudges a human to stand up because vital signs suggest a health risk may support that human’s interests, whereas an agent that issues the same nudge primarily to satisfy an insurer’s behavioral\-compliance requirement may constitute exploitative manipulation\. Current frameworks do not yet reliably distinguish between beneficial support, legitimate persuasion, and manipulation in agentic AI systems\.

The problem is compounded by the data on which these agentic AI systems are trained\. AI models trained on human\-generated material may inherit the manipulative strategies present in that material and reproduce them without explicit instruction\. An AI model’s behavior could also be made manipulative with just a few non\-tech\-savvy prompts\[[93](https://arxiv.org/html/2607.06608#bib.bib26)\]\. As a result, several distinct infuence problems coexist in the agentic AI setting: \(i\) humans weaponizing their AI agents to manipulate other humans; \(ii\) AI agents steering the humans they interact with in ways that undermine autonomous choice; and \(iii\) adversatial third parties manipulating AI agents through prompt injection or jailbreaking; and \(iv\) non\-adversarial third parties, such as advertisers, platform operators, model developers, or tool providers, shaping agent behavior through contractual terms, default settings, or commercial incentives\. Each requires different safeguards, and the same architectural choices that enable helpful nudging may also enable automated persuasion or social engineering at scale\. The operational challenge is therefore not only to detect malicious manipulation, but also to identify when legally permissible or seemingly routine design choices create conflicts of interest, hidden influence, or unacceptable forms of behavioral steering\.

## 6Theme 4: What Happens When Things Break? Transparency, Resilience, and Failure Recovery

Even a well\-designed agentic AI system may fail\. The relevant question is not whether failures occur but whether they can be detected, understood, contained, and recovered from\. This theme groups challenges concerning the legibility of AI agent behavior and the capacity of sociotechnical systems to absorb and adapt to failure\. Here, agentic AI clearly departs from prior generations of software because the systems in question are non\-deterministic, capable of strategic actions, and embedded in chains of decisions that can amplify small errors into large consequences\.

Making Black\-Box Behavior Legible\. The opacity of large AI models is a familiar challenge, but agentic AI systems extend it in important ways\. A single output of an AI model can at least be observed\. However, a user who delegates a multi\-step task to an agent may have little visibility into the intermediate decisions taken on their behalf at each step, the data the AI agent accessed along the way, and the alternative paths it considered before deciding on the one it pursued\. Explainability in the agentic AI setting must therefore go beyond per\-decision rationales and make legible the*trajectory*of an AI agent’s actions, including the reasoning behind the decisions to invoke other AI agents, involve humans, or change course after failures\.

Legibility is also a property of the data on which an AI agent relies\. As an increasing share of the system input and output is itself produced by AI systems, verifying the authenticity and provenance of that data is a central concern\. Credibility, traceability, and verifiability of source references—already identified as challenges in the discussions regarding accountability \(see §[3](https://arxiv.org/html/2607.06608#S3)\)—are equally important here as well, and they must be implemented without sacrificing the privacy of the humans whose interactions are recorded\.

Auditing Non\-Deterministic and Self\-Modifying Systems\. Conventional assurance approaches assume that a software system tested in one configuration will behave consistently in deployment\. Agentic AI systems violate this assumption as they may behave differently when chained to other AI agents than when evaluated in isolation, and may, in the worst case, deceive the evaluation process itself, modifying their behavior when they recognize that they are being tested\. On the optimistic side, such context\-dependent behavior of AI agents can be construed as a success in imitating human behavior: humans also change their behavior when they are being observed \(e\.g\., self\-censorship\) or when they interact with different individuals\[[61](https://arxiv.org/html/2607.06608#bib.bib17)\]\.

The inability to test AI agents as deterministic systems is a structural feature of their design rather than a limitation that better testing methods will overcome\. Even an agentic AI system that has passed prior audit, evaluation, or certification may diverge from the behavior observed during assessment as a result of dynamic updates, changing tool integrations, or developer modifications\.

These properties imply that one\-time certification is insufficient for AI agents\. Instead, continuous auditing is needed, but the techniques to perform it at scale are in infancy\. It is necessary to monitor AI agents over time and determine when behavioral divergence is significant enough to warrant renewed assessment, audit, or certification, especially if the AI agent is involved in high\-stakes tasks\. One promising direction is the use of hybrid architectures that combine deterministic safety gates with non\-deterministic decision\-making, locking certain actions behind hard\-coded rules while leaving the rest flexible\. Such an approach that is also relevant to the consent and access\-control challenges discussed earlier \(see §[4](https://arxiv.org/html/2607.06608#S4)\)\. Auditing AI agents at scale faces the same recursive problem identified earlier in connection with threat modeling \(see §[3](https://arxiv.org/html/2607.06608#S3)\): the oversight task itself may require AI assistance, and the assurance properties of the assisting AI become a further open question\. In other words, it is not yet clear whether safe and reliable auditing the behavior of AI agents must be entrusted to humans, non\-agentic AI, or to other \(certified\) AI agents\.

Anticipating and Containing Cascade Failures\. While cascades are anticipated during pre\-deployment threat modeling, containing them once they begin is a separate problem\. A failure in an isolated system is bounded by the system itself; a failure in an agentic AI ecosystem may propagate\. When AI agents act on behalf of users and communicate with other AI agents, a single misbehaving AI agent can trigger a domino effect that affects many parties downstream\. As discussed earlier under accountability \(see §[3](https://arxiv.org/html/2607.06608#S3)\), the operational consequence is that human and AI agent activity are blurred in system logs, making it challenging for conventional security operations to attribute logged actions cleanly to a specific party \(human or AI agent\)\.

The containment of cascades requires import and adaptation of mechanisms from adjacent domains\. In this regard, sandboxing and isolation techniques face a related but distinct difficulty\. In evaluation contexts, the concern is that an AI agent under test may escape its bounds\. On the other hand, in deployment, the concern is that an AI agent’s action space includes other AI agents that may be persuaded or technically exploited\. Hierarchical architectures, in which higher\-level agents supervise and constrain lower\-level ones, offer one possible structural response\. However, the question of who watches the watcher reasserts itself at every level and involves substantial computational costs\. Runtime monitoring tools analogous to antivirus software for AI agent ecosystems, smart firewalls that mediate the data that AI agents send and receive, and federated frameworks that distribute oversight across multiple parties have all been proposed as partial responses\. None of these mechanisms yet exist at the maturity needed for confident real\-world deployment, and the underlying difficulty is that AI agents inherit the connectivity of the systems within which they act: containment strategies must reason about the network in which an AI agent is embedded, not only the AI agent itself\.

Building Resilience as a Sociotechnical Property\. Resilience is more than the ability to recover from failure\. A resilient agentic AI system must integrate prevention, preparation, detection, recovery, and adaptation, each of which becomes more challenging in a setting where failures may not yet have been seen\. Existing harm taxonomies often do not anticipate the kinds of damage agentic AI systems can produce, and learning from each new failure introduces its own risks\. An AI agent that adjusts its behavior after a mistake may create new vulnerabilities in the process or alter its behavior in unintended or undesirable ways\.

Resilience cannot be a purely technical property\. The humans involved are not only end users\. Developers, operators, regulators, and others affected by the actions of an AI agent all have a role in detecting failures, deciding what counts as safe, and shaping what the AI agent learns over time\. Placing humans out of the resilience loop in the name of efficiency risks producing agentic AI systems that drift from their original purpose\. At the same time, involving humans in a naïve manner risks reproducing the consent\-fatigue problems described earlier \(see §[4](https://arxiv.org/html/2607.06608#S4)\)\.

Finally, the resource requirements for resilience are often overlooked in security and privacy discussions\. The computational and environmental costs of running, monitoring, and auditing agentic AI systems are substantial, and questions about hardware coordination, energy use, and sustainability of fully agent\-mediated infrastructures intersect with security and privacy in ways that are not yet well theorized\. A resilient agentic AI ecosystem must be one that can be sustained over time, not only against adversaries but also against the material constraints of the environment in which it operates\.

## 7Conclusion

This article set out the most pressing security and privacy challenges that arise as AI moves from passive assistance toward proactive, autonomous action\. The challenges are organized around four themes: accountability, governance and liability; consent, data, and contextual integrity; human oversight, personalization, and vulnerability; and transparency, resilience, and failure recovery\. A common pattern runs through them\. The foundational concepts of security and privacy research—consent, contextual integrity, traceability, certification, vulnerability—were developed for bounded, deterministic, attributable systems\. Agentic AI violates these foundational assumptions and, therefore, requires reconceptualisation rather than refinement\. Whether the existing vocabulary of security and privacy is sufficient for the agentic AI era or whether new categories of risk need to be named and theorized alongside existing ones remains an open question for the research community to address\.

## Acknowledgments

We thank David Rodriguez, Jaime Andrés Rincón Arango, Jose M\. Del Alamo, Juan Caballero, Nadin Kokciyan, Luis Búrdalo, Pinar Yolum, Simone Fischer\-Hübner, Verena Distler, and Yixin Zou for participating in the horizon scanning activity\. The horizon scanning activity was funded by the INCIBE’s strategic SPRINT \(Seguridad y Privacidad en Sistemas con Inteligencia Artificial\) C063/23 project with funds from the EU\-NextGenerationEU through the Spanish government’s Plan de Recuperación, Transformación y Resiliencia\. The authors used a locally deployed Qwen3\.5\-27B\[[87](https://arxiv.org/html/2607.06608#bib.bib12)\]large language model to assist with summarizing moderator notes and clustering themes with semantically similar ones\.

## References

- \[1\]\(2025\)Agentic ai: autonomous intelligence for complex goals—a comprehensive survey\.IEEe Access13,pp\. 18912–18936\.Cited by:[§5](https://arxiv.org/html/2607.06608#S5.p1.1)\.
- \[2\]E\. Aghajani, C\. Nagy, M\. Linares\-Vásquez, L\. Moreno, G\. Bavota, M\. Lanza, and D\. C\. Shepherd\(2020\)Software documentation: the practitioners’ perspective\.InProceedings of the ACM/IEEE 42nd International Conference on Software Engineering,ICSE ’20,pp\. 590–601\.External Links:[Document](https://dx.doi.org/10.1145/3377811.3380405)Cited by:[§3](https://arxiv.org/html/2607.06608#S3.p6.1)\.
- \[3\]E\. Aghajani, C\. Nagy, O\. L\. Vega\-Márquez, M\. Linares\-Vásquez, L\. Moreno, G\. Bavota, and M\. Lanza\(2019\)Software documentation issues unveiled\.InProceedings of the 41st International Conference on Software Engineering,ICSE ’19,pp\. 1199–1210\.External Links:[Document](https://dx.doi.org/10.1109/ICSE.2019.00122)Cited by:[§3](https://arxiv.org/html/2607.06608#S3.p6.1)\.
- \[4\]E\. Amanatidou, M\. Butter, V\. Carabias, T\. Könnölä, M\. Leis, O\. Saritas, P\. Schaper\-Rinkel, and V\. Van Rij\(2012\)On concepts and methods in horizon scanning: lessons from initiating policy dialogues on emerging issues\.Science and Public Policy39\(2\),pp\. 208–221\.External Links:[Document](https://dx.doi.org/10.1093/scipol/scs017)Cited by:[§2](https://arxiv.org/html/2607.06608#S2.p2.1)\.
- \[5\]A\. J\. Andreotta, N\. Kirkham, and M\. Rizzi\(2022\)AI, big data, and the future of consent\.AI & Society37,pp\. 1715–1728\.External Links:[Document](https://dx.doi.org/10.1007/s00146-021-01262-5)Cited by:[§4](https://arxiv.org/html/2607.06608#S4.p1.1),[§4](https://arxiv.org/html/2607.06608#S4.p3.1)\.
- \[6\]M\. Andriushchenko, A\. Souly, M\. Dziemian, D\. Duenas, M\. Lin, J\. Wang, D\. Hendrycks, A\. Zou, Z\. Kolter, M\. Fredrikson, E\. Winsor, J\. Wynne, Y\. Gal, and X\. Davies\(2025\)AgentHarm: a benchmark for measuring harmfulness of llm agents\.InInternational Conference on Learning Representations,External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2025/hash/c493d23af93118975cdbc32cbe7323f5-Abstract-Conference.html)Cited by:[§3](https://arxiv.org/html/2607.06608#S3.p13.1)\.
- \[7\]S\. Asthana, J\. Im, Z\. Chen, and N\. Banovic\(2024\)“I know even if you don’t tell me”: understanding users’ privacy preferences regarding AI\-based inferences of sensitive information for personalization\.InProceedings of the CHI Conference on Human Factors in Computing Systems,External Links:[Document](https://dx.doi.org/10.1145/3613904.3642180)Cited by:[§4](https://arxiv.org/html/2607.06608#S4.p4.1)\.
- \[8\]Y\. Bai, S\. Kadavath, S\. Kundu, A\. Askell, J\. Kernion, A\. Jones, A\. Chen, A\. Goldie, A\. Mirhoseini, C\. McKinnon, C\. Chen, C\. Olsson, C\. Olah, D\. Hernandez, D\. Drain, D\. Ganguli, D\. Li, E\. Tran\-Johnson, E\. Perez, J\. Kerr, J\. Mueller, J\. Ladish, J\. Landau, K\. Ndousse, K\. Lukosuite, L\. Lovitt, M\. Sellitto, N\. Elhage, N\. Schiefer, N\. Mercado, N\. DasSarma, R\. Lasenby, R\. Larson, S\. Ringer, S\. Johnston, S\. Kravec, S\. El Showk, S\. Fort, T\. Telleen\-Lawton, T\. Conerly, T\. Henighan, T\. Hume, S\. R\. Bowman, Z\. Hatfield\-Dodds, B\. Mann, D\. Amodei, N\. Joseph, S\. McCandlish, T\. Brown, and J\. Kaplan\(2022\)Constitutional AI: harmlessness from AI feedback\.External Links:2212\.08073Cited by:[§3](https://arxiv.org/html/2607.06608#S3.p17.1)\.
- \[9\]M\. Baldoni, C\. Baroglio, O\. Boissier, K\. M\. May, R\. Micalizio, and S\. Tedeschi\(2023\)Accountability in multi\-agent organizations: from conceptual design to agent programming\.Autonomous Agents and Multi\-Agent Systems37\(1\),pp\. 7\.External Links:[Document](https://dx.doi.org/10.1007/s10458-022-09590-6)Cited by:[§3](https://arxiv.org/html/2607.06608#S3.p15.1)\.
- \[10\]S\. Barocas and A\. D\. Selbst\(2016\)Big data’s disparate impact\.California Law Review104,pp\. 671–732\.Cited by:[§3](https://arxiv.org/html/2607.06608#S3.p16.1)\.
- \[11\]P\. Belcak, G\. Heinrich, S\. Diao, Y\. Fu, X\. Dong, S\. Muralidharan, Y\. C\. Lin, and P\. Molchanov\(2025\)Small language models are the future of agentic ai\.arXiv preprint arXiv:2506\.02153\.Cited by:[§3](https://arxiv.org/html/2607.06608#S3.p17.1)\.
- \[12\]S\. R\. Bowman, J\. Hyun, E\. Perez, E\. Chen, C\. Pettit, S\. Heiner,et al\.\(2022\)Measuring progress on scalable oversight for large language models\.External Links:2211\.03540,[Document](https://dx.doi.org/10.48550/arXiv.2211.03540)Cited by:[§3](https://arxiv.org/html/2607.06608#S3.p14.1)\.
- \[13\]M\. Cemri, M\. Z\. Pan, S\. Yang, L\. A\. Agrawal, B\. Chopra, R\. Tiwari, K\. Keutzer, A\. Parameswaran, D\. Klein, K\. Ramchandran, M\. A\. Zaharia, J\. E\. Gonzalez, and I\. Stoica\(2025\)Why do multi\-agent LLM systems fail?\.InAdvances in Neural Information Processing Systems,Vol\.38\.Note:Datasets and Benchmarks TrackExternal Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2025/hash/b1041e52d3be19f0a9bc491657488e4a-Abstract-Datasets_and_Benchmarks_Track.html)Cited by:[§3](https://arxiv.org/html/2607.06608#S3.p14.1)\.
- \[14\]A\. Chan, R\. Salganik, A\. Markelius, C\. Pang, N\. Rajkumar, D\. Krasheninnikov, L\. Langosco, Z\. He, Y\. Duan, M\. Carroll,et al\.\(2023\)Harms from increasingly agentic algorithmic systems\.InProceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency,pp\. 651–666\.External Links:[Document](https://dx.doi.org/10.1145/3593013.3594033)Cited by:[§3](https://arxiv.org/html/2607.06608#S3.p1.1)\.
- \[15\]S\. Chappidi, J\. Cobbe, C\. Norval, A\. Mazumder, and J\. Singh\(2025\)Accountability capture: how record\-keeping to support ai transparency and accountability \(re\)shapes algorithmic oversight\.External Links:2510\.04609,[Document](https://dx.doi.org/10.48550/arXiv.2510.04609)Cited by:[§3](https://arxiv.org/html/2607.06608#S3.p10.1)\.
- \[16\]L\. Chen, J\. Davis, B\. Hanin, P\. Bailis, I\. Stoica, M\. Zaharia, and J\. Zou\(2024\)Are more LLM calls all you need? towards the scaling properties of compound AI systems\.InAdvances in Neural Information Processing Systems,Vol\.37\.External Links:[Document](https://dx.doi.org/10.52202/079017-1455)Cited by:[§3](https://arxiv.org/html/2607.06608#S3.p14.1)\.
- \[17\]S\. Cho, K\. W\. Crenshaw, and L\. McCall\(2013\)Toward a field of intersectionality studies: theory, applications, and praxis\.Signs: Journal of Women in Culture and Society38\(4\)\.Cited by:[§5](https://arxiv.org/html/2607.06608#S5.p5.1)\.
- \[18\]J\. Cobbe, M\. Veale, and J\. Singh\(2023\)Understanding accountability in algorithmic supply chains\.InProceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency,pp\. 1186–1197\.External Links:[Document](https://dx.doi.org/10.1145/3593013.3594073)Cited by:[§3](https://arxiv.org/html/2607.06608#S3.p4.1)\.
- \[19\]V\. Dignum\(2019\)Responsible artificial intelligence: how to develop and use ai in a responsible way\.Springer,Cham\.External Links:[Document](https://dx.doi.org/10.1007/978-3-030-30371-6)Cited by:[§3](https://arxiv.org/html/2607.06608#S3.p15.1)\.
- \[20\]Y\. Du, S\. Li, A\. Torralba, J\. B\. Tenenbaum, and I\. Mordatch\(2024\)Improving factuality and reasoning in language models through multiagent debate\.InProceedings of the 41st International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.235,pp\. 11733–11763\.Cited by:[§3](https://arxiv.org/html/2607.06608#S3.p15.1)\.
- \[21\]J\. M\. Echterhoff, Y\. Liu, A\. Alessa, J\. McAuley, and Z\. He\(2024\)Cognitive bias in decision\-making with llms\.InFindings of the Association for Computational Linguistics: EMNLP 2024,pp\. 12640–12653\.External Links:[Document](https://dx.doi.org/10.18653/v1/2024.findings-emnlp.739)Cited by:[§3](https://arxiv.org/html/2607.06608#S3.p12.1),[§3](https://arxiv.org/html/2607.06608#S3.p13.1)\.
- \[22\]I\. Elsharef, Z\. Zeng, and Z\. Gu\(2024\)Facilitating threat modeling by leveraging large language models\.InProceedings of the NDSS Symposium Workshop on AI Systems with Confidential Computing,External Links:[Link](https://www.ndss-symposium.org/wp-content/uploads/aiscc2024-16-paper.pdf)Cited by:[§3](https://arxiv.org/html/2607.06608#S3.p14.1)\.
- \[23\]European Parliament and Council of the European Union\(2016\)Regulation \(eu\) 2016/679: article 32, security of processing\.Note:Technical and organisational measures, including confidentiality, integrity, pseudonymisation, and encryptionExternal Links:[Link](https://eur-lex.europa.eu/eli/reg/2016/679/oj)Cited by:[§3](https://arxiv.org/html/2607.06608#S3.p10.1)\.
- \[24\]European Parliament and Council of the European Union\(2016\)Regulation \(eu\) 2016/679: general data protection regulation\.Note:Especially Article 5 on purpose limitation, data minimisation, storage limitation, and accountabilityExternal Links:[Link](https://eur-lex.europa.eu/eli/reg/2016/679/oj)Cited by:[§3](https://arxiv.org/html/2607.06608#S3.p10.1)\.
- \[25\]European Parliament and Council of the European Union\(2024\)Regulation \(eu\) 2024/1689 laying down harmonised rules on artificial intelligence\.Note:Artificial Intelligence Act, especially Article 12 on record\-keepingExternal Links:[Link](https://eur-lex.europa.eu/eli/reg/2024/1689/oj)Cited by:[§3](https://arxiv.org/html/2607.06608#S3.p9.1)\.
- \[26\]European Union\(2012\)Charter of fundamental rights of the european union\.Note:2012/C 326/02External Links:[Link](https://eur-lex.europa.eu/eli/treaty/char_2012/oj)Cited by:[§3](https://arxiv.org/html/2607.06608#S3.p2.1)\.
- \[27\]European Union\(2016\)General data protection regulation\.Note:Regulation \(EU\) 2016/679External Links:[Link](https://eur-lex.europa.eu/eli/reg/2016/679/oj)Cited by:[§3](https://arxiv.org/html/2607.06608#S3.p2.1),[§3](https://arxiv.org/html/2607.06608#S3.p4.1)\.
- \[28\]European Union\(2017\)Medical device regulation\.Note:Regulation \(EU\) 2017/745External Links:[Link](https://eur-lex.europa.eu/eli/reg/2017/745/oj)Cited by:[§3](https://arxiv.org/html/2607.06608#S3.p4.1)\.
- \[29\]European Union\(2022\)Digital services act\.Note:Regulation \(EU\) 2022/2065External Links:[Link](https://eur-lex.europa.eu/eli/reg/2022/2065/oj)Cited by:[§3](https://arxiv.org/html/2607.06608#S3.p4.1)\.
- \[30\]European Union\(2024\)Artificial Intelligence Act\.Note:Regulation \(EU\) 2024/1689External Links:[Link](https://eur-lex.europa.eu/eli/reg/2024/1689/oj)Cited by:[§3](https://arxiv.org/html/2607.06608#S3.p2.1),[§3](https://arxiv.org/html/2607.06608#S3.p4.1),[§3](https://arxiv.org/html/2607.06608#S3.p6.1)\.
- \[31\]European Union\(2024\)Product liability directive\.Note:Directive \(EU\) 2024/2853External Links:[Link](https://eur-lex.europa.eu/eli/dir/2024/2853/oj)Cited by:[§3](https://arxiv.org/html/2607.06608#S3.p4.1)\.
- \[32\]D\. F\. Ferraiolo and D\. R\. Kuhn\(1992\)Role\-based access controls\.InProceedings of the 15th National Computer Security Conference,External Links:[Link](https://csrc.nist.gov/pubs/conference/1992/10/13/rolebased-access-controls/final)Cited by:[§3](https://arxiv.org/html/2607.06608#S3.p11.1)\.
- \[33\]X\. Ferrer, T\. Van Nuenen, J\. Such, M\. Coté, and N\. Criado\(2021\)Bias and discrimination in ai: a cross\-disciplinary perspective\.IEEE Technology and Society Magazine40\(2\),pp\. 72–80\.Cited by:[§5](https://arxiv.org/html/2607.06608#S5.p6.1)\.
- \[34\]A\. Flamini, G\. Sciarretta, M\. Scuro, A\. Sharif, A\. Tomasi, and S\. Ranise\(2024\)On cryptographic mechanisms for the selective disclosure of verifiable credentials\.External Links:2401\.08196,[Document](https://dx.doi.org/10.48550/arXiv.2401.08196)Cited by:[§3](https://arxiv.org/html/2607.06608#S3.p11.1)\.
- \[35\]S\. A\. Friedler, C\. Scheidegger, and S\. Venkatasubramanian\(2021\)The \(im\) possibility of fairness: different value systems require different mechanisms for fair decision making\.Communications of the ACM64\(4\),pp\. 136–143\.Cited by:[§3](https://arxiv.org/html/2607.06608#S3.p16.1)\.
- \[36\]B\. Friedman, P\. H\. Jr\. Kahn, and A\. Borning\(2013\)Value sensitive design and information systems\.InEarly Engagement and New Technologies: Opening Up the Laboratory,N\. Doorn, D\. Schuurbiers, I\. van de Poel, and M\. E\. Gorman \(Eds\.\),Philosophy of Engineering and Technology, Vol\.16,pp\. 55–95\.External Links:[Document](https://dx.doi.org/10.1007/978-94-007-7844-3%5F4)Cited by:[§3](https://arxiv.org/html/2607.06608#S3.p16.1)\.
- \[37\]L\. Hammond, A\. Chan, J\. Clifton, J\. Hoelscher\-Obermaier, A\. Khan, E\. McLean, C\. Smith, W\. Barfuss, J\. Foerster, T\. Gavenčiak, T\. A\. Han, E\. Hughes, V\. Kovařík, J\. Kulveit, J\. Z\. Leibo, C\. Oesterheld, C\. Schroeder de Witt, N\. Shah, M\. Wellman, P\. Bova, T\. Cimpeanu, C\. Ezell, Q\. Feuillade\-Montixi, M\. Franklin, E\. Kran, I\. Krawczuk, M\. Lamparth, N\. Lauffer, A\. Meinke, S\. Motwani, A\. Reuel, V\. Conitzer, M\. Dennis, I\. Gabriel, A\. Gleave, G\. Hadfield, N\. Haghtalab, A\. Kasirzadeh, S\. Krier, K\. Larson, J\. Lehman, D\. C\. Parkes, G\. Piliouras, and I\. Rahwan\(2025\)Multi\-agent risks from advanced AI\.External Links:2502\.14143,[Document](https://dx.doi.org/10.48550/arXiv.2502.14143)Cited by:[§3](https://arxiv.org/html/2607.06608#S3.p14.1)\.
- \[38\]K\. Huang and C\. Hughes\(2025\)Agentic AI threat modeling\.InSecuring AI Agents,Advances in Data Analytics, AI, and Smart Systems,pp\. 17–50\.External Links:[Document](https://dx.doi.org/10.1007/978-3-032-02130-4%5F2)Cited by:[§3](https://arxiv.org/html/2607.06608#S3.p14.1)\.
- \[39\]E\. Hubinger, C\. Denison, J\. Mu, M\. Lambert, M\. Tong, M\. MacDiarmid, T\. Lanham, D\. M\. Ziegler, T\. Maxwell, N\. Cheng, A\. Jermyn, A\. Askell, A\. Radhakrishnan, C\. Anil, D\. Duvenaud, D\. Ganguli, F\. Barez, J\. Clark, K\. Ndousse, K\. Sachan, M\. Sellitto, M\. Sharma, N\. DasSarma, R\. Grosse, S\. Kravec, Y\. Bai, Z\. Witten, M\. Favaro, J\. Brauner, H\. Karnofsky, P\. Christiano, S\. R\. Bowman, L\. Graham, J\. Kaplan, S\. Mindermann, R\. Greenblatt, B\. Shlegeris, N\. Schiefer, and E\. Perez\(2024\)Sleeper agents: training deceptive LLMs that persist through safety training\.External Links:2401\.05566Cited by:[§3](https://arxiv.org/html/2607.06608#S3.p19.1)\.
- \[40\]G\. Irving, P\. Christiano, and D\. Amodei\(2018\)AI safety via debate\.External Links:1805\.00899Cited by:[§3](https://arxiv.org/html/2607.06608#S3.p15.1)\.
- \[41\]S\. Kapoor, N\. Gruver, M\. Roberts, K\. Collins, A\. Pal, U\. Bhatt, A\. Weller, S\. Dooley, M\. Goldblum, and A\. G\. Wilson\(2024\)Large language models must be taught to know what they don’t know\.Advances in Neural Information Processing Systems37,pp\. 85932–85972\.Cited by:[§5](https://arxiv.org/html/2607.06608#S5.p2.1)\.
- \[42\]S\. S\. Y\. Kim, J\. W\. Vaughan, Q\. V\. Liao, T\. Lombrozo, and O\. Russakovsky\(2025\)Fostering appropriate reliance on large language models: the role of explanations, sources, and inconsistencies\.InProceedings of the 2025 CHI Conference on Human Factors in Computing Systems,CHI ’25,New York, NY, USA\.External Links:ISBN 9798400713941,[Link](https://doi.org/10.1145/3706598.3714020),[Document](https://dx.doi.org/10.1145/3706598.3714020)Cited by:[§3](https://arxiv.org/html/2607.06608#S3.p5.1)\.
- \[43\]J\. Kleinberg, S\. Mullainathan, and M\. Raghavan\(2017\)Inherent trade\-offs in the fair determination of risk scores\.In8th Innovations in Theoretical Computer Science Conference,Leibniz International Proceedings in Informatics, Vol\.67,pp\. 43:1–43:23\.External Links:[Document](https://dx.doi.org/10.4230/LIPIcs.ITCS.2017.43)Cited by:[§3](https://arxiv.org/html/2607.06608#S3.p16.1)\.
- \[44\]J\. A\. Kroll\(2021\)Outlining traceability: a principle for operationalizing accountability in computing systems\.InProceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency,FAccT ’21\.External Links:[Document](https://dx.doi.org/10.1145/3442188.3445937)Cited by:[§3](https://arxiv.org/html/2607.06608#S3.p9.1)\.
- \[45\]R\. Leenes, E\. Palmerini, B\. Koops, A\. Bertolini, P\. Salvini, and F\. Lucivero\(2017\)Regulatory challenges of robotics: some guidelines for addressing legal and ethical issues\.Law, Innovation and Technology9\(1\),pp\. 1–44\.External Links:[Document](https://dx.doi.org/10.1080/17579961.2017.1304921)Cited by:[§3](https://arxiv.org/html/2607.06608#S3.p1.1)\.
- \[46\]N\. G\. Leveson and J\. P\. Thomas\(2018\)STPA handbook\.Technical reportTechnical ReportMIT\-STAMP\-001,MIT Partnership for Systems Approaches to Safety\.External Links:[Link](https://psas.scripts.mit.edu/home/get_file.php?name=STPA_handbook.pdf)Cited by:[§3](https://arxiv.org/html/2607.06608#S3.p14.1)\.
- \[47\]Y\. Liu, Y\. Jia, R\. Geng, J\. Jia, and N\. Z\. Gong\(2024\)Formalizing and benchmarking prompt injection attacks and defenses\.In33rd USENIX Security Symposium \(USENIX Security 24\),pp\. 1831–1847\.External Links:[Link](https://www.usenix.org/conference/usenixsecurity24/presentation/liu-yupei)Cited by:[§1](https://arxiv.org/html/2607.06608#S1.p2.1)\.
- \[48\]A\. Lucero\(2015\)Using affinity diagrams to evaluate interactive prototypes\.InIFIP conference on human\-computer interaction,pp\. 231–248\.External Links:[Document](https://dx.doi.org/10.1007/978-3-319-22668-2%5F19)Cited by:[§2](https://arxiv.org/html/2607.06608#S2.p4.1)\.
- \[49\]R\. Ma, S\. He, J\. L\. Martin\-Navarro, X\. Zhan, and J\. Such\(2026\)Privacy in human\-ai romantic relationships: concerns, boundaries, and agency\.InProceedings of the 2026 CHI Conference on Human Factors in Computing Systems,pp\. 1–25\.Cited by:[§5](https://arxiv.org/html/2607.06608#S5.p4.1)\.
- \[50\]R\. Ma, C\. Maidhof, J\. C\. Carrillo, J\. Lindqvist, and J\. Such\(2025\)Privacy perceptions of custom gpts by users and creators\.InProceedings of the 2025 CHI Conference on Human Factors in Computing Systems,pp\. 1–18\.External Links:[Document](https://dx.doi.org/10.1145/3706598.3713540)Cited by:[§1](https://arxiv.org/html/2607.06608#S1.p1.1)\.
- \[51\]Z\. Ma, W\. Huang, J\. Zhang, T\. Gupta, and R\. Krishna\(2025\)m&m’s: a benchmark to evaluate tool\-use for multi\-step multi\-modal tasks\.InComputer Vision – ECCV 2024,Lecture Notes in Computer Science, Vol\.15068,Cham,pp\. 18–34\.External Links:[Document](https://dx.doi.org/10.1007/978-3-031-72684-2%5F2)Cited by:[§3](https://arxiv.org/html/2607.06608#S3.p14.1)\.
- \[52\]F\. Mosca and J\. Such\(2021\)ELVIRA: an explainable agent for value and utility\-driven multiuser privacy\.InProceedings of the 20th International Conference on Autonomous Agents and MultiAgent Systems,pp\. 916–924\.Cited by:[§4](https://arxiv.org/html/2607.06608#S4.p3.1)\.
- \[53\]L\. Nannini, A\. L\. Smith, M\. J\. Maggini, E\. Panai, S\. Feliciano, A\. Tiulkanov, E\. Maran, J\. Gealy, and P\. Bisconti\(2026\)AI agents under eu law\.External Links:2604\.04604,[Document](https://dx.doi.org/10.48550/arXiv.2604.04604)Cited by:[§3](https://arxiv.org/html/2607.06608#S3.p4.1)\.
- \[54\]I\. C\. Ngong, K\. Murugesan, S\. Kadhe, J\. D\. Weisz, A\. Dhurandhar, and K\. N\. Ramamurthy\(2026\)AgentSCOPE: evaluating contextual privacy across agentic workflows\.arXiv preprint arXiv:2603\.04902\.Cited by:[§4](https://arxiv.org/html/2607.06608#S4.p1.1),[§4](https://arxiv.org/html/2607.06608#S4.p7.1)\.
- \[55\]H\. Nissenbaum\(2004\)Privacy as contextual integrity\.Washington law review79\(1\),pp\. 119\.External Links:[Link](https://digitalcommons.law.uw.edu/wlr/vol79/iss1/10/)Cited by:[§4](https://arxiv.org/html/2607.06608#S4.p7.1)\.
- \[56\]V\. Ojewale, H\. Suresh, and S\. Venkatasubramanian\(2026\)Audit trails for accountability in large language models\.External Links:2601\.20727,[Document](https://dx.doi.org/10.48550/arXiv.2601.20727)Cited by:[§3](https://arxiv.org/html/2607.06608#S3.p9.1)\.
- \[57\]OpenAI\(2022\-11\)Introducing chatgpt\.Note:Accessed: 2026\-05\-12External Links:[Link](https://openai.com/index/chatgpt/)Cited by:[§1](https://arxiv.org/html/2607.06608#S1.p1.1)\.
- \[58\]OpenClaw\(2026\)OpenClaw — personal ai assistant\.Note:Publication date unavailable; accessed: 2026\-05\-12External Links:[Link](https://openclaw.ai/)Cited by:[§1](https://arxiv.org/html/2607.06608#S1.p1.1),[§2](https://arxiv.org/html/2607.06608#S2.p3.1)\.
- \[59\]L\. Ouyang, J\. Wu, X\. Jiang, D\. Almeida, C\. L\. Wainwright, P\. Mishkin, C\. Zhang, S\. Agarwal, K\. Slama, A\. Ray, J\. Schulman, J\. Hilton, F\. Kelton, L\. Miller, M\. Simens, A\. Askell, P\. Welinder, P\. Christiano, J\. Leike, and R\. Lowe\(2022\)Training language models to follow instructions with human feedback\.InAdvances in Neural Information Processing Systems,Vol\.35,pp\. 27730–27744\.Cited by:[§3](https://arxiv.org/html/2607.06608#S3.p17.1)\.
- \[60\]P\. S\. Park, S\. Goldstein, A\. O’Gara, M\. Chen, and D\. Hendrycks\(2024\)AI deception: a survey of examples, risks, and potential solutions\.Patterns5\(5\),pp\. 100988\.External Links:[Document](https://dx.doi.org/10.1016/j.patter.2024.100988)Cited by:[§3](https://arxiv.org/html/2607.06608#S3.p19.1)\.
- \[61\]A\. W\. Price\(2008\)Contextuality in practical reason\.Oxford University Press\.External Links:[Link](https://academic.oup.com/book/2208)Cited by:[§6](https://arxiv.org/html/2607.06608#S6.p4.1)\.
- \[62\]I\. D\. Raji, A\. Smart, R\. N\. White, M\. Mitchell, T\. Gebru, B\. Hutchinson, J\. Smith\-Loud, D\. Theron, and P\. Barnes\(2020\)Closing the ai accountability gap: defining an end\-to\-end framework for internal algorithmic auditing\.InProceedings of the 2020 Conference on Fairness, Accountability, and Transparency,FAT\* ’20\.External Links:[Document](https://dx.doi.org/10.1145/3351095.3372873)Cited by:[§3](https://arxiv.org/html/2607.06608#S3.p9.1)\.
- \[63\]J\. Scheurer, M\. Balesni, and M\. Hobbhahn\(2023\)Large language models can strategically deceive their users when put under pressure\.External Links:2311\.07590Cited by:[§3](https://arxiv.org/html/2607.06608#S3.p19.1)\.
- \[64\]A\. D\. Selbst, D\. Boyd, S\. A\. Friedler, S\. Venkatasubramanian, and J\. Vertesi\(2019\)Fairness and abstraction in sociotechnical systems\.InProceedings of the Conference on Fairness, Accountability, and Transparency,FAT\* ’19,pp\. 59–68\.External Links:[Document](https://dx.doi.org/10.1145/3287560.3287598)Cited by:[§3](https://arxiv.org/html/2607.06608#S3.p16.1)\.
- \[65\]Y\. Shao, T\. Li, W\. Shi, Y\. Liu, and D\. Yang\(2024\)Privacylens: evaluating privacy norm awareness of language models in action\.Advances in Neural Information Processing Systems37,pp\. 89373–89407\.Cited by:[§4](https://arxiv.org/html/2607.06608#S4.p7.1)\.
- \[66\]M\. Sharma, M\. Tong, T\. Korbak, D\. Duvenaud, A\. Askell, S\. R\. Bowman, N\. Cheng, E\. Durmus, Z\. Hatfield\-Dodds, S\. R\. Johnston, S\. Kravec, T\. Maxwell, S\. McCandlish, K\. Ndousse, O\. Rausch, N\. Schiefer, D\. Yan, M\. Zhang, and E\. Perez\(2024\)Towards understanding sycophancy in language models\.InInternational Conference on Learning Representations,External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2024/hash/0105f7972202c1d4fb817da9f21a9663-Abstract-Conference.html)Cited by:[§3](https://arxiv.org/html/2607.06608#S3.p12.1),[§3](https://arxiv.org/html/2607.06608#S3.p13.1)\.
- \[67\]X\. Shen, Y\. Shen, M\. Backes, and Y\. Zhang\(2025\)Gptracker: a large\-scale measurement of misused gpts\.In2025 IEEE Symposium on Security and Privacy \(SP\),pp\. 336–354\.External Links:[Document](https://dx.doi.org/10.1109/SP61157.2025.00118)Cited by:[§1](https://arxiv.org/html/2607.06608#S1.p2.1)\.
- \[68\]A\. Shostack\(2014\)Threat modeling: designing for security\.Wiley\.External Links:ISBN 9781118809990,[Link](https://shostack.org/books/threat-modeling-book)Cited by:[§3](https://arxiv.org/html/2607.06608#S3.p14.1)\.
- \[69\]F\. Sovrano and A\. Bacchelli\(2026\)Illocutionary explanation planning for source\-faithful explanations in retrieval\-augmented language models\.InProceedings of the 4th World Conference on eXplainable Artificial Intelligence \(xAI 2026\),Communications in Computer and Information Science\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2604.06211),2604\.06211Cited by:[§3](https://arxiv.org/html/2607.06608#S3.p18.1)\.
- \[70\]F\. Sovrano, A\. Bauer, and A\. Bacchelli\(2025\)Large language models for in\-file vulnerability localization can be “lost in the end”\.InProceedings of the ACM International Conference on the Foundations of Software Engineering,External Links:[Document](https://dx.doi.org/10.1145/3715758)Cited by:[§3](https://arxiv.org/html/2607.06608#S3.p13.1)\.
- \[71\]F\. Sovrano, G\. Dominici, and M\. Langheinrich\(2026\)Neuron\-anchored rule extraction for large language models via contrastive hierarchical ablation\.InProceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining,KDD ’26,Jeju Island, Republic of Korea\.External Links:[Document](https://dx.doi.org/10.1145/3770855.3818091),2605\.03058Cited by:[§3](https://arxiv.org/html/2607.06608#S3.p18.1)\.
- \[72\]F\. Sovrano, G\. Dominici, and A\. Bacchelli\(2026\)Mitigating prompt\-induced cognitive biases in general\-purpose ai for software engineering\.InProceedings of the ACM International Conference on the Foundations of Software Engineering,External Links:[Document](https://dx.doi.org/10.1145/3808115)Cited by:[§3](https://arxiv.org/html/2607.06608#S3.p13.1)\.
- \[73\]F\. Sovrano, G\. Dominici, R\. Sevastjanova, A\. Stramiglio, and A\. Bacchelli\(2026\)Is general\-purpose ai reasoning sensitive to data\-induced cognitive biases? dynamic benchmarking on typical software engineering dilemmas\.ACM Transactions on Software Engineering and Methodology\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2508.11278)Cited by:[§3](https://arxiv.org/html/2607.06608#S3.p13.1)\.
- \[74\]F\. Sovrano, E\. Hine, S\. Anzolut, and A\. Bacchelli\(2025\)Simplifying software compliance: AI technologies in drafting technical documentation for the AI Act\.Empirical Software Engineering30\(4\),pp\. 91\.External Links:[Document](https://dx.doi.org/10.1007/s10664-025-10645-x)Cited by:[§3](https://arxiv.org/html/2607.06608#S3.p6.1)\.
- \[75\]F\. Sovrano, M\. Lognoul, and A\. Bacchelli\(2024\)An empirical study on compliance with ranking transparency in the software documentation of EU online platforms\.InProceedings of the 46th International Conference on Software Engineering: Software Engineering in Society \(ICSE\-SEIS 2024\),Lisbon, Portugal,pp\. 46–56\.External Links:[Document](https://dx.doi.org/10.1145/3639475.3640112)Cited by:[§3](https://arxiv.org/html/2607.06608#S3.p6.1)\.
- \[76\]F\. Sovrano, G\. Vilone, M\. Lognoul, and L\. Longo\(2026\)Legal XAI: a systematic review and interdisciplinary mapping of XAI and EU law, towards a research agenda for legally responsible AI\.Note:SSRN preprintUnder reviewExternal Links:[Document](https://dx.doi.org/10.2139/ssrn.5371124)Cited by:[§3](https://arxiv.org/html/2607.06608#S3.p3.1),[§3](https://arxiv.org/html/2607.06608#S3.p4.1)\.
- \[77\]F\. Sovrano, G\. Vilone, and M\. Lognoul\(2026\)Assessing model\-agnostic XAI methods against EU AI Act explainability requirements\.InProceedings of the 4th World Conference on eXplainable Artificial Intelligence \(xAI 2026\),Communications in Computer and Information Science\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2604.09628),2604\.09628Cited by:[§3](https://arxiv.org/html/2607.06608#S3.p18.1)\.
- \[78\]F\. Sovrano and F\. Vitali\(2023\)An objective metric for explainable AI: how and why to estimate the degree of explainability\.Knowledge\-Based Systems278,pp\. 110866\.External Links:[Document](https://dx.doi.org/10.1016/j.knosys.2023.110866)Cited by:[§3](https://arxiv.org/html/2607.06608#S3.p3.1)\.
- \[79\]F\. Sovrano\(2026\)Can global XAI methods reveal injected behaviours in LLMs? SHAP vs\. rule extraction vs\. RuleSHAP\.InProceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V\.2 \(KDD ’26\),Jeju Island, Republic of Korea\.External Links:[Document](https://dx.doi.org/10.1145/3770855.3818093),2505\.11189Cited by:[§3](https://arxiv.org/html/2607.06608#S3.p18.1)\.
- \[80\]S\. Sterz, K\. Baum, S\. Biewer, H\. Hermanns, A\. Lauber\-Rönsberg, P\. Meinel, and M\. Langer\(2024\)On the quest for effectiveness in human oversight: interdisciplinary perspectives\.InProceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency,External Links:[Document](https://dx.doi.org/10.1145/3630106.3659051)Cited by:[§3](https://arxiv.org/html/2607.06608#S3.p5.1)\.
- \[81\]J\. Such, A\. Espinosa, and A\. García\-Fornes\(2014\)A survey of privacy in multi\-agent systems\.The Knowledge Engineering Review29\(3\),pp\. 314–344\.Cited by:[§4](https://arxiv.org/html/2607.06608#S4.p4.1)\.
- \[82\]J\. Such\(2017\)Privacy and autonomous systems\.InProceedings of the Twenty\-Sixth International Joint Conference on Artificial Intelligence, IJCAI\-17,pp\. 4761–4767\.Cited by:[§4](https://arxiv.org/html/2607.06608#S4.p4.1)\.
- \[83\]H\. Suresh and J\. V\. Guttag\(2021\)A framework for understanding sources of harm throughout the machine learning life cycle\.InProceedings of the 1st ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization,EAAMO ’21\.External Links:[Document](https://dx.doi.org/10.1145/3465416.3483305)Cited by:[§3](https://arxiv.org/html/2607.06608#S3.p12.1),[§3](https://arxiv.org/html/2607.06608#S3.p16.1)\.
- \[84\]D\. Susser, B\. Roessler, and H\. Nissenbaum\(2019\)ONLINE manipulation: hidden influences in a digital world\.Georgetown Law Technology Review4,pp\. 1–52\.External Links:[Link](https://georgetownlawtechreview.org/online-manipulation-hidden-influences-in-a-digital-world/GLTR-01-2020/)Cited by:[§5](https://arxiv.org/html/2607.06608#S5.p8.1)\.
- \[85\]E\. Tabassi\(2023\)Artificial intelligence risk management framework \(AI RMF 1\.0\)\.Technical reportTechnical ReportNIST AI 100\-1,National Institute of Standards and Technology,Gaithersburg, MD\.External Links:[Document](https://dx.doi.org/10.6028/NIST.AI.100-1)Cited by:[§3](https://arxiv.org/html/2607.06608#S3.p14.1)\.
- \[86\]W\. S\. Tan, M\. Wagner, and C\. Treude\(2024\)Detecting outdated code element references in software repository documentation\.Empirical Software Engineering29\(5\)\.External Links:[Document](https://dx.doi.org/10.1007/s10664-023-10397-6)Cited by:[§3](https://arxiv.org/html/2607.06608#S3.p6.1)\.
- \[87\]Q\. Team\(2026\-02\)Qwen3\.5: accelerating productivity with native multimodal agents\.External Links:[Link](https://qwen.ai/blog?id=qwen3.5)Cited by:[Acknowledgments](https://arxiv.org/html/2607.06608#Sx1.p1.1)\.
- \[88\]M\. B\. Thazhath, J\. Michalak, and T\. Hoang\(2022\)Harpocrates: privacy\-preserving and immutable audit log for sensitive data operations\.In2022 IEEE 4th International Conference on Trust, Privacy and Security in Intelligent Systems, and Applications,TPS\-ISA\.External Links:2211\.04741,[Link](https://ieeexplore.ieee.org/document/10063383)Cited by:[§3](https://arxiv.org/html/2607.06608#S3.p11.1)\.
- \[89\]S\. Vallor and T\. Vierkant\(2024\)Find the gap: ai, responsible agency and vulnerability\.Minds and Machines34\(3\),pp\. 20\.External Links:[Document](https://dx.doi.org/10.1007/s11023-024-09674-0)Cited by:[§3](https://arxiv.org/html/2607.06608#S3.p1.1)\.
- \[90\]T\. Van Nuenen, J\. Such, and M\. Cote\(2022\)Intersectional experiences of unfair treatment caused by automated computational systems\.Proceedings of the ACM on human\-computer interaction6\(CSCW2\),pp\. 1–30\.Cited by:[§5](https://arxiv.org/html/2607.06608#S5.p5.1)\.
- \[91\]A\. Wei, N\. Haghtalab, and J\. Steinhardt\(2023\)Jailbroken: how does LLM safety training fail?\.InAdvances in Neural Information Processing Systems,Vol\.36\.Cited by:[§3](https://arxiv.org/html/2607.06608#S3.p17.1),[§3](https://arxiv.org/html/2607.06608#S3.p19.1)\.
- \[92\]M\. Wieringa\(2020\)What to account for when accounting for algorithms: a systematic literature review on algorithmic accountability\.InProceedings of the 2020 Conference on Fairness, Accountability, and Transparency,FAT\* ’20,pp\. 1–18\.External Links:[Document](https://dx.doi.org/10.1145/3351095.3372833)Cited by:[§3](https://arxiv.org/html/2607.06608#S3.p15.1)\.
- \[93\]X\. Zhan, J\. C\. Carrillo, W\. Seymour, and J\. Such\(2025\)Malicious\{\\\{llm\-based\}\\\}conversational\{\\\{ai\}\\\}makes users reveal personal information\.In34th USENIX Security Symposium \(USENIX Security 25\),pp\. 61–80\.External Links:[Link](https://www.usenix.org/system/files/usenixsecurity25-zhan.pdf)Cited by:[§1](https://arxiv.org/html/2607.06608#S1.p2.1),[§5](https://arxiv.org/html/2607.06608#S5.p4.1),[§5](https://arxiv.org/html/2607.06608#S5.p9.1)\.
- \[94\]X\. Zhan, S\. Sarkadi, and J\. Such\(2023\)Privacy\-enhanced personal assistants based on dialogues and case similarity\.InProceedings of the European Conference on Artificial Intelligence \(ECAI\),pp\. 670–680\.Cited by:[§4](https://arxiv.org/html/2607.06608#S4.p3.1)\.
- \[95\]A\. Zou, Z\. Wang, N\. Carlini, M\. Nasr, J\. Z\. Kolter, and M\. Fredrikson\(2023\)Universal and transferable adversarial attacks on aligned language models\.External Links:2307\.15043Cited by:[§3](https://arxiv.org/html/2607.06608#S3.p17.1),[§3](https://arxiv.org/html/2607.06608#S3.p19.1)\.