Hierarchical Compositionality for An Assistive AI Agent

# Hierarchical Compositionality for An Assistive AI Agent
Source: [https://arxiv.org/html/2608.10330](https://arxiv.org/html/2608.10330)
\\JAIRTrack\\JAIRAE

andMohan Sridharan[m\.sridharan@ed\.ac\.uk](mailto:[email protected])University of EdinburghEdinburghUnited Kingdom

###### Abstract\.

AI agents are increasingly being developed to assist humans in various applications, and Large Language Models and other deep network architectures are considered to be state of the art for such agents\. These methods are impressive stochastic predictors, but they are resource\-hungry, opaque, and known to make arbitrary decisions in novel situations due to the narrow set of underlying representation and processing choices\. Our work seeks to explore the design of architectures for such AI agents based on core principles that can be traced back to the early pioneers of AI but are not fully utilized in modern AI methods\. We do so in this paper in the context of the core problem of AI agents addressing ambiguity in the objects being referred to by the human participants\. Humans address such ambiguity by heuristically leveraging compositional knowledge of domain context and the preferences of the other human participants\. Drawing inspiration from this observation, we describe an architecture that embeds the principle of hierarchical compositionality and uses simple heuristics to achieve the desired disambiguation\. Specifically, domain objects are represented in terms of primitive attributes drawn from human\-validated semantic feature norms, and a hierarchical combination ofattributesandconceptsautomatically identified from a limited observed history of interactions of an assistive agent with specific users\. The assistive agent then achieves the desired disambiguation by reasoning with knowledge of this compositional hierarchy; axioms governing domain dynamics; and models of semantic compatibility, session salience, and user\-specific thematic preference, requesting human clarification when necessary\. Experiments show that our approach consistently outperforms state of the art data\-driven baselines, supporting adaptation to specific user profiles\. Project website:[https://tianyi\-fu\.github\.io/HCAA/](https://tianyi-fu.github.io/HCAA/)

## 1\.Motivation

We are witnessing a rapid proliferation of AI agents being deployed to assist humans in various applications\. Deep networks andfoundation models\(FMs\) such as Large Language Models \(LLMs\) and Vision Language Models \(VLMs\) are considered state of the art for the design of such AI agents\. Although they have been reported to provide impressive performance in perception, reasoning, and interaction problems, these methods are opaque and resource\-hungry, requiring considerable training examples, storage, and computation\. The underlying stochastic prediction system captures the most likely patterns, which may lead to inconsistent and arbitrary predictions in novel situations\(doshi:corl24;huang2022language;ahn2022can;liu2023llm\+;kambhampati2024position;silver2022pddl;xiong2024can\)\.

Our work seeks to explore the design of architectures for such assistive AI agents based on fundamental principles such as iterative refinement \(i\.e\., compositionality\), ecological rationality, and interactive learning, which can be traced back to the early pioneers of AI but are not being fully leveraged by modern AI methods\(sridharan2025back\)\. In particular, supporting a diverse set of representations and processing methods, and rapidly acquiring predictive models of behavior are essential capabilities for effective operation\(mccarthy:MI69\)\. Also, humans use heuristics based on core capacities \(e\.g\., to sum, order, imitate, recognize, forget, track\) for robust decisions*in the wild*, i\.e\., under uncertainty or intractability; the use of such heuristics has also led to good performance in many practical application domains\(gigerenzer:MMM16\)\. Our long\-term objective is to determine if and how these fundamental principles should be introduced in architectures for assistive agents, with modern AI methods being one of many different tools that these agents use\.

As a step toward our long\-term objective, the representative use case considered in this paper is that of an AI agent that has to resolve the ambiguity in the objects being referred to by the human user\. Consider such an agent that has been asked by a human to “place a fruit on the table” in a kitchen containing apples, oranges, and bananas, as shown in Figure[1](https://arxiv.org/html/2608.10330#S1.F1)\. Such ambiguous statements are common in human conversations, and we achieve disambiguation using prior and acquired knowledge of domain and the preferences and habits of the human\(s\) we are interacting with\. It is challenging for AI agents based on deep networks to achieve such personalized disambiguation rapidly\. Interactive clarification may help resolve this ambiguity, but answering many questions from the AI agent will be a cognitive burden for the human user\. Some hybrid frameworks combining data\-driven models and knowledge\-based reasoning have also been explored in an attempt to address ambiguity in task specification\(guan2023leveraging;lin2024clmasp\), but effective personalized interaction remains an open problem\.

![Refer to caption](https://arxiv.org/html/2608.10330v1/fig_motivating.png)Figure 1\.Architecture overview and illustrative example of an AI agent receiving an ambiguous request\. Our architecture enables the agent to reason with prior knowledge, learned compositional concepts, and contextual information modeled using three heuristic factors, to identify and score valid candidates, resolving ambiguous references or triggering clarification as needed\. Attribute labels in the concept\-mapping panel are shortened for display; the full attribute strings are*fruit is juicy*,*can be red colored*, and*has a soft interior*\.In the design of our architecture, we specifically focus on embedding the principle of hierarchical compositionality, and on the use of simple heuristics\. We also limit the compositional hierarchy to representing and leveraging information about domain objects in the form of physical and functional attributes at different \(linked\) abstractions\. Doing so has been shown to support fast and reliable learning and adaptation in new situations\. For example, knowledge of a user’s regular consumption of oranges in the afternoon can also be considered as a preference for small fruits and healthy options for an afternoon snack\. If the AI agent is then asked to fetch a snack but there are no oranges available, it can use prior knowledge to offer the user apples instead of chips\. Future work could extend the compositional representation to temporal events, representing \(say\) the observed movement of a dog in terms of changes in the parts of the dog and using this knowledge to generate plausible motion for a previously unseen four\-legged animal\. In the design and use of our compositional hierarchy, we embed simple heuristics based on some core capacities\. We then explore the impact of learning and leveraging such a hierarchy of domain objects on disambiguating commands provided by a human interacting with an AI agent\. Specifically, our architecture:

- •Represents entities as hierarchical compositions of atomic attributes drawn from human\-validated semantic feature norms and compoundconceptsautomatically mined from these attributes, with personalized preferences of human users learned from observations of their behavior\.
- •Computationally and heuristically models semantic compatibility, session salience, and user\-specific thematic preference as complementary factors that capture contextual information about domain objects and the human users’ interaction with them\.
- •Performs non\-monotonic logical reasoning with current knowledge of concept hierarchy, axioms governing domain dynamics, and computed model of contextual information to achieve disambiguation, requesting clarification from humans only when needed\.

As the baseline for comparison in our implementation, we use the GPT\-5\.1\(singh2025openai\)LLM\. We also use Answer Set Programming \(ASP\)\(gelfond2014knowledge\)as the non\-monotonic logical reasoning paradigm\. We illustrate and evaluate our architecture’s capabilities in the context of an assistive agent aiding humans with different user profiles in a realistic physics\-based simulated environment\. We use qualitative and quantitative results to demonstrate clear performance gains over state of the art LLM baseline\.

The remainder of the paper is organized as follows\. We first motivate the need for our architecture based on a review of related work \(Section[2](https://arxiv.org/html/2608.10330#S2)\)\. We then describe the problem formulation and our architecture \(Section[3](https://arxiv.org/html/2608.10330#S3)\)\. This is followed by a description of the experimental setup and results \(Section[4](https://arxiv.org/html/2608.10330#S4)\), and a discussion of conclusions and future work \(Section[6](https://arxiv.org/html/2608.10330#S6)\)\.

## 2\.Related Work

We begin by reviewing work in command disambiguation, user modeling, and compositional representations in the context of AI agents assisting humans\.

### 2\.1\.Command Disambiguation and Personalization

With the increasing use of FMs in Human\-Robot Interaction \(HRI\), there has been work on addressing ambiguity in such models\(tellex2020robots;yu2018mattnet;deng2021transvg;huang2022language;ahn2022can;hemanthage2024divide\)\. This includes work on matching referring expressions to visual regions\(yu2018mattnet;deng2021transvg;hemanthage2024divide\), prompting LLMs to classify command ambiguity and generate clarification questions\(park2023clara\), and converting the cross\-modal attention of a VLM into a spatial signal that detects referential ambiguity and triggers clarification\(abrini2026clue\)\. However, these methods rely solely on information encoded in the FMs, visual features extracted from the scene, or dialogue history, without modeling individual user preferences\. When multiple candidates satisfy the linguistic description, they lack sufficient user\-specific signals to rapidly identify the intended referent\.

Clarification\-based approaches address ambiguity by posing questions to the user, ranging from enumerating individual candidates\(shridhar2020ingress;hatori2018interactively\)to selecting discriminative attributes or semantically informative questions for more targeted queries\(thomason2019improving;jiang2024llms;dogan2025model\)\. We instead draw inspiration from a framework that leverages the interplay between knowledge\-based reasoning and data\-driven learning, reasoning with prior knowledge to eliminate unsuitable candidates by constructing clarification queries\(mota2021answer\)\. However, even this framework relies on substantial human supervision, and it does not have the hierarchical compositional representation of knowledge that forms a key component of our architecture\.

A common limitation of much of the existing work based on modern AI methods is that the disambiguation method does not build on user\-specific cues\. In practice, however, the same ambiguous command may refer to different entities for different users and in different contexts\. The reliance of LLMs on statistical associations learned from large datasets leads to poor performance when such personalization needs to be achieved quickly\(kambhampati2024position;xiong2024can\)\. Even recent benchmarks assume no access to user history, either confining dialogue and questions to the current scene within a single task episode\(gao2022dialfred;padmakumar2022teach\)or deliberately imposing a zero\-context setting in which every preference\-type ambiguity must be resolved by asking the user\(ivanova2025ambik\)\. The approaches that exploit user\-specific evidence either reuse the past referring expressions recorded for each object\(roy2019leveraging\)or retain explicitly stated preferences in an LLM memory\(abugurain2024integrating\)\. These strategies do improve disambiguation for entities with recorded history, but do not transfer the accumulated evidence across entities\. Our architecture enables the AI agent to reason and learn with models of user\-specific preferences that are linked to the learned compositional concepts, requesting clarification from the human user only when necessary\.

More broadly, personalization to individual users continues to be an open problem for robots and agents controlled by FMs\(wu2025aligning\)\. Existing approaches learn behavioral patterns from interaction history through explicit feedback signals\(christiano2017deep;wirth2017survey\)or implicit signals in the form of command history\(fischer2001user;fu2025combining\)\. Beyond command\-level interactions, research has shown that stable per\-user regularities can also be recovered from sequences of everyday behavior\. The smart\-home studies cluster recognized daily\-activity sequences and recovered recurring routines specific to each user\(sepesy2021discovering\)\. However, these routines were primarily mined for activity monitoring and anomaly detection rather than for interpreting subsequent user commands\. Other work has focused on anticipating future behavior from egocentric video, e\.g\., predicting the next object a person will interact with\(furnari2017next\)or recommending which assistive action should be invoked\(abreu2024parse\)\. These studies show that user histories contain predictive regularities, but they do not use them to resolve entity\-level ambiguity in user commands\. In addition, research has explored transferring user preferences across assembly tasks by learning from demonstrations in a simpler canonical task and adapting the acquired knowledge to more complex tasks\(nemlekar2023transfer\)\. In practical domains, high\-level tasks are often linked to the specific target entities to be manipulated\. However, there is a scarcity of data needed to acquire preferences at the \(low\) level of primitive actions and specific entities\. Our architecture addresses this limitation by acquiring a hierarchical compositional representation of objects, enabling transfer of knowledge across related entities from limited interaction history\.

### 2\.2\.Compositional Hierarchical Representations

Hierarchical compositional representations have been studied for many decades in the context of human cognition\(fodor:book75;knoblich:PS01;piantadosi:PR16\)and probabilistic sequential decision making\(dietterich:icml98\); they have also received an information\-theoretic grounding\(elmoznino:icml25\), and used in different domains\. In visual recognition, low\-level features were recursively composed into shared parts across object categories\(fidler2007towards\), and shared semantic properties enabled zero\-shot transfer to unseen categories\(lampert2013attribute\)\. In robotics, encoded and learned ontologies \(and knowledge graphs\) have been used for planning\(beetz:icra18\), kinematic primitives have been hierarchically composed into transferable motor representations\(vzabkar2016motor\)and motion primitives are composed through graph structures for novel tasks\(tian2024gsc\)\. This principle has also been applied to referring expressions in human\-robot interaction\(gao2023compositional\)and preference modeling\(go2023compositional\)\. Our work is directed at understanding if and how a compositional hierarchy can be acquired and used in architectures governing AI agents\. In this paper, we do so in the context of disambiguating user commands\.

Researchers focusing on modern AI methods based on deep networks have also explored compositionality\. Some studies argue that complex target functions contain reusable lower\-dimensional components that deep networks can exploit\(danhofer2025position\), while others show that deep networks can learn composed functions without assuming such an explicit decomposition in advance\(jacot2025how\)\. Other studies have examined when such structure is actually learned and exploited by deep models\. For example, prior work shows that deep networks can generalize compositionally when the data contains learnable structure, the model can align with this structure, and the training distribution covers the relevant combinations\(Cagnetta2024HowDeep;boopathy2024breaking;redhardt2026scaling\)\. However, these deep networks and LLMs may still rely on seen combinations or statistical shortcuts rather than systematically combining learned concepts\(lippl2025when\)\. They can often answer the individual sub\-questions of a task but fail when the task requires composing them in a specific manner, and this compositionality gap does not necessarily close as models are provided additional data\(Press2023Ofir\)\. Even when the final output is correct, mechanistic studies show that the model’s internal computation may not capture the intermediate results\(khandelwal2026how\)\.

A related line of research attempts to make the intermediate structure of deep network models more explicit\. For example, concept\-based models introduce a concept layer so that the deep network first predicts human\-interpretable concepts and then uses them for classification or prediction\(pmlr\-v119\-koh20a\)\. Neurosymbolic methods combine neural perception with logical or probabilistic inference to enable use of concepts or symbolic variables in downstream reasoning\(chen2025neural\)\. These methods often rely on predefined concepts and concept\-level annotations, shifting much of the modeling burden to humans and limiting adaptation to fine\-grained, user\-specific preferences in changing contexts\. Also, they are developed primarily for classification or prediction in \(near\)static conditions, rather than for rapid adaptation to specific users and dynamic environments\. Our approach lies between fully implicit neural representations and explicit symbolic representations\. The basic set of attributes is chosen from an open\-source dataset of human\-validated features, but the concept\-level structures above it are learned from attribute co\-occurrence and user interaction history, allowing our architecture to capture subtle preferences that are difficult to specify in advance\. These structures can be audited and revised, and be combined with action constraints, context, and history for resolving ambiguous instructions\.

### 2\.3\.Decision Heuristics

There is a rich history on the design and use of heuristics in different disciplines; they are often viewed as shortcuts used to find solutions quickly or as approximations of optimal solutions\. The early pioneers of AI and modern mathematics, on the other hand, drew inspiration from the use of heuristics by humans to propose strategies for solving intractable problems\(katsikopoulos:JMM26\)\. One example of a related computational theory is Herb Simon’s work onBounded Rationalityand heuristics\(simon:PR56\)\.

Over time, bounded rationality has been incorrectly equated with maximizing expected utility in many disciplines \(e\.g\., finance, computing\)\(russell:aibook03\)\. The corresponding methods focus on optimal search in the presence ofrisk, i\.e\., a search over a set of known states and outcomes\. This includes methods for logical reasoning and probabilistic reasoning, and the modern AI methods based on FMs and other deep networks\. Another interpretation of bounded rationality is to be found in theheuristics and biasesprogram in Psychology, where the use of heuristics by humans came to be viewed as leading to biases in their decisions\(tversky:SCIENCE74\), leading to a well\-known dual\-process theory\(kahneman:book11\)that has also been used extensively in AI\. However, subsequent research in Psychology and other disciplines has shown that these interpretations do not match the original definition of bounded rationality, have no basis in human decision making, and do not give us any useful information about the process of making decisions\(gigerenzer:MMM16\)\.

Our work builds on the principle ofecological rationality, which is based on the original \(i\.e\., Herb Simon’s\) definition of bounded rationality and heuristics\(gigerenzer:bookchap20\)\. The focus here is on decision making underopen world uncertaintyor ”in the wild”, i\.e\., when the space of possible states and outcomes is not known in advance\. In such circumstances, the optimal solution is ill\-defined and probabilistic estimates may cease to be meaningful\. The behavior of a human or an AI system is characterized as a joint function of the internal cognitive processes and the environment, andadaptive satisficingis used as the guiding principle to designsimple heuristicssuch as tallying, lexicographic search, and fast and frugal \(FF\) trees in order to rapidly learn \(and revise\) predictive models and make rational decisions\. Instead of hacks or biases, heuristics are viewed as leveraging core \(human\) capacities such as counting, ordering, tracking, and collaborating, in order to ignore part of the information and make decisions more quickly, frugally, and accurately than more complex methods\(gigerenzer:MMM16\)\.

Modern AI methods and heuristics can be understood in terms of thebias\-variance tradeoff\(geman\)\. The AI methods are based on the belief that a large number of free parameters are necessary to generalize to different situation\. These parameters can lead these methods to overfit, resulting in lower bias but much high variance\. Heuristics, on the other hand, are based on limited free parameters\. They may lead to higher bias but still have lower overall error because of the much lower variance compared wtih modern AI methods\. Also, modern AI methods are largelyprescriptive, i\.e\., they describe what*should*be done in specific situations or to achieve specific outcomes\. Decision heuristics are both prescriptive anddescriptive, i\.e\., they are designed to also capture what people or agents do to achieve specific outcomes\. The descriptive design of these heuristics also supports the automatic generation of process\-level description as explanations of the decisions made\. In the long\-term, our work seeks to develop an adaptive toolbox of classes of such heuristics, FM, and other methods\. Given any particular problem, we can then match the characteristics of the problem with those of specific classes of tools, along with an algorithmic approach and out\-of\-distribution testing, to identify thetoolsuitable for the problem\.

Although decision heuristics have provided good performance on prediction problems in application domains such as finance, healthcare, and law\(brighton:ER12;durbach:DSS20;gigerenzer:MMM16;katsikopoulos:IJF21\), there is limited use of these methods in architectures for robots or AI agents, except in some related work in the cognitive systems community\(langley:acs22\)and prior work in our group\(dodampegama:FAI26\)\. This lack of attention to decision heuristics is potentially because their simplicity makes researchers doubt their suitability for addressing complex practical problems\. In this work, we use heuristic methods to aid in the design and use of our compositional hierarchy\.

![Refer to caption](https://arxiv.org/html/2608.10330v1/framework_2.png)Figure 2\.Three\-stage disambiguation framework\.Stage 1:The agent parses the input user command and applies ASP\-based feasibility filtering using domain knowledge \(Σ\\Sigma,Π\\Pi\) to obtain a candidate set𝒞t\\mathcal\{C\}\_\{t\}\.Stage 2:If\|𝒞t\|\>1\|\\mathcal\{C\}\_\{t\}\|\>1, candidates are scored by fusing semantic similarity \(Ss​e​mS\_\{sem\}\), session salience \(Ss​a​lS\_\{sal\}\), and user\-specific thematic preference \(St​h​e​mS\_\{them\}\)\.Stage 3:The agent selects the top candidate if the lead ratio exceeds thresholdθ\\theta, or invokes human clarification otherwise\. Each resolved interaction updates the user history, enabling continuous refinement of learned user preferences\.

## 3\.Problem Formulation and Architecture

Consider an embodied AI agent assisting a human in a home environment to perform tasks\{τ1,τ2,…,τn\}\\\{\\tau\_\{1\},\\tau\_\{2\},\\dots,\\tau\_\{n\}\\\}assigned sequentially\. A contiguous subsequence of thematically related tasks forms a*session*\(e\.g\., a sequence of commands constituting a routine for preparing for afternoon tea\)\. Session boundaries define the scope of short\-term contextual signals, e\.g\., salience evidence accumulates within a session and resets at each boundary\. Each taskτi\\tau\_\{i\}is executed as a sequence of fine\-grained actions such as*pickup*,*bring*, and*open*\. Disambiguation is required when: \(a\) the command contains an under\-specified reference, such as a category term \(e\.g\., “the drink”\) or a pronoun \(e\.g\., “it”\), that is compatible with multiple entities in the current state; or \(b\) the command is incomplete \(e\.g\., ”please fetch the …”\) because the human input was not processed properly\.

Figure[2](https://arxiv.org/html/2608.10330#S2.F2)outlines the flow of information and control in our architecture, organized in the form of three stages split over two phases\. In the first stage, the agent first parses any given command to construct a candidate set𝒞\\mathcal\{C\}of feasible interpretations of the input command\. This involves ASP\-based non\-monotonic logical reasoning \(Section[3\.1](https://arxiv.org/html/2608.10330#S3.SS1)\) with any prior knowledge \(if available\) of domain state and constraints to filter infeasible candidates\. If there are multiple feasible candidates, they are then ranked in the second stage using three heuristically\-modeled complementary signals \(Section[3\.3](https://arxiv.org/html/2608.10330#S3.SS3)\): \(i\) user\-independent semantic compatibility derived from WordNet\(miller1995wordnet\)and recent execution outcomes; \(ii\) session\-level salience based on recent references within the current session; and \(iii\) user\-specific thematic preference estimated from interaction history based on a hierarchical compositional representation \(Section[3\.2](https://arxiv.org/html/2608.10330#S3.SS2)\)\. In the third stage, if there is no clear candidate identified to resolve the ambiguity, the agent solicits clarification from a human\. Once the agent identifies a specific interpretation of the ambiguous command, ASP\-based reasoning is used to compute and execute a task plan that executes the user command\. This trace of interaction and observations is also added to the user\-specific history, enabling subsequent revision of the learned user preferences\. We now discuss the individual components of the architecture, along with the interplay between them, starting with the knowledge representation and reasoning component\.

### 3\.1\.Knowledge Representation and Reasoning with ASP

In our architecture, any domain’s transition diagram is represented using action language𝒜​ℒd\\mathcal\{AL\}\_\{d\}\(gelfond2013system\)\. Action languages are formal models of parts of natural language for describing prior knowledge about a domain’s transition diagrams\. The domain representation comprises a system description𝒟\\mathcal\{D\}, a collection of statements of𝒜​ℒd\\mathcal\{AL\}\_\{d\}, and a historyℋ\\mathcal\{H\}\.𝒟\\mathcal\{D\}has a sorted signatureΣ\\Sigmawith basic sorts, and the domain attributes \(statics and fluents\) and actions are described in terms of these basic sorts\.

Our domain includes basic sorts such asr​o​o​mroom,c​o​n​t​a​i​n​e​rcontainer,f​u​r​n​i​t​u​r​efurniture,f​o​o​dfood,h​u​m​a​nhuman, ands​t​e​pstep\(for temporal reasoning\) that are arranged hierarchically, e\.g\.,f​r​u​i​tfruitis a sub\-sort off​o​o​dfoodthat is a sub\-sort ofi​t​e​mitem\. Statics are domain attributes whose values cannot change, e\.g\.,n​e​x​t​\_​t​o​\(k​i​t​c​h​e​n,s​t​u​d​y\)next\\\_to\(kitchen,study\), and fluents are attributes whose values can change\. Fluents can be*inertial*, which obey inertia laws and are changed by actions, e\.g\.,l​o​c​\(i​t​e​m,r​o​o​m\)loc\(item,room\)andi​n​\_​h​a​n​d​\(a​g​e​n​t,i​t​e​m\)in\\\_hand\(agent,item\); and*defined*, which do not obey inertia laws and are not directly changed by the agent’s actions, e\.g\.,o​t​h​e​r​l​o​c​\(h​u​m​a​n,r​o​o​m\)otherloc\(human,room\)is the human’s location\. Actions include the agent’s actions, e\.g\.,m​o​v​e​\(a​g​e​n​t,r​o​o​m\)move\(agent,room\),p​i​c​k​u​p​\(a​g​e​n​t,i​t​e​m\)pickup\(agent,item\), ands​w​i​t​c​h​o​n​\(a​g​e​n​t,a​p​p​l​i​a​n​c​e\)switchon\(agent,appliance\)\.

For any given domain, prior knowledge of the domain’s dynamics is described based on the correspondingΣ\\Sigmausing three types of axioms:

\(1a\)m​o​v​e​\(A,R\)\\displaystyle move\(A,R\)𝐜𝐚𝐮𝐬𝐞𝐬​l​o​c​\(A,R\)\\displaystyle~\\mathbf\{causes\}~loc\(A,R\)\(1b\)¬a​t​\(A,R1\)\\displaystyle\\neg\\,at\(A,R\_\{1\}\)𝐢𝐟​a​t​\(A,R2\),R1≠R2\\displaystyle~\\mathbf\{if\}~at\(A,R\_\{2\}\),R\_\{1\}\\neq R\_\{2\}\(1c\)𝐢𝐦𝐩𝐨𝐬𝐬𝐢𝐛𝐥𝐞​g​i​v​e​\(A,O,U\)\\displaystyle\\mathbf\{impossible\}~give\(A,O,U\)~𝐢𝐟​l​o​c​\(A,R1\),o​t​h​e​r​l​o​c​\(U,R2\),R1≠R2\\displaystyle\\mathbf\{if\}~loc\(A,R\_\{1\}\),~otherloc\(U,R\_\{2\}\),R\_\{1\}\\neq R\_\{2\}where Statement[1](https://arxiv.org/html/2608.10330#S3.E1)\(a\), acausal law, implies that when an agent \(AA\) moves to a roomRR, its location becomesRR; Statement[1](https://arxiv.org/html/2608.10330#S3.E1)\(b\), astate constraint, implies that an agent \(AA\) cannot be in two places \(R1,R2R\_\{1\},R\_\{2\}\) at the same time; and Statement[1](https://arxiv.org/html/2608.10330#S3.E1)\(c\), anexecutability condition, prevents the agent \(AA\) from trying to give an object \(OO\) to a human user \(UU\) who is not in the same room as the agent\. In addition to such a system description𝒟\\mathcal\{D\}, the historyℋ\\mathcal\{H\}is a record of statements of the formo​b​s​\(f​l​u​e​n​t,b​o​o​l​e​a​n,s​t​e​p\)obs\(fluent,boolean,step\), which represent observations received at specific time steps, and of the formh​p​d​\(a​c​t​i​o​n,s​t​e​p\)hpd\(action,step\), which represent actions executed at specific time steps\.

In our architecture, ASP\-based non\-monotonic logical reasoning is used for inference, planning and diagnosis\. To perform such reasoning with knowledge, an ASP programΠ​\(𝒟,ℋ\)\\Pi\(\\mathcal\{D\},\\mathcal\{H\}\)is automatically constructed \(using a Python script\) to include statements from𝒟\\mathcal\{D\}andℋ\\mathcal\{H\}, inertia axioms, reality check axioms, closed world assumptions for defined fluents and actions, and helper relations to reason over time steps, e\.g\.,h​o​l​d​s​\(f​l​u​e​n​t,s​t​e​p\)holds\(fluent,step\)ando​c​c​u​r​s​\(a​c​t​i​o​n,s​t​e​p\)occurs\(action,step\)imply \(respectively\) that a fluent is true and that an action is part of a plan at a particular time step\. For planning and diagnosis \(in Stage 3\),Π​\(𝒟,ℋ\)\\Pi\(\\mathcal\{D\},\\mathcal\{H\}\)includes helper axioms to define goals and guide the search for plans or explanations \(diagnosis\)\. After executing each action in the plan, the correspondingh​p​d​\(a​c​t​i​o​n,s​t​e​p\)hpd\(action,step\)and any resulting observations of the formo​b​s​\(f​l​u​e​n​t,b​o​o​l​e​a​n,s​t​e​p\)obs\(fluent,boolean,step\)are appended toℋ\\mathcal\{H\}for subsequent reasoning\. Similarly, for inference and filtering of infeasible candidate interpretations of the user command \(in Stage 1\), the agent queriesΠ​\(𝒟,ℋ\)\\Pi\(\\mathcal\{D\},\\mathcal\{H\}\)to mentally simulate the execution of the command and compare the fluent values between current state and the resultant state\. Any entityo∈𝒪o\\in\\mathcal\{O\}that appears as an argument of a fluent literal whose value has changed between the two states is identified and collected in a setOtΔO^\{\\Delta\}\_\{t\}at timett\. This set serves as contextual input for subsequent disambiguation \(as described in Sections[3\.2](https://arxiv.org/html/2608.10330#S3.SS2)–[3\.3](https://arxiv.org/html/2608.10330#S3.SS3)\)\.

ASP is based on stable model semantics, and encodes*default negation*and*epistemic disjunction*; unlike classical negation “¬p\\lnot p”, which states that*p is believed to be false*, default negation “n​o​t​pnot~p” only implies that*p is not believed to be true*\. Each literal is true, false, or unknown, and the agent only believes that which it is forced to believe\. ASP supports non\-monotonic reasoning, i\.e\., the ability to revise previously held conclusions, which is essential for agents operating with incomplete knowledge and noisy observations\. All reasoning tasks are reduced to computinganswer setsofΠ\\Pi, and we use the SPARC system\(balai2013towards\)to compute these answer sets\. For example programs corresponding to our paper, please see\(code\-results\)\.

![Refer to caption](https://arxiv.org/html/2608.10330v1/compositional_hierarchy.png)Figure 3\.Overview of the three\-layered hierarchical and compositional representation\.ℒ0\\mathcal\{L\}\_\{0\}andℒ1\\mathcal\{L\}\_\{1\}are user\-independent and domain\-specific atomic attributes and concepts \(respectively\) that are constructed automatically before deployment and can be revised over time, with attributes drawn from human\-validated semantic feature norms and concepts mined from significant attribute combinations;ℒ2\\mathcal\{L\}\_\{2\}captures user\-specific workflow patterns from interaction snapshots conditioned by the action predicate of the current command\. This figure illustrates two example snapshots for*switch on*, with the patternc93→c21c\_\{93\}\\rightarrow c\_\{21\}indicating that this user tends to interact with entities that emit light \(e\.g\., light bulbs\) after interacting with printed reading material, e\.g\., book or newspaper\.
### 3\.2\.Compositional Hierarchy of Attributes and Concepts

While ASP filtering removes candidates violating known domain constraints, multiple feasible interpretations of the command often remain; these are based on the different objects \(i\.e\., entities\) that the ambiguous \(or missing\) part of the command may be referring to\. Ranking these feasible options requires estimating user preferences but the historyℋ\\mathcal\{H\}of interactions with the specific user is limited\. In an attempt to address the scarcity of data needed to resolve ambiguities in complex domains, our architecture includes a three\-layered hierarchical and compositional representation of objects in terms of their physical \(appearance, functional\) attributes:

- •ℒ0\\mathcal\{L\}\_\{0\}: domain objects \(entities\) are automatically represented in terms of fine\-grained, shared, atomic attributes drawn from NOVA, an open\-source semantic feature norm dataset\(suresh2025nova\);
- •ℒ1\\mathcal\{L\}\_\{1\}: statistically significant attribute combinations are identified automatically to form a domain\-level concept library𝒦\\mathcal\{K\}, with entities then being represented in terms of elements of this concept library; and
- •ℒ2\\mathcal\{L\}\_\{2\}: entities are further represented in terms of user\-specific workflow patterns and concept co\-occurrences from records inℋ\\mathcal\{H\}that correspond to actions \(i\.e\.,h​p​dhpdliterals\) and match the current command\.

Note thatℒ0\\mathcal\{L\}\_\{0\}andℒ1\\mathcal\{L\}\_\{1\}are user\-independent and domain\-specific; they are constructed before deployment\. Also, these two levels can be revised quickly over time but we keep them fixed for the work described in this paper\.ℒ2\\mathcal\{L\}\_\{2\}, on the other hand, is user\-dependent; it is constructed by gathering evidence dynamically, with each user maintaining a separate pattern set for each action \(please see Figure[3](https://arxiv.org/html/2608.10330#S3.F3)for examples\)\. We describe each layer \(i\.e\., level\) of the compositional hierarchy below\.

#### ℒ𝟎\\mathbf\{\\mathcal\{L\}\_\{0\}\}: Atomic Attributes\.

Let𝒪\\mathcal\{O\}be the set of all entities in the domain\. Each entityo∈𝒪o\\in\\mathcal\{O\}is represented as a set of atomic attributesA​\(o\)=\{a1,a2,…,am\}A\(o\)=\\\{a\_\{1\},a\_\{2\},\\dots,a\_\{m\}\\\}, capturing the properties that people regularly judge the entity to possess\. These attributes are drawn from*semantic feature norms*in cognitive science, i\.e\., features collected by asking human participants to describe a concept’s characteristic properties\(rosch1975;mcrae2005\)\. We use NOVA\(suresh2025nova\), a publicly\-available dataset of human\-validated feature norms covering787787everyday concepts\. These attributes describe objects as humans perceive and name them, e\.g\.,*emits\_light*for a lamp and*has\_printed\_text\_on\_the\_pages*for a book\. Evidence can thus transfer across entities that share attributes that would be considered \(by a human\) to indicate similarity\.

Attributes that are identified for only one entity are excluded, as they cannot contribute to cross\-entity transfer of knowledge\. The union𝒜=⋃o∈𝒪A​\(o\)\\mathcal\{A\}=\\bigcup\_\{o\\in\\mathcal\{O\}\}A\(o\)then defines the full vocabulary of atomic attributes for the domain\. At the end of this process, the attributes inℒ0\\mathcal\{L\}\_\{0\}are fine\-grained and shared, e\.g\.,*emits\_light*,*contains\_lightbulbs*, and*can\_be\_lit\_for\_light*appear across the lamp and the chandelier, while*has\_printed\_text\_on\_the\_pages*,*has\_printing\_on\_it*, and*can\_be\_carried\_in\_one\_hand*appear across the book and the newspaper\. However, these shared attributes lack the selectivity required to isolate specific functional preferences, motivating the discovery of higher\-level \(composite\) attributes\.

#### ℒ𝟏\\mathbf\{\\mathcal\{L\}\_\{1\}\}: Domain\-level Attributes\.

The next level in our hierarchy of attributes \(ℒ1\\mathcal\{L\}\_\{1\}\) is a library𝒦\\mathcal\{K\}of statistically significant composite attributes, which we refer to asconcepts\. In our architecture, the process for identifying these concepts starts by considering pairs of attributes\{ai,aj\}⊆A​\(o\)\\\{a\_\{i\},a\_\{j\}\\\}\\subseteq A\(o\), whereaia\_\{i\}andaja\_\{j\}may belong to the same or different dimensions, and retains pairs \(i\.e\., compositions\) that satisfy two heuristically\-modeled criteria:

1. \(1\)Significance: this criterion seeks to identify attributes whose joint occurrence is more significant that the occurrence of the individual attributes\. To do so, we first focus on attribute pairs and adapt the*lift*measure defined in the data mining literature\(brin1997dynamic\): \(2\)lift​\(ai,aj\)=Pr⁡\(ai,aj\)/\(Pr⁡\(ai\)⋅Pr⁡\(aj\)\)≥λ1\\displaystyle\\mathrm\{lift\}\(a\_\{i\},a\_\{j\}\)=\\Pr\(a\_\{i\},a\_\{j\}\)\\,/\\,\(\\Pr\(a\_\{i\}\)\\cdot\\Pr\(a\_\{j\}\)\)\\geq\\lambda\_\{1\}whereλ1\\lambda\_\{1\}is an experimentally\-determined satisficing threshold above which the joint occurrence of the attributes being examined is considered to be significant\.
2. \(2\)Frequency: this criterion seeks to identify attribute pairs that have support from a minimum number of entities in the domain\. It is computed in the form of a simple count of objects \(with the attribute pair under consideration\) exceeding a threshold: \(3\)\|\{o∈𝒪∣\{ai,aj\}⊆A​\(o\)\}\|\>θ\\displaystyle\|\\\{o\\in\\mathcal\{O\}\\mid\\\{a\_\{i\},a\_\{j\}\\\}\\subseteq A\(o\)\\\}\|\>\\thetawhereθ\\thetais an experimentally\-determined threshold\. Together, these two criteria help identify attribute pairs shared across entities\.

Next, the attribute pairs that pass both criteria are incrementally extended by considering additional attributes, with each such combination re\-evaluated using the same criteria, conditioned on the already\-confirmed subset\. For example,lift​\(ak∣\{ai,aj\}\)=Pr⁡\(ai,aj,ak\)/\(Pr⁡\(ai,aj\)⋅Pr⁡\(ak\)\)≥λ1\\mathrm\{lift\}\(a\_\{k\}\\mid\\\{a\_\{i\},a\_\{j\}\\\}\)=\\Pr\(a\_\{i\},a\_\{j\},a\_\{k\}\)\\,/\\,\(\\Pr\(a\_\{i\},a\_\{j\}\)\\cdot\\Pr\(a\_\{k\}\)\)\\geq\\lambda\_\{1\}\. This process continues until no further attribute passes both criteria or a predefined maximum size is reached for the concept\. In our experiments, we set this upper bound on the size of each concept to five to limit computational cost while also identifying useful concepts\. This yields concepts of varying sizes \(e\.g\., from pairs such as\{has\_printing\_on\_it, has\_printed\_text\_on\_the\_pages\}\\\{\\textit\{has\\\_printing\\\_on\\\_it, has\\\_printed\\\_text\\\_on\\\_the\\\_pages\}\\\}and triples such as\{has\_printing\_on\_it, has\_printed\_text\_on\_the\_pages, has\_a\_cover\}\\\{\\textit\{has\\\_printing\\\_on\\\_it, has\\\_printed\\\_text\\\_on\\\_the\\\_pages, has\\\_a\\\_cover\}\\\}\), each representing a more specific archetype to be considered for subsequent analysis\.

In practice, a single entity is often represented by multiple concepts in𝒦\\mathcal\{K\}, providing multiple pathways for evidence to transfer across structurally related entities\. Conversely, attributes that do not participate in any retained combination are not covered by𝒦\\mathcal\{K\}\. The final set𝒦\\mathcal\{K\}is a domain\-specific, user\- and category\-independent set of concepts\. As stated earlier, although this set can be revised over time, we decided to keep it static for the work in this paper\. In order to capture user\-specific habits, the compositional hierarchy is extended toℒ2\\mathcal\{L\}\_\{2\}by identifying workflow dependencies between these concepts\.

#### ℒ𝟐\\mathbf\{\\mathcal\{L\}\_\{2\}\}: User\-specific attributes\.

The next level of the concept hierarchy seeks to identify user\-specifichabitsby applying statistics to mine the individual user’s history of interacting with the agent\. Specifically, for any given user, the agent accessesℋu\\mathcal\{H\}\_\{u\}, the history maintained separately for that user\. It then retains only those records \(i\.e\.,h​p​dhpdliterals\) corresponding to an action that is the same as the current command\. For instance, if the current command is*switch\_on*, only records of*switch\_on*are retained from the history of interactions with this user, while records of other commands are discarded\. This process ends up identifying a contextually matched subset of the history\. Each record in this subset is then modeled as an*interaction snapshot*, which is made up of:

- •*Trigger concepts*: for each entityo∈𝒪o\\in\\mathcal\{O\}, letΓ​\(o\)⊆𝒞\\Gamma\(o\)\\subseteq\\mathcal\{C\}denote the set ofℒ1\\mathcal\{L\}\_\{1\}concepts describingoo, i\.e\., those concepts whose constituent attributes are all contained inoo’s attribute setA​\(o\)A\(o\)\. We can then formally describe the trigger concept set as: \(4\)⋃o∈Ot−1ΔΓ​\(o\),withΓ​\(o\)=\{k∈𝒞∣k⊆A​\(o\)\}\\displaystyle\\bigcup\_\{o\\in O^\{\\Delta\}\_\{t\-1\}\}\\Gamma\(o\),\\quad\\mathrm\{with\}\\quad\\Gamma\(o\)=\\\{k\\in\\mathcal\{C\}\\mid k\\subseteq A\(o\)\\\}whereOt−1ΔO^\{\\Delta\}\_\{t\-1\}is the set of entities that appear as an argument of a fluent literal whose value changed by the action executed by the agent at time\(t−1\)\(t\-1\); and
- •*Target concepts*: this is the set ofℒ1\\mathcal\{L\}\_\{1\}conceptsΓ​\(gt\)\\Gamma\(g\_\{t\}\)describing the entitygtg\_\{t\}\(ground target object\) that the agent acted on at stepttin the historical record under consideration\.

These snapshots capture the dependency between immediate world\-state transitions and the subsequent user selections\. Across all snapshots, the agent enumerates trigger\-target concept pairs and retains only those whose co\-occurrence is both frequent and statistically significant, applying the same ’lift’ criterion with a relaxed thresholdλ2\\lambda\_\{2\}to accommodate the smaller sample size \(i\.e\., limited number of examples\) for the current user\. The retained pairs form the user\-specific workflow pattern setPuP\_\{u\}\.

For example, in the context of Figure[3](https://arxiv.org/html/2608.10330#S3.F3), suppose a user’s interaction historyℋ\\mathcal\{H\}contains multiple records of the execution of the action*switch\_on*, and suppose that multiple entities in the domain can be switched on, e\.g\., the lamp, the chandelier, and the television\. In one record, the preceding instruction “Put the book on the table” changed the state of the book, the table, and the bookshelf, and the union of the corresponding concepts \(e\.g\.,c93​\{has\_printing\_on\_it, has\_printed\_text\_on\_the\_pages, has\_a\_cover\},…c\_\{93\}\\\{\\textit\{has\\\_printing\\\_on\\\_it, has\\\_printed\\\_text\\\_on\\\_the\\\_pages, has\\\_a\\\_cover\}\\\},\\ldots\) forms the trigger set\. The user then interacted with the lamp, whose concepts \(e\.g\.,c21​\{emits\_light, contains\_lightbulbs, can\_be\_lit\_for\_light\},…c\_\{21\}\\\{\\textit\{emits\\\_light, contains\\\_lightbulbs, can\\\_be\\\_lit\\\_for\\\_light\}\\\},\\ldots\) form the target set\. In another record, the trigger set derived from “Bring me the newspaper” again includesc93c\_\{93\}, and the selected chandelier again includesc21c\_\{21\}\. Across snapshots, the pair of concepts\(c93,c21\)\(c\_\{93\},c\_\{21\}\)co\-occurs significantly more than chance \(Pr⁡\(c93,c21\)≫Pr⁡\(c93\)⋅Pr⁡\(c21\)\\Pr\(c\_\{93\},c\_\{21\}\)\\gg\\Pr\(c\_\{93\}\)\\cdot\\Pr\(c\_\{21\}\)\) and is retained inPuP\_\{u\}; other pairs such as\(c72,c21\)\(c\_\{72\},c\_\{21\}\)are not found to be very significant and are thus discarded\.

### 3\.3\.Ranking Ambiguous Candidates and Disambiguation

The key part of Stage\-2 of the pipeline of our architecture \(in Figure[2](https://arxiv.org/html/2608.10330#S2.F2)\) is the filtering and ranking of candidate interpretations𝒞t\\mathcal\{C\}\_\{t\}of the input user command\. Recall that there are multiple interpretations when it is found that more than one domain object matches the under\-specified reference in the input user command\. Our architecture uses an approach comprising three steps for this filtering and ranking; the first step filters infeasible candidates based on prior domain knowledge, while the next two steps rank the remaining candidates\. The ranking uses a computational model of contextual information that considers three complementary factors: semantic compatibility, session salience, and user\-specific thematic preference, with only the last factor using the compositional hierarchy of concepts \(ℒ1\\mathcal\{L\}\_\{1\}andℒ2\\mathcal\{L\}\_\{2\}in Section[3\.2](https://arxiv.org/html/2608.10330#S3.SS2)\)

#### Step 1: Filtering\.

As outlined in Section[3\.1](https://arxiv.org/html/2608.10330#S3.SS1), the first step identifies candidates violating sort constraints inΣ\\Sigmaor executability conditions inΠ​\(𝒟,ℋ\)\\Pi\(\\mathcal\{D\},\\mathcal\{H\}\)\. These are filtered, i\.e\., removed from further consideration, because they are not valid choices given in the given domain\. After this filtering, if\|𝒞t\|=0\|\\mathcal\{C\}\_\{t\}\|=0, the agent requests clarification from the human; if\|𝒞t\|=1\|\\mathcal\{C\}\_\{t\}\|=1, this single candidate is selected immediately as the unambiguous interpretation and used for subsequent steps \(e\.g\., planning and plan execution\)\.

#### Step 2: Semantic similarity and salience\.

When\|𝒞t\|\>1\|\\mathcal\{C\}\_\{t\}\|\>1after Step 1 above, each candidatex∈𝒞tx\\in\\mathcal\{C\}\_\{t\}is first scored by heuristic measures of semantic similarity and salience that consider recent contextual information but not the learned compositional hierarchy of attributes, i\.e\., information inℒ1\\mathcal\{L\}\_\{1\}andℒ2\\mathcal\{L\}\_\{2\}are not used\. Specifically:

- •Semantic similarity\(Ss​e​mS\_\{sem\}\) measures how well each candidate fits the context established by the preceding action, using a WordNet\-based Wu–Palmer similarity score in\(0,1\]\(0,1\]\. To model this contextual information, the agent uses predefined associations between entities associated with the preceding action and the abstract context categories \(e\.g\.,*dining*,*reading*,*relaxing*\), weighted by association strength\. Summing these weights across the entities inOt−1ΔO^\{\\Delta\}\_\{t\-1\}and taking the highest\-scoring category yields a single context keywordwc​t​xw\_\{ctx\}based on the preceding action\. The Wu–Palmer similarity between each candidatex∈𝒞tx\\in\\mathcal\{C\}\_\{t\}andwc​t​xw\_\{ctx\}in WordNet then gives a score per candidate, which is normalized across𝒞t\\mathcal\{C\}\_\{t\}to sum to one; and
- •Salience\(Ss​a​lS\_\{sal\}\) measures how frequently each candidate has been the focus of attention in the recent past, within the current session\. Specifically, each candidatex∈𝒞tx\\in\\mathcal\{C\}\_\{t\}accumulates geometrically decayed weights over the recent action history, with more recent occurrences contributing more, and the resulting scores are normalized across𝒞t\\mathcal\{C\}\_\{t\}to sum to one\.

Let us consider, again, the example provided after the description of levelℒ2\\mathcal\{L\}\_\{2\}in Section[3\.2](https://arxiv.org/html/2608.10330#S3.SS2), in which the instruction “Put the book on the table” has just been executed\. Now suppose the user issues an under\-specified command “switch it on,” and Step 1 \(above\) leaves three candidates:*lamp*,*microwave*, and*television*\. Since the preceding action involvesbook\-likeentities predominantly associated with the*reading*context,wc​t​x=*reading*w\_\{ctx\}=\\emph\{reading\}is identified as the single context keyword\. Then, since*lamp*is found to be closest to this context \(*reading*\), it has the higherSs​e​mS\_\{sem\}score among the three candidates\. Independently, theSs​a​lS\_\{sal\}measure favors entities most recently referenced in the current session; assuming that the lamp was referenced in the recent past, we once again have*lamp*receive the highestSs​a​lS\_\{sal\}value among the three candidates\. Based on both measures,*lamp*is ranked first among the three candidates based on the domain\-specific measures\.

#### Step 3: User\-specific score\.

Human users interacting with the AI agent often exhibit distinct behavioral patterns\. The thematic measureSt​h​e​mS\_\{them\}estimates personalized preferences using the compositional concept hierarchy representing each object and relevant records from the history of this user’s interaction with the AI agent\. These records are identified using the filtering process described in Section[3\.2](https://arxiv.org/html/2608.10330#S3.SS2)\. It is a combination of two heuristic measures that use the hierarchy of concepts:

- •Concept evidence\(Sc​o​n​c​e​p​tS\_\{concept\}\) This gathers evidence of each candidate being the desired object, based on eachℒ1\\mathcal\{L\}\_\{1\}concept of the candidate and the user’s recent interaction with objects with similar concepts using the actionaaunder consideration\. Specifically, for a candidatexx, the agent starts with a score of 0 and identifies each of its conceptsc∈Γ​\(x\)c\\in\\Gamma\(x\)\. It also considers all past records of actionaain the user’s history \(ℋu\\mathcal\{H\}\_\{u\}\), and computes the proportion of these records that involve the conceptcc\. This proportion is added to the score of candidatexx, and the process is repeated for all entries inΓ​\(x\)\\Gamma\(x\)\. For example, if the user has habitually*eaten*apples, apple\-related concepts \(e\.g\.,\{sweet, crisp, can be red colored, …\}\\\{\\textit\{sweet, crisp, can be red colored, \.\.\.\}\\\}\) will dominate the past*eat*actions, so a candidate sharing more of them receives a higherSc​o​n​c​e​p​tS\_\{concept\}in the context of the eat action\. Since this signal can cease to be useful when the same action is applied equally to different types of entities \(e\.g\.,*switch\_on*for lamp, microwave, television\), we also consider the measure below\.
- •Pattern evidence\(Sp​a​t​t​e​r​nS\_\{pattern\}\) This gathers evidence in support of each candidatexxbased on set ofℒ2\\mathcal\{L\}\_\{2\}concepts for this user \(i\.e\.,PuP\_\{u\}\) \(see Section[3\.2](https://arxiv.org/html/2608.10330#S3.SS2)\)\. For each candidatexx, the agent starts with a score of 0 and accumulated evidence based on each patternp​t∈Pupt\\in P\_\{u\}whose target conceptct​a​r​g​e​tc\_\{target\}belongs toΓ​\(x\)\\Gamma\(x\)and whose trigger conceptct​r​i​g​g​e​rc\_\{trigger\}is part of the previous command\. This evidence is the product of two factors:*follow rate*and*lift*, wherelift​\(ct​r​i​g​g​e​r,ct​a​r​g​e​t\)\\mathrm\{lift\}\(c\_\{trigger\},c\_\{target\}\)is computed as in Equation[2](https://arxiv.org/html/2608.10330#S3.E2), with probabilitiesPr⁡\(ct​r​i​g​g​e​r,ct​a​r​g​e​t\)\\Pr\(c\_\{trigger\},c\_\{target\}\),Pr⁡\(ct​r​i​g​g​e​r\)\\Pr\(c\_\{trigger\}\)andPr⁡\(ct​a​r​g​e​t\)\\Pr\(c\_\{target\}\)estimated from this user’s historyℋu\\mathcal\{H\}\_\{u\}\. The follow rate considers past records \(for this user\) in which the trigger concept appeared at stept−1t\-1, and computes the fraction of these records in which the target concept appeared at steptt\. For instance, if a book\-like concept appeared at stept−1t\-120 times \(in this user’s history\) and was followed by a lamp\-like concept at steptt12 times, the follow rate is12/20=0\.612/20=0\.6\. Follow rate, by itself, can overemphasize the evidence from common targets, but lift \(computed as above\) can substantially emphasize small\-sample coincidences\. So the net evidence accumulated bySp​a​t​t​e​r​nS\_\{pattern\}is the product of follow rate and*lift*\. Continuing with our example, when book\-like and table\-like concepts both appear inOt−1ΔO^\{\\Delta\}\_\{t\-1\}, the agent looks upPuP\_\{u\}and finds two matched patterns, both with a lamp\-like target concept and evidence \(computed as described above\) of0\.550\.55and0\.300\.30respectively\.Γ​\(x\)\\Gamma\(x\)for the lamp candidate contains the lamp\-like concept, the target of both patterns, so the candidate accumulates0\.55\+0\.30=0\.850\.55\+0\.30=0\.85\.Γ​\(x\)\\Gamma\(x\)for the microwave and television candidates does not contain this concept, so they accumulate0\.

The thematic score then combines the individual scores:

\(5\)St​h​e​m​\(x\)=Sc​o​n​c​e​p​t​\(x\)\+Sp​a​t​t​e​r​n​\(x\)\.S\_\{them\}\(x\)=S\_\{concept\}\(x\)\+S\_\{pattern\}\(x\)\.

#### Score Fusion and Selection\.

Ss​e​mS\_\{sem\}is a similarity measure, whereasSs​a​lS\_\{sal\}andSt​h​e​mS\_\{them\}are essentially counts \(sum or fractions\)\. In order to combine these measure, we first make them comparable by normalizing each measure over the set of candidate \(𝒞t\\mathcal\{C\}\_\{t\}\) to obtain a value∈\[0,1\]\\in\[0,1\]\. The individual measures are then fused by computing their sum:

\(6\)S​c​o​r​e​\(x\)=ws​e​m​Ss​e​m​\(x\)\+ws​a​l​Ss​a​l​\(x\)\+wt​h​e​m​St​h​e​m​\(x\),Score\(x\)\\;=\\;w\_\{sem\}\\,S\_\{sem\}\(x\)\\;\+\\;w\_\{sal\}\\,S\_\{sal\}\(x\)\\;\+\\;w\_\{them\}\\,S\_\{them\}\(x\),where the weights are fixed constants shared across all ambiguity levels; since the normalized signals still differ in shape, i\.e\., salience concentrates its mass on the most recently mentioned candidate while the thematic signal is smooth, the weights encode the relative reliability of the signals \(their values are stated in Section[4\.1](https://arxiv.org/html/2608.10330#S4.SS1)\)\.

After all factors of contextual information are considered in ranking the available candidates, the agent selects the top\-ranked candidate only when its margin over the candidate ranked second exceeds a fixed threshold\. If this is not the case, the agent requests clarification from the human\. Once the disambiguation is eventually achieved, and the agent ends up executing actions to complete the tasks, the historyℋ\\mathcal\{H\}is updated with the new record and used for subsequent thematic scoring\.

## 4\.Experimental Setup and Results

We experimentally evaluated five hypotheses regarding our disambiguation pipeline:

- H1:The compositional concept representation enables preference evidence to transfer to entities with sparse or no direct interaction history, by sharing concepts with frequently observed entities\.
- H2:Aggregating preference statistics at the concept level \(ℒ1\\mathcal\{L\}\_\{1\}\) yields more accurate disambiguation than at the individual attribute level \(ℒ0\\mathcal\{L\}\_\{0\}\)\.
- H3:Semantic compatibility, session salience, and user\-specific thematic preference provide complementary evidence; their fusion outperforms any individual signal in isolation\.
- H4:As ambiguity increases, user\-specific workflow patterns \(ℒ2\\mathcal\{L\}\_\{2\}\) become increasingly necessary; concept\-level frequency alone \(ℒ1\\mathcal\{L\}\_\{1\}\) suffices under low ambiguity but degrades as the candidate set grows\.
- H5:The proposed architecture captures individual user habits more effectively than LLM baselines that receive the same information, both when forced to commit and when allowed to request clarification, with the advantage increasing in highly personalized scenarios\.

### 4\.1\.Experimental Setup

We created a household environment with6666entities, four rooms, and1212actions\. Each entity is represented by its atomic attributes from the NOVA semantic feature norms\(suresh2025nova\)\(Section[3\.2](https://arxiv.org/html/2608.10330#S3.SS2)\), yielding an attribute vocabulary of\|𝒜\|=3,983\|\\mathcal\{A\}\|=3\{,\}983across the6666entities\. We minedℒ1\\mathcal\{L\}\_\{1\}concepts by retaining attribute combinations of size22–55that co\-occur in at least10%10\\%of the entities and satisfy the lift thresholdλ1=2\.0\\lambda\_\{1\}=2\.0, producing2,5082\{,\}508concepts mapped to the6666entities\. Forℒ2\\mathcal\{L\}\_\{2\}workflow pattern mining, we setλ2=1\.2\\lambda\_\{2\}=1\.2with a minimum support of two snapshots; each user’s history yields138138–150150interaction snapshots from which these patterns are mined\.

The command set contains200200high\-level commands \(e\.g\., rearranging objects in specific configurations, fetching specific objects\), each annotated with a textual description and an ASP\-based goal, e\.g\., “has\(user, book\)” or “on\(cup, table\)”\. As with other such simulated domains, the agent has knowledge of the domain’s state \(i\.e\., full observability\)\. To capture realistic variation in user habits, five human participants not involved in our architecture’s design were recruited to provide descriptions of their household routines and preferences\. Based on their input, we constructed and verified1010interaction sessions per user, each containing1616sequential commands under different initial conditions\. Each session comprises a coherent multi\-step routine \(e\.g\., coffee preparation, livingroom relaxation, studying, cooking\)\. Sessions within the same user share consistent preferences, while sessions across users exhibit systematically different preferences to ensure that user\-specific history is necessary for accurate disambiguation\. In three sessions per user, the target objects are novel, appearing in no other session and coming from categories adjacent to familiar ones, e\.g\., a mug where the user’s routines feature cups, or an apricot where they feature peaches\. When such a session is held out for testing, these objects have no direct history of their own, so resolving them depends on carrying the user’s preferences across the concepts that the novel and familiar objects share\.

The ambiguity was generated by systematically replacing explicit object mentions with increasingly ambiguous references at four levels:

- A1Near hypernym \(e\.g\., “apple”→\\rightarrow“fruit”\)
- A2Higher hypernym \(e\.g\., “apple”→\\rightarrow“food”\)
- A3Broad hypernym \(e\.g\., “apple”→\\rightarrow“object”\)
- A4Pronoun \(e\.g\., “apple”→\\rightarrow“it/that”\)

The average number of candidates per command is6\.06\.0,16\.616\.6,52\.252\.2, and66\.066\.0\(out of6666entities\) for A1–A4 respectively\. The four levels weaken the evidence available for disambiguation: at A1 the near hypernym still constrains the entity type, whereas at A4 the pronoun carries no type information, so resolution must rely almost entirely on the session context and the user’s own habits\.

For each session, we keep the first command unambiguous and apply ambiguities to the remaining commands, yielding3,0003\{,\}000ambiguous commands in total \(55users×10\\times 10sessions×15\\times 15commands×4\\times 4levels\)\. Commands in each session are processed sequentially\. After each command, the symbolic world state is updated with the ground\-truth referent to isolate errors to subsequent steps\. For evaluation, each test session uses the remaining99sessions of the same user as history\.ℒ2\\mathcal\{L\}\_\{2\}workflow patterns are mined from records in these sessions whose action predicate matches the current instruction \(Section[3\.2](https://arxiv.org/html/2608.10330#S3.SS2)\)\.

We report results under two modes:*noask*, where a method must always commit to a selection\(the one with top score\), and*ask*, where it may request clarification when evidence is insufficient\. For the proposed method, clarification is triggered when the top candidate’s lead over the runner\-up falls below a confidence threshold \(Section[3\.3](https://arxiv.org/html/2608.10330#S3.SS3)\); we use a single threshold of0\.250\.25at all ambiguity levels\. All score\-based methods, including the ablations, share the same fusion weights\. We set\(ws​e​m,wt​h​e​m,ws​a​l\)\(w\_\{sem\},w\_\{them\},w\_\{sal\}\)empirically to\(0\.40,0\.45,0\.15\)\(0\.40,0\.45,0\.15\)for A1,\(0\.25,0\.62,0\.13\)\(0\.25,0\.62,0\.13\)for A2 and A3, and\(0\.35,0\.52,0\.13\)\(0\.35,0\.52,0\.13\)for A4, giving the semantic signal more weight at A1, where the hypernym still constrains the candidate type\.The average accuracy varies by less than1%1\\%across a broad region around these values \(wt​h​e​m∈\[0\.55,0\.80\]w\_\{them\}\\in\[0\.55,0\.80\],ws​a​l∈\[0\.10,0\.15\]w\_\{sal\}\\in\[0\.10,0\.15\]\)\. For LLM baselines under*ask*mode, the model is allowed to output a clarification request when uncertain; under*noask*mode, clarification is not allowed and the model is forced to commit to a candidate\. We used GPT\-5\.1 for all LLM\-driven baselines\.

### 4\.2\.Baselines

We considered the following baselines:

1. \(B0\)Random\.Select uniformly at random from the ASP\-filtered candidate set\.
2. \(B1\)LLM full\-information \+ CoT\.GPT\-5\.1 selects one candidate from the full unfiltered candidate list using chain\-of\-thought prompting\(wei2022chain\), given the complete world state, the current session, and the user’s cross\-session history, i\.e\., the same history available to our method\.
3. \(B2\)LLM full\-information \+ CoT, ASP feasibility filtering\.Identical toB1, except that ASP feasibility filtering is applied first and the LLM selects among the remaining feasible candidates; the LLM thus receives exactly the inputs available to our method\.
4. \(B3\)Semantic \+ Salience, ASP feasibility filtering\.Apply ASP feasibility filtering, then fuseSs​e​mS\_\{sem\}andSs​a​lS\_\{sal\}using the same score fusion and confidence threshold as the full system, but without thematic preference\. This isolates the marginal contribution ofSt​h​e​mS\_\{them\}\.
5. \(B4\)Object\-level history \(no compositional concept\), ASP feasibility filtering\.Apply ASP feasibility filtering and the same three\-signal scoring structure as the full system, but compute thematic preference directly over object identities rather than concept bundles\. This isolates the contribution of compositional concept\.
6. \(B5\)ℒ0\\mathcal\{L\}\_\{0\}attributes \(no concept mining\), ASP feasibility filtering\.Identical toB4, except that the thematic preference is computed over the rawℒ0\\mathcal\{L\}\_\{0\}atomic attributes of each entity rather than over object identities\. This isolates the contribution of concept mining\.\.
7. \(B6\)Concept\-level \(no pattern\), ASP feasibility filtering\.Apply ASP feasibility filtering and the same three\-signal scoring structure, but computeSt​h​e​mS\_\{them\}using only concept\-level frequency \(Sa​c​t​i​o​nS\_\{action\}\) without user\-specific workflow patterns \(Sp​a​t​t​e​r​nS\_\{pattern\}\) fromℒ2\\mathcal\{L\}\_\{2\}\. This isolates the contribution of workflow pattern mining\.
8. \(B7\)Full system, no ASP filtering\.Use the complete three\-signal scoring with compositional concepts but skip ASP feasibility filtering\. This isolates the contribution of symbolic state maintenance\.

Proposed\.Our proposed architecture applies ASP feasibility filtering to remove physically infeasible candidates, then scores remaining candidates by fusing three complementary signals: semantic similarity \(Ss​e​mS\_\{sem\}\), session salience \(Ss​a​lS\_\{sal\}\), and user\-specific thematic preference \(St​h​e​mS\_\{them\}\) computed through hierarchical concept pooling overℒ0\\mathcal\{L\}\_\{0\}/ℒ1\\mathcal\{L\}\_\{1\}/ℒ2\\mathcal\{L\}\_\{2\}\. An uncertainty\-aware decision policy selects the top candidate only when its margin exceeds a confidence threshold; otherwise, the agent requests clarification\.

For comparison with baselines, each specific experimental run with our architecture was repeated for each baseline with the same users, sessions, ambiguous command instances, candidate sets, and history splits\.

### 4\.3\.Evaluation Measures

We used three measures for evaluation:

1. \(1\)Overall accuracy\.Proportion of commands where the method selected the correct entity; under*ask*mode, clarifications count as incorrect\.
2. \(2\)Answer accuracy\.Accuracy computed only over commands where the method made a definite selection, excluding clarifications\.
3. \(3\)Clarification rate\.Proportion of commands where the method requested clarification rather than selecting a candidate\.

Table 1\.Comparing the proposed method withB0–B7under*noask*mode; \(↑\\uparrow\) indicates higher is better\. The proposed method outperforms all baselines at every ambiguity level\. The gap betweenB6and the proposed method is largest at A3–A4, indicating that workflow patterns become increasingly necessary as ambiguity increases; the LLM baselines \(B1–B2\) achieve substantially lower accuracy despite receiving the same interaction history\. Statistical tests conducted at the 99% level of significance\.Table 2\.Comparing the proposed method withB0–B7under*ask*mode; \(↑\\uparrow\) higher is better, \(↓\\downarrow\) lower is better\. The LLM baselines \(B1–B2\) ask markedly more often than the proposed method yet answer less accurately\. The proposed method asks the least, is the most accurate when it commits, and attains the highest overall accuracy\. Statistical tests conducted at the 99% level of significance\.Ambiguity Level→\\rightarrowB3:Ss​e​mS\_\{sem\}\+Ss​a​lS\_\{sal\}B5:ℒ0\\mathcal\{L\}\_\{0\}attr\.ProposedOverall accuracy \(%\)\(a clarification counts as not completed\)A1↑\\uparrow27\.625\.929\.732\.027\.128\.957\.354\.266\.7A2↑\\uparrow7\.119\.722\.524\.419\.120\.938\.243\.654\.2A3↑\\uparrow2\.218\.119\.713\.311\.612\.927\.636\.047\.6A4↑\\uparrow1\.316\.018\.812\.010\.712\.026\.726\.744\.4Average↑\\uparrow9\.619\.922\.720\.417\.118\.737\.440\.153\.2Answered accuracy \(%\)\(over the commands each method chose to answer\)A1↑\\uparrow27\.664\.570\.149\.849\.450\.380\.775\.482\.6A2↑\\uparrow7\.149\.354\.735\.633\.235\.969\.363\.672\.7A3↑\\uparrow2\.227\.231\.229\.727\.829\.658\.655\.669\.0A4↑\\uparrow1\.323\.427\.428\.626\.328\.154\.144\.766\.3Average↑\\uparrow9\.641\.145\.935\.934\.235\.965\.759\.872\.7Clarification \(%\)↓\\downarrow–46\.246\.146\.156\.654\.744\.233\.926\.6

\(a\)

−5\-50551010151520202525303035354040454550505555606065650101020203030404050506060Clarification Rate \(%\)Overall Accuracy \(%\)B0B1B2B3ProposedB4B5B6B7

\(b\)

0224466881010202030304040505060607070Number of historical sessionsAverage accuracy \(%\)B6 \(concept, no pattern\)B1 \(LLM\)B4 \(object\)Proposed

Figure 4\.\(a\) Clarification–accuracy tradeoff under*ask*mode; the ideal point is the upper\-left corner\. The LLM baselines \(B1–B2\) and the scoring ablations \(B3–B6\) clarify on roughly half of the commands with much lower accuracy, while the proposed method attains the highest accuracy at the lowest clarification rate\. \(b\) Average*noask*accuracy \(over the four ambiguity levels\) versus the number of historical sessions used to compute the user’s preference; accuracy increases up to a point,B1improves only marginally, the object baseline \(B4\) stays flat because object identity does not transfer to novel entities, and the workflow\-pattern channel activates only after a few sessions, so the proposed method coincides with the concept ablation \(B6\) at very low history and leads once patterns become reliable\.\(a\)

A1A2A3A410102020303040405050606070708080Ambiguity LevelOverall Accuracy \(%\)B4: Object\-levelB5:ℒ0\\mathcal\{L\}\_\{0\}attributesB1: LLM\+CoTB6: Concept\-level \(no pattern\)Proposed

\(b\)

A1A2A3A4303040405050606070708080Ambiguity LevelOverall Accuracy \(%\)Concept evidence only \(Sc​o​n​c​e​p​tS\_\{concept\}\)Pattern evidence only \(Sp​a​t​t​e​r​nS\_\{pattern\}\)

Figure 5\.\(a\) Overall accuracy \(%\) by ambiguity level under*noask*mode\. The concept representation \(B6\) far exceeds object\-level history \(B4\), rawℒ0\\mathcal\{L\}\_\{0\}attributes \(B5\), and the LLM baseline \(B1\) at every level; among the three history representations, object identity \(B4\) is the weakest because the test targets do not occur in the user’s history, whereas concepts transfer through shared attributes, and the full system, which adds workflow patterns, is the most accurate\. \(b\) Ranking the ASP\-filtered candidates by each thematic statistic alone under*noask*mode\. Concept frequency alone is ahead at A1; pattern evidence is dominant at A3–A4\.Table 3\.Effect of history source on the proposed method under*noask*mode\. Results report accuracy \(%\), with \(↑\\uparrow\) indicating higher is better\. The full pipeline is kept fixed; only the user history used to computeSt​h​e​mS\_\{them\}is changed\. In the mismatched setting each user is paired with a different user’s history\.Statistical tests conducted at the 99% level of significance\.Table 4\.Accuracy \(%\) under*noask*mode, grouped by how often the command’s target entity occurs in the user’s history; \(↑\\uparrow\) higher is better\.B4becomes effective only on frequent targets, while the proposed method keeps accuracy high even on rare targets\. Statistical tests conducted at the 99% level of significance\.
### 4\.4\.Experimental Results

For any given user, the1010sessions were split so that one session was held out for testing and the remaining nine were used as historical information, with the held\-out session excluded when computingSt​h​e​mS\_\{them\}\. We ran ten such trials per user, each holding out a different session, across all five users, yielding3,0003\{,\}000ambiguous commands evaluated under both*noask*and*ask*modes\.

#### Overall comparison\.

Tables[1](https://arxiv.org/html/2608.10330#S4.T1)and[2](https://arxiv.org/html/2608.10330#S4.T2)summarize the results comparing the proposed method withB0–B7\. The proposed method led to significantly higher accuracy than all baselines at every ambiguity level under both modes, and degraded the least from A1 to A4, e\.g\., from74\.2%74\.2\\%to54\.7%54\.7\\%, well above the strongest LLM baseline \(B2\) at every level\. Removing ASP feasibility filtering \(B7\) reduced accuracy under both modes\.

#### Comparison with the LLM baselines \(H5\)\.

Both LLM baselines received the full world state and the same interaction history as our method\. Under*noask*, they remained significantly less accurate,B1reached30\.9%30\.9\\%on average against62\.7%62\.7\\%, with both LLM baselines well below the proposed method at every level, and even with ASP feasibility filtering,B2remained far below the proposed method\. Under*ask*, the LLM baselines requested clarification markedly more often than the proposed method, e\.g\., about46%46\\%against26\.6%26\.6\\%of the commands, yet answered less accurately, placing the proposed method alone in the upper\-left region of Figure[4](https://arxiv.org/html/2608.10330#S4.F4)\(a\)\. To evaluate the impact of historical knowledge, we further repeated the experiment for different numbers of historical sessions \(from0to99\) and computed theaverage*noask*accuracy, as summarized in Figure[4](https://arxiv.org/html/2608.10330#S4.F4)\(b\)\. Note that the accuracy increased only marginally for the LLM baselineB1; it rose substantially for the proposed method and was consistently highest, with the gap overB1widening as more history was added\. These findings indicate that the advantage came from the compositional hierarchy used to organize and represent the existing user and environmental information rather than from the information available, and they supportH5\.

#### Effect of the thematic signal and its representation \(H3, H2\)\.

Removing the thematic signal \(B3\) reduced the average accuracy from62\.7%62\.7\\%to26\.4%26\.4\\%, supportingH3\. Figure[5](https://arxiv.org/html/2608.10330#S4.F5)\(a\) summarizes the comparison of three representations for this signal\. Object identities \(B4\) were adequate at A1 but collapsed at A3–A4, since their statistics covered only frequently used entities and did not transfer to rarely seen targets\. Rawℒ0\\mathcal\{L\}\_\{0\}attributes \(B5\) improved on object identities only marginally on average, since nearly all candidates shared most attributes\. Mined concepts \(B6\) were the most accurate, since a concept was general enough to transfer evidence across entities yet specific enough to separate candidates\. The resulting order, objects<<attributes<<concepts<<concepts\+\+patterns, supportsH2\.

#### Concept frequency vs\. workflow patterns \(H4\)\.

Figure[5](https://arxiv.org/html/2608.10330#S4.F5)\(b\) summarizes the ranking of candidates by each thematic statistic alone\. Note that concept frequency led at A1, e\.g\.,67\.3%67\.3\\%against61\.9%61\.9\\%, but the order reversed from A2, with patterns stronger at A3–A4, since concept frequency helped only while the hypernym constrained the candidate type\. Accordingly, adding workflow patterns to the concept\-only system \(B6\) raised the accuracy at every level, with the largest gains at the higher ambiguity levels A3–A4 \(Table[1](https://arxiv.org/html/2608.10330#S4.T1)\)\.

#### Preference transfer and user\-specificity \(H1\)\.

We grouped the test commands by how often the target appeared in the history, i\.e\., rare \(≤5\\leq 5\), medium \(66–1010\), and frequent \(1111–2020\)\. On rare targets, the proposed method nearly tripled the object\-level accuracy, e\.g\.,44\.6%44\.6\\%against16\.2%16\.2\\%\(Table[4](https://arxiv.org/html/2608.10330#S4.T4)\), since evidence transferred from other entities that shared the same concepts\. Note that the object baseline \(B4\) stayed nearly flat as more historical sessions were added, since object identity did not transfer to novel entities, whereas the proposed method kept improving, as summarized in Figure[4](https://arxiv.org/html/2608.10330#S4.F4)\(b\)\. This benefit depended on the right user’s history; computingSt​h​e​mS\_\{them\}from a size\-matched history of a different user reduced the average accuracy from62\.7%62\.7\\%to16\.0%16\.0\\%\(Table[3](https://arxiv.org/html/2608.10330#S4.T3)\), below the history\-freeB3at every level\. These findings supportH1\.

lampTVmicrowave00\.10\.10\.20\.20\.30\.30\.40\.40\.50\.50\.60\.60\.190\.240\.42fused score\(a\) Semantic \+ thematic“Could you switch on …”\(reading context\)semanticthematicsalience

bananaappleorange00\.10\.10\.20\.20\.30\.30\.40\.40\.50\.50\.60\.6filtered0\.300\.55fused score\(b\) Thematic decides“Please put a fruit on the table\.”

cupthermoskettle00\.10\.10\.20\.20\.30\.30\.40\.40\.50\.50\.60\.60\.300\.270\.43fused score\(c\) Salience decides“Please give me the hot beverage\.”

Figure 6\.Fused per\-candidate scores for three under\-specified commands, decomposed into semantic, thematic, and salience contributions; the selected candidate is shown in bold\.

## 5\.Execution Traces

We present three execution traces to illustrate how the agent combines the semantic compatibility, thematic preference, and session salience signals to resolve under\-specified commands\. Figure[6](https://arxiv.org/html/2608.10330#S4.F6)reports the fused per\-candidate scores for each command\. The same three scenarios are illustrated in the supplementary video\.

### 5\.1\.Execution Example 1: Context and Signal Fusion

Consider a living room that contains a bookshelf, a table, and three switchable appliances, a lamp, a television, and a microwave, with the agent in the same room\. Earlier, the user instructed‘‘Please put the book on the table’’, which the agent executed using the ASP\-based planoccurs\(walk\(agent1, living\_room\), 0\),occurs\(walktowards\(agent1, bookshelf\), 1\),occurs\(grab\(agent1, book, bookshelf\), 2\),occurs\(walktowards\(agent1, table\), 3\),occurs\(put\(agent1, book, table\), 4\)to achieveon\(book, table\)\. The objects changed by this command are recorded asOt−1ΔO^\{\\Delta\}\_\{t\-1\}:book,bookshelf, andtable\. The user then speaks to the agent from the other end of the room, and the agent only hears‘‘Could you switch on …’’\. ASP\-based feasibility reasoning keeps the switchable applianceslamp,television, andmicrowave\. The recent action of placing the book on the table is associated with the reading context, identified using semantically relevant data from WordNet, so thelampthat shares properties with this context receives the highest semantic compatibility score\. A user\-specific workflow pattern mined from this user’s history \(Section[3\.2](https://arxiv.org/html/2608.10330#S3.SS2)\), in which a book\-like trigger concept is followed by a lamp\-like target concept, gives thelampthe highest thematic preference score\. No device was used recently, so session salience does not contribute\. The two signals reinforce each other, and the fused score ranks thelampfirst \(Figure[6](https://arxiv.org/html/2608.10330#S4.F6)\(a\)\)\. As its lead over the second\-ranked candidate exceeds the threshold, the agent commits toholds\(switched\_on\(lamp\)\)without requesting clarification, after which ASP\-based reasoning computesoccurs\(walk\(agent1, living\_room\), 0\),occurs\(walktowards\(agent1, lamp\), 1\),occurs\(switchon\(agent1, lamp\), 2\), which the agent is ready to execute\. The per\-candidate scoring trace for this example is reported in Figure[5\.1](https://arxiv.org/html/2608.10330#S5.SS1)\.

\[COMMAND\]“Could you switch on …”

\[GOAL\]switched\_on\(agent, switchable\_item\)

\[FILTER\]switchon accepts only sort \#switch\_furniture, 3 of 66 entities qualify candidates = lamp, television, microwave

\[CONTEXT\]previous command = “put the book on the table”, on\(book, table\) changed = book, bookshelf, table bookreading 0\.9, studying 0\.5, relaxation 0\.1bookshelfreading 0\.6, storage 0\.5, studying 0\.2tabledining 0\.8, cooking 0\.3, reading 0\.1 totals: reading 1\.6, dining 0\.8, studying 0\.7, storage 0\.5, cooking 0\.3 context word = reading

\[SEMANTIC\]Similarity to “reading” \(WordNet\) lamp 0\.475, television 0\.300, microwave 0\.225 the lamp fits a reading situation best

\[THEMATIC\]Concept evidence: past switch\-on commands lamp 0\.08, television 0\.50, microwave 0\.43 Pattern evidence: trigger concepts of the previous command c93\{has printing on it, has printed text on the pages, …\}c45\{has a flat surface, supports other objects, …\} target concept c21 \{emits light, contains lightbulbs, can be lit for light\}, carried by the lamp lampc93 to c210\.55c45 to c210\.30total0\.85 television, microwave: 0\.00 combined: lamp 0\.50, television 0\.27, microwave 0\.23 the mined pattern outweighs the concept evidence

\[SALIENCE\]nothing referenced this session lamp 0\.00, television 0\.00, microwave 0\.00

\[SCORE\]weights: semantic 0\.40, thematic 0\.45, salience 0\.15 lamp0\.4750\.500\.00= 0\.42television0\.3000\.270\.00= 0\.24microwave0\.2250\.230\.00= 0\.19

\[DECIDE\]lamp 0\.42 over television 0\.24, lead 0\.75, threshold 0\.25 committed without asking

\[COMMIT\]switched\_on\(lamp\)

Figure 7\.Scoring trace for Execution Example 1\.
### 5\.2\.Execution Example 2: Preference from Interaction History

In the kitchen, an orange already sits on the table, and an apple and a banana are within reach\. For the command‘‘Please put a fruit on the table’’, ASP\-based feasibility reasoning first removes theorangefrom further consideration and leaves theappleand thebanana\. The semantic compatibility score is nearly the same for the remaining candidates, although there is a slight preference for theapple, and neither fruit has been handled recently\. Thebanana, however, shares concepts that recur in this user’s records of this action, accumulated in the concept channelSconceptS\_\{\\mathrm\{concept\}\}, so its thematic preference score is high enough to outweigh the difference in the semantic compatibility score \(Figure[6](https://arxiv.org/html/2608.10330#S4.F6)\(b\)\), with the scoring trace reported in Figure[5\.2](https://arxiv.org/html/2608.10330#S5.SS2)\. The agent proceeds to place the banana on the table and commits toon\(banana, table\), a case a model without persistent user history cannot resolve\.

\[COMMAND\]“Please put a fruit on the table\.”

\[GOAL\]on\(fruit, table\)

\[FILTER\]orange dropped, on\(orange, table\) already holds candidates = banana, apple

\[CONTEXT\]previous command = “put the orange on the table”, changed = orange, table context word = dining

\[SEMANTIC\]Similarity to “dining” \(WordNet\) banana 0\.49, apple 0\.51 near tie, slight edge to the apple

\[THEMATIC\]Concept evidence: 98 past put commands bananac34 \{has a soft interior, is soft on the inside, …\}31 of 98 = 0\.32c41 \{elongated, is cylindrical with a curved top, …\}24 of 98 = 0\.24other concepts0\.19total0\.75applec19 \{crisp, crunchy, makes a crunchy sound, …\}8 of 98 = 0\.08c23 \{can be red colored, is sweet, is round, …\}5 of 98 = 0\.05other concepts0\.08total0\.21 Pattern evidence: no pattern matched, 0\.00 for both combined: banana 0\.78, apple 0\.22 this user keeps acting on the concepts the banana carries

\[SALIENCE\]nothing referenced this session banana 0\.00, apple 0\.00

\[SCORE\]weights: semantic 0\.40, thematic 0\.45, salience 0\.15 banana0\.490\.780\.00= 0\.55apple0\.510\.220\.00= 0\.30

\[DECIDE\]banana 0\.55 over apple 0\.30, lead 0\.83, threshold 0\.25 history outweighs the semantic edge of the apple

\[COMMIT\]on\(banana, table\)

Figure 8\.Scoring trace for Execution Example 2\.
### 5\.3\.Execution Example 3: Resolution by Salience

A cup, a thermos, and a kettle stand together on the kitchen counter, and the cup has just been filled from the kettle\. For the command‘‘Please give me the hot beverage’’, the feasible candidates are thecup, thethermos, and thekettle, comparable vessels for which the user shows no thematic preference, so the semantic compatibility and thematic preference scores are similar across them\. Session salience, computed from the maintained world state, assigns the highest weight to the most recently used candidate\. Since thecupwas just filled, it receives the highest fused score and is identified as the intended referent \(Figure[6](https://arxiv.org/html/2608.10330#S4.F6)\(c\)\), with the scoring trace reported in Figure[5\.3](https://arxiv.org/html/2608.10330#S5.SS3), and the agent commits tohas\(user, cup\)\.

\[COMMAND\]“Please give me the hot beverage\.”

\[GOAL\]has\(user, drinkware\)

\[FILTER\]nothing dropped, all three can be handed over candidates = cup, thermos, kettle

\[CONTEXT\]previous command = “fill the cup from the kettle”, changed = cup context word = dining

\[SEMANTIC\]Similarity to “dining” \(WordNet\) cup 0\.35, thermos 0\.28, kettle 0\.37 highest for the kettle, the wrong entity

\[THEMATIC\]Concept evidence: past give commands cup 0\.32, thermos 0\.35, kettle 0\.33 Pattern evidence: no pattern matched, 0\.00 for all three combined: cup 0\.32, thermos 0\.35, kettle 0\.33 same concepts across the three vessels, no separation

\[SALIENCE\]recency this session, each step back discounted by 0\.9 cupfilled one step back0\.90thermosnot referenced0\.00kettlenot referenced0\.00 relative: cup 1\.00, thermos 0\.00, kettle 0\.00

\[SCORE\]weights: semantic 0\.40, thematic 0\.45, salience 0\.15 cup0\.350\.321\.00= 0\.43thermos0\.280\.350\.00= 0\.27kettle0\.370\.330\.00= 0\.30

\[DECIDE\]cup 0\.43 over kettle 0\.30, lead 0\.43, threshold 0\.25 salience alone separates the three

\[COMMIT\]has\(user, cup\)

Figure 9\.Scoring trace for Execution Example 3\.

## 6\.Conclusion

This paper presented an architecture for personalized command disambiguation in household environments that combines ASP\-based feasibility filtering, compositional concept representation, multi\-signal scoring over user\-specific interaction history, and uncertainty\-aware clarification\. The experimental evaluation on3,0003\{,\}000ambiguous commands from five users supported all five hypotheses\. Compositional concept representation is more accurate than statistics over either object identities or rawℒ0\\mathcal\{L\}\_\{0\}attributes, particularly under high ambiguity\. The two components of user\-specific scoring, concept frequency and workflow patterns, provide complementary preference evidence and are more effective in combination\. Shared concepts enable preference transfer to rarely observed entities, and the learned preferences are user\-specific, i\.e\., a mismatched history performs worse than no history\. ASP filtering improves accuracy under both modes and restricts the selection to feasible interpretations\. The proposed method outperforms all baselines, including LLM baselines that receive the same interaction history, under both evaluation modes, while requesting clarification far less often\.

Our architecture opens up multiple directions for further research\. We plan to extend the compositional concept representation to novel object recognition and to integrate it with robot motion planning\. We also aim to leverage the compositional hierarchy to generate more targeted clarification questions\. Furthermore, we will evaluate the architecture in larger domains and on physical robot platforms under partial observability, using interaction histories logged over long\-term use\. The ultimate goal is to develop assistive agents that rapidly understand individual user preferences from limited interaction history and few clarification exchanges\.

## References