Can We Trust AI-Inferred User States. A Psychometric Framework for Validating the Reliability of Users States Classification by LLMs in Operational Environments
Summary
This paper empirically tests the psychometric reliability of LLM-based user state classification, finding that only 31 of 213 metrics met reliability criteria, questioning trust in real-time adaptive systems.
View Cached Full Text
Cached at: 05/18/26, 06:34 AM
# Can We Trust AI-Inferred User States. A Psychometric Framework for Validating the Reliability of Users States Classification by LLMs in Operational Environments Source: [https://arxiv.org/abs/2605.15734](https://arxiv.org/abs/2605.15734) [View PDF](https://arxiv.org/pdf/2605.15734) > Abstract:The use of large language models to assess user states in conversational and adaptive systems is based on the assumption that the metrics used for such assessment are stable and interpretable at the level of individual scores\. This paper empirically tests this assumption, focusing on the psychometric reliability of artificial intelligence \(AI\) measures of user states\. This study employed replication evaluation procedures to assess the repeatability of a broad set of metrics across three different bimodal large language models \(GPT\-4o audio, Gemini 2\.0 Flash, Gemini 2\.5 Flash\)\. Analyses include both individual score reliability and aggregated reliability, allowing us to distinguish metrics potentially useful for real\-time adaptation from those that retain their value only in aggregated analyses\. The results demonstrate that metric reliability cannot be considered a default property in interpretive domains\. The lack of stability at the level of individual scores precludes the interpretation of such scores as indicators of user state in real\-time adaptive systems, even if these metrics demonstrate stability after aggregation\. At the same time, the study indicates that individually unstable metrics can retain analytical utility in post\-hoc studies, identifying rules governing interactions and their relationships with user experience parameters such as satisfaction, trust, and engagement\. The main contribution of this work, besides quantifying the severity of the problem \(only 31 of 213 metrics met the criteria\), is the proposal of a replicable evaluation framework, enabling measurable evaluations of metric applicability\. This approach supports more responsible AI design of adaptive systems, in which the interpretation of results requires explicit validation of reliability and monitoring for violations over time\. ## Submission history From: Izabella Krzeminska Dr \[[view email](https://arxiv.org/show-email/287eac86/2605.15734)\] **\[v1\]**Fri, 15 May 2026 08:43:26 UTC \(2,921 KB\)
Similar Articles
Evaluating LLMs as Human Surrogates in Controlled Experiments
This paper evaluates whether off-the-shelf LLMs can reliably simulate human responses in controlled behavioral experiments by comparing LLM-generated data with human survey responses on accuracy perception. The findings show that while LLMs capture directional effects and aggregate belief-updating patterns, they do not consistently match human-scale effect magnitudes, clarifying when synthetic LLM data can serve as behavioral proxies.
Examining Human-Like Behaviors in LLMs: A Multi-Dimensional Analysis of Model Behaviors, User Factors, and System Prompts
This paper presents a multi-dimensional analysis of human-like behaviors in LLMs, examining prevalence, effects, and controllability across 21,000 conversations from four models, finding that behaviors vary by model and user factors, with implications for responsible design.
Human Psychometric Questionnaires Mischaracterize LLM Behavior
This paper finds that human psychometric questionnaires fail to reliably predict LLM behavior in real-world interactions, and proposes generation-based profiling as a more accurate alternative.
Towards Trust Calibration in Socially Interactive Agents: Investigating Gendered Multimodal Behaviors Generation with LLMs
This paper investigates the use of LLMs to generate multimodal behaviors (verbal, vocal, gestural, facial) for trust calibration in socially interactive agents. The study finds that while LLMs can produce coherent behaviors aligned with intended trustworthiness traits, they also reproduce societal gender stereotypes.
Re-Centering Humans in LLM Personalization
This paper studies the gap between synthetic and human data for evaluating LLM personalization across three stages: attribute extraction, relevance matching, and response generation. Results show models perform worse on real human data, and the authors introduce lightweight training interventions to improve alignment.