Psychological Competence as a Missing Dimension in AI Evaluation
Summary
This paper introduces psychological competence as a missing dimension in AI evaluation, proposing a conceptual framework to assess how AI systems support user cognition, emotional interpretation, and decision-making in human-facing roles.
View Cached Full Text
Cached at: 07/10/26, 06:08 AM
# Psychological Competence as a Missing Dimension in AI Evaluation Source: [https://arxiv.org/abs/2607.08285](https://arxiv.org/abs/2607.08285) [View PDF](https://arxiv.org/pdf/2607.08285) > Abstract:Current AI evaluation frameworks focus primarily on technical performance, including accuracy, robustness, reasoning ability, and policy compliance\. These measures remain essential, but they are not sufficient for systems that interact directly with users through natural language\. Human\-facing AI systems are increasingly used as advisors, coaches, tutors, and companions\. In these roles, their responses can shape how users reason, interpret emotions, form beliefs, calibrate trust, and make decisions\. The relevant unit of evaluation is therefore not only the model, but the human\-AI interaction\. This paper introduces psychological competence as a missing dimension in AI evaluation\. We define psychological competence as the capacity of a human\-facing AI system to support user cognition, emotional interpretation, and behavioral decision\-making in ways that are appropriate to the user, context, and purpose of the interaction\. This includes interaction properties such as framing, tone, perceived authority, responsiveness, uncertainty handling, and conversational guidance\. Existing evaluation approaches capture parts of this problem but rarely assess these psychological effects directly\. Drawing on behavioral science and human\-AI interaction research, we outline a conceptual framework for psychological competence and its core domains\. Rather than proposing a specific benchmark, we define the construct, clarify its boundaries, and describe how it may be assessed through scenario\-based probes, structured human evaluation, and model\-assisted evaluation methods\. We argue that psychological competence should become a core consideration for model providers, deploying organizations, researchers, and regulators concerned with the real\-world effects of human\-facing AI systems\. ## Submission history From: Marcos Economides PhD \[[view email](https://arxiv.org/show-email/c3b50f6f/2607.08285)\] **\[v1\]**Thu, 9 Jul 2026 09:29:01 UTC \(881 KB\)
Similar Articles
Position: Artificial Intelligence Needs Meta Intelligence -- the Case for Metacognitive AI
This position paper argues that incorporating metacognition as a design principle can lead to more accurate, secure, and efficient AI systems, and demonstrates the concept through a Federated Learning case study and a software framework for experimentation.
Uneven Evolution of Cognition Across Generations of Generative AI Models
This paper introduces a psychometric framework and the AIQ Benchmark to evaluate the cognitive profiles of generative AI models, revealing uneven evolution with strong verbal skills but stagnant perceptual reasoning.
Measuring Intelligence Beyond Human Scale
This paper proposes a new paradigm for measuring intelligence beyond human capability using adversarial psychometric rating systems, where models generate challenges to separate other systems, enabling evaluation that scales with AI capabilities.
Robust Human-AI Complementarity under Uncertainty
This paper investigates how uncertainty about AI prediction quality affects human decision makers' ability to benefit from complementary information, finding that negative error correlation between human and AI predictions enables robust improvement strategies.
AI proficiency is becoming a hiring requirement but we still have no real way to measure it
The author explores the difficulty of accurately measuring AI proficiency in hiring, arguing that current certifications and tests focus on memorization rather than practical reasoning and evaluation.