Measuring Intelligence Beyond Human Scale

arXiv cs.AI Papers
intelligence-measurement ai-evaluation benchmarking adversarial-psychometrics superhuman-ai scalable-evaluation ai-safety

Summary

This paper proposes a new paradigm for measuring intelligence beyond human capability using adversarial psychometric rating systems, where models generate challenges to separate other systems, enabling evaluation that scales with AI capabilities.

arXiv:2607.07040v1 Announce Type: new Abstract: How can we measure intelligence beyond human capability? Human-authored benchmarks saturate, and above human capability, examiners may not know which tasks are both hard and verifiable. We argue that this difficulty is inherent to absolute-scale evaluation and propose a new paradigm based on relative measurement in which models generate public challenges that separate other systems. Aggregating these outcomes yields an adversarial psychometric rating system that can scale with the systems being measured. We describe practical protocols that reduce incentives for private-information attacks, support judge-free adjudication, and naturally scale with agent capabilities. We instantiate the framework across verifiable and open-ended, non-verifiable domains, illustrating how model-generated evaluation can continue to measure systems beyond the human frontier.
Original Article
View Cached Full Text

Cached at: 07/09/26, 07:55 AM 

# Measuring Intelligence Beyond Human Scale
Source: [https://arxiv.org/abs/2607.07040](https://arxiv.org/abs/2607.07040)
[View PDF](https://arxiv.org/pdf/2607.07040)

> Abstract:How can we measure intelligence beyond human capability? Human\-authored benchmarks saturate, and above human capability, examiners may not know which tasks are both hard and verifiable\. We argue that this difficulty is inherent to absolute\-scale evaluation and propose a new paradigm based on relative measurement in which models generate public challenges that separate other systems\. Aggregating these outcomes yields an adversarial psychometric rating system that can scale with the systems being measured\. We describe practical protocols that reduce incentives for private\-information attacks, support judge\-free adjudication, and naturally scale with agent capabilities\. We instantiate the framework across verifiable and open\-ended, non\-verifiable domains, illustrating how model\-generated evaluation can continue to measure systems beyond the human frontier\.

## Submission history

From: Elad Hazan \[[view email](https://arxiv.org/show-email/532769a8/2607.07040)\] **\[v1\]**Wed, 8 Jul 2026 06:19:10 UTC \(40 KB\)

Similar Articles

We measured how AI capabilities INTERACT as models scale. Below 3.5B, reasoning and truthfulness fight. Above it, they cooperate. The transition is engineerable. (2 papers + interactive dashboard + 7 falsifiable predictions)

Reddit r/artificial

Researchers discovered a critical scale (~3.5B parameters) where the trade-off between reasoning and truthfulness in AI models flips from antagonistic to cooperative. They provide a framework, interactive dashboard, and open-source steering tool to identify and correct misaligned outputs at small scales.

Scaling Participation in Modular AI Systems

arXiv cs.AI

This paper introduces scaling participation, a new paradigm for building modular AI systems through contributions from diverse stakeholders, where small models collaborate to outperform monolithic LLMs by up to 15.4% across various tasks, demonstrating emergent capabilities and improved diversity benefits.