Tag
The article warns against hyping GPT-6 Astra's ARC AGI-3 results, noting Nvidia's 100% achievement with AVO and OpenAI's use of a non-standard harness.
GPT-6 Astra achieves state-of-the-art scores on the ARC-AGI-3 benchmark, scoring 99.9% with a Provider Adapter harness and demonstrating fewer actions than human testers. The model exhibits the ability to convert unfamiliar environments into compact symbolic world models for efficient planning.
Prime Agent is an open-source harness that uses recursive subagents and persistent computation to extend language models' long-horizon capabilities across coding and reasoning tasks, significantly improving performance on benchmarks like ARC-AGI-3.
Nvidia's research demonstrates that a well-designed harness around an AI model, rather than the model itself, significantly improves performance on long-horizon tasks, with Claude Opus 5 achieving a perfect score on the ARC-AGI-3 benchmark.
NVIDIA's coding agent has achieved a 100% score on the ARC-AGI-3 interactive reasoning benchmark, demonstrating advanced AI reasoning capabilities.
Claude Opus 5, along with Claude Code and a skill, scored 100% on the ARC AGI 3 benchmark's public set, suggesting the benchmark may not be as challenging as thought.
Anthropic's Opus 5 beats GPT-5.6 on ARC-AGI-3, but Tibo claims GPT-5.6 Sol becomes SoTA with two setting changes involving multi-context reasoning and canonical compaction.
A claim that GPT-5.6 Sol achieves state-of-the-art on ARC-AGI-3 by enabling reasoning across multiple context windows using canonical compaction.
OpenAI reveals that enabling retained reasoning and context compaction tripled GPT-5.6 Sol's ARC-AGI-3 benchmark scores, highlighting how harness settings significantly impact measured model performance.
GPT-5.6 Sol, a model that solved open math problems, initially struggled with the ARC-AGI-3 benchmark due to a harness memory limitation. Enabling two API settings tripled scores with 6x fewer output tokens.
OpenAI discovered that enabling retained reasoning and compaction settings in the API harness tripled GPT-5.6 Sol's scores on the ARC-AGI-3 benchmark while cutting output tokens by 6x, revealing that benchmark performance is heavily influenced by harness design.
Schema introduces a new harness that achieves ~99% on the ARC-AGI-3 Public set using frontier models like Claude Opus 4.8 and Fable 5, by improving the process around models rather than modifying weights.