Tag
Mozilla's report indicates that open-weights AI models from Chinese companies are only 4.4 months behind closed frontier models in performance but at a fraction of the cost, leading many organizations to adopt open models for routine tasks.
Analyzes the gap between open weights and closed source LLMs using the Artificial Analysis Intelligence Index and other benchmarks, finding that the gap is shrinking on some metrics but stable on others.
The paper introduces the Capability Frontier, a Pareto frontier over models that corrects for biases in single-model and single-run evaluations, showing that standard benchmarks miss up to 82% of model performance and that collective LLM capabilities are substantially underestimated.
This paper introduces Agents' Last Exam, a benchmark that tests AI agents on real expert work across 55 digital work areas. Current best agents fail most tasks, averaging only 2.6% pass rate on the hardest tier, revealing a large gap between benchmark scores and real-world automation readiness.
An analysis from LessWrong examining the performance gap between open-source and proprietary AI models.