Tag
Introduces Long-Horizon-Terminal-Bench, a benchmark of 46 long-horizon terminal tasks with dense reward-based grading, evaluating AI agents on planning, long-context, and debugging. Even the strongest model achieves only 15.2% pass@1, showing significant room for improvement.
Cohere and Cohere Labs released North Mini Code, an open weights 30B-A3B parameter model optimized for code generation, agentic software engineering, and terminal tasks, with strong benchmark results on SWE-Bench and Terminal-Bench.
Cohere Labs released North Mini Code, a 30B-parameter (3B active) open-weights model optimized for code generation, agentic software engineering, and terminal tasks, licensed under Apache 2.0.
This paper introduces TerminalWorld, a benchmark for evaluating AI agents on real-world terminal tasks, derived from 80,870 terminal recordings. Current systems achieve at most 62.5% pass rate, highlighting challenges in authentic terminal workflows.