testing

Gergely Orosz tests new AI models like Fable and GPT-6 by asking them to write articles in his style from his interviews, but finds they fail spectacularly, producing lots of words without understanding.

Kent C. Dodds is offering early access to Kody Koala for developers working on agents with MCP support to ensure compatibility.

The article demonstrates how reducing unnecessary test isolation allows Preact's test suite of over 1000 tests to run in about 1 second, contrasting with slower modern test runners.

OpenCode's CEO secretly tested version 5.6 during a government regulatory blackout, and reportedly more upset about losing that than losing Fable 5.

Read-only agents are easier to test than write-access agents; production data write access remains an unsolved eval problem for many teams.

The author argues that building AI agents is no longer the hard part; the real challenges are deployment, testing, version control, and operational management, which remain fragmented in the ecosystem.

An Anthropic engineer shares an autonomous code loop architecture that runs on a closed laptop, opening pull requests and running tests without chat prompts. The system uses a five-file folder with contract, schedule, rubrics, and state components, and a subscription window for Fable 5 reasoning loops closes on July 12.

A tweet arguing that software rewrites can be good, cheap, and fast, especially as AI models improve at filling testing and verification gaps.

A 140 GB IQ2_XXS REAP quantized version of GLM 5.2 for coding has been created, and the author is looking for testers.

AdaStop is a cost-aware early stopping framework for DNN test selection that optimally stops labeling when the marginal fault discovery rate falls below a threshold, achieving 65-84% fault discovery using only 9-31% of the labeling budget.

The article questions the wisdom of companies laying off human employees and replacing them with AI agents without proper testing, suggesting a safer approach like parallel pilot runs. It discusses the potential risks and unknowns of such rapid AI adoption.

The author recounts the results of asking Reddit to try breaking their AI widget, sharing unexpected outcomes and lessons from the community stress test.

eve now ships with built-in eval capabilities for agents, addressing the need for first-class testing in AI agent development.

The author asks how developers are regression-testing AI agent workflows, noting common failure modes and sharing their work on adding eval support to Runme for recording tasks, scoring trajectories, and comparing against baselines.

Cargo-nextest is a next-generation test runner for Rust that offers up to 3x faster test execution, per-test isolation, and first-class CI support.

Google is testing a new dedicated inbox section inside the Gemini app for Workspace users, featuring filters to triage emails and tasks in an Inbox Zero workflow, positioning Gemini as a consolidated desktop workspace.

A planetary-scale test for locally-run AI models is being conducted, likely benchmarking their performance across different environments.

Anthropic published a detailed guide on how to make Claude work autonomously for 6 hours, covering failure modes, testing with Puppeteer MCP, and session protocols.

A developer used AI to build a PHP engine in Rust, passing 17% of PHP's official test suite and rendering WordPress, despite having no prior Rust or compiler experience.

Looking for testers for a custom agent harness.