testing

Dinamos is a free, open-source platform that enables testing system architectures with Chaos Engineering without spending on AWS.

Dan Luu reflects on his experiences using AI coding agents, noting that while AI can fabricate convincing but false repros, it still offers productivity gains in areas like debugging and testing. He discusses the balance of trust, automation, and quality in modern software development.

Cultivar is a new tool designed for testing and evaluating AI agent skills, providing a framework for benchmarking agent performance.

PyTorch explains how its tests are dynamically generated at import time, affecting CI failure names, and offers a blog to help contributors understand the testing workflow.

A tweet by @svpino demonstrates how to create an agentic loop using Claude Code to implement a Fibonacci function and run tests automatically, highlighting key flags like -p, --allowedTools, and --max-turns.

A discussion on defining and testing boundaries for tool-using AI agents to prevent them from crossing security or ethical lines even when not obviously jailbroken.

Replay QA is a tool that helps developers find bugs before users do, by capturing and replaying sessions.

The author emphasizes that the key to making AI agents write good code lies in following classic software engineering practices from 1990s textbooks: writing tests, doing CI/CD properly, top-down design, and modular decoupling.

html.contact is a backend service for forms that allows you to test it before paying.

This paper proposes a modular LLM-based pipeline that generates diverse test scenarios for autonomous driving systems using historical failure records (e.g., NHTSA crash data), enabling effective failure discovery within limited testing budgets.

Built an MCP that enables Claude to shop across ~25k stores; seeking testers and honest feedback.

A developer argues that the harness (critics, scaffolding) around an AI model is more important than the model itself, sharing an example where a 27B model with good critics became usable for coding work.

Harbor Framework now integrates with LangChain's Deep Agents, LangSmith Sandboxes, and Observability, enabling running agents in isolated, reproducible environments for deterministic testing.

Meta hired contractors through Covalen to pose as teenagers and send high-risk prompts (suicide, sex, drugs) to rival chatbots including ChatGPT, Gemini, and Character.AI, as part of a safety benchmarking project called Cannes. Over 45,000 prompts were used in August 2025 alone, with the targeted companies unaware of the testing.

PyTorch introduces a Cross-Repository CI Relay (CRCR) that automatically triggers and tracks CI in out-of-tree backends, providing a unified dashboard for CI health.

The article requests testers for the Canvas patch for GNU Emacs, providing building instructions for MS Windows and a test code snippet.

The article reports initial manual results from experiments testing procedural skill transfer in small AI models, providing insights into how skills can be transferred across models.

This article introduces a new natural-language testing system for AI agents that uses simulated isolates to automatically generate multi-turn simulations and evaluate agent behavior, helping developers catch regressions from prompt changes.

Test That is a new Rust library for test assertions, forked from GoogleTest Rust, offering a cleaner API, optional dependencies, and improved matcher composability.

A multi-tweet analysis of ~15 agentic-loop papers concludes that the verifier, not the model, is the key predictor of success, with examples showing that robust, non-gamable checks (e.g., compilers, tests, verifiable rewards) dramatically improve performance, while failures stem from lack of such verifiers or gaming vulnerabilities.