Tag
The paper introduces the 'stale' benchmark to study semantic coordination failures in parallel LLM-agent development, showing that interference is frequent in controlled tasks but rare in real-world reviewed pull requests.
The user asks for ways to verify the correctness of AI-generated code, highlighting doubts and inefficiencies when using multiple models for checks.
The article highlights the unparalleled thoroughness of AI agents in testing and QA software, using advanced simulations and deployments to achieve quality beyond human limits.
The blog post explores the effectiveness of generative testing versus unit tests in discovering bugs, using the Rust regex crate as a case study. It demonstrates how a custom fuzzer found multiple bugs and provides techniques for improving testing approaches.
This article discusses how to write tests and key points of code review in business projects, emphasizing that testing is foundational and mentioning that future content will cover critical design.
Benchmarked GPT-5.6 Luna vs GPT-6 Astra across 50 real PRs, showing Astra found 92 bugs vs Luna's 69, with Luna catching 75% of bugs at 3.6% of the cost. Includes detailed evaluation breakdown and upcoming comparison with Fable 5.1.
Benchmarked GPT-6 Astra vs GPT-5.6 Sol on 50 real PRs, finding Sol detected more bugs while Astra had higher precision and lower latency. Feedback is sought for future evaluations.
Antithesis is a bug-finding tool that wraps systems and runs them in hostile simulations to detect bugs and provide perfect step-by-step reproduction.
The article critiques change-detector tests in software development, advocating for behavior-based testing while acknowledging practical challenges in service-oriented systems.
A division by zero vulnerability was identified in the FFmpeg multimedia tool through the use of a custom fuzzer, as reported in a software issue.
Gergely Orosz criticizes the developer tendency to trivialize difficult migrations while being sensitive to similar trivializations, citing a five-year estimate for migrating Enzyme to React Testing Library.
The article discusses programmable property-based testing, a software testing technique that enables customizable test case generation based on defined properties.
The article discusses composable tests in software testing, explaining how composing tests can improve readability, maintainability, and efficiency while preserving properties like isolation and specificity.
This paper presents DNSSE, a hybrid framework combining LLM-guided schedule prediction, symbolic constraint solving, and coverage-guided stochastic mutation for verifying distributed parallel AI programs. It detects 2.9x more concurrency bugs than the baseline and raises branch coverage from 68.6% to 91.6% on realistic benchmarks.
MetaSpace is a framework that applies metamorphic testing to evaluate spatial cognition in embodied agents, generating test cases from execution trajectories and encoding logical/physical constraints as Prolog rules. Benchmarking shows state-of-the-art MLLM-driven agents score far below human levels on spatial cognition, with directional tasks being especially weak.
Checksum AI is an AI-powered testing tool designed to assist coding agents in software development and testing processes.
A field note argues that a green test suite doesn't prove tests are meaningful; AI agents should be asked to make tests fail to verify they actually catch bugs.
freeCodeCamp announces a comprehensive course on software testing with Playwright, covering fundamentals like the testing pyramid, TDD, API testing, and AI-powered tools like KaneAI.
A blog post by Antithesis detailing bugs found in several open-source Raft consensus implementations, including HashiCorp Raft and OpenRaft, emphasizing the difficulty of testing distributed systems and the need for better tooling.
Antithesis describes its custom deterministic hypervisor and subtree deduplication technique for efficient, repeatable whole-system fuzz testing at cloud scale.