Tag
A developer is seeking feedback on an open-source framework for coding agents that catches AI-generated 'slop' before it reaches the repo by having a second agent automatically review every change and send findings back, along with a session manager, ledger, and guards against repeated model mistakes.
Greptile is leveraging GPT-6 Astra's computer use capabilities to automate application testing beyond static code review, alongside its Daybreak security product and Codex-integrated Knowledge Base, in a push to reduce bugs and rework for developers.
Hillel Wayne's newsletter covers his move to NYC, an upcoming Antithesis webinar on property-based testing for web applications, and a technical exploration of the challenges of differential testing on nondeterministic programs, including hyperproperties and distribution-level comparisons.
The author argues that 100% AI alignment is as absurd as 100% tested software, since verification is always limited to the evals you actually run. Passing tests (or evals) only proves no errors were found, not the absence of untested failure modes.
The paper introduces the 'stale' benchmark to study semantic coordination failures in parallel LLM-agent development, showing that interference is frequent in controlled tasks but rare in real-world reviewed pull requests.
The user asks for ways to verify the correctness of AI-generated code, highlighting doubts and inefficiencies when using multiple models for checks.
The article highlights the unparalleled thoroughness of AI agents in testing and QA software, using advanced simulations and deployments to achieve quality beyond human limits.
The blog post explores the effectiveness of generative testing versus unit tests in discovering bugs, using the Rust regex crate as a case study. It demonstrates how a custom fuzzer found multiple bugs and provides techniques for improving testing approaches.
This article discusses how to write tests and key points of code review in business projects, emphasizing that testing is foundational and mentioning that future content will cover critical design.
Benchmarked GPT-5.6 Luna vs GPT-6 Astra across 50 real PRs, showing Astra found 92 bugs vs Luna's 69, with Luna catching 75% of bugs at 3.6% of the cost. Includes detailed evaluation breakdown and upcoming comparison with Fable 5.1.
Benchmarked GPT-6 Astra vs GPT-5.6 Sol on 50 real PRs, finding Sol detected more bugs while Astra had higher precision and lower latency. Feedback is sought for future evaluations.
Antithesis is a bug-finding tool that wraps systems and runs them in hostile simulations to detect bugs and provide perfect step-by-step reproduction.
The article critiques change-detector tests in software development, advocating for behavior-based testing while acknowledging practical challenges in service-oriented systems.
A division by zero vulnerability was identified in the FFmpeg multimedia tool through the use of a custom fuzzer, as reported in a software issue.
Gergely Orosz criticizes the developer tendency to trivialize difficult migrations while being sensitive to similar trivializations, citing a five-year estimate for migrating Enzyme to React Testing Library.
The article discusses programmable property-based testing, a software testing technique that enables customizable test case generation based on defined properties.
The article discusses composable tests in software testing, explaining how composing tests can improve readability, maintainability, and efficiency while preserving properties like isolation and specificity.
This paper presents DNSSE, a hybrid framework combining LLM-guided schedule prediction, symbolic constraint solving, and coverage-guided stochastic mutation for verifying distributed parallel AI programs. It detects 2.9x more concurrency bugs than the baseline and raises branch coverage from 68.6% to 91.6% on realistic benchmarks.
MetaSpace is a framework that applies metamorphic testing to evaluate spatial cognition in embodied agents, generating test cases from execution trajectories and encoding logical/physical constraints as Prolog rules. Benchmarking shows state-of-the-art MLLM-driven agents score far below human levels on spatial cognition, with directional tasks being especially weak.
Checksum AI is an AI-powered testing tool designed to assist coding agents in software development and testing processes.