Tag
This paper argues for layered verification of LLM-generated argument graphs, combining deterministic checks, solvers, and LLM reviewers, and shows that LLM reviewers detect structural defects well but are inconsistent, over-reject correct graphs, and slower than exact checks.
作者分享用 Claude Code 做变异测试时遇到的"252/252 全杀其实是假阳性"事故：测试运行器的复制清单漏了数据文件，导致每次变异体都因同一缺失文件而"死亡"，并给出控制变异体、日志点名等防止假杀的实践建议。
qarunbook is a Product Hunt launch for a test-planning tool that lets teams and AI agents work through structured test plans over MCP, tracking fixes and retests until issues are verified as passed.
AI voice quality-check tool Onepin launches to catch pronunciation errors on acronyms, version numbers, and brand names in TTS output, fixing individual words instead of regenerating entire lines.
LangChain introduces an Engine that learns how your agent works by reading traces and repositories, then tests for agent-specific weaknesses such as hallucinations and system prompt violations, returning a list of verified issues to fix before they impact users.
The article critiques a research paper arguing that coding agents can replace human code review, highlighting that human aspects like confusion, skepticism, and noticing missing elements are crucial and not replicable by LLMs.
The post discusses capping the size of AI coding agent-generated pull requests to maintain review quality, seeking input on effective thresholds and trade-offs.
This paper presents InterviewPlayground, a clinician-grounded evaluation platform for AI-assisted psychiatric intake using simulated patients, showing that an LLM interviewer recovers more clinical items but makes more unsupported inferences compared to clinicians.
RespanAI launches Prompt Simulations, a tool for testing AI prompts by generating realistic user scenarios and running multi-turn conversations to identify failures before real users encounter them.
QAgent automates quality assurance for AI agents by scoring correctness, detecting hallucinations against ground truth, verifying policy adherence, and benchmarking RAG to prevent bad responses from reaching customers.
This article discusses how to write tests and key points of code review in business projects, emphasizing that testing is foundational and mentioning that future content will cover critical design.
The article discusses the importance of integrating QA into RL task generation through an iterative process to improve AI pipelines and model evaluation, emphasizing the value of intuition in eval design.
The article discusses methods and challenges for evaluating AI agents in production environments, focusing on quality assurance for real-world conversations beyond pre-defined evaluation sets.
The article demonstrates how a NousResearch Hermes local AI agent on NVIDIA RTX Spark autonomously identifies and fixes website issues and runs QA testing on device.
The article discusses the challenge of AI-generated code 'slop' due to human review bottlenecks and argues that software engineering must evolve to focus on system design rather than code readability.
The paper presents a locally deployed multi-agent AI system for structuring radiology reports and performing quality assurance, with radiologist evaluation showing favorable performance.
The article promotes using Replay QA for automated testing of web applications, especially for AI-generated code, highlighting its ease of use and features like continuous QA and root-cause analysis.
Xiaomi highlights over 1,000 days and 4.28 million kilometers of real-world testing for its vehicles, emphasizing meticulous quality and thorough road validation.
DoorDash shared how they automated evaluation of their Ask DoorDash AI assistant, enabling 2,000 daily graded sessions and cutting test time from six hours to 20 minutes while halving error rates.
A discussion on the next bottleneck for AI coding agents: verifying that AI-generated applications are actually correct, and who should be responsible for checking the output.