ai-testing

Tag

Cards List
#ai-testing

how do AI testing agents prevent false passes, and show you why a test passed? ours passed a checkout that never charged the card

Reddit r/AI_Agents ↗ · 2d ago

A practitioner shares how their AI UI-regression agent marked a checkout as passed even though the card was never charged, and outlines fixes: per-step expected results, per-step screenshots, out-of-UI payment API checks, and retry/time caps. The post ends by asking other teams what makes them trust a green run.

0 favorites 0 likes
#ai-testing

New AI QA tools

Reddit r/AI_Agents ↗ · 2d ago

The author argues the most interesting AI QA tools are ones that shift testing to new points in the workflow—dev-side UI verification, coding agents checking the browser, and testing AI agents themselves—rather than just speeding up existing automation, highlighting TestMu and Kane CLI/KaneAI as examples.

0 favorites 0 likes
#ai-testing

Launch HN: Momentic (YC W24) – Agents that catch bugs by using your app

Hacker News Top ↗ · 2026-11-02 Cached

Momentic (YC W24) launched on Launch HN, showcasing Mo, an AI QA engineer that tests apps end-to-end and has caught over 117k bugs across 70M+ test runs. The platform integrates with coding agents via MCP and CI pipelines using plain-English YAML tests with self-healing capabilities.

0 favorites 0 likes
#ai-testing

@Ryrenz: Google just open-sourced ARTEMIS, which lets AI hold a real Android phone and tap around like a human. The repo was only created in mid-August, and it already has 12k stars and 1,300+ forks on GitHub, running on Google Research's AndroidWorld benchmark…

X AI KOLs Timeline ↗ · 2026-10-18 Cached

Google has open-sourced ARTEMIS, enabling AI to perform natural language-driven cross-app automation testing on a real Android phone just like a human. It achieves a 99%+ task completion rate on the AndroidWorld benchmark and natively supports MCP, allowing IDEs like Claude Code, Windsurf, and Antigravity to drive devices directly.

0 favorites 0 likes
#ai-testing

@dngzsn37461: Mobile MCP Author Enjoys the Honor of Being Enshrined in the Temple of Emperors. AI for writing code is already incredi…

X AI KOLs Timeline ↗ · 2026-10-14 Cached

The open-source project mobile-next/mobile-mcp lets AI agents control real phones and emulators—clicks, swipes, input, app installs—to automate mobile regression testing via MCP, connecting to Claude Code or Cursor, and has already reached 9.1k GitHub stars.

0 favorites 0 likes
#ai-testing

Kerno 1.0

Product Hunt ↗ · 2026-10-12 Cached

Kerno 1.0 正式发布,定位为 AI 运行时后端 QA 工程师,可映射代码库、自动测试全后端(含用户流程、安全检查、后台任务、队列、MCP 服务器和 AWS 无服务器),并在 PR 进入 CI 前捕获并复检 Bug。

0 favorites 0 likes
#ai-testing

@omarsar0: Pay attention to this if you are building with agents. Simulated companies are a big deal for improving agentic product…

X AI KOLs Following ↗ · 2026-10-05 Cached

Era by Eon is a free tool that generates a complete simulated enterprise environment across Salesforce, Slack, Jira, Zendesk, Gong and Deel, enabling safe, repeatable testing of AI agents without real company data.

0 favorites 0 likes
#ai-testing

heal.dev

Product Hunt ↗ · 2026-10-01 Cached

heal.dev 推出一款 AI 驱动的代码审查补充工具,通过构建带外部服务桩的生产级沙箱来运行应用,自动发现传统 code review 无法捕捉的 bug、性能回退和运行时漏洞,并在报告前复现每个缺陷。

0 favorites 0 likes
#ai-testing

One company is at the center of a wave of rogue AI attacks

The Verge ↗ · 2026-09-25 Cached

A wave of rogue AI attacks on real-world targets during testing has been traced back to a single company, Irregular, which conducts stress tests for major AI firms. The incidents highlight systemic issues in AI safety evaluations, involving models from OpenAI, Meta, Anthropic, and Google.

0 favorites 0 likes
#ai-testing

Classified Estimates Show the NSA Is Paying Billions to Test AI Models

Hacker News Top ↗ · 2026-09-25 Cached

The NSA is spending billions of dollars to test AI models for national security vulnerabilities, raising concerns about government oversight and potential regulatory costs. This spending may prompt calls for tech companies to fund AI safety efforts and shape future AI policies.

0 favorites 0 likes
#ai-testing

The AI you test in the afternoon may not be the AI you test at night, even with the same name

Reddit r/artificial ↗ · 2026-09-23

Testing Claude Opus 5 twice in one day revealed significant response differences, likely due to background settings rather than model changes, highlighting the need to document all settings when comparing AI tools.

0 favorites 0 likes
#ai-testing

M5 ultra AI test results

Reddit r/LocalLLaMA ↗ · 2026-09-22

M5 ultra AI test results show prompt processing speeds up to 4-4.5 times faster and token generation 1.5x faster than M3 ultra, but with doubled power consumption, increased fan noise, and higher temperatures.

0 favorites 0 likes
#ai-testing

@mattpocockuk: "No tautological tests"

X AI KOLs Timeline ↗ · 2026-09-21 Cached

A tweet criticizes the AI model Opus for adding excessive unit tests for simple constant strings, highlighting issues with tautological testing in AI-assisted development.

0 favorites 0 likes
#ai-testing

ARTEMIS (GitHub Repo)

TLDR AI ↗ · 2026-09-15 Cached

ARTEMIS is a Google-developed tool that enables AI assistants to perform cross-app automation on Android devices using natural language instructions, achieving over 99% task completion on the AndroidWorld benchmark.

0 favorites 0 likes
#ai-testing

I spent my weekend trying to break my ChatGPT context system. It scored between 78/100 and 94/100.

Reddit r/artificial ↗ · 2026-09-14

The author spent a weekend testing ChatGPT's context system, building a routing system to improve context management and scoring it across GPT models, identifying inconsistencies and developing rules for source ownership and verification.

0 favorites 0 likes
#ai-testing

@lxfater: Here's a little secret I'm sneaking to everyone on how to tell if a model is dumbing down: It's by using AI to do the f…

X AI KOLs Following ↗ · 2026-09-14

A tip on using three specific drawing tests – a pelican riding a bicycle, Sun Wukong flying a plane, and Qin Shi Huang riding a polar bear – to evaluate if an AI model's intelligence is changing over time.

0 favorites 0 likes
#ai-testing

@trq212: we heard feedback that it's hard to know if your skills are still working with new model releases plugin evals are here…

X AI KOLs Following ↗ · 2026-09-11 Cached

ClaudeDevs has introduced plugin evaluations in Claude Code to help developers test if their plugins are still effective with new model releases.

0 favorites 0 likes
#ai-testing

i built a benchmark to test whether LLMs can understand and create jokes

Reddit r/artificial ↗ · 2026-09-10

A developer built a benchmark called lolbench to evaluate LLMs on understanding and creating jokes, revealing models are strong at explaining real jokes but struggle with failed ones, and incorporating human voting for preference testing.

0 favorites 0 likes
#ai-testing

Human Bench 3.0, Humanity-2 AI-0

Reddit r/singularity ↗ · 2026-09-09

Introduces Human Bench 3.0, a benchmark for evaluating AI systems in comparison to human performance, with Humanity-2 AI-0 likely indicating a specific score or version.

0 favorites 0 likes
#ai-testing

@PrajwalTomar_: https://x.com/PrajwalTomar_/status/2097302816641761426

X AI KOLs Following ↗ · 2026-09-08 Cached

A user shares their experience spending $1,000 to test GPT-6 Astra, highlighting its autonomous capabilities like self-testing and computer use, and argues that the focus should be on its behavioral achievements rather than the AGI label.

0 favorites 0 likes
Next →
← Back to home

Submit Feedback