Tag
Close shares how they evolved their CI pipeline to run nearly 58,000 integration tests in under 3 minutes using orchestrated, dynamically split pytest workers across multiple runners, replacing sequential runs and simple fixed test splitting.
ggml-org details how they optimized llama.cpp's CI by storing compiler caches in Hugging Face Buckets and offloading workloads to HF Jobs, overcoming GitHub Actions' cache and runner limits for a project merging thousands of PRs.
A practitioner shares their deterministic evaluation harness for QA/coding agents — fixed corpora, injected bugs, hidden tests, and pass^k/token metrics — and asks the community how others balance smoke tests vs full benchmarks, whether LLM-as-a-judge adds value, and whether tools like Promptfoo or DeepEval reduce maintenance burden.
A developer building an independent verification prototype for coding agents shares two reproduced cases where pytest reported passing despite failing tests (exit status 0, config-based deselection) and asks the community how they verify agent-generated changes before accepting them.
A practitioner asks whether AI agents can handle large legacy migrations (Java 8 and outdated Spring upgrades) at insurers or banks, weighing agent autonomy against bank-grade change-control processes and highlighting Orq AI's scoped-workflow software factory as a candidate approach.
Momentic (YC W24) launched on Launch HN, showcasing Mo, an AI QA engineer that tests apps end-to-end and has caught over 117k bugs across 70M+ test runs. The platform integrates with coding agents via MCP and CI pipelines using plain-English YAML tests with self-healing capabilities.
Simantic (YC F26) Launch HN post announcing a firmware simulation layer that lets engineers and AI agents test and validate firmware on virtual devboards without physical hardware, positioned as a faster alternative to Renode and QEMU.
A comparison of seven popular LLM evaluation tools, split between in-code libraries (promptfoo, DeepEval, Ragas) and tracing/experiment platforms (Langfuse, Arize Phoenix, LangSmith, Braintrust), with guidance on when to use each.
Vercel 发布 Vercel CI(Beta),利用 Turborepo 的任务依赖图只运行受代码变更影响的任务,并借助 Hive 计算层按需分配算力,官方称 git push 到 CI 通过的速度提升 2-6 倍。
Kestrix 是一个仓库无关的 CI 控制平面,使用 HCL 定义 CI/CD 管道并解析为 DAG,仅在相关仓库发生变更时触发任务,提供 React Flow 日志界面和仅发起出站 HTTPS 请求的远程运行器。
Vercel 介绍 Vercel CI,通过 Turborepo 智能缓存跳过任务、全球集群基础设施以及面向 AI 代理的 MCP 与 CLI 支持,实现 2-6 倍更快的 CI,主打「Agentic CI」理念,现已在所有计划中进入 Beta。
licenseal 是一款免费的开源 CLI 工具,可读取 lockfile 与注册表元数据,在 10 种生态系统中检查所有依赖(含传递依赖)的许可证兼容性,并为 CI 提供明确判定结果。
CodeRabbit launched CodeRabbit Secure, whose Deep Scan traces vulnerabilities across linked repos and replays exploits in a throwaway sandbox to confirm real risk before flagging, while tracking which merges introduced or fixed each bug.
Standfast is a tool that helps you know whether your Mac's GitHub Actions runners will be assigned work, launched on Product Hunt.
作者运行了一套由编码代理组成的小型流水线,在 7 天内于 13 个 Rust 仓库中合并了约 316 个 PR,并分享了关键经验:结果校验应依赖测试与 CI 而非模型本身。项目已开源,为单一 Rust 二进制,兼容 Claude Code、Codex 与 Ollama。
GitHub reported degraded performance affecting Actions, Pull Requests, and Webhooks on Oct 21, 2026, with investigation ongoing.
GitHub reported degraded availability affecting GitHub Actions, GitHub Pages, and Webhooks on Oct 19, 2026, causing delays in workflow processing, site builds, and webhook delivery before resolving the incident. A root cause analysis will follow.
A recap of BazelCon 26 highlighting hallway-track discussions on the value of tests, criticizing frontier coding agents for over-generating unit tests while weighing the trade-offs between unit and e2e testing based on business risk tolerance.
Origin, Cursor's code hosting platform, now supports over 20 integrations, letting users connect repos to tools for CI, deployments, code review, security, monitoring, and project management.
Kerno 1.0 正式发布,定位为 AI 运行时后端 QA 工程师,可映射代码库、自动测试全后端(含用户流程、安全检查、后台任务、队列、MCP 服务器和 AWS 无服务器),并在 PR 进入 CI 前捕获并复检 Bug。