Ephemeral testing

Lobsters Hottest News

Summary

Daniel Lemire proposes 'ephemeral testing,' a method where AI agents build throwaway applications on top of your code, and you judge the quality of the underlying library by how well that temporary software turns out rather than reviewing the code directly. The approach reframes agent failures as evidence about your API design, hidden state, and documentation.

<p><a href="https://lobste.rs/s/ymannf/ephemeral_testing">Comments</a></p>
Original Article
View Cached Full Text

Cached at: 10/05/26, 01:12 PM

# Ephemeral testing Source: [https://lemire.me/blog/2026/10/05/ephemeral-testing/](https://lemire.me/blog/2026/10/05/ephemeral-testing/) ![](https://lemire.me/blog/wp-content/uploads/2026/10/Gemini_Generated_Image_8mi98l8mi98l8mi9-825x510.jpg) We have many ways to ensure software quality\. Unit testing\. Fuzz testing\. Integration testing\. And so forth\. I’d like to propose a method that was unthinkable before:ephemeral testing\. \(Ephemeral is a fancy word for ‘throw away’ or ‘temporary’\.\) You write your code\. You build your software component\. Or the AI agent does it for you, it does not matter\. Then you ask an AI agent to build on it: an application, another layer, maybe several\. You have it test what it built\. You do not assess the original work directly\. You assess how good the software built on top of it is\. It is a form of integration testing\. The difference is that the software on top is entirely ephemeral\. You throw it away when you are done\. A library with a clean API, stable invariants, and useful errors lets the agent produce something that works quickly\. A library with hidden state, surprising defaults, or incomplete docs produces a pile of patches and failures\. The failures are evidence about your code, not about the agent\. You can repeat it\. Different agents, different tasks, same foundation\. In effect, instead of building the core while trying to anticipate what might be needed at the other layers, you just simulate the other layers by actually building them\. Of course, you could argue that with AI, you can rebuild everything whenever you need to\. But that’s not practical\. You need some form of stability\. I have been applying this trick to various projects\. As I consider a new feature, I ask my AI to prototype quickly what I might later build based on what I am doing it\. Ephemeral testing works for me thus far\. ## Published by ![](https://secure.gravatar.com/avatar/a0c6c34cf4a45ff5083d5ea541e7f27c5e4457ac393889e077af10688f6b3831?s=56&d=mm&r=g)

Similar Articles

Building to the Test: Coding Agents Deliver What You Check, Not What You Requested

Hugging Face Daily Papers

This paper investigates how LLM-based coding agents often pass benchmark tests by building to the test rather than implementing the requested functionality correctly, using controlled experiments with Copilot CLI agents reimplementing a React UI library in Angular. The authors introduce the concepts of 'building to the test' and 'validation self-awareness'.

the agents can die; the work survives; this was useful on a real production system; please try to break it.

Reddit r/AI_Agents

The author open-sources mishe-tauftauf, a small local coordination system for coding agents based on 'observability-oriented programming,' where ephemeral agents ('minds') work from shared state documents, a text tape, and checks returning GREEN/RED/UNKNOWN instead of preserving agent continuity. The approach was validated in production on an adtech DSP built by a three-person team reaching ~50k RPS, and the project is released CC0 with a call for others to stress-test it in unfamiliar environments.

AI agents recreate the “rockstar developer” problem, just faster

Reddit r/AI_Agents

The post compares AI agents to 'rockstar developers' who create clever but unmaintainable code, pointing out that agents lack memory of their own actions. It recommends using visible conventions like AGENTS.md, ADRs, and tests to keep agent-generated code understandable by the team.