In a Probabilistic System, No Change Is Small

Author: Matt Pocock (TypeScript Educator, Founder of Total TypeScript, Creator of the Eval Tool Evalite; This article is part of his AI Engineer Roadmap course)

As we discussed in our previous article, building applications with LLMs requires a fundamental shift in how you think about software development. You’re no longer designing a deterministic system where inputs map to predictable outputs; you’re dealing with a probabilistic system that is inherently unpredictable.

The key tool for managing this uncertainty is evals. Evals are unit tests for AI engineers. They are your way of extracting predictability from a probabilistic system. They are an essential part of productizing any AI application.

Let’s break down what evals are and why AI applications need them so badly.

1. Why Traditional Tests Aren’t Enough

Traditional software testing relies on a deterministic relationship between inputs and outputs. Each component has clear, well-defined responsibilities.

But systems powered by LLMs are different. Every input goes through a complex, often opaque transformation.

In AI systems, no change is “small.” Their attention mechanisms and transformation pipelines are non-transparent. Writing “flaps” versus “Flaps” when a butterfly flaps its wings can change the output. To put it mildly, building robust systems with them requires extraordinary care.

2. Manual QA Isn’t Enough

It’s easy to get an impressive AI demo running quickly. Getting that AI system into production is not.

Specifically, it’s hard to know whether a change to your application makes it better or worse. You make a change, test a few common prompts, and see if it “feels” better. This is a dangerous way to work.

Pure manual QA often works for deterministic software. You say, “I added a new page,” and the QA team can rigorously test that new page and perform smoke tests on the previous ones.

But in a probabilistic system, this is fatal. When any single change can affect the entire system, you need a way to determine whether your system is getting better or worse.

This is especially true for major changes, like switching which model you’re using or redesigning your prompt architecture.

3. Evals

The key is automation. You need to evaluate your application every time you make a change, or when the underlying model changes.

1. The Traditional Deterministic System

In a deterministic system, automated testing is straightforward. You feed in an input and check the output:

const output = myNormalSystem(input);
// Fails if output doesn't match
assert(output === "my-desired-output");

These assertions either “pass” or “fail.” Typically, an application must pass every test before it’s considered shippable.

2. The Probabilistic System

Writing tests for AI isn’t as direct.

Suppose your application generates articles. You want to check if the output meets production-ready standards. You might need to write assertions for:

• Factuality: Checking if all statements in the output are factually correct.
• Writing Style: Ensuring the text is elegant and well-written.
• Adherence to the Prompt: Ensuring the output actually corresponds to what the user requested.

These are qualitative metrics. They aren’t pass/fail; they require scores. Every time you change your application, you need to know if that change made the system 5% better or 50% worse.

This is what evals do — they give you a score, allowing you to see how well your AI system is performing.

4. Three Types of Evals

You can run three main types of evals on your AI system.

1. Deterministic Evals

There are deterministic evals, which can be written as simple assertions:

const article = writeArticleWithLLM(prompt);
// Article should be longer than 300 words
assert(article.length >= 300);
// Article should be shorter than 2000 words
assert(article.length <= 2000);

These are traditional pass/fail checks. You’d feed a variety of prompts into your system each time, checking if they pass these tests.

They’re simple to write but can only cover a portion of what you want to evaluate.

2. Human Evaluation

For more probabilistic metrics, you have two options.

You can use human evaluation to check if the system is operating correctly. This is often your only option early on, when you don’t have much data.

It’s expensive and time-consuming — but all AI systems will, to some degree, rely on human input.

3. Using an LLM as a Judge

Another technique is to send the results of a prompt to another LLM, using that LLM as a judge. This is currently a very popular method for evaluating AI systems.

Suppose you want to ensure your application tells the truth. You could send the system’s output along with some ground truth answers to an LLM to perform this check.

Using an LLM as a judge makes certain evaluations possible — but at a cost. Running an LLM is expensive, so you need to think carefully about the frequency at which you run them. For example, running your evals on every file change could become prohibitively expensive.

A common strategy is to split your evals into two sets — a small one for local testing and a large one that runs daily.

5. How Do I Continuously Improve My Evals?

Your evals are your method for monitoring and improving your AI system. This also means that the dataset you use to evaluate the system is critical.

You need to ensure your evals represent the data your system will see in production. If you’re building a classifier, you need to ensure your evals cover all the edge cases the system will encounter.

This means it’s crucial to build observability and feedback systems into your application. Once deployed, your users are the judges of whether your system is usable. Simple feedback buttons — like thumbs up and thumbs down — can provide incredibly valuable information about your system’s performance.

The Data Flywheel

Vercel (the creator of v0) has written about the AI Native Flywheel, describing the importance of evals in the AI development process.

The best data source for your evals is your users. By carefully monitoring how users interact with your applications, you can build a feedback loop that helps you continuously improve your system. For example:

1. A user asks your application, “Create a classy React app for me.”
2. Your application generates some React code. But it doesn’t make the UI look “classy”; instead, it uses class in the code (a pun on “classy”).
3. The user gives the response a thumbs down. Perhaps they also leave a comment explaining why.
4. You take the “Create a classy React app for me” prompt, create a new eval for it, and add it to your eval suite.
5. You improve the system until it passes this eval.
6. You redeploy. The next time a user encounters this prompt, they get a better response.

This is the data flywheel in action. By carefully monitoring your system and building in feedback loops, you can ensure your system is always improving.

6. How Do I Run Evals?

There are many ways to run evals. Numerous startups have entered this space, offering tools to run your evals and view results online.

Braintrust is a popular choice. They provide a cloud platform to run evals and share results with your team, along with many other features. You can write evals in TypeScript using their SDK. However, they rate-limit your evals — which can be frustrating when you want to iterate quickly.

I maintain a library called Evalite. It’s a lightweight eval runner based on the TypeScript test runner Vitest.

Evalite is designed to let you run evals locally. It’s not tied to any cloud platform, so you can run it as many times as you want. If you’re just starting out, this is a great option.

7. What Evals Look Like in Practice

Think of an eval as a function:

const score = runEval({
  // 1. The prompts we want to test
  data: [
    "Fish species in the Mediterranean",
    "Story of the first Moon landing",
    "Are Krakens real?",
  ],

  // 2. A function that generates output from a prompt
  task: async (topic) => {
    return generateArticle(topic);
  },

  // 3. Scorers we use to generate the final score
  scorers: [
    // Check if the output is long enough
    length,
    // Use an LLM to check accuracy
    factualAccuracy,
    // Use an LLM to check writing style
    writingStyle,
  ],
});

// 4. A score between 0-100%
console.log(score);

We pass in a set of prompts (1), then the task (2), then the methods for scoring the output (3).

Finally, we get back a score telling us how well this function performed (4).

This is essentially what an eval is. This API is roughly inspired by Braintrust’s autoevals library.

8. It’s Evals, Stupid

This diagram, based on Vercel’s AI Native Flywheel, summarizes the impact of evals on your application.

Evals should be the center of your feedback loop. As more users use your application (distribution), they generate more data (usage). You can use this data to improve your application (data), and then run your evals again (evals).

These evals allow you to respond to new technologies and models, putting you on a path of continuous improvement.

Links

Original article: https://www.aihero.dev/what-are-evals Previous article (The AI Engineer Mindset): https://www.aihero.dev/the-ai-engineer-mindset Evalite: https://www.evalite.dev/ Braintrust: https://braintrust.dev/ Vercel’s AI Native Flywheel: https://vercel.com/blog/eval-driven-development-build-better-ai-faster autoevals library: https://github.com/braintrustdata/autoevals

#Evals #AIEngineering #TypeScript