Automatic tool approvals powered by @typesafeai’s Jev. https://eve.dev/docs/guides/evaluate#evaluate-tool-approvals…

Automatic Model Selection – eve

Source: https://eve.dev/docs/guides/evaluate Choose agent models automatically or evaluate typed questions in your tools and application code.

Useautofromeve/modelsto choose an agent model from an allowlist before inference begins. It uses theAI SDK evaluation API, so the evaluator can be a Vercel AI Gateway model ID or an evaluation model from an installed provider. Useevaluatefromeve/aito ask typed questions in your own tools or application code.

The AI SDK evaluation model specification is experimental and can change in patch releases.

By default,autoevaluates withtypesafe\-ai/jev. Like other AI SDK model strings, it uses Vercel AI Gateway unless the application has configured a different global default provider.

import { defineAgent } from "eve";
import { auto } from "eve/models";

export default defineAgent({
  model: auto({
    options: {
      "openai/gpt-5.6-sol": "Difficult reasoning and engineering tasks",
      "openai/gpt-5.6-luna": "Routine tasks where fast completion matters",
    },
  }),
});

Configure Gateway authentication as you would for any other AI SDK model. eve does not add a TypeSafe credential or transport layer. Duringeve dev, a Gateway evaluator uses the same connection selected through/loginas Gateway language models. A configured AI SDK default provider still owns string model resolution during development. The TUI footer displaysdynamic modelwhen the agent usesauto, then adds the resolved model for the current turn, such asdynamic model · openai/gpt\-5\.6\-luna.

Install the provider package yourself and pass its evaluation model. The provider owns its credentials and settings.

pnpm add @ai-sdk/typesafe-ai

import { typeSafeAi } from "@ai-sdk/typesafe-ai";
import { defineAgent } from "eve";
import { auto } from "eve/models";

export default defineAgent({
  model: auto({
    model: typeSafeAi.evaluationModel("jev-latest"),
    options: {
      "openai/gpt-5.6-sol": "Difficult reasoning and engineering tasks",
      "openai/gpt-5.6-luna": "Routine tasks where fast completion matters",
    },
  }),
});

Any provider that implements the AI SDKExperimental\_EvaluationModelcontract works here.

An option’s key is the value shown to the evaluator. A string value describes a Gateway model whose ID is the key. Use the object form when the selected model is a provider instance, an alias, or needs a reasoning override.

import { anthropic } from "@ai-sdk/anthropic";
import { defineAgent } from "eve";
import { auto } from "eve/models";

export default defineAgent({
  reasoning: "medium",
  model: auto({
    options: {
      "openai/gpt-5.6-sol": "Hard problems",
      my_secret_model: {
        model: anthropic("sonnet-5"),
        description: "Routine work that can use the direct Anthropic provider",
        reasoning: "low",
      },
    },
  }),
});

The evaluator sees option keys, descriptions, and recent text messages. It never receives provider credentials or serialized language model instances. When it selectsmy\_secret\_model, eve resolves the key back to the authored Anthropic model.

Supported reasoning values are"provider\-default","none","minimal","low","medium","high", and"xhigh". An omitted value inherits the agent’s reasoning setting.

Useevaluateto ask choice, score, or boolean questions about the state you pass to it. It defaults totypesafe\-ai/jevand uses the same authentication asauto, including the Gateway connection selected through/loginduringeve dev. Passmodelto use another evaluation model ID or a provider instance. A configured AI SDK default provider takes precedence over the local Gateway connection.

import { evaluate } from "eve/ai";
import { defineTool } from "eve/tools";
import { z } from "zod";

export default defineTool({
  description: "Choose the team that can help with a customer request.",
  inputSchema: z.object({ request: z.string().min(1).max(8000) }),
  async execute({ request }, ctx) {
    const result = await evaluate({
      state: { request },
      questions: {
        team: {
          type: "choice",
          instructions: "Select the team best suited to handle the request.",
          criteria: {
            billing: "Invoices, payments, and refunds",
            support: "Product questions and troubleshooting",
          },
        },
      },
      abortSignal: ctx.abortSignal,
    });
    return { team: result.answers.team.choice };
  },
});

The choice above is typed as"billing" \| "support". Each question appears under its authored key inresult\.answers. Results also include token usage, warnings, provider metadata, and response metadata.

evaluateaccepts AI SDK evaluation options, includingmaxRetries,headers, andproviderOptions. Pass anabortSignalto cancel the request. Input and answer validation, retries, and provider errors follow AI SDK semantics.

You can also callevaluateoutside a tool; it does not require an active eve session. Each call performs its own evaluation.autouses this function and adds the per-turn routing behavior described below.

Useauto\(\{ model? \}\)when an evaluation model should decide whether a tool call can run automatically or needs human approval. It accepts the same AI SDK evaluation model strings and provider instances described above and defaults totypesafe\-ai/jev:

import { defineTool } from "eve/tools";
import { auto } from "eve/tools/approval";
import { z } from "zod";

export default defineTool({
  description: "Deploy an application.",
  inputSchema: z.object({ environment: z.string() }),
  approval: auto({ model: "typesafe-ai/jev" }), // Uses AI SDK string-model resolution
  execute: ({ environment }) => deploy(environment),
});

The evaluation model reviews the tool name and input for dangerous effects. A caution, failed review, or incomplete input requires human approval. SeeHuman-in-the-loop approvalsfor classifier options and data handling.

autoevaluates at the firststep\.startedevent, after the incoming prompt is available and before the selected language model runs. It reuses that choice for later tool-loop steps in the same turn. A new turn makes a new choice, and child sessions route from their own prompts.

The evaluator receives up to eight recent user and assistant text messages, capped at 16,000 characters. Requests without user text and latest messages over the limit fail before provider I/O.

Evaluation validation, retries, provider errors, and model resolution follow AI SDK semantics. Cancelling the active turn aborts evaluation and prevents the choice from being retained.