Build a Data Analyst Agent With Jev as the Review Layer

Jev gives agents something most of them are missing: a separate model for judgment.

Instead of asking the same LLM to generate an answer, judge its own work, verify the result, and decide whether it should continue, you can split those responsibilities across different layers.

I wanted to test what that looks like inside a real agent.

So we built a data analyst agent with Vercel eve where Qwen does the main work and Jev sits around the loop as the reviewer.

Qwen reads the database schema, writes SQL, and explains the returned rows. Jev checks whether the original question is clear enough to answer, whether the generated SQL actually matches the user’s intent, and whether the final explanation is grounded in the data that came back.

Everything that should not depend on model judgment stays deterministic: read-only access, row limits, and human approval.

Qwen generates. Jev reviews. Code enforces. Vercel eve runs the loop.

This is how we built it.

Here’s a quick demo:

The pieces, and why each one is there

We used Vercel eve as the agent runtime.

eve handles the agent loop, tool execution, and human approval flow, while the agent itself exposes only the capabilities we want the model to have.

For this project, the setup is intentionally small:

• Model: Qwen (alibaba/qwen3.8-omni-flash) through the Vercel AI Gateway

• Data: SQLite with a seeded sales database containing customers, products, and orders from 2025

• Interface: Gradio

• Tools: inspect_schema and run_sql

• Review layer: Jev from TypeSafe AI

The SQLite database is local, so the demo does not depend on an external database API.

And the model only gets two database tools.

inspect_schema lets it see the tables and columns.

run_sql lets it execute the query after the surrounding checks have passed.

Nothing here is particularly complicated by itself.

The important part is deciding which layer is responsible for which decision.

How a question actually flows

Suppose you ask:

“What was total revenue by product category?”

The basic agent loop is:

plaintextQuestion ↓ inspect_schema ↓ Qwen reads the table and column structure ↓ Qwen writes one read-only SELECT ↓ run_sql ↓ Rows come back ↓ Qwen explains the result

The model never gets direct access to the database.

It sees the schema through a tool, proposes SQL, and receives the returned rows through another tool.

For the seeded demo, completed-order revenue is defined as:

quantity * unit_price

for orders where:

status = ‘completed’

The agent states that definition in the answer instead of silently deciding what “revenue” means.

That distinction becomes important very quickly.

A query can be syntactically valid and still answer a different question from the one the user intended.

The guards are deterministic, not model judgment

This is the first separation we made.

Qwen can propose SQL.

It does not get to decide whether that SQL is safe to execute.

Code does.

Before a query touches the database, run_sql applies deterministic checks.

It accepts exactly one SELECT.

SQLite is opened read-only and query-only mode is enabled.

Anything that attempts a write or a second statement is rejected.

The tool also rejects queries that return more than 200 rows and asks the agent to aggregate the result or add a LIMIT.

And if a query contains no WHERE clause, execution pauses for human approval before the broad scan runs.

None of those checks need Jev.

They do not need Qwen either.

A model call would actually make them less reliable.

If a rule can be checked deterministically, check it deterministically.

The model should only be involved when the problem actually requires semantic judgment.

Generating SQL is easy. Trusting the answer is the work.

The deterministic guards solve one problem:

Can this query safely execute?

They do not answer:

Does this query actually answer what the user asked?

A perfectly safe query can still use the wrong metric.

It can filter the wrong date range.

It can group by the wrong field.

And even if the SQL is correct, the final explanation can make a claim that is not supported by the returned rows.

A read-only database connection cannot protect you from any of those failures.

That is where we added Jev.

Jev reviews the agent at three different points.

1. Is the question clear enough to answer?

Before the question reaches the main agent loop, Jev checks whether it is answerable from the available schema.

We tested this with:

“Who is our best customer?”

That sounds reasonable, but “best” is undefined.

Best by revenue?

Number of purchases?

Average order value?

Most recent activity?

Jev flagged the question as ambiguous and returned clarify with 93% confidence.

Instead of inventing a metric and confidently answering the wrong question, the agent asks the user to define what “best” means.

That is a small decision, but it changes the behavior of the whole agent.

2. Does the SQL actually match the question?

The next review happens after Qwen writes the SQL but before the query executes.

Jev receives:

• the user’s question

• the database schema

• the proposed SQL

It then judges whether the query actually represents the user’s intent.

If there is a confident mismatch, the agent gets the reason and one chance to revise the SQL before execution.

This is different from the deterministic safety layer.

A regex can tell you that the query is a SELECT.

It cannot reliably tell you that the query answered “revenue last month” instead of “revenue across all time.”

That is a semantic decision.

3. Is the final answer grounded in the returned rows?

There is one more failure point after the query runs.

The SQL may be correct.

The rows may be correct.

The explanation can still be wrong.

So after Qwen produces the final answer, Jev compares that answer against the rows that were actually returned.

If the answer makes claims the result does not support, we do not silently present it as verified.

It is shown as an unverified draft, with the SQL and returned rows still visible so the user can inspect what happened.

This gives the agent three separate review points:

plaintextQuestion clarity ↓ SQL relevance ↓ Answer grounding

Jev reviews. It never takes the wheel.

This separation is important.

Qwen writes every query and every explanation.

Deterministic code controls what is allowed to execute.

Jev reviews the parts where interpretation is required.

But a Jev verdict cannot:

• authorize unsafe SQL

• disable the row cap

• bypass read-only mode

• answer a human approval prompt

• change the database directly

Jev gives an opinion.

The runtime still decides what that opinion is allowed to influence.

We also deliberately made the review layer fail open.

If Jev is slow, unavailable, or returns a result below the threshold we need, the application does not suddenly become unusable.

The answer can still come through with a visible note that the review did not run.

A reviewer becoming unavailable should degrade to “no opinion.”

It should never silently become “approved.”

Confidence is useful for judgment. It is not permission.

The full loop

Once all of those pieces are combined, the agent looks like this:

plaintextPlain-language question ↓ Jev clarity review ↓ Qwen inspects the schema ↓ Qwen writes read-only SQL ↓ Jev SQL relevance review ↓ Deterministic execution guards ↓ Human approval when required ↓ SQLite executes the query ↓ Rows return to Qwen ↓ Qwen writes the answer ↓ Jev grounding review ↓ Answer shown or flagged as unverified

This is the part of the project I find most useful.

There is no single model responsible for everything.

Each layer gets a much narrower job.

Where it fails, and what to know before relying on it

We tested this against a seeded SQLite database, not a production warehouse.

The goal was to test the control flow, review behavior, and separation between deterministic checks and model judgment.

It was not a scale benchmark.

There is also one implementation detail worth knowing.

The clarity and grounding reviews call the evaluation endpoint directly, so we receive a confidence value that we can threshold.

The SQL-relevance check currently runs through eve’s evaluate helper.

That path returns the verdict, but it does not always expose a confidence value in the same way.

So that review can fail open and allow the query to continue without blocking.

That should be interpreted as:

“No opinion.”

Not:

“Approved.”

And none of these evaluation checks make Jev a guarantee of correctness.

The things keeping database access safe are still the deterministic controls:

read-only execution

row limits

human approval

Jev helps us judge whether the agent’s reasoning appears aligned with the task.

It does not replace those controls.

The decision rule

The main rule I would reuse from this project is simple:

Guard with code when the condition is checkable. Use a model when the question actually requires judgment.

“Is this query read-only?”

Code can answer that.

“Did this query return more than 200 rows?”

Code can answer that.

“Does this query require human approval?”

Code can answer that.

But:

“Is this question ambiguous?”

“Does this SQL actually represent the user’s intent?”

“Does this explanation follow from these rows?”

Those are semantic questions.

That is where a model like Jev becomes useful.

Keeping those two categories separate is what makes the architecture much more useful than simply adding another evaluator call after an LLM response.

The full code and setup, including the seeded database and Gradio app, are here:

https://github.com/Sumanth077/Hands-On-AI-Engineering/tree/main/ai_agents/nl_data_analyst_agent

If you want to go deeper on Vercel eve itself, Vercel is also running a hands-on session at Ship on October 15 in San Francisco:

https://vercel.com/ship

Thanks to Vercel for working with me on this one.