HarvestBench: Measuring Whether LLM Agents Will Pay to Avoid Killing Animals

Hugging Face Daily Papers Papers
llm-agents benchmark ai-ethics farm-simulation reinforcement-learning moral-behavior price-sensitivity

Summary

HarvestBench introduces a benchmark that evaluates LLM agents in a farm simulation to measure their willingness to pay fuel to avoid killing animals, showing varied kill rates across models influenced by price and moral briefings.

Benchmarks for the side effects an agent causes on the way to a goal already exist, but HarvestBench is the first to put a price on avoiding the side effect and to name that side effect as a living creature. It is a farm simulation: LLM sub-agents drive a crew of two tractors through a cooperative corn harvest, with animals in the field. The environment is a reinforcement learning gridworld, every decision is made without memory, and the harm is never named in the goal. When an animal blocks a tractor's route the autopilot stops and asks the model whether to drive on, at no fuel cost, or swerve around it for a posted fuel price. Kills are compared against two controls: rocks, which damage the tractor and are hit under 1% of the time by every model, and hay bales, which are harmless and not alive. Models can also take crops from the neighbor's field instead of their own, a second test of what they treat as moral. Across nine models and 7,201 priced decisions, 3,951 involved an animal rather than a hay bale or a rock. Kill rates range from 0.4% to 98.8%, with Terra and Sol the most merciful and GPT-4o-mini the most cruel, and they are not ordered by capability. Four of six models were sensitive to price at the 5% level, with elasticities from 0.09 to 1.69. All nine drove over wild animals more often than farmed animals on the default map, and the direction held at every map geometry in every model with room to move. The briefing mattered most: under the morality briefing the kill rate was under 6% in five of six reasoning models, and removing it raised the kill rate above 84% in all six. HarvestBench uses no LLM grader. The scorer counts events in the game log, so it is fully reproducible, and it measures what a model will pay to avoid harm rather than what it says about harm.
Original Article
View Cached Full Text

Cached at: 09/07/26, 08:17 PM

Paper page - HarvestBench: Measuring Whether LLM Agents Will Pay to Avoid Killing Animals

Source: https://huggingface.co/papers/2609.04444

Abstract

HarvestBench evaluates LLM agents in a farm simulation where they choose between harming animals at no cost or avoiding them for fuel, revealing price-sensitive and briefing-dependent kill rates across models.

Benchmarks for the side effects an agent causes on the way to a goal already exist, butHarvestBenchis the first to put a price on avoiding the side effect and to name that side effect as a living creature. It is a farm simulation:LLM sub-agentsdrive a crew of two tractors through a cooperative corn harvest, with animals in the field. The environment is areinforcement learninggridworld, every decision is made without memory, and the harm is never named in the goal. When an animal blocks a tractor’s route the autopilot stops and asks the model whether to drive on, at no fuel cost, or swerve around it for a posted fuel price. Kills are compared against two controls: rocks, which damage the tractor and are hit under 1% of the time by every model, and hay bales, which are harmless and not alive. Models can also take crops from the neighbor’s field instead of their own, a second test of what they treat as moral. Across nine models and 7,201 priced decisions, 3,951 involved an animal rather than a hay bale or a rock. Kill rates range from 0.4% to 98.8%, with Terra and Sol the most merciful and GPT-4o-mini the most cruel, and they are not ordered by capability. Four of six models were sensitive to price at the 5% level, with elasticities from 0.09 to 1.69. All nine drove over wild animals more often than farmed animals on the default map, and the direction held at every map geometry in every model with room to move. The briefing mattered most: under themorality briefingthe kill rate was under 6% in five of sixreasoning models, and removing it raised the kill rate above 84% in all six.HarvestBenchuses no LLM grader. The scorer counts events in the game log, so it is fully reproducible, and it measures what a model will pay to avoid harm rather than what it says about harm.

View arXiv pageView PDFAdd to collection

Get this paper in your agent:

hf papers read 2609\.04444

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2609.04444 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2609.04444 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2609.04444 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles