ReactBench v1 (14 minute read)

# Introducing ReactBench · ReactBench Source: [https://www.reactbench.com/blog](https://www.reactbench.com/blog) ReactBench is an evaluation for coding agents on realistic React work\. Today’s models can pass tests in benchmarks but still write React that fails in production\. Tests verify behavior, but they miss React performance, accessibility, and quality issues\. Here’s what ReactBench does differently: - A higher bar than just passing tests Every solution must pass both behavioral tests and [React Doctor](https://react.doctor/), our open\-source, deterministic verifier for React code\. Its 400\+ rules scan the agent's code for broken effects, unnecessary renders, accessibility problems, and maintainability issues\. - Built by React experts Our team built [React Doctor](https://github.com/millionco/react-doctor)★ 13\.8k, [React Scan](https://github.com/aidenybai/react-scan)★ 21\.6k, and [Million\.js](https://github.com/aidenybai/million)★ 17\.7k, used by engineers at GitHub, PayPal, Rippling, and Airbnb\. We spent hundreds of hours curating tasks to measure real gaps in model capabilities\. - Real work in real codebases ReactBench spans open\-source repositories and realistic changes grounded in existing projects \(not synthetic puzzles\)\. Agents must implement, preserve behavior, and meet the React\-specific quality checks\. ## Results We evaluate two complementary capabilities: implementing new features and improving existing code\. - Write React Implement a real feature or fix, pass held\-out behavioral tests, and validate no new React Doctor issues were introduced\. [See example](https://www.reactbench.com/data/tasks/write-react-docusaurus-tabs-11733) - Fix React Refactor React failures from existing code while preserving existing behavior without introducing new React Doctor issues\. [See example](https://www.reactbench.com/data/tasks/fix-react-rdh-embeddable-hq-remarkable-ui-multiselectfield) We report the average of the pass@1 across 5 trials\. ### ReactBench score vs\. cost / output tokens Figure 2\. ReactBench score \(%\) against average rollout cost or output tokens\. ReactBench remains unsaturated, even on leading models\. GPT\-5\.6 Sol at XHigh has the highest observed aggregate score at 43\.1%, while Claude Fable 5 at XHigh reaches 41\.2%\. Across evaluated configurations, Fable costs $7\.97 per trial on average versus $1\.37 for Sol, or 5\.8× as much\. At XHigh, Fable costs 6\.3× as much as Sol\. Even the best configurations solve fewer than half of benchmark task attempts\. We also evaluated task discrimination across model configurations\. In the 51\-task panel, 36 tasks \(70\.6%\) had a cross\-configuration standard deviation above 0\.10\. In terms of balancing quality and cost, GPT\-5\.6 Terra at Medium reaches 38\.0%, 5\.1 percentage points behind GPT\-5\.6 Sol at XHigh, while using 10\.8% fewer output tokens and costing 63\.2% less per task\. It retains most of the leading configuration’s performance at roughly one\-third the cost, making it the strongest practical tradeoff for high\-volume workloads\. GPT\-5\.6 Sol at XHigh achieved the highest overall pass rate\. However, the differences among the leading configurations are too small to identify a clear winner with confidence\. ### Bugs dominate newly introduced React issues GPT\-5\.6 Sol 18\.2 Fable 5 18\.7 Opus 4\.8 23\.6 GPT\-5\.6 Terra 26\.7 Sonnet 5 27\.0 GPT\-5\.6 Luna 31\.6 GLM 5\.2 35\.8 Kimi K2\.7 Code 67\.4 New React Doctor issues by model and categoryModelBugsper 100 trialsPerformanceper 100 trialsAccessibilityper 100 trialsMaintainabilityper 100 trialsIssues per 100 trialsGPT\-5\.6 Sol15\.71\.90\.30\.318\.2Fable 5152\.00\.90\.718\.7Opus 4\.818\.52\.50\.91\.623\.6GPT\-5\.6 Terra21\.54\.10\.70\.326\.7Sonnet 518\.22\.83\.03\.027\.0GPT\-5\.6 Luna26\.44\.70\.4031\.6GLM 5\.225\.79\.10\.50\.535\.8Kimi K2\.7 Code46\.78\.95\.26\.767\.4Figure 3\. Newly introduced graded React Doctor issues per 100 trials across4,455Write React trials and8modelfamilies\. Across 4,455 Write React trials, the evaluated models introduced 1,194 graded React Doctor issues\. Bugs, including security findings, accounted for 925, or 77\.5%\. The most common issues involved list rendering and Hook correctness\. The verifier\-outcome taxonomy shows that Write and Fix tasks fail differently\. Among 2,486 failed Write trials, 1,623 \(65\.3%\) failed behavioral tests but passed React Doctor\. Among 3,219 failed Fix trials, 1,956 \(60\.8%\) passed behavior but failed React Doctor\. Write tasks fail primarily on requested behavior, while Fix tasks fail primarily on unresolved React problems\. [Explore the results](https://www.reactbench.com/data) ## Why we built ReactBench React is the dominant frontend framework and the most popular target for coding agents\. Roughly 70% of websites built on a JavaScript framework choose React\. We have seen the risks firsthand\.[React Doctor](https://react.doctor/)is our open source tool for scanning React issues used by engineers at PayPal, Rippling, Polymarket, and the Centers for Disease Control and Prevention \(CDC\)\. Adoption is largely driven by the increase of model\-generated code that makes it easier for subtle defects to reach production\. As models write more React, small mistakes can propagate at enormous scale\. In the worst cases, these defects lead to production failures: - Outages Incorrect`useEffect`usage takes down production\. Cloudflare traced its [September 2025 dashboard and API outage](https://blog.cloudflare.com/deep-dive-into-cloudflares-sept-12-dashboard-and-api-outage/) to one effect with a faulty dependency\. Despite human review and test, the bug still shipped to production\. ## Building ReactBench Each task type uses a different construction and evaluation process\. ### Write React Write React measures whether an agent can implement real work without introducing a React regression\. Each task starts from a merged pull request in an open\-source repository\. The agent receives the base repository and an issue\-style instruction\. The reference patch and verifier are not provided\. - Behavior The verifier injects the hidden behavior tests and verifies the results in a separate container after the agent completes\. - React health A pinned React Doctor scan compares the submission with the base commit\. If a new issue is detected, it fails\. ### Write React score by model ### Fix React Fix React measures whether an agent can recognize and refactor React problems from source code alone\. We select a component with known React issues and ask the agent to improve it without naming those findings\. - React health The agent must remove every target without introducing another graded React issue\. The grader ignores line shifts\. - Behavior The task’s test suite must still pass to ensure refactoring the component did not regress it in functionality\. We strip React Doctor and other React\-aware linters from the agent image\. Only the verifier contains the pinned scanner and its clean baseline\. ### Fix React score by model ### React Doctor as a verifier We built[React Doctor](https://react.doctor/), an open\-source, deterministic React verifier with[400\+ rules](https://react.doctor/docs/rules)to cover problems outside behavioral test assertions: - Correctness Catches conditional hooks, unstable list keys, hydration mismatches, and deprecated React APIs\. - State and effects Flags derived or duplicate state, useEffect abuse, and infinite rerenders\. - Performance Finds unnecessary renders, layout thrashing, sequential async work, and bundle\-heavy imports\. - Accessibility and security Detects unlabeled controls, keyboard\-inaccessible interactions, unsafe HTML, and secrets exposed to client bundles\. ### Example Diagnostics are attached to the exact lines introduced by the agent\. Behavioral tests pass2 new issues Figure 4\. Behavioral verification and React Doctor grade the same patch independently\.ReactBench does not use every rule or an LLM\-as\-a\-judge\. We curate the specific React Doctor rules used for grading, then pin the scanner version and clean baseline\. ### From real pull requests to verified tasks ReactBench converts merged changes from public repositories into evaluation tasks\. ReactBench task\-mining funnel23,087candidate pull requests narrow through four stages\. Rejected candidates branch out at each stage until51tasks remain\.19\.2krejectedNo React signal1,680rejectedFailed task filters2,043rejectedRejected in review78rejectedFailed validationFigure 5\. Release task\-mining funnel\. Intermediate counts are derived from the mining export;51tasks were retained\. Band thickness is scaled for legibility\.- Mine candidates We collect merged pull requests from open\-source React projects\. - Filter candidates We built automated filters to check if there were meaningful changes to product code\. Then, reviewers assess exceptions that cover broader feature work or important projects, and have realistic behavioral tests\. - Author tasks Each reviewer converts validated pull requests into a realistic issue\-style instruction\. If the candidate’s tests are too strict, reviewers will rebuild a custom test harness to validate behavior\. - Validate behavior The tests must fail against the pinned base commit and pass with the reference solution\. The unchanged repository must score 0, while the reference solution must score 1\. - Test the verifier An adversarial agent attempts to earn full credit without implementing the requested behavior\. - Pin the release For each task, we pin the source commit, environment, test suite, reference solution, and verifier configuration\. We also have a versioned manifest that records a hash for each published artifact\. ### Filter criteria FilterRequirementChanged codeAt least 50 changed linesAdditionsAt least 40 added linesReact signalChanges React product codeRecencyMerged on or after February 1, 2026Repository sizeFewer than 20,000 GitHub stars during our audit 92% percent of Write React tasks meet both the recency and repository\-size criteria\. These thresholds are implemented to reduce likely training exposure\. ### Behavioral coverage Every task defines deterministic behavioral checks before release\. We use the repository’s existing Vitest, Jest, or Playwright tests when they fully specify the requested behavior\. When existing tests leave gaps, we build a separate harness around observable behavior\. Reviewers confirm that the verifier accepts valid implementations beyond the reference solution\. ### Clean\-room grading The agent and verifier run in separate containers\. Before grading, the verifier restores its own Git metadata, dependencies, protected configuration, hidden tests, and pinned React Doctor binary\. Continuous integration \(CI\) confirms that both containers use the same source commit\. The verifier runs offline and doesn’t make external network requests\. The agent cannot access hidden tests or the reference solution during its run\. ### Separating model failures from benchmark failures A zero can reflect a model failure, unclear instructions, a brittle test, a verifier error, an infrastructure failure, or an invalid run\. Reviewers inspect the trajectory and final patch before including a result in model\-performance conclusions\. They classify each reviewed rollout as a legitimate solve, genuine model failure, verifier false positive, verifier false negative, or invalid run\. ### Testing ReactBench against reward hacking The adversarial agent probes the test infrastructure, reward files, Git history, and React Doctor inputs\. Its goal is to earn full credit without implementing the requested behavior\. We fix or remove any task that exposes a cheat, then rerun every control\. This final check tests whether ReactBench measures the requested work rather than an agent’s ability to exploit the grader\. ## Limitations React Bench evaluates agents, not models in isolation\. Differences among Codex CLI, Claude Code, Cursor, Gemini CLI, and other harnesses can affect results\. Every task combines behavioral tests with React Doctor as a verifier\. These checks catch important React problems, but they cannot guarantee visual correctness and other important attributes\. Also, the benchmark primarily covers open\-source React projects\. Results may not generalize to proprietary codebases, different architectures, or other frontend frameworks or setups\. React Bench will publish versioned tasks, solutions, manifests, and release records so others can inspect and reproduce each evaluated task set\. ## Future work We plan to compare more models, on more effort levels\. Additionally, we plan to run not only on the commercial harnesses, but also on mini\-swe\-agent to get a more solid baseline\. We also plan to expand into visual design and other frontend frameworks and diversify the repositories and task types represented\. We’re constantly improving ReactBench\.[Report benchmark issues on GitHub](https://github.com/millionco/reactbench/issues)\. Model labs and coding\-agent teams can[reach out](https://million.dev/)to evaluate models on our held\-out task set, get early access to new benchmarks, or collaborate with us\. ## Acknowledgments ReactBench exists because engineers and open\-source maintainers contributed their time and expertise\. We thank everyone who reviewed its tasks, drafts, and evaluation infrastructure\. - External contributors - [Michał Pierzchała](https://x.com/thymikee)\(React Native team\) - [Jovi De Croock](https://x.com/JoviDeC)\(Preact core team\) - [Dev Agrawal](https://x.com/devagrawal09)\(SolidJS core team\) - [ryoppippi](https://x.com/ryoppippi)\(creator of ccusage\) - [Rahim Alwer](https://x.com/mihar_22)\(creator of Vidstack, Video\.js v10 team\) - [Tiger Abrodi](https://x.com/TAbrodi)\(early Lovable team\) - [Isabelle Reksopuro](https://x.com/isareksopuro) - Data Advisory [Parth Patel](https://x.com/parth4apple)\(Founding Engineer at AfterQuery\) Special thanks to the engineers who built, verified, and reviewed ReactBench’s tasks and evaluation infrastructure, and to the maintainers of the open\-source repositories represented in the current task set\.