WeightBench: Evaluating Language Models as Weight-Space Programmers
Summary
WeightBench introduces a benchmark evaluating LLMs on 42 Weight-Space Programming tasks—reading, fixing, and writing neural network weights—across five tracks. Results show frontier models like GPT-6.1-sol and Claude Opus 5.5 can read small networks and write control policies, but still struggle to patch language models or perform data-free compression.
View Cached Full Text
Cached at: 11/19/26, 03:40 AM
Paper page - WeightBench: Evaluating Language Models as Weight-Space Programmers
Source: https://huggingface.co/papers/2611.18553
Abstract
The WEIGHTBENCH benchmark evaluates the capacity of large language models to perform Weight-Space Programming tasks, which involve reading, modifying, and generating the weights of various neural network architectures.
Recent work shows that large language models (LLMs) can operate directly on the parameters of other neural networks. They can read what a network was trained to do, fix a network that fails, and write working networks from scratch, all withoutgradient descent. This capability has been termedWeight-Space Programming (WSP). Exploring this new paradigm systematically requires a robust benchmark. We introduce WEIGHTBENCH, a benchmark that evaluates LLMs of increasing capability on 42 WSP tasks across five tracks: reading, fixing and writing weights, recognizing edits that are impossible, anddata-free compression. The networks whose weights the LLMs read and edit range from 2K-parametercontrol policiesto 7B-parameter language models. Experimental benchmark results show that frontier agents can already read small networks and write workingcontrol policieswithout ever running them, but cannot yet patch language models, compress them without data, or tell an impossible edit from a hard one. On all 42 tasks, Claude Opus 5.5 reaches a macro score of 0.535 (Sonnet 5.5 0.466, Haiku 5.5 0.402, DeepSeek v4-flash 0.215); on the 38 tasks it was run on, GPT-6.1-sol reaches 0.568 against Opus’s 0.546. Code, the task registry, our OS-level sandbox harness, and the live leaderboard are publicly available at https://github.com/Aayush-Marasini/WeightBench.
View arXiv pageView PDFGitHub1Add to collection
Get this paper in your agent:
hf papers read 2611\.18553
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2611.18553 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2611.18553 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2611.18553 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
Weight-Space Programming builds neural networks like a 1990s researcher
A blog post explores Weight-Space Programming, where frontier LLMs directly read and write neural network weights, showing that an LLM hand-crafting MNIST CNN weights essentially rediscovers classic pre-deep-learning techniques.
Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights
A research blog post finding that frontier LLMs can natively read and write neural network weights — diagnosing trained policies, editing weights to fix reasoning loops and boost Qwen3.5-0.8B's MATH-500 from 7.9% to 48.1%, and hand-writing small game-playing networks — a capability the authors call Weight-Space Programming.
Training Vision Models for LLM-Driven Weight-Space Programming
The paper presents a benchmark of LLM-driven weight-space programming on vision models (CNNs and ViTs) across five LLMs and four datasets, introducing Edit-Ready Training (ERT), a method that rehearses closed-form weight edits to substantially improve LLM edit success, achieving the highest mean scores for GPT-6 Astra and Claude Opus 5.5.
Weight-Space Programming builds neural networks like a 1990s researcher (20 minute read)
The author runs a Weight-Space Programming experiment where Claude Fable 5.2 hand-sets all 5,994 weights of a CNN to classify MNIST without any fitting or search, finding that the resulting network resembles hand-crafted pre-deep-learning techniques — a pattern also observed in the Go network built by GPT-6 Astra in the original WSP post.
Weight-space programming: an LLM edited another model's weights directly and lifted its math score from 7.9% to 48.1%
MIT researchers showed an LLM (GPT-6 Astra) can directly edit another model's weights to fix a repetitive habit, lifting a small Qwen model's math score from 7.9% to 48.1% by modifying ~0.016% of parameters without training, with caveats and independent reproduction still pending.