dbreunig/nesformer

Source: https://github.com/dbreunig/nesformer

nesformer

An NES CPU compiled into the weights of a stock Qwen3.5 model.

The NES’s 6502 CPU (the core of its 2A03 chip), its 2 KB of work RAM, and a cartridge’s program ROM run inside a standard Qwen3_5ForCausalLM checkpoint. Nobody trained these weights; a compiler computed every one. When you play a game, the model predicts the CPU’s execution log one token at a time. A small host program plays the rest of the console around it: it draws the picture, reads the controller, and fires the vblank interrupt.

nesformer builds on Qwenputer by Divyansh Agrawal, which compiled a CHIP-8 and then a RISC-V CPU into Qwen3.5-4B. It reuses Qwenputer’s compiler and its central trick: the model’s Gated DeltaNet state is the machine’s memory.

Super Mario Bros. (World 1-1)	Full Game by Doug Fraker (title screen and first level)

Every CPU token in these runs came from the model. For the bundled test games, every frame also matches a conventional 6502 emulator’s output exactly (see How it was checked).

Quick start

1. Folder layout

Unzip nesformer.zip and nesformer-weights.zip into the same folder. The weights go next to the nesformer/ package, not inside it:

nesformer/              <- run commands from here
├── nesformer/          <- the Python package
├── chipformer/         <- Qwenputer's compiler
├── roms/               <- test ROMs (put your own here too)
├── examples/           <- sample GIFs and frame sheets
├── tests/
├── LICENSE
├── LICENSE.chipformer
├── README.md
├── pyproject.toml
└── weights/            <- from nesformer-weights.zip
    ├── config.json
    ├── generation_config.json
    ├── model.safetensors
    ├── nesformer.json
    └── SHA256SUMS

The CLI looks for weights/ beside the nesformer/ package. To keep the weights elsewhere, pass --weights /path/to/weights to run, check or trace, or give the path to info.

To confirm the weights unzipped correctly, run shasum -a 256 -c weights/SHA256SUMS from the top folder.

2. Install

You need Python 3.11 or newer.

python3 -m venv .venv
source .venv/bin/activate          # Windows: .venv\Scripts\activate
pip install numpy scipy numba pillow

3. Play a game

python -m nesformer run roms/thwaite.nes --frames 600 --show \
  --buttons "start@90-95+200-205,a@230-233+250-253+270-273+290-293+310-313+330-333+350-353+370-373+390-393+410-413+430-433+450-453+470-473+490-493+510-513+530-533+550-553+570-573+590-593"

--show draws each frame in your terminal as the model generates it. When the run ends, the CLI saves an animated GIF named after the ROM (here thwaite.gif) in the current folder.

Expect it to be slow. The first run spends about 20 seconds compiling the engine with numba. After that, a frame takes 1–6 seconds, so 600 frames (10 seconds of NES time) take around 25 minutes.

Playing your own ROMs

Point run at any .nes file that uses the NROM board (iNES mapper 0), such as Super Mario Bros., Donkey Kong or Excitebike. You supply the ROM; nesformer only ships the three freely licensed test ROMs in roms/.

python -m nesformer run roms/smb.nes --frames 600 --show \
  --buttons "start@60-65,right@200-600,a@250-265+300-315+360-375+420-435+480-495+540-555"

That script starts the game, then walks Mario right and jumps about once a second. The CLI loads the ROM into the model’s state at startup, so the same weights run any NROM game without a rebuild. Games on other boards stop with a message: see Limitations.

Controller scripts

--buttons takes a comma-separated list of entries. Each entry names one or more buttons joined by +, then @, then one or more frame ranges joined by +:

start@60-65                 press Start during frames 60 to 65
right@200-600               hold Right from frame 200 to frame 600
a@250-265+300-315           press A twice
right+b@200-600             hold Right and B together (run, in Super Mario Bros.)

The buttons are a, b, select, start, up, down, left and right. Frames count from power-on at 60 per second, so frame 600 is 10 seconds in. Games ignore input on screens where you can’t move, so holding a button a little early does no harm. In Super Mario Bros., a longer A press makes a higher jump.

Commands

Options for run:

run, check and trace all accept --weights, --engine, --buttons and --no-skip. check --tokens N sets how far to compare (default 50,000). trace --tokens N sets how many tokens to print (default 40). build --out DIR writes the checkpoint somewhere other than weights/.

How it works

What’s in the weights, and what isn’t

This split follows Qwenputer’s CHIP-8 build, whose host injects timer ticks and draws the display from the trace. The host never sees the CPU’s registers or RAM except through the trace.

The memory trick works like this. Each DeltaNet head keeps a 128 × 128 state matrix that the delta rule updates at every token. With the gate held open and a one-hot key, an update overwrites exactly one row of that matrix, and a one-hot query reads it back. So every row is a memory cell, and the cartridge, RAM and registers all live in rows.

The checkpoint

weights/ is an ordinary Hugging Face folder. It loads with Qwen3_5ForCausalLM.from_pretrained and needs no custom code.

nesformer.json records which layer and head holds each part of the machine. A run writes the cartridge, registers and RAM into those DeltaNet states, then decodes greedily.

Layer plan

The trace

Each instruction becomes a record, and the record’s last token commits it. As in Qwenputer’s RISC-V core, the PC is implied rather than written: each final token carries its instruction’s length, three DeltaNet counters tally the instructions of each length since the last jump, and the PC is the jump target plus that offset. Excerpts from Full Game:

SEI            P(000100 px0 L1)
LDX #$40       X($40 c0 v0 px0 L2)
STX $4017      IOW($4017=$40 L3)
TXS            S($FF L1)
BIT $2002      IOR($2002) IN($00) P(000110 px0 L3)       <- the host injects IN
BPL $8017      J($8017)
JSR $8279      PSH($FE) PSH($FD) J($8279)                 <- the return address is pushed in-circuit
RTS            PUL($FC) LO($7B) PUL($FD) J($827C)
STA ($72),Y    LO($08) HI($03) ST($00 a7=0 L2)            <- pointer bytes latched, then the store
STA $4014      IOW($4014=$02 L0) DM(0:$FF) … DM(255:$00) NT(L3)   <- OAM DMA

Most instructions take a single token; real game code averages 1.1–1.3 tokens per instruction. nesformer/spec.py documents every token family.

How it was checked

• The reference CPU (nesformer/cpu.py) matches nestest’s golden log for 3,348 instructions, and the py65 emulator on 5.7 million instructions of game code.

• The exact circuit, run in Qwenputer’s evaluator with its bf16 exactness checks, predicts every reference token: all of nestest plus 283,610 positions of two games’ boot sequences.

• The checkpoint in stock transformers (bf16, CPU) predicts 2,651 positions of nestest and Full Game with no mismatches.

• The weights, free-running, generate traces identical to the reference CPU’s:

  Run	Tokens	Frames	NES time	Result
  nestest (official opcodes)	4,586	—	—	identical, ending in the expected HALT
  Thwaite, check	700,000	271	4.5 s	identical trace and frames
  Thwaite, run	1,916,527	600	10.0 s	all 300 saved frames match
  Full Game, run	2,645,885	420	7.0 s	all 210 saved frames match

• The idle-loop skip changes nothing visible: frames and I/O writes, cycle timestamps included, match a run without it.

Speed

The default engine computes Qwen3.5’s forward pass from model.safetensors, compiled with numba and skipping the all-zero blocks. It ran at about 1,650 tokens/s on one CPU core of the machine it was built on. The plain numpy engine does about 210, and stock transformers about 10. A real NES would need about 700,000 tokens per second.

Games spend most of each frame in a loop that waits for vblank. When the host sees a loop body repeat exactly, with no RAM or I/O writes, it advances the clock to just before the next PPU event, and the remaining iterations still run. That cuts the token count by 2.6× to 4.2×. Overall, nesformer runs 150–290× slower than the real console.

Limitations

• NROM cartridges only (iNES mapper 0). The CPU can address only 32 KB of program ROM at a time. Bigger games carry a bank-switching chip (a mapper, such as MMC1 or MMC3) that swaps which part of the ROM the CPU sees. nesformer’s ROM reads happen inside the model, from a fixed 32 KB table, so it can’t follow a bank switch yet. Supporting a mapper means storing the whole ROM in the state, keeping the bank registers there too, and adding one stage that turns each CPU address into a ROM address.
• Official opcodes only. An unofficial opcode emits HALT.
• Code must run from the cartridge ROM. A jump into RAM emits HALT.
• No IRQs (APU frame IRQ, DMC). BRK and NMI work.
• Read-modify-write instructions on I/O registers don’t produce an I/O write, and 4020–7FFF reads as 0 (no PRG-RAM).
• The PPU and APU aren’t in the weights. The PPU renders one scanline at a time, so mid-scanline effects aren’t reproduced. Sprite-0 hit and sprite overflow are approximate, and there’s no sound.

Tests

pip install pytest py65
python -m pytest -q tests

The tests take a few minutes. They check the reference CPU against nestest’s log, the circuit against the reference, the weights free-running against the reference, and the idle-loop skip.

Running the checkpoint in stock transformers

pip install "torch>=2.6" "transformers>=5.22"
python -m nesformer run roms/thwaite.nes --frames 3 --engine hf

nesformer/hfcheck.py shows the whole procedure: load the checkpoint, decode one padding token, write the machine’s state into the cache’s DeltaNet states, then feed tokens one at a time.

Files

Credits

• Qwenputer and its chipformer compiler: MIT, © 2026 Divyansh Agrawal (LICENSE.chipformer). That license also covers the code adapted from Qwenputer in nesformer/core.py and nesformer/weights.py. Its last paragraph refers to Qwenputer’s CHIP-8 ROMs, which this repository does not include.
• Test ROMs: nestest by Kevin Horton; Full Game by Doug Fraker (MIT); Thwaite by Damian Yerrick (GPL-3.0, source at pinobatch/thwaite-nes).

License

nesformer is released under the MIT License (see LICENSE). The ROMs in roms/ keep their own licenses, listed in roms/SOURCES.md.