@GoodfireAI: Neural networks do math by rotating shapes. We found a shape-rotating calculator hidden inside an LLM – and it’s used f…

Neural networks do math by rotating shapes.

We found a shape-rotating calculator hidden inside an LLM – and it’s used for more than just math! (1/6)

How does Llama encode numbers? Not on a number line or in binary, but as positions on several circles in parallel.

This might sound odd, but it’s just a Fourier decomposition (a common mathematical technique). Prior work shows this is true across LLMs! (2/6)

These circles form the inputs and outputs of the model’s geometric “calculator”.

To add two numbers, the calculator solves smaller problems in parallel, one for each circle.

Each circle gets its own little addition problem. (3/6)

The same calculator handles a wide range of tasks, including:

• arithmetic (“7+9”)
• weekdays (“nine days after Friday”)
• months (“six months after August”)

Llama built this mechanism from scratch in training, and uses it with striking elegance and flexibility. (4/6)

How do we know Llama is really using this geometric calculator?

We can steer it, manipulating the circles inside the network and watching the answer change.

The proof is in the steering. (5/6)

This is a glimpse of how neural geometry can lead us to discover mechanisms we’d otherwise miss - in this case, neural computation.

Understanding this machinery paves the way for better debugging, control, and design of AI.

Read the full post: https://goodfire.ai/research/a-geometric-calculator… (6/6)

getting one of those for the office rn

they definitely rotate shapes

we’re big fans of causal evidence :)

thanks for reading. more to come soon!

it’s circles all the way down

we think so!

interesting hypothesis! the role of layernorm is definitely underappreciated

Fourier would have too

it turns out that the addition module always uses base-10, even for months/hours/weekdays

Yes! full paper here