@che_shr_cat: 1/ A 5M-parameter model just beat frontier LLMs on hard logical puzzles at less than 1/100,000th of the inference cost.…

1/ A 5M-parameter model just beat frontier LLMs on hard logical puzzles at less than 1/100,000th of the inference cost.

How? By scaling test-time compute in continuous latent space rather than discrete token space.

Let’s unpack how this works.

2/ The paper: Probabilistic Tiny Recursive Model (PTRM) by Amin Sghaier, Ali Parviz, and @jm_alexia.

It solves a massive issue in Tiny Recursive Models (TRMs): they get trapped in suboptimal basins of their latent space during reasoning.

3/ Instead of a single deterministic rollout, PTRM runs K parallel trajectories.

At each recursion step, it injects small Gaussian noise directly into the latent state: z_t = rec(x, z_{t-1} + noise)

This noise helps the trajectory escape local attractors.

4/ How do we choose the right trajectory at the end?

The authors repurposed the pre-trained Q-head (originally used only for halting during training) as a zero-shot verifier.

The path with the highest Q-head confidence score is selected. No retraining needed.

5/ You might expect gradient-guided search (like Langevin sampling) to work better than random noise.

But empirical ablation showed that pushing latents along gradients yielded no benefit over pure Gaussian noise.

Pure noise is simple, fast, and requires zero backprop.

6/ The results are wild: • PPBench accuracy: 62.6% -> 91.2% • Sudoku-Extreme: 98.75% • Cost: $0.001 per solution vs $38.51 for a frontier LLM ensemble

This proves that massive autoregressive models aren’t the only path to deep reasoning.

7/ The main limitation? The verifier bottleneck.

PTRM is bounded by the accuracy of the Q-head. In tasks like Maze-Hard, there is a large gap between the theoretical oracle performance (95.6%) and the Q-head selected performance (85.1%).

8/ This work shows that test-time scaling can happen entirely in continuous latent space.

As we hit the limits of token-based search (MCTS, RL), exploring continuous, multi-step perturbations in compact models looks incredibly promising.

9/ Read my full breakdown here: https://arxiviq.substack.com/p/probabilistic-tiny-recursive-model…

Check out the paper: https://arxiv.org/abs/2605.19943

What are your thoughts on continuous vs. discrete test-time scaling? Let me know!

10/ I also made a visual breakdown of how these latent trajectories escape bad basins.

1/ We have been treating GPU memory all wrong.

What if the GPU didn’t need to store your model at all?

MegaTrain enables full-precision training of 100B+ LLMs on a single GPU by turning VRAM into a transient, stateless cache.

The secret? Inverting the memory hierarchy.