Spectral Rewiring for Exploration, Purification, and Model Merging

Hugging Face Daily Papers Papers

Summary

This paper introduces SAR, a training-free method that projects RL updates onto a compact reasoning core in spectral space, enabling purification, improved exploration, and stronger model merging.

Reinforcement learning has become a standard post-training recipe for large language models, but dense full-parameter updates create two deployment-relevant bottlenecks: suppressed reasoning performance, often reflected by premature saturation of test-time scaling, and interference when consolidating multiple capabilities through multi-domain training or model merging. We show that the reasoning-effective component of these updates is largely concentrated in the base model's spectral space, motivating Subspace-Aligned Rewiring (SAR), a post-hoc editing method that retains this spectral core while removing orthogonal components. SAR therefore preserves reasoning gains and filters residual update directions that suppress performance or amplify cross-domain interference. Across several model families and scales, SAR extracts compact reasoning cores using as little as approximately 0.58% of total parameters: it preserves over 99% of post-training performance and improves high-k exploration in mathematical reasoning, and generalizes to agentic coding by improving six of seven open benchmarks on an in-house model. SAR also purifies mixed-domain training updates by releasing suppressed coding capability while maintaining math reasoning and instruction following. It further enables model merging across experts, yielding cross-domain generalization that surpasses previous merging baselines and even the best single-domain experts. Overall, SAR shows that extracting reasoning-effective updates from parameter geometry can serve as a training-free mechanism to improve reasoning and multi-domain performance.
Original Article
View Cached Full Text

Cached at: 07/20/26, 09:42 AM

Paper page - Spectral Rewiring for Exploration, Purification, and Model Merging

Source: https://huggingface.co/papers/2607.03065 RL post-training works incredibly well, but its parameter update is still a black box.

🧐Our question: can we understand and locate the effective part of an RL update, remove the noisy directions, and even improve the model after training?

We find the answer is yes.

📌The reasoning-effective part of an RL update can be represented as a compact rewiring matrix in the base model’s spectral space.

🤖 This leads to SAR: a training-free post-hoc method that projects the raw RL update onto this compact reasoning core, enabling us to understand, purify, and merge RL-trained models.

Key results: 1️⃣ The reasoning core of RL is highly compact: With less than 1% spectral parameters, SAR can recover or improve full-RL gains.

2️⃣ Dropping noisy directions helps: For math, SAR solves the exploration degradation often observed after RL. It also improves agentic coding on large-scale in-house models.

3️⃣ SAR purifies mixed-domain RL: On a 32B model jointly trained for math, code, instruction following, and chat, SAR improves coding and math exploration while keeping instruction following stable.

4️⃣ SAR makes model merging stronger: After SAR purification, merged models can surpass the best single-domain experts. We observe the same trend on production-scale in-house models.

Overall, SAR is training-free, broadly useful, and gives us a new geometric lens on what RL is really changing inside reasoning models.

Similar Articles

Spectral Rewiring for Exploration, Purification, and Model Merging

arXiv cs.LG

The paper introduces Subspace-Aligned Rewiring (SAR), a post-hoc editing method that retains the spectral core of RL updates to preserve reasoning gains, remove interference, and enable model merging across experts, achieving strong performance with minimal parameters.

Sparsity Curse: Understanding RLVR Model Parameter Space from Model Merging

arXiv cs.LG

This paper investigates the 'sparsity curse' in merging RLVR (Reinforcement Learning with Verifiable Reward) models, finding that sparse updates cause near-orthogonal parameter directions that hinder aggregation, and proposes SAR-Merging, which uses Fisher information and sparsification to resolve conflicts and improve merging performance on math and coding tasks.

Reinforcing Dual-Path Reasoning in Spatial Vision Language Models

Hugging Face Daily Papers

This paper introduces SR-REAL, a unified framework for spatial vision-language models that combines linguistic deduction and 3D geometric reasoning via reinforcement learning, enabling robust multi-step spatial reasoning across diverse tasks.