Spectral Rewiring for Exploration, Purification, and Model Merging
Summary
This paper introduces SAR, a training-free method that projects RL updates onto a compact reasoning core in spectral space, enabling purification, improved exploration, and stronger model merging.
View Cached Full Text
Cached at: 07/20/26, 09:42 AM
Paper page - Spectral Rewiring for Exploration, Purification, and Model Merging
Source: https://huggingface.co/papers/2607.03065 RL post-training works incredibly well, but its parameter update is still a black box.
🧐Our question: can we understand and locate the effective part of an RL update, remove the noisy directions, and even improve the model after training?
We find the answer is yes.
📌The reasoning-effective part of an RL update can be represented as a compact rewiring matrix in the base model’s spectral space.
🤖 This leads to SAR: a training-free post-hoc method that projects the raw RL update onto this compact reasoning core, enabling us to understand, purify, and merge RL-trained models.
Key results: 1️⃣ The reasoning core of RL is highly compact: With less than 1% spectral parameters, SAR can recover or improve full-RL gains.
2️⃣ Dropping noisy directions helps: For math, SAR solves the exploration degradation often observed after RL. It also improves agentic coding on large-scale in-house models.
3️⃣ SAR purifies mixed-domain RL: On a 32B model jointly trained for math, code, instruction following, and chat, SAR improves coding and math exploration while keeping instruction following stable.
4️⃣ SAR makes model merging stronger: After SAR purification, merged models can surpass the best single-domain experts. We observe the same trend on production-scale in-house models.
Overall, SAR is training-free, broadly useful, and gives us a new geometric lens on what RL is really changing inside reasoning models.
Similar Articles
Spectral Rewiring for Exploration, Purification, and Model Merging
The paper introduces Subspace-Aligned Rewiring (SAR), a post-hoc editing method that retains the spectral core of RL updates to preserve reasoning gains, remove interference, and enable model merging across experts, achieving strong performance with minimal parameters.
Sparsity Curse: Understanding RLVR Model Parameter Space from Model Merging
This paper investigates the 'sparsity curse' in merging RLVR (Reinforcement Learning with Verifiable Reward) models, finding that sparse updates cause near-orthogonal parameter directions that hinder aggregation, and proposes SAR-Merging, which uses Fisher information and sparsification to resolve conflicts and improve merging performance on math and coding tasks.
FUSAR-R1: A Large-Scale Reasoning Model for Intelligent Interpretation of SAR Images
The paper proposes FUSAR-R1, a large-scale reasoning model for SAR image interpretation that uses chain-of-thought reasoning and reinforcement learning to achieve better performance than existing models.
Rebellious Student: Reversing Teacher Signals for Reasoning Exploration with Self-Distilled RLVR
This paper introduces RLRT, a method that reverses teacher signals in self-distillation to reinforce successful student deviations, enhancing reasoning exploration in large language models.
Reinforcing Dual-Path Reasoning in Spatial Vision Language Models
This paper introduces SR-REAL, a unified framework for spatial vision-language models that combines linguistic deduction and 3D geometric reasoning via reinforcement learning, enabling robust multi-step spatial reasoning across diverse tasks.