From Proprietary to Open-Source: Bridging the Distribution Gap via Multi-Agent Protocol Distillation in Agentic Search

Hugging Face Daily Papers Papers
agentic-search knowledge-distillation multi-agent reinforcement-learning open-source proprietary-models

Summary

This paper introduces Multi-Agent Protocol Distillation (MAPD), a framework that distills knowledge from proprietary agentic search models into open-source models using a structured protocol to bridge the distribution gap, achieving state-of-the-art results on seven QA benchmarks.

Agentic search enables large language models to solve knowledge-intensive tasks by interleaving multi-step reasoning with retrieval, yet optimizing this with outcome-based reinforcement learning (RL) provides only sparse supervision. Knowledge distillation can supply denser guidance, and advanced proprietary models with their strong reasoning capabilities are promising teachers. While distilling from proprietary models can densify this supervisory signal, conventional logit-matching is precluded by hidden logits and mismatched tokenizers, whereas raw natural language trajectory imitation transfers superficial stylistic artifacts rather than core reasoning competence. To address the heterogeneous distillation problem and bridge the distribution gap, we propose Multi-Agent Protocol Distillation (MAPD), a joint distillation and RL framework uses a structured, style-normalized protocol as an intermediate representation. An offline multi-agent system (MAS) decomposes each query, retrieves supporting evidence, repairs failed searches, and converts the resulting exploration trace into a JSON protocol containing the task type, reasoning plan, and extractive grounding facts. During training, the protocol is provided only to a privileged branch of the student policy, whose token distributions furnish a dense distillation signal alongside the sparse RL objective. Extensive evaluations across seven QA benchmarks demonstrate that MAPD consistently outperforms competitive distillation and RL, achieving average success rates of 39.4\% on Qwen3-1.7B and 44.4\% on Qwen3-4B. Crucially, the framework generalizes robustly across diverse proprietary teachers while effectively mitigating the student policy from style drift and verbosity degeneration.
Original Article
View Cached Full Text

Cached at: 07/28/26, 06:33 AM

Paper page - From Proprietary to Open-Source: Bridging the Distribution Gap via Multi-Agent Protocol Distillation in Agentic Search

Source: https://huggingface.co/papers/2607.24280

Abstract

Agenticsearchenableslargelanguagemodelstosolveknowledge-intensivetasksbyinterleavingmulti-stepreasoningwithretrieval,yetoptimizingthiswithoutcome-basedreinforcementlearning(RL)providesonlysparsesupervision.Knowledgedistillationcansupplydenserguidance,andadvancedproprietarymodelswiththeirstrongreasoningcapabilitiesarepromisingteachers.Whiledistillingfromproprietarymodelscandensifythissupervisorysignal,conventionallogit-matchingisprecludedbyhiddenlogitsandmismatchedtokenizers,whereasrawnaturallanguagetrajectoryimitationtransferssuperficialstylisticartifactsratherthancorereasoningcompetence.Toaddresstheheterogeneousdistillationproblemandbridgethedistributiongap,weproposeMulti-AgentProtocolDistillation(MAPD),ajointdistillationandRLframeworkusesastructured,style-normalizedprotocolasanintermediaterepresentation.Anofflinemulti-agentsystem(MAS)decomposeseachquery,retrievessupportingevidence,repairsfailedsearches,andconvertstheresultingexplorationtraceintoaJSONprotocolcontainingthetasktype,reasoningplan,andextractivegroundingfacts.Duringtraining,theprotocolisprovidedonlytoaprivilegedbranchofthestudentpolicy,whosetokendistributionsfurnishadensedistillationsignalalongsidethesparseRLobjective.ExtensiveevaluationsacrosssevenQAbenchmarksdemonstratethatMAPDconsistentlyoutperformscompetitivedistillationandRL,achievingaveragesuccessratesof39.4\%onQwen3-1.7Band44.4\%onQwen3-4B.Crucially,theframeworkgeneralizesrobustlyacrossdiverseproprietaryteacherswhileeffectivelymitigatingthestudentpolicyfromstyledriftandverbositydegeneration.

View arXiv pageView PDFAdd to collection

Get this paper in your agent:

hf papers read 2607\.24280

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2607.24280 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2607.24280 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2607.24280 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

OpenThoughts-Agent: Data Recipes for Agentic Models

Hugging Face Daily Papers

This paper introduces OpenThoughts-Agent, an open-source data curation pipeline for training agentic language models, achieving a 44.8% average accuracy across seven benchmarks and outperforming prior open datasets through systematic experiments.