From Production Traffic to Post-Training: Building a Self-Hosted LLM That Covers the Corporate Request Mix
Summary
A research paper presents a method for training a smaller self-hosted LLM using separate GRPO experts merged via SLERP, which outperforms a larger baseline on enterprise tasks and serves half of platform traffic at lower cost.
View Cached Full Text
Cached at: 09/02/26, 11:45 AM
Paper page - From Production Traffic to Post-Training: Building a Self-Hosted LLM That Covers the Corporate Request Mix
Source: https://huggingface.co/papers/2609.01572 Authors:
,
,
,
,
,
,
,
,
,
,
,
,
Abstract
A smaller self-hosted LLM trained with separate GRPO experts merged via SLERP outperforms a much larger baseline on instruction following, function-calling, and internal tasks while serving half of platform traffic at lower cost.
Data-residency constraints force enterprises to self-host LLMs, but continuous adoption of newer models without decommissioning their predecessors expands the serving fleet, fragmenting a finite GPU pool. We consolidate traffic from over 200 internal applications onto a single model by closing quality gaps identified through production error analysis along three axes:instruction following,function-calling, and internal task distribution. Quality is tracked by offline benchmarks stratified to production traffic and scored bydeterministic verifiersorcalibrated LLM judges. Rather than optimising all objectives jointly, which introduces cross-domainreward interference, we train a separateGRPOexpert per axis and merge them via two-stageSLERP. Each expert’s reward exposes a distinct failure mode, namelysemantic collapse,over-calling, andverbosity hacking, each requiring a domain-specific fix. In non-reasoning mode the recipe surpasses a {sim}7times larger by total parameters baseline on the in-house Arena with 69.6 to 65.8,instruction followingwith 0.85 to 0.83, andfunction-callingwith 0.79 to 0.77, while lifting general dialogue benchmarks. The model absorbs 50% of platform traffic, 116M requests per month, at a fraction of the serving cost.
View arXiv pageView PDFProject pageAdd to collection
Get this paper in your agent:
hf papers read 2609\.01572
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2609.01572 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2609.01572 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2609.01572 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
We stopped optimizing our LLM stack manually — it optimizes itself now
The article describes a company's transition to a self-optimizing LLM stack that uses production traces to automatically route requests and fine-tune models, resulting in significant cost reductions and performance improvements.
Efficient Benchmarking in Production: A Study of an Evolving LLM Agent
This paper explores efficient recurring evaluation methods for production LLM agents, comparing techniques like adaptive testing and fixed subsets, and provides practical recommendations for deployment.
A Year in LLM Serving: Workload Evolution, Caching and Load-Balancing
This paper analyzes a one-year production trace from Chutes to study LLM serving workloads, revealing temporal evolution and user-model interactions to improve serving system benchmarking.
After talking to 20+ teams running LLMs in production, 3 pain points kept coming up independently
Based on conversations with over 20 teams, the author identifies three recurring pain points when using LLMs in production: enterprise-only basics, lack of agent observability, and slow support for new models.
AutoLLMResearch: Training Research Agents for Automating LLM Experiment Configuration -- Learning from Cheap, Optimizing Expensive
This paper introduces AutoLLMResearch, an agentic framework that automates the configuration of expensive LLM experiments by learning from low-fidelity environments and extrapolating to high-cost settings. It aims to reduce computational waste and reliance on expert intuition in scalable LLM research.