@j_golebiowski: How to replace a 744B agent with a 1.7B model using your own traces, straight from dlt. Four steps:
Summary
This tweet explains how to use dlt to collect traces and replace a 744B parameter agent with a much smaller 1.7B model in four steps.
View Cached Full Text
Cached at: 07/09/26, 05:49 PM
How to replace a 744B agent with a 1.7B model using your own traces, straight from dlt. Four steps:
-
Point a dlt verified source at your traces (Pydantic, Arize, Langfuse). It pulls them from wherever they live and normalizes every vendor format to one OTel standard.
-
Agent Cost & Usage shows what each agent costs, per model, per person, per customer. Pick one high-volume, well-scoped agent that’s expensive to run.
-
The agent_distillation transform lands a training-ready dataset. We train on validated synthetic data drawn from your traces, not the raw traces, which is why the small model can match the big one.
Does it survive messy production traces? We trained on deliberately degraded ones. The fine-tuned model holds where direct training collapses.
- Swap one URL. The 1.7B model serves behind the same OpenAI-compatible API, 50-90% cheaper. Full walkthrough:
Similar Articles
@adithya_s_k: You can now finetune models on agent traces directly with TRL Claude Code traces Codex traces OpenClaw traces Pi traces…
TRL now supports fine-tuning models on agent traces from various sources like Claude Code, Codex, OpenClaw, and Pi, moving towards a standardized stack for training agentic models.
Split my agent into a cheap router model and a premium synthesis model, bill dropped about 75%
A developer splits their AI agent's LLM calls into a cheap router model (GPT-OSS 120B) for tool-picking and a premium model (gpt-5.4) for synthesis, cutting costs by ~78% while maintaining output quality.
@henrytdowling: This is an amazing writeup on how you are probably not using your agent traces correctly
The article discusses common errors in using AI agent traces and offers advice on proper implementation.
$16 refactor, 400 steps, 95% routed to open MoE
A developer built a routing layer on vLLM to route simple agent steps to a cheap open-source MoE model (21B active) and hard steps to Opus, reducing costs to $15.60 for a 400-step refactor with 93.4% success rate.
Switched my local agent from Qwen3.8 27B to Ornith 1.5 35B-A3B on two 5070 Tis: about 180 tok/s vs 60, same scores on my tests
A user benchmarks local agent models on dual RTX 5070 Tis, finding that Ornith 1.5 35B-A3B (a ~3B-active-parameter MoE) matches Qwen3.8 27B on their tests while running about 3x faster (~180 tok/s) and scaling to 512K/1M contexts with YaRN.