@samsja19: do not delete your production trace, turn them into fuel for your next post training
Summary
Advocates using production traces as data for AI post-training, highlighting the growing scale of data spending.
View Cached Full Text
Cached at: 07/07/26, 12:14 AM
do not delete your production trace, turn them into fuel for your next post training
will depue (@willdepue): A Stargate for Data
Labs are on a trajectory towards >$100B/year of data spend by 2030. As we begin the trillion-dollar compute project, we need to think about the equivalent civilizational-scale effort for the other core ingredient: data.
At the foundation of the scaling
Similar Articles
@martin_casado: This tackles a very hard, very important problem in AI systems. Basically how do you expose your traces at scale to age…
A tweet by Martin Casado highlighting a solution to the difficult problem of exposing traces at scale to AI agents, balancing cost and AI leverage.
I'm tired of manually debugging traces
A developer builds a debugging tool for AI agents that compares replays against reference runs to identify where behavior first drifted, expressing frustration with manual trace debugging.
@LangChain: Head of AI @nlarusstone on the patterns @benchling uses to look at production traces.
Head of AI at Benchling discusses patterns for analyzing production traces in a tech talk.
@hwchase17: Detecting issues in production agent traces is hard. You have to do it cheaply (because of volume) but also accurately …
Harrison Chase announces a post-trained model for detecting issues in production agent traces, claiming SOTA accuracy at 10-100x cheaper rates than frontier models.
@Vtrivedy10: https://x.com/Vtrivedy10/status/2066571435871551655
A joint study by LangChain Labs and Fireworks AI demonstrates fine-tuning an open Qwen model to create a trace judge that detects 'perceived error' in production traces, achieving frontier performance at up to 100x lower cost. The model is evaluated on two internal datasets and shows generality across applications.