A practical recipe for building agent trajectory datasets

If you are trying to train or fine-tune a tool-using agent, I would not start by collecting random chat logs. I would start by defining what a good trajectory looks like. For me, a useful trajectory has at least six parts: the task, the agent’s reasoning state, the tool call, the tool input, the observation returned by the environment, and the final answer. If any of these are missing, the data becomes much less useful for training. Tool-using agents need to learn the connection between intent, action, environment feedback, and correction. A practical pipeline could look like this. Record trajectories in a structured format. Generate tasks that require actual tool use, not just text completion. Run the agent in a sandbox where tools can be called safely. Save both successful and failed runs, because failures are useful for evaluation and refinement. Score each trajectory on success, efficiency, coherence, and tool-use correctness. Filter out malformed or low-signal traces. Select a diverse subset so the dataset covers different tools, task depths, and recovery patterns. For promising but flawed trajectories, rerun or repair them with explicit diagnostics. The key is to treat agent traces as data assets, not debug logs. Debug logs are written for humans after something happened. Training trajectories should be designed, generated, evaluated, and cleaned with model learning in mind. This also creates a better feedback loop. If your model keeps making bad tool calls, you can synthesize more trajectories around that failure mode. If it struggles with long tasks, you can generate deeper traces. If it overuses tools, you can score and select for efficiency. I think agent trajectory datasets will become one of the main bottlenecks for training practical agents, and OpenDCAI/DataFlow is one open-source project moving in that direction.