@rohanpaul_ai: New Stanford + Carnegie paper. Screen recordings of real work look like free training data for agents. They aren't, bec…

X AI KOLs Timeline Papers

Summary

The paper presents Task Model Induction, a method that decomposes screen recordings of real computer work into separate tasks and goal trees to better train AI agents, improving performance by 30% on unseen tasks compared to current summarizers.

New Stanford + Carnegie paper. Screen recordings of real work look like free training data for agents. They aren't, because nobody works in one clean sequence of steps. People switch between unrelated tasks and redo the same fix until it passes. Most tools flatten that into one list, so the agent learns a workflow nobody performed. Task Model Induction, from Stanford and CMU, first pulls a recording apart into the separate tasks that were running, then rebuilds each as a goal tree with its loops intact. It works out the goals and the running order in separate passes and merges them, because asking one model for both at once flattens the detail. That recovers 74.9% of what actually happened, more than double the best current summarizer, and skills built from it did 30% better on unseen tasks. It's weakest where sessions get messy, on error fixing and dead ends. So if you're mining demonstrations for agent skills, feed the goal structure, not the transcript. – arxiv. org/abs/2608.20319 Title: "Inducing Task Models from Computer-Use Traces"
Original Article
View Cached Full Text

Cached at: 08/24/26, 01:43 AM

New Stanford + Carnegie paper.

Screen recordings of real work look like free training data for agents. They aren’t, because nobody works in one clean sequence of steps.

People switch between unrelated tasks and redo the same fix until it passes. Most tools flatten that into one list, so the agent learns a workflow nobody performed.

Task Model Induction, from Stanford and CMU, first pulls a recording apart into the separate tasks that were running, then rebuilds each as a goal tree with its loops intact.

It works out the goals and the running order in separate passes and merges them, because asking one model for both at once flattens the detail.

That recovers 74.9% of what actually happened, more than double the best current summarizer, and skills built from it did 30% better on unseen tasks.

It’s weakest where sessions get messy, on error fixing and dead ends.

So if you’re mining demonstrations for agent skills, feed the goal structure, not the transcript.

– arxiv. org/abs/2608.20319

Title: “Inducing Task Models from Computer-Use Traces”

Similar Articles