Introducing SWE-ZERO-12M-trajectories: the largest agentic trace dataset in the open, 5.7x larger than the previous largest.

112B tokens · 12M trajectories · 122K PRs · 3K repos · 16 languages

https://t.co/beIxnrHnlR

AlienKevin/SWE-ZERO-12M-trajectories · Datasets at Hugging Face

Source: https://huggingface.co/datasets/AlienKevin/SWE-ZERO-12M-trajectories

https://huggingface.co/datasets/AlienKevin/SWE-ZERO-12M-trajectories#swe-zero-12m-trajectoriesSWE-ZERO 12M Trajectories

The largest agentic-coding trace dataset to date:112 B tokensof execution-free agentic trajectories covering122 K pull requests,3 K repositories, and16 programming languages.

https://huggingface.co/datasets/AlienKevin/SWE-ZERO-12M-trajectories#motivationMotivation

Agentic mid-training has become a standard ingredient for frontier coding models:

• Code World Model(Sept 2025) mid-trains on 5 T tokens, using 3 M agentic trajectories from 10.2 K Docker images across 3.15 K repos.
• Kimi-Dev(Dec 2025) mid-trains on ~150 B tokens of “high-quality and real-world data” to instill BugFixer and TestWriter priors.
• DeepSeek-V4(May 2026) incorporates agentic data during mid-training to enhance coding capability.

All three rely oncontainerized executionto verify trajectories, and that is where the data ceiling sits. Kimi-Dev was first to call it out:

“while repository snapshots from GitHub are available, not all snapshots are equipped with an executable Docker environment.”

The current largest containerized collection,SWE-rebench-V2, ships 32 K tasks with pre-built images, and concedes the ceiling by additionally releasing120 K+ tasks that could not be containerized(4× more) with only install instructions and fail-to-pass metadata, because building those images is prohibitively expensive.

SWE-ZERO(prior work) was the first to propose anexecution-freepipeline for agentic SWE training data, framing the two Docker-imposed bottlenecks:

• Data scalability: a large fraction of real-world repositories and PRs are discarded because intricate or legacy build configurations do not work in modern containerized environments.
• Training scalability: orchestrating thousands of task-specific Docker images creates massive infrastructure overhead that complicates large-scale optimization and RL.

SWE-ZERO sidesteps the Docker requirement by instructing the model not to run repo-specific commands likepytest,npm test, or build scripts that depend on the project’s installed environment. The model is still free to explore and edit the repository using standard POSIX commands that need no setup:grep,find,cat,sed,ls,git, and so on. None of those require the repo to be built or its dependencies installed, so the pipeline can run against any GitHub PR snapshot and unlocks the long tail of PRs that prior pipelines must throw away.

SWE-ZERO released9 B tokens, and we scale this10× to 112 B tokensusing the same execution-free recipe. Coverage:

• 12,290,800rollouts
• 122,908unique PRs
• 3,222repositories
• 16programming languages

To our knowledge it is now5.7× larger than the previous largest agent-trace dataset,open\-thoughts/AgentTrove(111.06 B vs 19.41 B tokens):

DatasetReleaseRolloutsTotal tokensricdomolm/mini\-coder\-trajs\-400k2025-09-30396,5155.26 Bnvidia/Nemotron\-Terminal\-Corpus2026-02-19366,1546.23 Bnvidia/SWE\-Zero\-openhands\-trajectories(original SWE-ZERO release)2026-04-17318,1158.92 Bopen\-thoughts/AgentTrove2026-04-271,696,84719.41 B**AlienKevin/SWE\-ZERO\-12M\-trajectories(this dataset, 10× scale-up)2026-05-1212,290,800****111.06 B**

https://huggingface.co/datasets/AlienKevin/SWE-ZERO-12M-trajectories#experiment-validationExperiment validation

Diversity check (MinHash-64).We follow the diversity protocol from the Code World Model paper. It measures the mean Jaccard similarity between rollouts in the same PR, on bash-command shingles. Anything under 0.5 is considered diverse. On our corpus, we measure0.2730, well under the threshold.

Training check.To verify the trajectories actually transfer into model capability, we trained Marin-8B base on three subsets of an earlier snapshot of this dataset (@ab6295c3, 1.6 M rollouts / 18.4 B tokens) and evaluated onSWE-bench Verified(100 random tasks, mini-swe-agent v1 + Harbor, 3 runs per model, temperature 1.0):

Training dataRolloutsTokensMean resolve rateMarin-8B Basen/an/a0.0 %SWE-ZERO training10,000~100 M3.3 % ± 0.6SWE-ZERO training50,000~570 M4.0 % ± 2.0SWE-ZERO training100,000~1.1 B5.3 % ± 1.5

Resolve rate scales consistently with training data through the 1 B-token point we tested. For methodological simplicity these runs use the standard SFT masked-loss recipe, but this corpus is intended as amid-training dataset. Downstream models should be further post-trained on dedicated SFT datasets such asnvidia/SWE\-Hero\-openhands\-trajectoriesandricdomolm/mini\-coder\-trajs\-400k. Full methodology, per-task breakdown, and other experiment details:marin-community/marin#4898 (comment).

https://huggingface.co/datasets/AlienKevin/SWE-ZERO-12M-trajectories#methodologyMethodology

Each rollout starts from a real GitHub PR snapshot innebius/SWE\-rebench\-V2\-PRs, which provides real-world software-engineering tasks spanning 3,222 repos. We sample100 independent multi-turn rollouts per PRfromricdomolm/mini\-coder\-1\.7b(a compact 1.7 B-parameter code model that scores 50.4 pass@100 on SWE-bench Verified) at temperature 1.0. Trajectories follow themini\-swe\-agentv1format: amessageslist where the assistant issues one bash command per turn and the harness returns the command output.

Following SWE-ZERO, generation is fullyexecution-free: no container is built, no test is executed, no verifier is consulted. That is what unlocks the scale.

https://huggingface.co/datasets/AlienKevin/SWE-ZERO-12M-trajectories#schemaSchema

fieldtypeinstance\_idstring - PR identifierrepostring -owner/name``messageslist[{role, content}] - multi-turn agentic trajectorytrajectory\_formatstring -mini\-swe\-agent\-1``exit\_statusstring -Submitted,incomplete, etc.duration\_secfloat64 - wall-clock generation time

https://huggingface.co/datasets/AlienKevin/SWE-ZERO-12M-trajectories#sampling-settingsSampling settings

SettingValueMax turns per rollout15Rollouts per PR100Sampling temperature1.0Max model length (vLLM)32,768 tokensMax sequences (vLLM)256Max total tokens per rollout32,768

https://huggingface.co/datasets/AlienKevin/SWE-ZERO-12M-trajectories#limitationsLimitations

This is amid-training corpus, not an SFT dataset. Treat it accordingly.

• Skill acquisition, not task completion.The goal of this corpus is to instillagentic tool-use priorsin a base model at scale: how to drive a bash shell, navigate a repository, read and edit files, and interleave thinking with actions. It is not designed to teach the model tosolveSWE-bench-style tasks end-to-end. That is what downstream RL and SFT stages are for.
• **No verification.**Because SWE-ZERO is execution-free, no trajectory has been checked against ground-truth tests. Patches are not known to compile, apply cleanly, or pass. Correctness was deliberately traded for scale and repo coverage.
• **Low submission rate.**Most rollouts do not reach a successfulSubmittedexit status; many exitincompleteor hit other terminal conditions. If you filter to clean submissions only, you will lose the bulk of the corpus.
• **15-turn cap.**Trajectories are truncated at 15 model turns to keep p99 token cost bounded and throughput high. Many tasks are not solvable in 15 turns and the trajectory simply ends mid-investigation.

That said, prior work shows these limitations are forgiving in practice. BothSERAandNemotronTerminalreport that downstream SFT performance is robust to incomplete, failed, and length-truncated traces. Concretely, NemotronTerminal’s Table 7 shows that**no filtering (12.4 %)significantly surpasses bothcomplete-only (6.74 %)andsuccess-only (5.06 %)filtering strategies. Our own experiments with Qwen3-1.7-Base corroborate this: the model reaches9 %**on SWE-bench Verified when trained on 100K samples from this corpus truncated to 8K tokens, even though most rollouts do not end with a patch submission. Full details inmarin-community/marin#5611 (comment).

https://huggingface.co/datasets/AlienKevin/SWE-ZERO-12M-trajectories#related-datasetsRelated datasets

• SWE\-ZERO\-96K\-trajectories- 96 K rollouts from the 1B scaling run (#4666)
• SWE\-ZERO\-multilang\-300\-trajectories- 300 rollouts from multilang validation (#4653)

https://huggingface.co/datasets/AlienKevin/SWE-ZERO-12M-trajectories#citationCitation

@misc{li2026swezero12m,
  author       = {Xiang Li},
  title        = {{SWE-ZERO-12M-trajectories: 112B Tokens of Execution-Free Agentic Trajectories}},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/datasets/AlienKevin/SWE-ZERO-12M-trajectories}},
}

Downloads last month1,192