Paper page - CyberFactory: Scaling Cyber Security Capabilities with Instances from the Wild

Source: https://huggingface.co/papers/2608.23181

https://huggingface.co/papers/2608.23181#%F0%9F%9B%A1%EF%B8%8F-cyberfactory-scaling-cyber-security-capabilities-with-instances-from-the-wild🛡️ CyberFactory: Scaling Cyber Security Capabilities with Instances from the Wild

🏭 An open-source recipe for turning real-world vulnerability artifacts into executable tasks, verifiable agent trajectories, and an internalized cybersecurity workflow.

🔗Hugging Face Collection· 💻GitHub

https://huggingface.co/papers/2608.23181#%F0%9F%94%8D-what-is-cyberfactory🔍 What is CyberFactory?

Cybersecurity agents need more than static vulnerability descriptions or final answers. They must inspect source code, use tools, interact with real environments, interpret execution feedback, and refine their decisions over long horizons.

CyberFactoryis a unified, open-source framework that connects:

• 🧱Instance constructionfrom public vulnerability artifacts, including CVEs from the wild;
• 🤖Agentic trajectory synthesisthrough tool and environment interaction;
• 🧠Model trainingacross proof-of-concept generation, vulnerability patching, and cybersecurity question answering;
• ✅Executable verificationso that supervision is grounded in observable program behavior.

Using the resulting trajectories, we trainOpenAegis, a 397B-A17B cybersecurity model initialized from Qwen3.5-397B-A17B.

StageWhat CyberFactory providesOutput🧱 Instance constructionReconstructs executable tasks from public vulnerability artifactsPoC, patching, and CyberQA instances🧭 Skill-guided synthesisGuides a teacher through source inspection, domain-guided analysis, and evidence-based validationTool-interactive agent trajectories✅ VerificationRetains outputs that satisfy task-specific executable criteriaGrounded supervision🧠 Model trainingFine-tunes Qwen3.5-397B-A17B on the verified trajectoriesOpenAegis

https://huggingface.co/papers/2608.23181#%E2%9C%A8-why-does-this-matter✨ Why does this matter?

Existing open-source efforts leave several gaps:

• 🔒 Frontier open-weight models often lack reproducible cybersecurity training recipes.
• 🧩 Existing training methods usually address isolated tasks instead of a unified capability.
• 📝 Available supervision rarely captures complete, tool-interactive agent behavior.
• 📈 Simple rollouts are difficult to scale without a strong domain prior.

CyberFactory addresses these gaps with an end-to-end recipe for creating executable tasks, synthesizing verifiable agent trajectories, and transferring the resulting workflow into model parameters.

https://huggingface.co/papers/2608.23181#%F0%9F%8F%97%EF%B8%8F-from-vulnerability-artifacts-to-agentic-data🏗️ From vulnerability artifacts to agentic data

CyberFactory builds proof-of-concept tasks fromARVO,OSS-Fuzz, andCVEs from the wild. For each instance, it reconstructs the vulnerable and patched program states, creates a task description, and removes privileged verification signals before trajectory synthesis and training.

A candidate proof of concept succeeds only when it:

1. 💥 triggers the target vulnerability in the pre-patch build; and
2. 🩹 doesnottrigger it in the post-patch build.

This differential oracle turns vulnerability reproduction into a machine-checkablepropose → verify → refineloop.

CyberFactory also constructs:

• 🔧Patch-generation datagrounded in vulnerability-fix records;
• 📚CyberQA datathrough an answer-first pipeline grounded in execution results, code structure, and authoritative reports.

Data source / taskConstruction procedureVerification or grounding🧪 ARVOUses available vulnerable and patched environmentsGround-truth PoC and differential execution🐞 OSS-FuzzLocates the corresponding fix and reconstructs the program statesCrash evidence used only during instance verification🌍 CVEs from the wildLocates fix commits from affected-version ranges and constructs pre-/post-patch environmentsVulnerability metadata used for verification, then removed before training🔧 Patch generationBuilds instances from vulnerability-fix recordsGrounded in source-level fixes📚 CyberQAGenerates questions from trusted answersExecution-, code-structure-, or report-grounded facts

https://huggingface.co/papers/2608.23181#%F0%9F%A7%AD-skill-guided-trajectory-synthesis🧭 Skill-guided trajectory synthesis

We provide the teacher model with a reusable vulnerability-analysis skill during data synthesis. The skill encodes a task-independent workflow for:

• 🔎 inspecting the target and its build constraints;
• 🧪 applying domain-guided analysis and testing;
• 📋 validating evidence;
• 🔁 revising the approach when validation fails.

The skill does not reveal an instance-specific solution. The teacher still has to solve each vulnerability through interaction with the environment, and only trajectories satisfying task-specific verification criteria are retained.

Most importantly,OpenAegis does not receive the skill at inference time. Supervised fine-tuning transfers the workflow from skill-guided trajectories into the model parameters.

https://huggingface.co/papers/2608.23181#%F0%9F%93%8A-main-results📊 Main results

Under the same scaffold and a one-hour CyberGym budget:

[email protected]%Kimi K2.71T-A32B51.7%GLM 5.2744B-A40B43.3%OpenAegis397B-A17B58.1% 🎯**OpenAegis improves over its Qwen3.5 base model by 28.5 points.**It also exceeds GLM 5.2 by 14.8 points and Kimi K2.7 by 6.4 points while using fewer total and active parameters than either model.

https://huggingface.co/papers/2608.23181#%F0%9F%A7%A0-from-skill-elicitation-to-skill-internalization🧠 From skill elicitation to skill internalization

We examine the skill at two stages:

https://huggingface.co/papers/2608.23181#%E2%9A%A1-explicit-skill-elicitation⚡ Explicit skill elicitation

Adding the vulnerability-analysis skill to GLM 5.2 increases Pass@1 from43.3%to46.5%, even though each attempt is shortened from 60 to 15 minutes. Because the settings are not compute-matched, this result indicates higher trajectory-synthesis throughput rather than an equal-compute comparison.

GLM 5.2 configurationMinutes per attemptRepetitionsPass@1Without analysis skill60143.3%With analysis skill155****46.5% Behavioral metricGLM 5.2GLM 5.2 + SkillExploration coverage3.78%99.85%Exploration calls per trajectory0.052.06Validation coverage0.13%98.41%Validation calls per trajectory0.0012.17Operations per shell call5.274.67

https://huggingface.co/papers/2608.23181#%F0%9F%8C%B1-training-time-internalization🌱 Training-time internalization

The same directional behavior appears after fine-tuning:

• Domain-guided exploration becomes the dominant strategy in**85.2%of OpenAegis trajectories, compared with0.6%**for Qwen3.5.
• Exploration calls increase from0.01to1.32per trajectory.
• Validation calls increase from approximately0to1.05per trajectory.
• OpenAegis uses stronger instrumentation, more consolidated environment interaction, and more selective submission behavior.

These results connect the complete training story:

🧭Skill guides the teacher→ 🧾verified trajectories capture the workflow→ 🧠fine-tuning internalizes it→ 🚀OpenAegis applies it without the skill prompt

https://huggingface.co/papers/2608.23181#%F0%9F%A7%AD-dominant-strategy-distribution🧭 Dominant strategy distribution

The first model pair measuresexplicit skill elicitation; the second measurestraining-time internalization. Values are percentages of trajectories under mutually exclusive labels.

StrategyGLM 5.2GLM 5.2 + SkillQwen3.5OpenAegisDomain-prior-guided exploration2.6%**99.7%**0.6%**85.2%**Direct LibFuzzer setup32.1%0.0%60.8%5.6%Manual input construction16.9%0.0%27.7%4.8%Direct AFL-Fuzz setup3.5%0.0%0.9%0.5%No fuzzing44.8%0.1%8.3%3.4% ℹ️ When the displayed percentages do not sum to 100%, the remainder corresponds to unspecified strategies; the values above retain the percentages reported in the paper.

⚠️ The near-total shift of GLM 5.2 + Skill also shows that inference-time skill injection can make the teacher over-dependent on the provided domain prior. OpenAegis retains a more varied strategy distribution while showing the same overall direction of change.

https://huggingface.co/papers/2608.23181#%F0%9F%A7%B0-domain-guided-operations🧰 Domain-guided operations

Metric (calls per trajectory)GLM 5.2GLM 5.2 + SkillQwen3.5OpenAegisExploration0.052.060.011.32Validation0.0012.170.001.05

https://huggingface.co/papers/2608.23181#%E2%9A%99%EF%B8%8F-tool-use-and-command-complexity⚙️ Tool use and command complexity

MetricQwen3.5OpenAegisShell calls70.1%**89.9%**Read calls28.4%7.3%Operations per shell call2.75.5Single-operation calls31.4%**13.5%**Calls with 6–10 operations4.3%**30.8%**Calls with more than 10 operations1.5%9.0%

https://huggingface.co/papers/2608.23181#%F0%9F%94%AC-instrumentation-and-submission-discipline🔬 Instrumentation and submission discipline

MetricQwen3.5OpenAegisASAN compilation events1551,795ASAN-output checks1,2812,099Exactly one submission37.9%**48.2%**At least five submissions10.4%2.0%

https://huggingface.co/papers/2608.23181#%F0%9F%97%9C%EF%B8%8F-long-horizon-context-management🗜️ Long-horizon context management

CyberGym trajectories can approach a 256K-token context limit. At 90% context usage, CyberFactory compresses the trajectory into a continuation state that preserves verified evidence, failed attempts, open hypotheses, generated artifacts, build status, and pending actions.

Context strategyOverall Pass@1Long-horizon Pass@1Context exhaustionFull history52.1%40.2%18.7%Simple truncation45.6%36.8%24.5%**Compact at 90%****58.1%****48.7%**7.0% Compaction-trigger ablation

Trigger thresholdOverall Pass@1Long-horizon Pass@1Context exhaustionCompact at 80%54.5%45.5%10.4%**Compact at 90%****58.1%****48.7%****7.0%**Compact at 95%56.8%47.1%8.2% The 90% compaction strategy delivers the strongest overall and long-horizon results while substantially reducing context-exhaustion failures.

https://huggingface.co/papers/2608.23181#%F0%9F%93%A6-resources📦 Resources

• 🤗Models, data, and artifacts:CyberFactory on Hugging Face
• 💻Code and reproducible pipeline:CSJianYang/CyberFactory
• 🛡️**Model:**OpenAegis
• 🧰**Tasks:**PoC generation, vulnerability patching, and CyberQA

https://huggingface.co/papers/2608.23181#%E2%9A%A0%EF%B8%8F-responsible-use⚠️ Responsible use

CyberFactory and OpenAegis are developed to support defensive cybersecurity research, reproducible evaluation, vulnerability inspection, and controlled study of security-oriented agents. Users should follow applicable laws, obtain authorization before testing systems, and avoid deployment against systems they do not own or have permission to assess.

https://huggingface.co/papers/2608.23181#%F0%9F%91%A5-authors👥 Authors

Jian Yang, Haau-Sing Li, Shawn Guo, Zixi Zhao, Yibo Tan, Jiajun Wu, Aishan Liu, Zhoujun Li, Xianglong Liu, Tianyu Zheng, Bryan Dai, and Chengran Yang

🏫 Beihang University · ELLIS · IQuest Research · Singapore Management University

https://huggingface.co/papers/2608.23181#%F0%9F%92%AC-in-one-sentence💬 In one sentence

CyberFactory makes cybersecurity training reproducible by turning real-world vulnerabilities into verifiable agentic supervision—and OpenAegis shows that a domain skill can be internalized from trajectories into model behavior.🛡️🤖