MindForge: Teaching Small Language Models Whole-Life-Cycle Software Engineering via Source-Free Program Synthesis

Hugging Face Daily Papers Papers
small-language-models software-engineering program-synthesis fine-tuning open-source code-generation

Summary

MindForge is an automated pipeline that converts open-source command-line programs into source-free training environments for small language models. Fine-tuning Qwen3.6-27B on trajectories generated by GLM-5.2 significantly improves its performance on software engineering benchmarks, achieving results comparable to larger models.

Coding agents have made substantial progress on software engineering tasks that modify existing codebases, including bug fixing and feature implementation. However, constructing a complete program from scratch remains a major challenge: even the frontier models evaluated on ProgramBench fully resolve fewer than 1% of tasks. One obstacle is the lack of scalable training environments for this from-scratch setting, spanning the whole software engineering life cycle, as existing environment-construction frameworks focus only on a single phase in software development. To address this gap, we introduce MindForge, an automated pipeline that converts open-source command-line programs into source-free environments that expose only a compiled reference executable and its documentation. Using MindForge, we construct training environments from repositories disjoint from those in ProgramBench, and curate a high-quality data recipe consisting of program synthesis trajectories using GLM-5.2 as the teacher agent. Fine-tuning Qwen3.6-27B on these trajectories increases its ProgramBench average test pass rate from 37.98% to 49.51%, achieving performance comparable to substantially larger frontier models. Moreover, the fine-tuned model consistently improves over the base model across all seven unseen software engineering benchmarks, spanning long-horizon repository generation and translation, bug fixing, feature implementation, and cross-language issue resolution, with absolute gains of 31.00 points on RepoZero-C2Rust, 14.16 on DeepSWE, 10.70/4.56 on NL2Repo-Bench (with/without tests), 5.04 on SWE-bench Verified, 5.93 on SWE-bench Pro, 5.22 on SWE-bench Multilingual, and 4.94 on FeatBench.
Original Article
View Cached Full Text

Cached at: 07/30/26, 05:49 PM

Paper page - MindForge: Teaching Small Language Models Whole-Life-Cycle Software Engineering via Source-Free Program Synthesis

Source: https://huggingface.co/papers/2607.27146

Abstract

Codingagentshavemadesubstantialprogressonsoftwareengineeringtasksthatmodifyexistingcodebases,includingbugfixingandfeatureimplementation.However,constructingacompleteprogramfromscratchremainsamajorchallenge:eventhefrontiermodelsevaluatedonProgramBenchfullyresolvefewerthan1%oftasks.Oneobstacleisthelackofscalabletrainingenvironmentsforthisfrom-scratchsetting,spanningthewholesoftwareengineeringlifecycle,asexistingenvironment-constructionframeworksfocusonlyonasinglephaseinsoftwaredevelopment.Toaddressthisgap,weintroduceMindForge,anautomatedpipelinethatconvertsopen-sourcecommand-lineprogramsintosource-freeenvironmentsthatexposeonlyacompiledreferenceexecutableanditsdocumentation.UsingMindForge,weconstructtrainingenvironmentsfromrepositoriesdisjointfromthoseinProgramBench,andcurateahigh-qualitydatarecipeconsistingofprogramsynthesistrajectoriesusingGLM-5.2astheteacheragent.Fine-tuningQwen3.6-27BonthesetrajectoriesincreasesitsProgramBenchaveragetestpassratefrom37.98%to49.51%,achievingperformancecomparabletosubstantiallylargerfrontiermodels.Moreover,thefine-tunedmodelconsistentlyimprovesoverthebasemodelacrossallsevenunseensoftwareengineeringbenchmarks,spanninglong-horizonrepositorygenerationandtranslation,bugfixing,featureimplementation,andcross-languageissueresolution,withabsolutegainsof31.00pointsonRepoZero-C2Rust,14.16onDeepSWE,10.70/4.56onNL2Repo-Bench(with/withouttests),5.04onSWE-benchVerified,5.93onSWE-benchPro,5.22onSWE-benchMultilingual,and4.94onFeatBench.

View arXiv pageView PDFAdd to collection

Get this paper in your agent:

hf papers read 2607\.27146

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2607.27146 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2607.27146 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2607.27146 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

Language Models Learn to Improve Language Models

Hugging Face Daily Papers

LM2LM trains LLMs with reinforcement learning to propose effective language-model improvements, using a frozen research world model as a scalable reward signal with periodic real executions for reliability, enabling Qwen3.8-27B to outperform GPT-6-sol at 33% lower data-generation cost.

GraphForge: Training Working Agents with Graph-Anchored Workspace Synthesis

arXiv cs.CL

GraphForge introduces an evidence-graph framework that grounds both agent task generation and verification in real files, enabling synthesis of verifiable training trajectories for working agents. Fine-tuning Qwen3.6-27B on 2,169 GraphForge trajectories improves GDPVal, Workspace-Bench-Lite, and SpreadsheetBench II, with rejection fine-tuning yielding further gains.