Compile by Training: Turning Natural-Language Specifications into Local Neural Functions
Summary
This paper introduces 'Compile by Training', a method that compiles natural-language function descriptions into local neural programs using teacher model synthesis and fine-tuning, achieving 83.6% semantic accuracy on FuzzyBench.
View Cached Full Text
Cached at: 09/04/26, 07:56 AM
Paper page - Compile by Training: Turning Natural-Language Specifications into Local Neural Functions
Source: https://huggingface.co/papers/2609.04199 What if LLMs built reusable tools instead of solving the same task over and over?
**Program-as-Weights (PAW)**compiles a natural-language function description into a small neural program that runs on a shared 0.6B local interpreter. Our original compiler does this in a single forward pass, so compilation takes only seconds.
This work, Compile by Training, adds a higher-accuracy mode: teacher models synthesize task-specific examples at compile time, then about a minute of finetuning specializes the program. On a hard subset of FuzzyBench where the fast PAW compiler did not produce exact-match outputs, Compile by Training reaches 83.6% semantic accuracy.
Both compilers produce reusable PAW programs that can be called locally like regular Python functions.
https://huggingface.co/papers/2609.04199#examplesExamples
🗣️Claudish— English ↔ Claude-style prose https://programasweights.com/claudish
🤖Avatar Director— natural language → action programs for a live 3D character https://programasweights.com/avatar
🔒PII Masker— detect and mask personal information locally https://programasweights.com/pii
Try the higher-accuracy compiler: https://programasweights.com/playground?compiler=paw-ft-bs48
Similar Articles
@neural_avb: Watch this 45 min video to learn how to create synthetic datasets and train tiny (100M params) local language models th…
A 45-minute video tutorial on creating synthetic datasets and training tiny (100M parameter) local language models for narrow tasks, with code and resources provided.
Beyond Compilation: Evaluating Faithful Natural-Language-to-Lean Statement Formalization
This paper presents a benchmark and evaluation protocol for faithful natural-language-to-Lean statement formalization, revealing a 29-point gap between compile-pass and consensus-faithfulness, and decomposing the effects of expert drafting, context search, and elaboration feedback.
I built a compiler that turns computation graphs into the weights of a vanilla transformer — no training anywhere [P]
A compiler that converts computation graphs defined in Python into the weights of a standard transformer architecture (Phi-3) without any training, enabling vanilla HuggingFace deployment with no custom code.
@KirkDBorne: "How to Build and Fine‐Tune a Small Language Model: A Step-by-Step Guide for Beginners, Researchers, and Non-Programmer…
A step-by-step guide on building and fine-tuning small language models, designed for beginners, researchers, and non-programmers, with hands-on examples and Colab notebooks.
Syntax Without Semantics: Teaching Large Language Models to Code in an Unseen Language
This paper introduces PyLang, a programming language absent from all pretraining corpora, and shows that LLMs fine-tuned on it can learn syntax but fail to transfer algorithmic reasoning, resulting in an 'implementation fidelity gap' where models understand algorithms but cannot express them in an unfamiliar language.