@lateinteraction: dspy.GEPA used in pretraining data curation in the new Microsoft AI effort :-)
Summary
GEPA-optimized LLM judges from dspy are used for data filtering in Microsoft's MAI-Thinking-1 model pre-training pipeline.
View Cached Full Text
Cached at: 06/03/26, 11:55 PM
dspy.GEPA used in pretraining data curation in the new Microsoft AI effort :-)
Lakshya A Agrawal (@LakshyAAAgrawal): Excited to see the use of GEPA-optimized LLM judges for data filtering in MAI-Thinking-1 model’s pre-training pipeline!
Similar Articles
@harold_matmul: dspy.GEPA used in pretraining data curation in the new Microsoft AI effort :-)
The article explains how GEPA (Genetic-Pareto Optimization) within DSPy is used for efficient prompt tuning, specifically applied to pretraining data curation at Microsoft AI, allowing researchers to replace manual prompt engineering with automated compute-driven optimization.
@harold_matmul: it was my idea :) Using GEPA is a very natural workflow for creating LLM programs. The iteration speed is very quick, a…
A user thanks for the GEPA tool, highlighting its natural workflow for LLM programs, fast iteration, and ability to bias optimization with data-derived priors.
Microsoft's new MAI models
Microsoft announced two new LLMs: MAI-Thinking-1 (35B reasoning model) and MAI-Code-1-Flash (5B code model), both trained on enterprise-grade, clean data without third-party distillation, with MAI-Thinking-1 claimed to be preferred over Sonnet 4.6 in blind evaluations.
@_akhaliq: SpenseGPT Practical One-shot Pruning Enabling Sparse and Dense GEMMs for LLM Inference
SpenseGPT introduces a practical one-shot pruning method for LLMs that enables both sparse and dense GEMMs during inference, improving efficiency.
Exploring Autonomous Agentic Data Engineering for Model Specialization
This paper formalizes Autonomous Agentic Data Engineering, where LLMs act as autonomous data engineers to curate and optimize training data for specialized domains, showing a 57.29% improvement in student model performance using GPT-5.2.