rl-post-training

Neon and Castform team up to show how RL post-trained open-weights models combined with Lakebase Search can beat frontier models like GPT-5.6 Sol on retrieval tasks at 100x lower cost and latency.

Introduces PIRL (Policy Improvement Reinforcement Learning) and its practical implementation PIPO, a closed-loop framework that verifies policy updates by comparing performance with a historical anchor, enabling correction or reinforcement of previous updates. Experiments show consistent gains in mathematical reasoning, code generation, tool use, and self-distillation when applied on top of existing RL algorithms like PPO and GRPO.

Proposes Flow-Map GRPO, an online RL post-training framework for deterministic few-step flow-map generators, introducing Anchored Stochastic Flow Map Composition (ASFMC) to enable stochastic optimization without altering original model parameterization. Experiments on FLUX-based MeanFlow and sCM show improvement across reward-based, perceptual, and task-level metrics.

The author evaluates Ornith-9B against its base Qwen3.5-9B, finding that RL post-training improves token efficiency and sustained coding coherence but sacrifices single-turn judgment and robustness to misleading inputs, making it a narrower upgrade at 9B compared to the 35B version.

This paper introduces STAR, a method for spatiotemporally adaptive reward allocation in RL post-training for text-to-image diffusion models, improving compositional alignment and text rendering by focusing policy updates on relevant latent regions.