I taught my 1B to follow instructions. It got worse at following instructions...
Summary
The author trained 1B, 2B, and 3B models with the same SFT recipe and observed that instruction-following (IFEval) regressed for the 1B and 2B models but improved for the 3B, possibly due to different learning rates or model capacity.
Similar Articles
Finetuned a Early 2023-Era Model on 2 Instruction Following Datasets and it Became Good
A finetuned Pythia-6.9B model on two instruction-following datasets for 550 steps becomes capable in 13 languages, showing significant improvement over the base model.
Shared SFT Lessons Across Alignment, Model Organisms, and Toy Models
This paper studies transferring lessons about supervised fine-tuning (SFT) across alignment training, model organisms, and toy models, showing that techniques like training on reasons for behavior and mixing on-model data can improve generalization and capability preservation.
SEIF: Self-Evolving Reinforcement Learning for Instruction Following
This paper introduces SEIF, a self-evolving reinforcement learning framework that enhances LLM instruction-following capabilities through iterative difficulty adaptation and co-training of instructor and follower components.
Video-IFBench: Evaluating Instruction Following of Multimodal LLMs in Video Understanding Scenarios
Video-IFBench introduces a comprehensive benchmark for evaluating how well multimodal large language models follow diverse instructions in video understanding tasks, covering various constraints and task types.
LIFT: Scoped Constraints for Evaluating and Training Instruction Following
LIFT presents a new instruction-following benchmark (LIFT-Bench) and training counterpart (LIFT-Train) using scoped constraints on structured document outlines, showing that frontier models struggle on harder subsets and that training with LIFT-Train is less susceptible to reward hacking than existing IFTrain approaches.