code-llms

Proposes CodeBlock, a structure-aware sparse supervision framework for supervised fine-tuning of code LLMs. It selects high-quality instruction-response pairs and partitions code responses into syntactically coherent coding items, applying loss only to selected items to achieve stronger pass@1 rates using only 1.9% of supervised response tokens.