POLARIS: Guiding Small Models to Write Long Stories

arXiv cs.CL Papers
creative-writing small-language-models reinforcement-learning grpo long-form-generation fine-tuning llm-as-judge

Summary

POLARIS is a training recipe using GRPO with LLM-as-judge rewards and human-reference injection to improve long-form story generation in small models. Applied to Qwen3.5-9B, the resulting POLARIS-9B model matches Qwen3.5-27B performance on creative writing benchmarks while better adhering to length instructions.

arXiv:2606.04095v1 Announce Type: new Abstract: Small open-weight models struggle at long-form creative writing: their generated stories either fall far short of the requested length, or their quality significantly degrades as length increases, especially when compared to frontier models. We present POLARIS (Policy Optimization with LLM-as-a-judge rewards and Anchored-Reference Injection for Storywriting), a lower-compute GRPO recipe with two key ingredients: a frontier LLM judge with a structured Story Quality rubric as the online reward, and human-reference injection (HRI), where a teacher-forced human-written story serves as a high-reward anchor within each GRPO group. By applying our training recipe to Qwen3.5-9B, using a dataset of approximately 1.4K prompt-story pairs derived from 100 short-story anthologies and 4 A100 GPUs, we obtain POLARIS-9B. Across five benchmarks spanning in-distribution and out-of-distribution prompts and rubrics, POLARIS-9B is competitive with much larger open-weight models while following length instructions more closely. A blinded human evaluation confirms that POLARIS-9B is preferred to the base Qwen3.5-9B and on par with Qwen3.5-27B. Despite training only on stories up to 4k words, POLARIS-9B preserves quality on prompts requesting stories up to 3 times the training length, a regime where most open-weight models degrade substantially in quality, length adherence, or both. More broadly, our results suggest that length generalization is a meaningful stress test for creative-writing models and a useful lens for distinguishing otherwise close models.
Original Article
View Cached Full Text

Cached at: 06/05/26, 02:12 AM 

# POLARIS: Guiding Small Models to Write Long Stories
Source: [https://arxiv.org/abs/2606.04095](https://arxiv.org/abs/2606.04095)
[View PDF](https://arxiv.org/pdf/2606.04095)

> Abstract:Small open\-weight models struggle at long\-form creative writing: their generated stories either fall far short of the requested length, or their quality significantly degrades as length increases, especially when compared to frontier models\. We present POLARIS \(Policy Optimization with LLM\-as\-a\-judge rewards and Anchored\-Reference Injection for Storywriting\), a lower\-compute GRPO recipe with two key ingredients: a frontier LLM judge with a structured Story Quality rubric as the online reward, and human\-reference injection \(HRI\), where a teacher\-forced human\-written story serves as a high\-reward anchor within each GRPO group\. By applying our training recipe to Qwen3\.5\-9B, using a dataset of approximately 1\.4K prompt\-story pairs derived from 100 short\-story anthologies and 4 A100 GPUs, we obtain POLARIS\-9B\. Across five benchmarks spanning in\-distribution and out\-of\-distribution prompts and rubrics, POLARIS\-9B is competitive with much larger open\-weight models while following length instructions more closely\. A blinded human evaluation confirms that POLARIS\-9B is preferred to the base Qwen3\.5\-9B and on par with Qwen3\.5\-27B\. Despite training only on stories up to 4k words, POLARIS\-9B preserves quality on prompts requesting stories up to 3 times the training length, a regime where most open\-weight models degrade substantially in quality, length adherence, or both\. More broadly, our results suggest that length generalization is a meaningful stress test for creative\-writing models and a useful lens for distinguishing otherwise close models\.

## Submission history

From: Rishanth Rajendhran \[[view email](https://arxiv.org/show-email/70dede27/2606.04095)\] **\[v1\]**Tue, 2 Jun 2026 18:00:07 UTC \(7,952 KB\)

Similar Articles

StoicLLM: Preference Optimization for Philosophical Alignment in Small Language Models

arXiv cs.CL

This research paper investigates using preference optimization (ORPO, AlphaPO) on small language models like Llama-3.2-3B and Qwen-3-4B to align them with Stoic philosophy using micro-datasets. The study finds that while 300 examples can effectively encode Stoic virtues, small models still struggle with outward-facing cosmopolitan duties.

Running Qwen3.6 35b a3b on 8gb vram and 32gb ram ~190k context

Reddit r/LocalLLaMA

The author shares a high-performance local inference configuration for running Qwen3.6 35B A3B on limited hardware (8GB VRAM, 32GB RAM) using a modified llama.cpp with TurboQuant support, achieving ~37-51 tok/sec with ~190k context.