open-ended-generation

This paper investigates the effects of inference-time interventions and weight consolidation on open-ended AI generation, finding that training on value-filtered candidates improves mean quality but does not exceed classic heuristic performance.

This paper proposes a human-grounded framework to measure distributional breadth (cultural reach) of LLM-generated content in open-ended tasks, introducing metrics LLM Coverage and In-Boundary Rate. Experiments show current LLMs produce plausible but narrow content concentrated near the center of human response space.

This paper introduces Rubrics as Privileged Information (RuPI), extending on-policy self-distillation to open-ended generation by conditioning the teacher on rubrics as soft privileged information. The method outperforms rubric-as-reward RL and reference-completion distillation across multiple LLMs and benchmarks like HealthBench.

CalibratedRubric is a task-adaptive framework for building compact, measurable rubric banks for open-ended LLM evaluation, using Bayesian measurability filtering and IRT-based selection to improve human-gold agreement and rank fidelity across financial, healthcare, general, and legal benchmarks.

EarlyDx is a new large-scale benchmark for evaluating LLMs on open-ended, evidence-supported diagnosis generation at emergency department admission, built from 154,834 MIMIC-IV encounters. It reveals that even frontier and medical-specialized models struggle to synthesize admission-time evidence, with post-training only partially improving inference-dependent recall.

SERPO introduces a self-evolving rubric policy optimization framework for test-time reinforcement learning in open-ended generation, replacing answer voting with a closed loop that co-evolves response evidence, query-specific rubrics, and policy parameters, achieving significant improvements on health and research benchmarks.

This paper introduces a two-level meta-rubric framework for evaluating factual completeness in open-ended generation, instantiated as the GAMUT benchmark. It features 1,813 questions across 10 domains and finds the benchmark challenging and discriminative, with top models scoring 58.7%.

This paper introduces a unified framework for test-time diverse generation in large language models, categorizing methods by where diversity is injected (surface-level vs. specification-level). It proposes specification-level methods that generate diverse intermediate specifications, achieving better output diversity across five open-ended tasks and four backbone models while maintaining quality.

This paper introduces G-Zero, a verifier-free framework that enables autonomous large language model self-improvement through co-evolutionary training using intrinsic rewards and hint-based guidance. It aims to overcome the limitations of proxy LLM judges in open-ended tasks by deriving supervision from internal distributional dynamics.