Tag
LOCUS is a task-aware low-rank post-training method that reduces output token length in language models while maintaining preference alignment, achieving up to 39.84% reduction on Pythia-2.8B with minimal parameter updates.
BridgeAlign proposes a preference-alignment pipeline for humanities and social sciences, generating 210k synthetic preference samples and enabling Qwen3-8B to achieve strong results across 17 benchmarks.
This paper systematically studies the limitations of steering vectors for controlled text generation, finding that their effectiveness varies across traits, degrades on task transfer, and suffers from composition tradeoffs.
This paper introduces Personalized Active Preference Alignment (PAPA), a method for fine-tuning diffusion models using real-time user feedback without a parameterized reward model, enhancing efficiency in personalized tasks like recommendations and image generation.
DiARC is a method that improves the reasoning ability of large language models on ARC-like tasks by constructing preference pairs from positive and negative samples, outperforming baselines across multiple benchmarks.
This paper introduces CHILLGuard, a fine-grained Chinese LLM content safety guardrail built on a new 5-macro, 31-micro category risk taxonomy and a scalable multi-stage data construction pipeline. The model achieves state-of-the-art performance, improving F1 score by 15.92% over existing baselines.
TuneJury is an open-source pairwise reward model for text-to-music generation that provides calibrated preference scoring and generalizes across multiple downstream applications.
This paper introduces Spectral Souping, a framework for efficiently aligning LLMs with individual user preferences by discovering a universal spectral representation that enables merging of specialized policies at inference time without costly retraining.
This paper introduces Implicit Preference Alignment (IPA), a data-efficient post-training framework that improves hand motion generation in human image animation without requiring paired preference data. It utilizes implicit reward maximization and hand-aware local optimization to enhance generation quality while reducing data curation costs.
Arch-Router is a compact 1.5B model that aligns LLM routing with human preferences by mapping queries to user-defined domains and action types, outperforming proprietary models in subjective evaluations.