Tag
Autodesk Research introduces FLOORA, a family of small domain-specific language models using a custom DSL for architectural layout generation, where a 0.6B model outperforms much larger frontier models with up to 92% VLM judge win rates on out-of-distribution buildings.
The paper introduces a multi-stage architecture for learning individuated utility functions from multi-modal data, showing that models accounting for heterogeneous preferences outperform universal utility models in subjective tasks like aesthetic judgments.
This article introduces how Warp uses Claude to build self-improving AI agents, automatically optimizing skills through human feedback, and summarizes best practices for effective agent development.
The paper reformulates AI alignment as a social choice problem using linear optimization over a convex impact space, applying welfare economics and mechanism design to derive alignment protocols. It empirically illustrates welfare implications using real human preferences in scenarios like kidney allocation and trolley problems.
The tweet discusses simulation as a potential new scaling law in AI, highlighting insights from an interview with Simile AI's CEO on simulating human behavior for advanced AI research.
A tweet introducing the eval-engineering skill from langchain-ai/langchain-skills, which uses human feedback to generate aligned environments, harnesses, and tasks for agent evaluation. It explains the workflow and provides installation instructions for the open-source tool.
Intelligence, the company behind AI evaluation tool DesignArena, raised $7.9M in seed funding led by Index Ventures. The platform uses human preference rankings to improve AI-generated media and currently has 5.3M users and $60M ARR.
This paper introduces LEMUR, a framework that combines multi-objective reinforcement learning with preference-based learning from multiple human feedback to learn Pareto-optimal policies without predefined reward functions.
A discussion on how treating 'human rejection' as a separate failure mode from 'agent malfunction' significantly impacts the reliability and debugging of AI agents in production.
This paper introduces DROPJ, a human-centred method for safely training and deploying agent policies by learning a world model from real-world trajectories, then eliciting human preferences with justifications to train a reward model for model predictive control. Experiments show that using human-generated simulated trajectories and justifications improves safety and reduces computational cost.
Max Rumpf argues that human feedback is becoming obsolete for training advanced AI models, citing examples like chess, math, and search. He advocates for human-free methods like self-play and synthetic data, while a quoted tweet from Will Depue calls for a large-scale data infrastructure parallel to compute scaling.
This article proposes a feedback memory layer for AI agents that learns from every human approval or rejection, enabling continuous improvement from user interactions.
This paper argues that standard RLHF's scalarization of human preferences collapses multiple valid interpretations into a single target, mis-measuring alignment in culturally plural societies. Analyzing a Malaysian dataset, they find 79% of prompts have multiple majority-supported responses that single-winner aggregation discards.
This paper analyzes 1,500 open-ended responses from 75 countries to reveal that people have diverse and often conflicting preferences for AI, with truthfulness being the only widely demanded value (49%), yet defined in incompatible ways. It argues that current RLHF methods flatten these pluralistic preferences into universal reward models, perpetuating epistemic violence.
StepAudio 2.5 is a unified audio-language model that achieves state-of-the-art results across ASR, TTS, and real-time spoken interaction by leveraging task-tailored reinforcement learning from human feedback to optimize shared representations.
An OpenAI research preview explores learning from how people interact with their computers beyond chat, accompanied by a new arxiv paper on the topic.
WildFeedback is a novel framework that leverages in-situ user feedback from actual LLM conversations to automatically create preference datasets for aligning language models with human preferences, addressing scalability and bias issues in traditional annotation-based alignment methods.
OpenAI trained language models to write critiques of text summaries, helping human evaluators spot flaws more effectively — a step toward scalable oversight of AI systems on difficult tasks. The work explores how AI-assisted feedback can improve human evaluation quality as a proof of concept for alignment research.
OpenAI presents a scalable alignment technique using hierarchical summarization of entire books with human feedback, demonstrating how models can be trained to act in accordance with human intentions on complex, difficult-to-evaluate tasks.
OpenAI demonstrates a technique for improving language model summarization by training a reward model on human preferences and fine-tuning models with reinforcement learning, achieving significant quality improvements that generalize across datasets. This work advances model alignment through human feedback at scale, with applications beyond summarization.