Tag
Matt Pocock proposes a simple workflow for getting an AI agent to match your writing style: a scorer app where you rate sample sentences one by one, then co-write an article sentence-by-sentence so the agent tests hypotheses and builds a style profile over time.
The paper proposes PACE (Proxy Alignment and Curvature Expansion), a pre-optimization diagnostic that uses repeated human ratings to predict the human-gain curve induced by a fixed proxy, modeling proxy optimization as a KL-regularized exponential tilt with first-order alignment and second-order correction terms, and shows reduced held-out gain-prediction error across multiple benchmarks.
WeFair 提出 fidelity-aware 人类反馈对齐框架(示范→偏好→验证三阶段),用于短视频搜索的个性化查询重写,WeFair-2B 已在微信短视频搜索系统部署,日均服务超 1 亿查询,线上 A/B 测试显示有效播放率相对提升 2.593%。
该论文提出了一种人类反馈引导的多智能体逆强化学习框架(HFG-MAIRL),通过局部马尔可夫博弈和Bradley–Terry偏好反馈对交互式驾驶行为进行奖励建模,在AIDE数据集上超越了现有IRL基线。
RSI Arena introduces a two-stage study where ten frontier AI agents autonomously post-train language models from a shared base under fixed GPU and API budgets, guided by human preference feedback. The strongest agent (GPT-6 Astra) closed 76% of the base-to-Instruct gap, and most models improved in human preference from stage 1 to stage 2, demonstrating a step toward recursive self-improvement with human feedback (RSIHF).
A tutorial-style article accepted to IEEE Signal Processing Magazine that surveys the mathematical foundations and algorithmic frameworks for aligning LLMs with human feedback, covering SFT, RLHF, DPO, and their links to inverse reinforcement learning and discrete choice models. It serves as a comprehensive resource for researchers and practitioners on alignment methodologies.
The author argues that scaling laws are missing a parameter H — human effort/time invested in producing learning signal — and that most recent AI gains (ChatGPT's RLHF, coding/math verifiable tasks, data vendor value) stem from scaling H rather than model size or data alone.
该论文提出福利感知偏好获取方法(WAPA),在有限人工反馈预算下,通过社会 welfare 目标来分配人工标注,以改善多元对齐中不同个体偏好的监督质量,实验证明仅获取 20% 人工标签即可恢复 75–86% 的效用差距。
Omar (@omarsar0) discusses Kled V3, an AI data-collection platform that lets labs order custom datasets (image, video, audio, text, annotation) from a network of 500,000+ opt-in contributors within 72 hours, enabling tight identify-failure→collect-data→retrain loops.
EPFL and University of Helsinki researchers introduce Uncertainty-Normalized Margin DPO (UNM-DPO/ULNM-DPO-WR), which combines preference-strength margins with a learned prompt-dependent uncertainty scale and length normalization, achieving strong win rates over DPO and SimPO on HelpSteer2/3 and AlpacaEval with a Llama-3.1-8B-Instruct policy.
Autodesk Research introduces FLOORA, a family of small domain-specific language models using a custom DSL for architectural layout generation, where a 0.6B model outperforms much larger frontier models with up to 92% VLM judge win rates on out-of-distribution buildings.
The paper introduces a multi-stage architecture for learning individuated utility functions from multi-modal data, showing that models accounting for heterogeneous preferences outperform universal utility models in subjective tasks like aesthetic judgments.
This article introduces how Warp uses Claude to build self-improving AI agents, automatically optimizing skills through human feedback, and summarizes best practices for effective agent development.
The paper reformulates AI alignment as a social choice problem using linear optimization over a convex impact space, applying welfare economics and mechanism design to derive alignment protocols. It empirically illustrates welfare implications using real human preferences in scenarios like kidney allocation and trolley problems.
The tweet discusses simulation as a potential new scaling law in AI, highlighting insights from an interview with Simile AI's CEO on simulating human behavior for advanced AI research.
A tweet introducing the eval-engineering skill from langchain-ai/langchain-skills, which uses human feedback to generate aligned environments, harnesses, and tasks for agent evaluation. It explains the workflow and provides installation instructions for the open-source tool.
Intelligence, the company behind AI evaluation tool DesignArena, raised $7.9M in seed funding led by Index Ventures. The platform uses human preference rankings to improve AI-generated media and currently has 5.3M users and $60M ARR.
This paper introduces LEMUR, a framework that combines multi-objective reinforcement learning with preference-based learning from multiple human feedback to learn Pareto-optimal policies without predefined reward functions.
A discussion on how treating 'human rejection' as a separate failure mode from 'agent malfunction' significantly impacts the reliability and debugging of AI agents in production.
This paper introduces DROPJ, a human-centred method for safely training and deploying agent policies by learning a world model from real-world trajectories, then eliciting human preferences with justifications to train a reward model for model predictive control. Experiments show that using human-generated simulated trajectories and justifications improves safety and reduces computational cost.