Tag
This article explores the challenge of applying reinforcement learning to tasks that lack clear verifiability, citing Dario Amodei's prediction about achieving a 'country of geniuses in a data center' and discussing techniques such as RLVR, RLHF, Constitutional AI, and rubric-based rewards from Scale AI.
An analysis discussing the limitations of reinforcement learning with verifiable rewards (RLVR) in math and coding, and the challenge of extending RL to subjective or unverifiable tasks like planning or scientific discovery. It explores techniques such as RLHF and Constitutional AI as alternatives for alignment.
This paper introduces a categorical framework for constructing verifiable, local truth-preserving foundation models using composable foundries, implemented in the Odyssey system, and scheduled for a tutorial at ICML 2026.
Data2Story is a multi-agent framework that transforms datasets into verifiable multimodal stories, acting as an end-to-end data journalist with evidence grounding and multimodal generation.
This paper introduces Data Journalist Agent (Data2Story), a multi-agent framework that automates data journalism by generating evidence-grounded, multimodal news stories while ensuring transparency and verifiability.
ScientistOne introduces Chain-of-Evidence, a verifiability framework for autonomous research agents that ensures every claim is traceable to evidence, achieving zero hallucinated references, perfect score verification, and the highest method-code alignment across 75 papers while matching or exceeding human expert performance on frontier research tasks.
This paper proposes a fine-grained concept bottleneck model framework that grounds each concept in localized visual evidence, enabling direct verification of concept correctness and improving transparency in medical imaging tasks.
OpenAI publishes a report on mechanisms to improve verifiability in AI development, addressing how stakeholders can verify organizations' claims about AI system properties and safety practices.
Andrej Karpathy discusses the December 2024 shift where LLMs reached new reliability, coining 'vibe coding' for floor-raising and 'agentic engineering' for ceiling-raising, and argues that verifiability is key to AI's jagged capabilities.