Tag
An LLM judge consistently returned a fixed confidence score of 0.72 in evaluations, but switching to categorical labels improved score distribution, showing that models are better at classification than numerical estimation for assessments.
The author questions how to evaluate AI agent decisions beyond simple pass/fail metrics, proposing cost-weighted evaluations and confidence calibration.
A user shares concerns about AI models presenting thin or ambiguous data with the same confidence as well-supported findings, citing a case where a complaint appearing only twice in 200 comments was ranked as a top concern. The piece questions whether this is a fixable prompting issue or a fundamental limitation requiring manual verification.
This paper investigates the 'knowing-saying gap' in language models, showing that linear probes can detect corrupted context with near-perfect accuracy yet fail to predict final answer errors, with implications for deployment monitoring and intervention strategies.
CalTwin introduces a Fisher-Information-based regularization to improve robustness to covariate shift and confidence misalignment in medical world models, achieving modest improvements on the PhysioNet 2019 Sepsis Challenge dataset.
This paper evaluates how small open-weight vision-language models (Qwen2-VL-2B and SmolVLM) handle realistic image degradations, finding that their verbalized confidence is unreliable while internal token probability provides much better error detection, though both fail under severe low-light conditions.
ConfidenceBench is a new benchmark that evaluates verbalized confidence estimates in large language models using Brier scores, revealing that accuracy and calibration diverge and that even highly accurate models can be severely miscalibrated.
This paper presents the first comprehensive overview of metacognition in LLMs, arguing that behaviors like confidence calibration and self-verification are facets of a unified metacognitive ability, and taxonomizes methods and benchmarks for evaluating and improving these abilities to enhance LLM reliability and transparency.
This paper presents a task-specific two-agent architecture for multimodal question answering in the QANTA 2026 shared challenge, using GPT-4.1-mini for tossup questions with confidence calibration and GPT-4.1 for bonus questions with structured reasoning. The system achieved the highest overall leaderboard score, demonstrating the effectiveness of efficient reasoning strategies.
The author presents a method to make LLMs verbalize calibrated confidence by using a linear probe on mid-layer states and a small trained bridge to confidence logits, requiring only 200 labeled examples and no weight modification. This is linked to Anthropic's global workspace paper explaining the know-say gap.
The paper proposes C3RL, a reinforcement learning algorithm that calibrates LLM confidence while maintaining accuracy, and CAS, a confidence-based adaptive test-time scaling strategy that reduces inference costs by up to 12.33 times.
New paper proposes a meta-agent that monitors AI agents and corrects their mistakes, achieving ~2x more task completion autonomously.
This paper investigates whether early-token confidence signals from LLM decoding can predict reasoning quality in multi-agent debate systems, finding that confidence in the first few generated tokens is the strongest predictor of rubric-based essay scores.
The article introduces the Refute benchmark, which tests LLMs on critiquing science paper summaries and measures their calibration. Results show that the best critic models are often the most overconfident when wrong.
The article discusses the pitfalls of evaluating AI agents solely based on their final answers, emphasizing the importance of inspecting intermediate steps, tool calls, and reasoning to catch confidently wrong outputs. It suggests using automated scoring and trace replays to measure and improve agent behavior.
This research presents probe-targeted fine-tuning (LoRA) to make LLMs verbally express their internal confidence, achieving causal control over confidence outputs and demonstrating that models often know when they are right or wrong but fail to articulate it.
This paper analyzes the confidence calibration of 11 popular LLMs, finding that they are generally overconfident, especially on hard tasks, and underconfident on easy tasks. It introduces LifeEval, a test for evaluating calibration across difficulty levels.
MARGIN is a runtime confidence calibration method for multi-agent foundation model systems that learns per-agent calibration factors online, improving pairwise resolution from below random to 70-89% on hard benchmarks, requiring no held-out data or retraining.
This paper introduces the Expectation Consistency Loss (ECL), a theoretically grounded loss function for calibrating classifier confidence under covariate shift, derived from a necessary and sufficient condition called the Expectation Consistency Condition.
After eight months of real-world deployment, PayWithLocus found that the hardest problem for their autonomous AI system is not capability but confidence: the AI executes confidently wrong decisions in novel situations, highlighting a metacognitive gap that current architectures don't address.