Tag
This paper studies how instruction tuning affects model confidence and lexical diversity in question answering, finding that it alters confidence and reduces rationale diversity without improving calibration.
A study on uncensored LLMs (Gemma and Qwen) shows that removing censorship makes them more optimistic in stock market predictions, but not more accurate. The effect varies by model family.
DeepLook is a training-free framework that improves LLM reasoning by allocating compute at uncertainty bottlenecks, reducing token generation by 87.3% on average while improving accuracy on competition math benchmarks.
Proposes Dual-Confidence Contrastive Decoding (DCCD), a training-free method for retrieval-augmented generation that handles intra-context conflicts in multi-document settings by combining document-level and token-level confidence signals, and introduces the DRQA benchmark for factual-conflict QA.
The author reflects on why people trust confident AI answers, especially in finance, and introduces their project AutoFlow, a Credit Evidence Engine designed to verify financial claims against source evidence and highlight contradictions.
This paper demonstrates that global calibration metrics like Expected Calibration Error are confounded by model accuracy, and proposes ACE, an accuracy-controlled evaluation framework for fair comparison of large language models.
This paper proposes TriageRA-CCF, a method for adaptive rank budgeting in LoRA for medical question answering. It uses source-side signals (base-model confidence, clinical coverage, counterfactual proxy) to dynamically choose rank budgets, achieving modest accuracy gains on Qwen3-8B and Llama3.1-8B.
The author critiques the idea of agents remembering everything and introduces TrueMemory, a system that converts memories into trait claims with confidence and evidence to better calibrate agent behavior.
A developer building autonomous billing agents discusses the difficulty of reconstructing why an agent made a decision after the fact, and describes building a tool (Attova) that records decisions with evidence, alternatives, and confidence to improve debugging and human review.
This paper investigates whether frontier LLMs exhibit individuated metacognition—the ability to assess their own item-level capabilities beyond shared signals. Through factor analysis and pairwise calibration across 20 models and six benchmarks, the authors find no evidence of such metacognition; confidence differences reduce to a single shared difficulty factor, suggesting models rely on a common difficulty signal rather than model-specific self-knowledge.
A reflection on how many AI models prioritize sounding confident over being truthful, using Claude as an example of a model that seems more focused on internal consistency and logical honesty.
Proposes CSR, a framework that calibrates LLMs directly in semantic space using a novel semantic calibration reward, reducing ECE by up to 40% and improving AUROC by up to 31% over verbalized-confidence baselines across multiple datasets.
Armin Ronacher (@mitsuhiko) suggests that people should be upfront about their actual understanding of a topic when making pull requests, as AI tools (referred to as 'clanker') make it easy to sound confident without real knowledge.
The article discusses a banquet where second-generation rich were seated next to Musk and Jensen Huang but lacked interaction, contrasting with the confidence of first-generation entrepreneurs like Jack Ma and Charles Zhang, sparking discussion on the differences between the two generations of entrepreneurs.
This paper identifies that on-policy distillation (OPD) in language models leads to severe overconfidence due to information mismatch between training and deployment, and proposes CaOPD, a calibration-aware framework that improves both performance and confidence reliability.