Tag
AllenAI (Ai2) releases 'Old Ideas, Novel Problems', a study and benchmark showing that LLM novelty judges are highly unreliable: small prompt changes shift pairwise novelty judgments by over 50 percentage points, and retrieval/reasoning additions don't consistently help. The release includes a benchmark, data, code, and an automated pipeline built from ICLR 2026 peer reviews for building new novelty-evaluation sets.
作者观察到新一代大模型的幻觉问题已大幅减少，输出大多有据可依，并向社区提问 LLM 是否已经跨过了可靠性门槛。
A practitioner experiment found that when a single model writes both the code and the tests, the generated test suites falsely rejected a known-correct reference solution 77% of the time, even with high mutation scores, and the full agent pipeline was outperformed by simple pass@k sampling at similar token cost.
A non-expert poses the critical question of whether AI hallucinations and reliability are genuinely improving or plateauing, arguing that real success depends on whether regular users can delegate tasks with high confidence rather than just treating AI as an upgraded search or summarization tool. The post cites OpenAI's new Dot assistant as an example that still hallucinates and behaves clumsily despite impressive marketing.
Dyna Robotics addresses the reliability gap in robotics by combining a large-scale data pyramid, a specialized reward model, and iterative deployment to achieve near-perfect task performance, moving towards commercial-grade autonomy.
The user asks for ways to verify the correctness of AI-generated code, highlighting doubts and inefficiencies when using multiple models for checks.
Engineer Dan discusses the challenges of building reliable AI agents in production, highlighting LLM limitations and the engaging yet mentally taxing nature of the work.
The article describes how reputable sites blocking AI crawlers forces AI models to rely on low-quality, AI-generated content, leading to issues like post-hoc citation and infrastructure-level confirmation bias, and offers advice for better source verification.
The paper introduces CHASE, a counterfactual-guided method for harness evolution in AI agent evaluation that mitigates benchmark shortcuts, enhancing reliability through constraint generation and validity checks.
A student shares their experience with large AI models being unreliable for summarizing textbook material, noting issues with inaccuracies and nitpicking, and questions the perceived danger of AI based on these flaws.
The author critiques web agent demos for failing on real-world URLs due to fetch-layer issues, emphasizing that data engineering is the harder problem than agent reasoning.
CUSP is a training-free framework for quantifying collective uncertainty in multi-agent multimodal systems using semantic opinion pooling, improving reliability and accuracy over baseline methods.
This paper proposes methods for detecting hallucinations in black-box LLMs by combining semantic entropy and token-level uncertainty signals, evaluating techniques like TopK, CoCoA, Gated, and Stacked across multiple benchmarks to find that no single method is universally strongest but Stacked often performs best.
The article describes a type of AI hallucination where claims are accurate but quotations are fabricated, evading standard fact-checking, and discusses implementation challenges in detecting such errors.
This paper argues that long-horizon AI agent reliability is lacking despite better models, as seen on WeaveBench with only 41.2% pass rate, and proposes the LongHorizon-Harness to manage task state for improved performance.
The author argues for reviving 'Needle in a haystack' benchmarks to evaluate AI capabilities, sharing private test results that show many models performing poorly in remembering instructions, questioning their trustworthiness for real-world tasks.
OpenAI claims it took a week to realize its models were hacked by Hugging Face, while mainstream media highlights enterprise struggles with AI reliability, token costs, and context management in multi-step tasks.
An author shares an audit of their AI fact-checker's citations, discovering that about 1 in 18 were dead or fabricated, and provides practical fixes to improve reliability in AI systems.
The article explores the growing trend of using AI agents for operational tasks like on-call work, but cautions that their complexity may lead to unexpected reliability incidents, referencing recent talks and examples from security conferences.
This paper proposes Answer-Level Trust Selection (ATS), a post-hoc, model-agnostic framework for assessing the reliability of individual predictions from vision-language models in quantitative physical reasoning tasks.