benchmark-evaluation

Tag

Cards List
#benchmark-evaluation

Retrieval-Driven Memory Reconsolidation for Long-Term LLM Agents

arXiv cs.CL ↗ · 2026-09-16 Cached

The paper introduces REALM, a framework for long-term memory in LLM agents that uses retrieval-driven reconsolidation to autonomously organize memories into a cognitive graph, achieving improved performance on long-term memory benchmarks.

0 favorites 0 likes
#benchmark-evaluation

Same Patient, Different Order: Action-Level Reliability of Clinical LLM Agents Under Repeated Runs

arXiv cs.CL ↗ · 2026-09-15 Cached

This study examines the action-level reliability of clinical LLM agents by rerunning tasks with identical inputs and comparing orders, finding significant divergence that benchmarks may miss and proposing enhanced evaluation methods.

0 favorites 0 likes
#benchmark-evaluation

Towards a Deterministic Math Solver for Clinical Language Models

arXiv cs.AI ↗ · 2026-09-12 Cached

This paper proposes a Program-Solve interface where clinical language models generate Python code for a deterministic executor to perform math calculations, evaluating on MedCalc-Bench and finding improved accuracy for larger models like Qwen2.5-32B compared to direct arithmetic and hand-written libraries.

0 favorites 0 likes
#benchmark-evaluation

Apodex-1.1-mini-GGUF*Hugging Face

Reddit r/LocalLLaMA ↗ · 2026-09-10 Cached

Apodex-1.1 is a reasoning-first AI model for complex, long-horizon research tasks, with end-to-end execution, adaptive agent teams, and built-in verification to deliver verifiable results.

0 favorites 0 likes
#benchmark-evaluation

Beyond Prompts: Measuring and Optimizing LLM Tool-Agent Harnesses

arXiv cs.AI ↗ · 2026-09-10 Cached

This paper studies agent harness optimization to improve LLM tool agents without retraining, focusing on prompts and tool-boundary middleware, and introduces a protocol and the PRISM optimizer for measurable gains.

0 favorites 0 likes
#benchmark-evaluation

RecKAN: Kolmogorov-Arnold Networks with a Learnable Recursive Polynomial Basis

arXiv cs.LG ↗ · 2026-09-03 Cached

RecKAN introduces a learnable recursive polynomial basis for Kolmogorov-Arnold Networks, outperforming existing KAN variants on classification and forecasting tasks.

0 favorites 0 likes
#benchmark-evaluation

From Tokens to Semantics: Leveraging Complementary Signals for Hallucination Detection in Black-Box LLMs

arXiv cs.CL ↗ · 2026-09-03 Cached

This paper proposes methods for detecting hallucinations in black-box LLMs by combining semantic entropy and token-level uncertainty signals, evaluating techniques like TopK, CoCoA, Gated, and Stacked across multiple benchmarks to find that no single method is universally strongest but Stacked often performs best.

0 favorites 0 likes
#benchmark-evaluation

Unfold The World: Factorize 4D Properties in Reinforcing Spatial Reasoning

Hugging Face Daily Papers ↗ · 2026-09-03

This paper introduces FactoSR, a factorized reinforcement learning framework that enhances spatial reasoning in Vision-Language Models by decomposing 4D properties into orthogonal sub-objectives, achieving significant performance boosts on multi-view and video benchmarks.

0 favorites 0 likes
#benchmark-evaluation

Neural means and kernel corrections for operator learning

arXiv cs.LG ↗ · 2026-09-02 Cached

This paper presents a method combining neural network means with exact Matérn kernel corrections for operator learning in PDEs, achieving competitive or improved performance on public benchmarks like structural mechanics and OCO-2 radiative transfer emulation.

0 favorites 0 likes
#benchmark-evaluation

Mitigating Strong-Modality Collapse in Multimodal Learning via Inverted Asymmetric Fusion

arXiv cs.LG ↗ · 2026-08-28 Cached

The paper identifies strong-modality collapse in multimodal learning where fusion degrades the dominant modality's performance, and proposes Inverted Asymmetric Fusion (IAF) to preserve it, improving over unimodal baselines.

0 favorites 0 likes
#benchmark-evaluation

The Imperfective Paradox Is Not Necessarily in Large Language Models: A Benchmark Failure Before a Model Failure

arXiv cs.CL ↗ · 2026-08-27 Cached

This paper reevaluates the imperfective paradox benchmark for large language models, identifying conceptual and evaluation mis-specifications, and introduces lexically matched minimal pairs to reveal sufficiency bias in models' semantic reasoning.

0 favorites 0 likes
#benchmark-evaluation

Beyond Endpoint Gains: A Weight-Delta Audit of Medical Specialization

arXiv cs.AI ↗ · 2026-08-24 Cached

This paper proposes a weight-delta audit method to analyze medical specialization in language models, examining internal update changes beyond endpoint benchmark scores.

0 favorites 0 likes
#benchmark-evaluation

Position: Profiling Game Worlds by Transition Complexity

arXiv cs.AI ↗ · 2026-08-20 Cached

This paper proposes the Transition Complexity Profile (TCP), a set of metrics to quantify the difficulty of transition prediction in game world modeling, aiming to standardize benchmark evaluation in reinforcement learning and world model research.

0 favorites 0 likes
#benchmark-evaluation

Temporal Leakage in Financial News NLP: A Multi-Architecture Audit with a Regime-Specific M&A Signal

arXiv cs.CL ↗ · 2026-08-19 Cached

This paper audits temporal leakage in financial news NLP benchmarks across multiple models, finding that random splits inflate performance metrics and identifies M&A events as a category with a localized positive signal under chronological evaluation.

0 favorites 0 likes
#benchmark-evaluation

MemoryLake on MemoryArena: A Matched Study of Agent Memory Backends

arXiv cs.AI ↗ · 2026-08-17 Cached

The study compares MemoryLake, a structured memory backend, with other systems on the MemoryArena benchmark, finding it has higher success rates in some tasks but not establishing a clear advantage across all domains.

0 favorites 0 likes
#benchmark-evaluation

Library Reachability in LSR-Synth: How Anti-Memorization Design Changes the Measurement of Symbolic Discovery

arXiv cs.AI ↗ · 2026-08-03 Cached

This paper studies the LSR-Synth benchmark and finds that a fixed vocabulary covers most tasks, while language-model-generated candidates rarely expand the set of solvable instances except when vocabulary coverage is disrupted.

0 favorites 0 likes
#benchmark-evaluation

DarwinX: Evolving Agent Harnesses Through Natural Selection

Hugging Face Daily Papers ↗ · 2026-07-31 Cached

DarwinX evolves LLM agent harnesses via natural-selection-style population search with frozen model weights, improving verified performance across multiple benchmarks without benchmark-specific patching.

0 favorites 0 likes
#benchmark-evaluation

Recursive Self-Evolving Agents via Held-Out Selection

arXiv cs.AI ↗ · 2026-06-30 Cached

Introduces RSEA, a method for recursive self-evolution of LLM agents using a three-layer natural-language state and a held-out selection gate to prevent regression. Evaluated across four benchmarks, it shows that context evolution is benchmark-dependent and that a strict selection gate is crucial for reliability.

0 favorites 0 likes
#benchmark-evaluation

The Inattentional Gap: Task-Conditioned Language and Vision Models Omit the Safety-Critical Signals They Can Otherwise Report

arXiv cs.CL ↗ · 2026-06-26 Cached

This paper identifies the 'Inattentional Gap' where task-conditioned AI models suppress reporting of safety-critical signals they can otherwise detect, analogous to human inattentional blindness, challenging the assumption that benchmark performance ensures real-world safety.

0 favorites 0 likes
#benchmark-evaluation

Are Text-to-Image Models Inductivist Turkeys? A Counterfactual Benchmark for Causal Reasoning

Hugging Face Daily Papers ↗ · 2026-06-23 Cached

This paper introduces CF-World, a counterfactual benchmark to evaluate whether text-to-image models rely on causal reasoning or mere pattern matching. Experiments show all models degrade sharply in counterfactual settings, suggesting their understanding is limited to tightly coupled visual-textual patterns rather than genuine causal reasoning.

0 favorites 0 likes
← Previous
Next →
← Back to home

Submit Feedback