explainability

This paper introduces a physics-inspired framework for structural attribution in cyber-physical IoT systems, using an undirected energy-based representation to provide dependency-aware explanations without requiring a directed causal graph. Experiments on an industrial IoT testbed demonstrate higher attribution accuracy, robustness, and scalability compared to existing graph-based methods.

Introduces AIriskEval-edu-db2, a new dataset for pedagogical risk assessment in AI-generated explanations for K-12 education, with 1,639 explanations and structured risk annotations. Includes validation experiments comparing LLMs for risk detection and explainability.

TokenScope is an interactive interpretability tool for decoder-only large language models that provides token-level metrics, attention patterns, and counterfactual branching during code generation, enabling systematic investigation of model behavior.

This paper investigates the cross-domain generalization failure of lightweight ML models for IIoT intrusion detection, finding they rely on coarse port features and that adversarial robustness does not correlate with cross-network performance.

The article discusses the challenges that arise when AI agents transition from demos to production, focusing on the need for operational control planes that provide idempotency, approval tracking, and operational explainability rather than just model reasoning.

This paper introduces Tree of Evidence (ToE), a hierarchical and explainable claim verification framework that dynamically retrieves and aggregates multi-source evidence using reinforcement learning. Experiments show 4-24 percentage point improvements over baselines, especially against adversarially poisoned inputs from Generative Engine Optimization.

This paper presents a five-arm ablation methodology for diagnosing which component of retrieval-warmed energy-based reasoning (RW-EBR) drives performance gains, applied to structured reasoning tasks like graph reachability and Sudoku. The method separates effects of class-prior bias, stochastic warm-starting, and graph-aligned value reuse.

ProvenAI introduces a framework for decomposing transparency in multi-hop question answering into three independently measurable layers: answer correctness, citation fidelity, and per-document influence, revealing a citation-influence gap where cited sources may have weak influence while uncited sources significantly shape the output.

Microsoft Research and collaborators introduce generative causal testing (GCT), a method that distills black-box brain prediction models into testable explanations and validates them with fMRI experiments, revealing specific brain region responses to language concepts.

This paper introduces methods to decompose location embeddings from geographic implicit neural representations into human-interpretable features, such as sparse latent concepts, natural language concepts, and visual features, revealing geographic structures like forests and urban areas.

This paper evaluates the robustness of multi-sensor fusion for cattle posture classification under temporal distribution shift, finding that multimodal models suffer significant performance drops and that simpler single-sensor models generalize better, highlighting shortcut learning issues.

This paper studies how pruning attention layers in LLMs affects explanation faithfulness and confidence calibration, finding that accuracy often remains high but interpretability and reliability degrade, highlighting a misalignment between model confidence, interpretability, and accuracy.

This paper analyzes why machine learning, particularly neural networks, remains opaque in its learning process by framing it as a complex dynamical system, identifying three key properties that contribute to learning opacity, and arguing that some sources may be irreducible.

This paper proposes a definition of good explanations based on counterfactuals and prior beliefs, and discusses the inherent difficulties in explaining LLM outputs under this definition.

This paper proposes Behavior Forecasters, a learned approach that predicts an LRM's future behavior (e.g., answer consistency and input sensitivity) from its reasoning trajectory, outperforming GPT-5.4 and Claude Opus 4.6 at lower cost.

This paper proposes training Behavior Forecasters to predict large reasoning model outputs from single trajectories, outperforming large language models like GPT-5.4 and Claude Opus-4.6 at lower computational cost, bypassing traditional explainability methods.

This paper examines counterfactual behavior in ML models through a geometric lens, showing that models with similar predictive performance can differ substantially in counterfactual outcomes due to the interaction between decision-boundary proximity and local data support. The findings identify counterfactual behavior as a distinct dimension from predictive performance, with implications for model selection and reliability of counterfactual explanation methods.

Researchers from the University of Michigan introduce MechSim, a mechanism-grounded neuro-symbolic reasoning framework that enables LLM agents to reason about the internal assumptions, dependencies, and execution behavior of scientific simulators rather than treating them as black boxes. The framework improves explanation quality and decision-making reliability across high-stakes domains like healthcare, finance, and public policy.

GridVQA-X introduces a diagnostic framework to evaluate cross-modal explainability by distinguishing genuine spatial-relational reasoning from cross-modal shortcuts in multimodal models.

Introduces SafeRx-Agent, a knowledge-grounded multi-agent framework for safe and explainable medication recommendation that generates fine-grained ATC code predictions while controlling drug interactions and contraindications, evaluated on MIMIC-III and MIMIC-IV datasets.