Tag
This paper proposes PTTSD, a probabilistic framework for depression severity detection from clinical interview transcripts that models uncertainty and provides temporal interpretability, achieving competitive performance on benchmark datasets.
The paper proposes UASPL, a method that integrates predictive reliability into sample selection for self-paced learning using evidential neural networks, improving classification performance and interpretability.
This paper presents a large-scale evaluation of nine uncertainty estimation methods for LLMs across 22 languages, finding that prompting models to reason in English improves uncertainty estimation for low-resource languages and that the choice of method depends on model scale.
Proposes an uncertainty-aware reinforcement learning framework for algorithmic trading that integrates distributional, epistemic, and aleatoric uncertainty using SHAP-weighted reconstruction, MC Dropout, and LSTM consensus. Outperforms traditional models on five major US stock indices.
Introduces SALT, a benchmark with deterministic ground truth for evaluating LLM uncertainty at fine-grained atomic levels in long-form generation. Analysis of over 50 LLMs reveals insights into confidence functions, error propagation, and trade-offs with reasoning.
This paper proposes ASK+, a method for uncertainty-gated assistance from small language models (SLMs) to reinforcement learning agents in partially observable environments (POMDPs). By providing trajectory-aware context and structured chain-of-thought reasoning, ASK+ significantly improves success rates over baselines, demonstrating that prompt design and selective gating dominate model scale.
This paper investigates whether LLMs can approximate swarm intelligence effects through intra- and inter-model aggregation, finding significant error reductions up to 37 percentage points in MAPE across eight estimation tasks.
This paper presents a factorised study of probe-based uncertainty estimation in LLMs, showing that raw hidden states and attention features perform well in-domain but structured features are more robust under distribution shift, and provides pretrained probes as off-the-shelf baselines.
This paper evaluates confidence interval methods for classifier performance metrics under conditions typical of social science text classification, providing guidance on accurate interval estimation for metrics like recall and precision when using large language models.
This paper proposes a probabilistic framework for Alzheimer's disease progression forecasting that combines ordinal diagnosis prediction, multi-horizon trajectory generation, and decomposed uncertainty estimation using a Temporal Fusion Transformer encoder and an autoregressive Mixture Density Network. The model outperforms baselines on ADNI data, achieving near-nominal 90% credible interval coverage with clinically meaningful uncertainty signals.
This paper presents a systematic review and benchmark of 24 black-box uncertainty estimation methods for large language models across 4 models and 4 dataset settings, finding that no single method dominates but hybrid methods that combine multiple uncertainty signals perform well.
This paper proposes a prompt-based uncertainty decomposition method for LLM agents that separates action confidence from request uncertainty, enabling proactive clarification seeking in underspecified tasks. The method is evaluated on new clarification-augmented benchmarks across five LLM backbones, showing significant improvements.
This paper investigates verbalized methods for extracting LLM confidence in machine translation outputs, comparing them with internal token probabilities. The study finds that while both approaches perform similarly in error detection and calibration, there is little correlation between internal and verbalized confidence measures.
This paper presents theoretical bounds for uncertainty estimation and generalization in modern deep learning models.
This paper proposes EcoTab, a table-aware stepwise routing framework that separately estimates uncertainty for table tokens and text tokens to dynamically route reasoning steps between small and large models, achieving a better accuracy-efficiency trade-off on table reasoning tasks.
Helicase is an autonomous multi-agent LLM system that constructs supply chain knowledge graphs with uncertainty guidance. It decomposes complex queries into executable plans and outperforms baseline methods on the new SCQA benchmark.
This paper proposes a gated hyper-heuristic for job shop scheduling that uses regret-normalized rollout labels and contextual KNN uncertainty estimates to reduce label generation costs and avoid switching away from strong default rules unless the predicted improvement is credible. Experiments show the gated selector achieves low mean relative percentage deviation while significantly reducing computational cost.
Proposes CSR, a framework that calibrates LLMs directly in semantic space using a novel semantic calibration reward, reducing ECE by up to 40% and improving AUROC by up to 31% over verbalized-confidence baselines across multiple datasets.
This paper compares cross-validation ensembles to deep ensembles for uncertainty estimation in medical image segmentation. Deep ensembles outperform cross-validation ensembles in calibration and failure detection, while cross-validation ensembles better approximate inter-rater variability.
This paper investigates disagreement-based drift detection in ensembles of incremental decision trees, finding that while effective in neural networks, the method underperforms loss-based detectors for tree ensembles due to limited model plasticity.