Competence-Gated Pooling of Language Models and Priors for Event Forecasting
Summary
This paper introduces competence-gated pooling to determine when language models should influence forecasts that already have external predictions, improving accuracy by learning domain-specific weights and deferring when external sources are stronger.
View Cached Full Text
Cached at: 09/14/26, 10:37 PM
Paper page - Competence-Gated Pooling of Language Models and Priors for Event Forecasting
Source: https://huggingface.co/papers/2609.12101 When should an LLM actually influence a forecast that already has a market, crowd, or statistical prediction?
Most forecasting work asks how accurate an LLM is in isolation. We instead study relative competence: the model’s marginal value beyond an already-available forecast.
We introduce competence-gated pooling, which learns domain-specific source weights from resolved outcomes and defers when the external source is stronger. Across 2,357 resolved forecasting questions, the gate improves Brier score from 0.0771 → 0.0732 and significantly outperforms global forecast combinations. On a strong live-market subset, however, it correctly finds no significant gain from adding the LLM.
We also find that self-reported LLM confidence does not reliably predict when the model beats the external forecast—while outcome-estimated competence enables substantially better selective prediction.
Takeaway: don’t ask only “Is the model good?” Ask “Does it add information beyond what we already know?”
Similar Articles
World Models Meet Language Models: On the Complementarity of Concrete and Abstract Reasoning
This paper proposes Privileged-Future On-Policy Self-Distillation (PF-OPSD) for controlled concrete reasoning, combining world models' visual simulation with language models' abstract reasoning to improve prediction accuracy and robustness on two new benchmarks.
Forecasting Future Behavior as a Learning Task
This paper proposes training Behavior Forecasters to predict large reasoning model outputs from single trajectories, outperforming large language models like GPT-5.4 and Claude Opus-4.6 at lower computational cost, bypassing traditional explainability methods.
Competing Learning in Hybrid Architectures
The paper identifies a Competing Learning phenomenon in hybrid language models where recurrence and attention layers compete to learn the same functions, and proposes a staged training procedure that improves language modeling loss, common-sense reasoning, and retrieval tasks on models up to 1B parameters.
Task-Level Natural Language Priors as Learning Signals for Low-Resource LLM Training
The paper proposes Prior-Guided Tuning (PGT) and Contrastive Prior Steering (CPS) to use natural language priors as auxiliary learning signals, improving low-resource LLM training performance on tasks like AmbiMath, Jigsaw, and MNLI/HANS.
Co-Evolving Acting and Forecasting for Language Agents
CoPE 论文提出通过在成功轨迹上训练多步动作目标,使语言智能体的行动与预测能力协同演化,从而在不增加推理开销的情况下提升样本效率,相比 GRPO 与 ECHO 在多项基准上取得更强表现。