surprisal

该论文研究语言模型在处理花园路径句后是否会保留被放弃的句法分析残余，发现残余效应主要储存在名词短语的状态中，并可通过激活修补（activation patching）在32B和70B模型中近乎完全消除该残余。

This paper evaluates whether Cantonese-adapted language models better predict Cantonese reading using eye-tracking data, comparing models like CKIP GPT-2 and CantoneseLLM-7B. Results indicate that more extensive Cantonese-specific training improves predictive fit, though performance varies by information-theoretic measure.

This paper investigates whether five open-weight LLMs exhibit human-like sensitivity to psycholinguistic factors in anaphor resolution, using surprisal and comprehension accuracy as behavioral measures. Results show selective cognitive alignment, with some models matching human discourse sensitivity but not semantic interference effects.

This paper investigates whether language models' next-word prediction aligns with human cognitive processing by analyzing EEG signals and event-related potentials, finding that only surprisal correlates with human brain responses, especially for open-class words.

This study demonstrates that contextual semantic relevance, measuring how strongly an incoming word relates to its recent semantic context, reliably predicts fMRI BOLD responses during naturalistic speech comprehension across two datasets, whereas surprisal (local probabilistic expectation) does not. The findings support that slow hemodynamic responses are especially sensitive to contextual semantic integration rather than local prediction.

STARE addresses policy entropy collapse in GRPO-based reinforcement learning for large language models by introducing surprisal-guided token-level advantage reweighting and target-entropy regulation, achieving 4%-8% accuracy gains on AIME benchmarks.

Introduces trajectory extrapolation error, a measure derived from transformer LM hidden states that predicts human reading times independently of and orthogonally to surprisal, revealing a dissociable component of incremental processing cost.

This paper tests the Parse Multiplicity Mismatch Hypothesis, proposing that language models underpredict human processing difficulty in garden path sentences because they can consider more simultaneous parses. Using RNNGs with beam search, they find reducing the number of active parses increases predicted garden path effects, but not enough to fully capture human data.