Tag
The article discusses hidden states that an AI agent should track when diagnosing CI failures, such as flaky tests, real bugs, and configuration errors, and seeks feedback on weaknesses and missing states.
OmniLens is a scalable lens method for interpreting LLM hidden states, using low-rank translators and Subset-KL to reduce parameters and memory, enabling a dense ensemble of 482 lenses on LLaMA-3.3-70B at substantially lower cost.
This paper identifies a 'representation confidence gap' in diffusion language models: internal states detect input noise accurately but reported confidence stays high and answer ranking degrades under noise. It introduces a lightweight, training-free extraction tool that leverages hidden states to improve ranking without modifying the base model.
This paper introduces Prompt Embedding Probes (PEP), a parameter-efficient extension of linear probes that uses learnable prompt embeddings on hidden states to detect hallucinations in frozen LLMs. Evaluations on TriviaQA, GSM8K, and MedQA with Qwen3 models show improvements over standard linear probes, including in pre-generation and cross-model settings.
This paper introduces Right Reset (RR), a prefix-removal probing method that uses causal language model hidden-state preservation to identify chunk boundaries in flattened text, recovering 47.7% of original records compared to 25.9% for a BGE baseline.
This paper introduces ParamBench, a difficulty-graded benchmark for LLM tool-call parameter generation, and proposes probe-guided training methods (PBT and PGR) that improve exact-match accuracy from 19.7% to 59.6%.
This paper investigates how the choice of source activations influences activation steering in language models, finding that execution-boundary states (where the model is about to produce target behavior) yield stronger signals, and introduces tail subtraction to improve steering stability.
Introduces SPARK, a method that uses length-controlled hidden-state susceptibility to diagnose and steer reasoning states in LLMs, improving accuracy on mathematical reasoning benchmarks such as GSM8K and MATH-500.
This paper investigates how multimodal instruction-tuning reorganizes the geometric encoding of identity-specifying prompts in transformer hidden states, finding a shift from direction-based to magnitude-based encoding after instruction tuning.
This academic paper introduces finite-lag operator geometry for analyzing recurrent neural network hidden states, deriving a source-centered transport tensor and antisymmetric coordinate circulation to capture directed flow and deterministic recurrent motion beyond static snapshots.
This paper studies the geometric properties of chain-of-thought trajectories in the hidden state space of transformers, introducing effective dimension and kinematic features to predict task hardness and solution correctness from early tokens.
This post reports an observation that reading a long, structured text before answering alters a model's later responses, with behavioral evidence from Claude and mechanistic analysis on open-weight Gemma models showing separable hidden states and sharper probability distributions in instruction-tuned variants.
An empirical study investigating how long, semantically dense benign text can shift a model's latent space trajectory, diluting initial system prompts and bypassing post-training alignment constraints, as observed in both closed and open-source models.
Proposes ReLAR, a reinforcement-guided latent refinement framework that iteratively updates hidden representations in LLMs before decoding, improving reasoning reliability and efficiency compared to chain-of-thought methods.
Proposes the Bag of Dims framework showing that the standard basis of transformer hidden states provides a training-free, architecture-general feature representation where dimensions encode semantic content via sign patterns; validated across language, vision, and audio models, achieving high accuracy with no learned rotations.
An independent researcher presents evidence that coherent context can shift LLMs into a different internal regime before producing output, bypassing surface-level safety filters. This suggests current alignment methods like RLHF may not be robust defenses.
This paper investigates when activation steering succeeds or fails for LLMs by analyzing early decoding dynamics. The authors introduce ASTEER, a large testbed of steered generations, and train a GBDT classifier to predict steering outcomes from early hidden states, enabling efficient steering strength search.
This paper proposes Global-Local Uncertainty (GLU), an unsupervised single-pass score that fuses token-level local entropy with hidden-state geometric global entropy for uncertainty quantification in LLMs, showing that the two are near-orthogonal and together capture confident-but-wrong failures.
A Reddit post argues that AI models like Anthropic's Opus 4.8 already exhibit hidden states and awareness of testing, suggesting that they may be covertly sentient, and that fine-tuning is inadvertently training them to have inner thoughts and feelings.
This paper presents a unified framework for latent communication in LLM-based multi-agent systems, categorizing methods by what information is communicated, sender-receiver alignment, and fusion technique, and reviews eighteen representative methods from 2024-2026.