Tag
This paper reports that deep reinforcement learning agents using frozen, randomly initialized CNN feature extractors spontaneously develop extremely sparse fully-connected representations, compressing task-relevant information through very few neurons without any sparsity-inducing objective.
This paper evaluates hand-off compression in a two-agent LLM relay for travel planning, comparing methods like JSON extraction and narrative summarization, finding that structured representations preserve constraints better.
This paper establishes a formal correspondence between the SIGReg objective used in Joint-Embedding Predictive Architectures (JEPAs) and the Active Inference framework, showing that under certain conditions the training objective becomes an exact variational free energy. It organizes non-contrastive regularisers into an entropy-estimator hierarchy and proves theoretical results including exact information bottleneck and preserved surprise bounds for SIGReg.
Proposes GenDa, a unified framework for unsupervised reinforcement learning that addresses non-stationary skill semantics and brittle generalization via skill relabeling and a complementary information bottleneck, significantly improving data efficiency and generalizability.
This paper proposes a linguistic-invariant spoofing detection framework that uses teacher-student adversarial learning and a variational information bottleneck to mitigate linguistic bias, achieving up to a 36.2% relative reduction in equal error rate across nine datasets.
The article introduces SynIB, a scalable objective based on the information bottleneck principle that targets synergistic information in multimodal learning by penalizing confident predictions when a modality is masked, improving performance on tasks requiring cross-modal reasoning.
The paper challenges the assumption that cosine alignment between supervised latents and visual targets improves accuracy in vision-language models, finding a strong negative correlation. It introduces PRISM diagnostics revealing that answers are decoded downstream from latents, not within them, and that the auxiliary loss reshapes the language model via shared parameters.
Introduces Stepwise Confidence Attribution (SCA), a framework for assigning step-level confidence to reasoning traces from black-box LLMs without internal access, using the Information Bottleneck principle to distinguish legitimate variability from errors. Experiments show SCA reliably identifies low-confidence steps and improves self-correction success rates by up to 13.5% over answer-level feedback.
This paper introduces an Information Bottleneck Adapter (IB-Adapter) for Vision-Language-Action (VLA) models to improve robustness against unseen visual disturbances without requiring extra data, achieving up to 30% improvement with minimal parameter overhead.