Tag
A call for 2 emergency reviewers for ICLR, requiring expertise in RL, agent memory, and/or information theory, with reviews due by Nov 2 AoE.
The paper introduces 'model information,' an information-theoretic measure of how much a trained diffusion model's outputs depend on its training dataset, and uncovers two scaling regimes: a nonlinear regime marking the transition from memorization to imperfect generalization, and a linear generalization regime. It proposes a principled scaling law that induces extrapolatable scaling laws for memorization and sample quality.
本文提出数据差异化框架（DDF），结合互信息冗余惩罚、成本归一化的边际效用评分和治理输入，由SDSAE代理评估管线执行，用于量化候选数据资产的边际价值，并在信用卡欺诈检测基准上验证了其区分度与推荐稳定性。
提出Uncertainty-Aware Allocation (UAA)框架，通过兄弟节点语义分散度估计不确定性来动态分配LLM树搜索的每步扩展预算，在保持精度的同时将token使用量减少最多90%、延迟降低最多9倍，且可跨四种树搜索算法和开源/闭源模型泛化。
该论文将主动推理（Active Inference）与贝叶斯 UCBVI 放在统一框架下比较，发现 AIF 的信息增益衰减过快，需用平方根校准才能达到 √T 遗憾界；校准后的 AIF 在对抗性奖励设定下表现出 ℓ1 置信集优势，而在固定奖励下并不优于 Bayes-UCBVI。
A personal essay on how Edgar Allan Poe's 1843 story 'The Gold-Bug' introduced the author to cryptography, explaining substitution ciphers, frequency analysis, and the story's influence on figures like Claude Shannon and the foundations of information theory.
本文提出 CIG-SFT，一个信息论框架，通过估计每个示范 token 的任务条件信息增益来加权监督信号，从而在数学推理与代码生成基准上提升 SFT 的泛化能力，且无需额外数据或在线采样。
论文提出一种6G无线接入网的隐私保护意图履约与验证架构，仅依赖声明的意图类别和标准O1小区级计数器进行资源分配与验证，并通过数据处理不等式给出隐私泄露上界，在四个运营商网络的生产数据上验证了检测性能与隐私权衡。
该论文提出 Task-Sufficient Contraction 概念，研究在编码器、码本或失真目标确定之前，如何依据任务预先约简信源并保证下游问题族结果不变，给出了基于 regret 合并状态和二次损失下投影的精确刻画。
This paper analyzes how VLA policies should choose which backbone layers form the latent interface, finding that layer fusion often underperforms the best single layer and proposing InfoNCE as a compute-efficient proxy for layer selection that cuts GPU costs 9-33x while reducing selection regret.
During a talk setup for Kristen Nygaard's birthday celebration, Alan Kay encountered audio feedback that reminded him of Claude Shannon's noisy channel coding theorem, prompting an improvised commentary on its relevance to real-world technical issues.
This paper characterizes the minimal recurrent behavioral memory required for imitating an expert under partial observability, using information-theoretic measures and experimental validation.
This blog post provides a playful and beginner-friendly introduction to coding theory, using simple examples like flashlight communication to explain basic concepts.
This article surveys the connection between lossless compression algorithms and machine learning, introducing a design framework that makes compression-based methods competitive with conventional baselines and particularly effective for malware classification.
This paper introduces CoVer, a co-training framework for code generation that addresses self-play RL failures by using information-gain rewards and diversity-pruned tests, achieving significant pass rate improvements on benchmarks.
The paper introduces ePID, a scalable pipeline using supervised embeddings to compute partial information decomposition for symptom networks, enabling the separation of redundant and synergistic information in mental health data.
This paper introduces a certified, differentiable complexity estimator to control grokking in neural networks, accelerating the process and providing insights into algorithmic information dynamics of learning.
This paper introduces a rate-distortion theoretical framework for factual hallucination in closed-book question answering, distinguishing between errors from missing coverage and compression distortion under finite memory.
This paper introduces a synthetic benchmark to evaluate information-theoretic metrics for assessing the informativeness of text annotations in multimodal time series forecasting, demonstrating their utility for annotation auditing without model training.
This paper identifies sufficient sample sizes for maximum-likelihood support recovery of sparse binary signals in high-SNR regimes, revealing an information-theoretic threshold and trade-offs between measurement sparsity and computational cost.