mutual-information

SynAct提出一种主动学习框架，通过在重复采样中计算决策与最终结果之间的互信息，将稀缺的专家反馈精准分配到最影响结果的决策点上，并将修正蒸馏为可复用规则，在四个基准、三个模型家族的全部16个设置中平均提升11.7个百分点。

该论文证明，仅靠在发布时移除信息内容（以互信息衡量）无法普遍保证开源权重模型的抗篡改能力——函数保持的重参数化会改变梯度下降几何结构，使得即使互信息为零，攻击者也可能在一次梯度步骤内恢复被移除的能力。作者提出认证需要对攻击动力学施加约束，而非仅依赖发布时的互信息。

ALICE is a foundation model for zero-shot, in-context mutual information estimation, trained only on synthetic distributions and using rectified-flow velocity fields to estimate MI without per-distribution training. It achieves competitive accuracy across biology, genetics, and neuroscience benchmarks.

This paper introduces MI-SARSA, a reinforcement learning algorithm that models bounded rationality by incorporating mutual-information regularization to balance policy complexity and reaction time under cognitive constraints.

The paper proposes Independent Token Sampling (ITS), a query-efficient framework for detecting training data in diffusion large language models by selecting weakly dependent tokens to reduce approximation errors, achieving improved performance over state-of-the-art baselines.

This paper introduces a synthetic benchmark to evaluate information-theoretic metrics for assessing the informativeness of text annotations in multimodal time series forecasting, demonstrating their utility for annotation auditing without model training.

Entropic Scree is a new diagnostic tool for assessing signal strength and structure in dirty tabular data using mutual information, with a preprint and upcoming Python and R packages.

This study compares mutual information and data-based sensitivity analysis for feature selection in bank telemarketing, showing that sensitivity analysis achieves good prediction with fewer features while mutual information is better for cost reduction.

CutClean is a privacy-aware pruning method for neural networks that reduces private information leakage while increasing sparsity, using auxiliary privacy heads to quantify and mitigate privacy risks.

This paper proposes MIDAS, a unified framework for incomplete multimodal sentiment analysis that uses mutual information disentanglement and uncertainty-aware fusion to robustly represent and integrate modalities under missing-data conditions.

An intuitive, visual introduction to information theory covering entropy, mutual information, and channel capacity, assuming only basic probability. The paper explains fundamental limits of compression and transmission.

Proposes MI-EPO, an information-theoretic framework for multi-objective alignment of large language models that uses mutual information to enhance exploration and ensure generated responses are distinguishable and aligned with different preference vectors, achieving stable trade-offs across conflicting objectives.

InfoShield introduces a privacy-preserving method for speech representations in mental health screening using information-theoretic optimization, reducing sensitive attribute inference while maintaining diagnostic accuracy. A novel TimeAwareMINE estimator addresses temporal-static misalignment in sequential speech.

InfoAtlas is a foundation model that directly estimates mutual information in a single forward pass, achieving 100x speedup over traditional neural estimators while matching accuracy. It is pretrained on synthetic data and generalizes to real-world scenarios.

This paper introduces PromptNCE, a method that uses large language models and contrastive prompts to estimate pointwise mutual information zero-shot, achieving high correlation with human-derived ground truth across three datasets.

OmniISR proposes a unified framework combining centralized and federated learning via intermediate supervision and regularization at hidden layers, offering theoretical convergence guarantees and reducing the CL–FL gap by 22.60%.

This paper introduces a spectral diagnostic method to detect hidden coalitions in multi-agent AI systems by analyzing internal neural representations via mutual information, addressing critical AI safety and alignment challenges.

ArXiv preprint quantifies layered RTL and LTR constraints in the Voynich Manuscript, showing 97 % of cross-boundary mutual information lies in specific grapheme transitions and that simple generative models cannot simultaneously reproduce all observed structural signatures.