Tag
ALICE is a foundation model for zero-shot, in-context mutual information estimation, trained only on synthetic distributions and using rectified-flow velocity fields to estimate MI without per-distribution training. It achieves competitive accuracy across biology, genetics, and neuroscience benchmarks.
This paper introduces MI-SARSA, a reinforcement learning algorithm that models bounded rationality by incorporating mutual-information regularization to balance policy complexity and reaction time under cognitive constraints.
The paper proposes Independent Token Sampling (ITS), a query-efficient framework for detecting training data in diffusion large language models by selecting weakly dependent tokens to reduce approximation errors, achieving improved performance over state-of-the-art baselines.
This paper introduces a synthetic benchmark to evaluate information-theoretic metrics for assessing the informativeness of text annotations in multimodal time series forecasting, demonstrating their utility for annotation auditing without model training.
Entropic Scree is a new diagnostic tool for assessing signal strength and structure in dirty tabular data using mutual information, with a preprint and upcoming Python and R packages.
This study compares mutual information and data-based sensitivity analysis for feature selection in bank telemarketing, showing that sensitivity analysis achieves good prediction with fewer features while mutual information is better for cost reduction.
CutClean is a privacy-aware pruning method for neural networks that reduces private information leakage while increasing sparsity, using auxiliary privacy heads to quantify and mitigate privacy risks.
This paper proposes MIDAS, a unified framework for incomplete multimodal sentiment analysis that uses mutual information disentanglement and uncertainty-aware fusion to robustly represent and integrate modalities under missing-data conditions.
An intuitive, visual introduction to information theory covering entropy, mutual information, and channel capacity, assuming only basic probability. The paper explains fundamental limits of compression and transmission.
Proposes MI-EPO, an information-theoretic framework for multi-objective alignment of large language models that uses mutual information to enhance exploration and ensure generated responses are distinguishable and aligned with different preference vectors, achieving stable trade-offs across conflicting objectives.
InfoShield introduces a privacy-preserving method for speech representations in mental health screening using information-theoretic optimization, reducing sensitive attribute inference while maintaining diagnostic accuracy. A novel TimeAwareMINE estimator addresses temporal-static misalignment in sequential speech.
InfoAtlas is a foundation model that directly estimates mutual information in a single forward pass, achieving 100x speedup over traditional neural estimators while matching accuracy. It is pretrained on synthetic data and generalizes to real-world scenarios.
This paper introduces PromptNCE, a method that uses large language models and contrastive prompts to estimate pointwise mutual information zero-shot, achieving high correlation with human-derived ground truth across three datasets.
OmniISR proposes a unified framework combining centralized and federated learning via intermediate supervision and regularization at hidden layers, offering theoretical convergence guarantees and reducing the CL–FL gap by 22.60%.
This paper introduces a spectral diagnostic method to detect hidden coalitions in multi-agent AI systems by analyzing internal neural representations via mutual information, addressing critical AI safety and alignment challenges.
ArXiv preprint quantifies layered RTL and LTR constraints in the Voynich Manuscript, showing 97 % of cross-boundary mutual information lies in specific grapheme transitions and that simple generative models cannot simultaneously reproduce all observed structural signatures.