Need to estimate rank or perform dimensionality reduction on big, messy tabular data? The Entropic Scree is an information-theoretic upgrade to PCA

Reddit r/ArtificialInteligence 08/21/26, 02:44 AM Papers

dimensionality-reduction information-theory open-source preprint tabular-data data-analysis pca

Summary

The Entropic Scree is an information-theoretic upgrade to PCA for robust rank estimation and dimensionality reduction on messy tabular data, with open-sourced code and a preprint released.

Here's a new rank estimation method I've been working on. It's basically an upgraded Principal Component Analysis (PCA) built on information theory instead of linear variance. It's robust to mixed data types, highly non-linear generative processes, low signal to noise ratios, and sparsity (more variables than samples). It's especially useful if you need to find the exact rank of a dataset to explicitly size a neural bottleneck (like an autoencoder). I just open-sourced the code and put up the preprint. I'd love to hear what you guys think or if you end up testing it on your own data! GitHub (R Code): https://github.com/tjleestjohn/Entropic-Scree Preprint: https://doi.org/10.5281/zenodo.22028087

Original Article

Similar Articles

Mapping intrinsic rank and informational gravity in complex tabular data: I developed a non-parametric, model-agnostic, information-theoretic diagnostic to bypass the limits of linear, rank, and Euclidean baselines. [R]

Reddit r/MachineLearning

The Entropic Scree is a non-parametric, information-theoretic diagnostic framework that maps intrinsic rank and informational gravity in complex tabular data, addressing limitations of standard PCA and other baselines.

How to assess if there is a strong signal in your dirty data [Project]

Reddit r/MachineLearning

Entropic Scree is a new diagnostic tool for assessing signal strength and structure in dirty tabular data using mutual information, with a preprint and upcoming Python and R packages.

Breaking Structural Isolation: Scalable Graph Clustering via Community-Aware Sampling and Structural Entropy

arXiv cs.LG

This paper proposes SCISE, a scalable unsupervised graph clustering framework that uses community-aware sampling and structural entropy to overcome structural isolation in mini-batch training, achieving state-of-the-art results on benchmark datasets.

Discrete Ricci Curvature on Protein Contact Graphs for Lightweight Fold Classification

arXiv cs.LG

This paper investigates discrete Ricci curvature on protein contact graphs as a lightweight structural descriptor for fold classification, showing that a 22-dimensional curvature feature outperforms mean-pooled ESM-2 embeddings on CATH and SCOPe benchmarks.

Data eccentricity, asymptotics of Gaussian RBF reproducing kernel Hilbert space, and kernel PCA

arXiv cs.LG

This paper proves that for large bandwidths, the Gaussian RBF reproducing kernel Hilbert space becomes asymptotically isometric to Euclidean space, causing kernel PCA to converge to linear PCA. A measure of data eccentricity predicts convergence behavior in top principal directions.

Similar Articles

Mapping intrinsic rank and informational gravity in complex tabular data: I developed a non-parametric, model-agnostic, information-theoretic diagnostic to bypass the limits of linear, rank, and Euclidean baselines. [R]

How to assess if there is a strong signal in your dirty data [Project]

Breaking Structural Isolation: Scalable Graph Clustering via Community-Aware Sampling and Structural Entropy

Discrete Ricci Curvature on Protein Contact Graphs for Lightweight Fold Classification

Data eccentricity, asymptotics of Gaussian RBF reproducing kernel Hilbert space, and kernel PCA

Submit Feedback