Tag
This paper shows that the Johnson–Lindenstrauss lemma can be uninformative about retained geometry in high dimensions and derives exact limits for features like distance recovery, nearest-neighbor rankings, and covariance shape in Gaussian models.
The Entropic Scree is an information-theoretic upgrade to PCA for robust rank estimation and dimensionality reduction on messy tabular data, with open-sourced code and a preprint released.
The Entropic Scree is a non-parametric, information-theoretic diagnostic framework that maps intrinsic rank and informational gravity in complex tabular data, addressing limitations of standard PCA and other baselines.
This paper studies how smoothing the affinity matrix in t-SNE via a row-wise power transform affects neighborhood preservation, finding that sharpening improves nearest neighbor retention while smoothing enhances broader local neighborhoods.
The paper presents a unified geometric framework for understanding differentiable embeddings like t-SNE and UMAP, showing that existing diagnostics derive from a single object and proposing new curvature-based and integral-path trust measures.
This paper studies using Whisper for Persian speech emotion recognition, showing PCA-based dimensionality reduction improves performance and efficiency, while ASR fine-tuning offers only modest gains.
This paper systematically compares tabular foundation models (TabPFN) with classical regression approaches across 85 soil spectroscopy tasks, finding that TabPFN combined with PLS-derived features achieves the best predictive performance from field-scale to global spectral libraries.
FADEx introduces a novel local per-instance feature attribution method for explaining dimensionality reduction techniques, using Taylor expansion and Singular Value Decomposition to provide model-agnostic explanations and distortion analysis.
FloDR is a dimensionality reduction method based on a normalising flow that creates an invertible embedding, preserving both local and global structure while providing diagnostic tools like conditional spread and hidden contrast.
CLOE is a new semi-supervised anomaly detection method combining an autoencoder with a Christoffel Function-based detector, using a novel loss function to improve representation learning. It achieves state-of-the-art results on high-dimensional tabular data while maintaining simplicity.
This paper proposes incorporating symmetries into affinity kernels for spectral embedding, proving convergence of invariant graph Laplacians on quotient manifolds with improved sample complexity.
This paper introduces ODIN, a novel autoencoder architecture that enforces orthogonality and importance ordering of latent dimensions, recovering PCA-like interpretability in a fully non-linear regime. The method integrates geometric constraints into the training objective, theoretically grounded and empirically validated on synthetic and real-world datasets.
Proposes a method to embed graphs in high-dimensional space and search for informative 2D viewpoints that optimize aesthetic and readability metrics, enabled by a novel differentiable surrogate for edge crossings. Introduces an interactive system, DataFly, for exploring multiple candidate viewpoints.
Proposes GraphDR-LinUCB, a method for contextual bandits with graph-structured arms that projects features onto the graph's low-frequency spectral subspace. Achieves the first regret bound for spectral-projection-based contextual bandits and demonstrates 15x regret reduction on real datasets over full-dimensional LinUCB.
This paper presents exact dimensionality reductions using Schur complement and Sylvester's determinant identity to reduce computational complexity from O(N^3) to O(k^3+N^2k) per step for non-smooth NML estimation, achieving over 14,000x speedup while maintaining numerical precision.
Proposes Distance-Adaptive Representation (DAR) which reduces key-value dimensionality for distant tokens while preserving full dimensionality for nearby tokens, improving KV cache efficiency without performance loss.
This study reveals that LLM text embeddings are hijacked by high-frequency tokens (e.g., periods, articles) and proposes EmbedFilter, which performs SVD on the unembedding matrix and subtracts the projection component to release true semantics, achieving zero-training-cost dimensionality reduction and retrieval efficiency gains.
The paper identifies that LLM text embeddings overly express high-frequency uninformative tokens and proposes EmbedFilter, a linear transformation that filters out this subspace to improve semantic representations and enable dimensionality reduction.
A short mathematical write-up on Principal Component Analysis (PCA), explaining the concept and its applications.
ScaleMAP is a new nonlinear dimensionality reduction method that preserves local density and neighborhood structure by rescaling embedding distances based on original-space local radii, achieving better density preservation than DensMAP while maintaining UMAP-level neighborhood preservation.