Tag
Proposes a method to embed graphs in high-dimensional space and search for informative 2D viewpoints that optimize aesthetic and readability metrics, enabled by a novel differentiable surrogate for edge crossings. Introduces an interactive system, DataFly, for exploring multiple candidate viewpoints.
Proposes GraphDR-LinUCB, a method for contextual bandits with graph-structured arms that projects features onto the graph's low-frequency spectral subspace. Achieves the first regret bound for spectral-projection-based contextual bandits and demonstrates 15x regret reduction on real datasets over full-dimensional LinUCB.
This paper presents exact dimensionality reductions using Schur complement and Sylvester's determinant identity to reduce computational complexity from O(N^3) to O(k^3+N^2k) per step for non-smooth NML estimation, achieving over 14,000x speedup while maintaining numerical precision.
Proposes Distance-Adaptive Representation (DAR) which reduces key-value dimensionality for distant tokens while preserving full dimensionality for nearby tokens, improving KV cache efficiency without performance loss.
This study reveals that LLM text embeddings are hijacked by high-frequency tokens (e.g., periods, articles) and proposes EmbedFilter, which performs SVD on the unembedding matrix and subtracts the projection component to release true semantics, achieving zero-training-cost dimensionality reduction and retrieval efficiency gains.
The paper identifies that LLM text embeddings overly express high-frequency uninformative tokens and proposes EmbedFilter, a linear transformation that filters out this subspace to improve semantic representations and enable dimensionality reduction.
A short mathematical write-up on Principal Component Analysis (PCA), explaining the concept and its applications.
ScaleMAP is a new nonlinear dimensionality reduction method that preserves local density and neighborhood structure by rescaling embedding distances based on original-space local radii, achieving better density preservation than DensMAP while maintaining UMAP-level neighborhood preservation.
Proposes DIVE, a compression adapter for embedding dimensionality reduction that uses self-limiting gradient updates and head-wise NT-Xent contrastive loss to prevent overfitting on small datasets, outperforming existing methods on BEIR benchmarks.
This paper proposes a hybrid quantum-classical workflow for plant phenomics classification under small-data regimes, using supervised latent restructuring (PCA + LDA) to improve geometric separability before quantum kernel alignment. Experiments show improved separability but highlight compression trade-offs and the difficulty of achieving strong quantum performance.
This paper introduces an unsupervised framework for modeling acquisition-related variability in structural connectomes using hybrid latent space modeling, eliminating the need for manual capacity tuning by architecturally annealing encoder outputs.
This paper proposes Spectra, a method using spectral occupancy to analyze and control the realized capacity of latent graph models, arguing that rank is not equivalent to model capacity.
This post highlights the Johnson–Lindenstrauss Lemma, explaining its importance for ML engineers in understanding dimensionality reduction, random projections, and embedding efficiency.
This article introduces a polynomial autoencoder that improves upon PCA for compressing transformer embeddings by using a quadratic decoder to capture nonlinear variance. Benchmarks on BEIR show it significantly outperforms standard PCA and Matryoshka embeddings in retrieval quality while maintaining high compression ratios.
Spectral Tempering (SpecTemp) proposes a learning-free method for embedding compression in dense passage retrieval that adaptively determines optimal spectral scaling based on signal-to-noise ratio analysis, outperforming fixed hyperparameter approaches like PCA and whitening.