Tag
This paper analyzes the embedding space of Masked Diffusion Language Models (MDLMs) and finds it is hyperspherical, making linear interpolation suboptimal. The authors introduce Spherical Soft-Masking (S-SM), which uses SLERP and Fréchet means on the hypersphere, improving MAUVE and perplexity over prior soft-masking approaches.
This paper examines the embedding geometry of GPT-2 Small around the token 'Trump', comparing discretized and continuous nearest neighbor approaches to understand representational structure.
Analyzes the 64-D embedding manifold of Google AlphaEarth across 12.1M U.S. samples, shows non-Euclidean structure and poor vector arithmetic, then builds an agentic system with geometry-aware tools that outperforms parametric baselines on environmental queries.
Systematic study shows LLM-based dense retrievers outperform BERT baselines on typos and poisoning but remain vulnerable to semantic perturbations, with embedding geometry predicting robustness.