Tag
This paper introduces coherence, a geometric constraint for neural representations inspired by grid cells and head direction cells in the brain. Coherence ensures that features respond to geometrically connected regions of the data manifold, improving interpretability; the authors propose a differentiable objective (Coh) and validate it on synthetic data, rotated MNIST, and BERT token embeddings.
This paper investigates how large language models reorganize representational geometry during in-context learning, showing that ICL performance correlates with the geometric structure of tasks and that successful ICL involves increasing separability of representations.
This paper proposes Physics-Informed Multi-Scale Mamba (PIMSM), a state-space architecture that aligns model memory with physical timescales to improve robustness under distribution shift in scientific time series, demonstrating improvements on fMRI and weather forecasting tasks.
This paper uses shape analysis tools to characterize how different data augmentation strategies reshape the geometry of neural network representations, finding that augmentation strength and type lead to distinct, well-behaved trajectories in shape space.