representation-learning

Introduces CANDOR, a chance-calibrated discordance measure for frozen foundation encoders that corrects for prevalence bias, revealing that no encoder is completely blind but all are weak for certain fine-grained findings.

CAMMAR introduces a representation learning framework that organizes Arabic metaphorical meaning into nested lexical, cultural, and metaphorical subspaces using a staged semantic curriculum, achieving strong metaphor detection (AUC up to 0.84) on a new span-annotated dataset.

This paper studies oversmoothing in hypergraph neural networks from a dynamical-systems perspective, proposing a reaction-diffusion framework (HNRD) that preserves node-discriminative variation and achieves depth-robust propagation. Experiments show consistent improvement over baselines.

This paper applies Wilsonian renormalization group theory to analyze Transformer attention as a perturbation of the MLP residual-stack fixed point, determining whether attention is relevant or irrelevant based on data correlation length. Experiments on synthetic Markov chains confirm that attention's relevance depends on the spectral structure of the data-generating process, with the first-layer head dominating the transition.

LLM4EHR proposes a clinical foundation model that temporally aligns Electronic Health Record time series with medical event sequences using a domain-adapted large language model and a regularized contrastive objective, improving downstream prediction tasks.

AV-JEPA extends LeJEPA to audio-visual self-supervised learning, achieving cross-modal alignment in latent space without decoders, contrastive negatives, or complex losses, and obtains competitive classification on VGGSound and AudioSet.

This paper proposes a method to decompose the response distribution of language models into structured, strategy-conditioned representations using a latent variable, addressing posterior collapse with a model-directed reconstruction objective.

This paper proposes FaStR, a method that factorizes the transition kernel in reinforcement learning using CP decomposition into separate state, action, and next-state encoders, improving sample efficiency especially in high-dimensional locomotion tasks.

Introduces a novel 'cake' representation for game levels over time that implicitly encodes dynamic information, along with a generative approach (Playtrace Reconstructive Partitioning, PRP) that outperforms six state-of-the-art PCG methods in Sokoban by producing valid levels without sacrificing solution diversity.

This paper introduces a novel method using sparse autoencoders (SAEs) to learn interpretable features from intermediate network activations for out-of-distribution (OOD) detection, achieving state-of-the-art performance and providing insights into how distribution shifts affect learned representations.

This paper introduces a white-box instrument using hidden deterministic finite automata to separately measure reward success and latent-state learning in reinforcement learning agents, finding that high reward does not imply task understanding.

This paper argues that within-class variance in language model representations is not incomplete neural collapse but allocated information storage, and that the allocation obeys an information floor law. Across 14 models, macro-category structure carries only 4–12% of representational variance, while within-token context dominates at 79–91%.

Proposes a novel multilevel graph prompt learning framework for crystal property prediction that captures local chemical semantics and global structural symmetry via node-level and graph-level soft prompts, improving state-of-the-art GNN performance by 3%-15% and enabling cross-property knowledge transfer.

Researchers introduce the Unified Hand Action Space (UHAS), a representation enabling a single policy to control robotic hands with different kinematic structures and numbers of fingers, advancing cross-embodiment dexterous manipulation.

A step-by-step annotated implementation and explanation of Joint Embedding Predictive Architectures (JEPA) for self-supervised learning, covering I-JEPA, V-JEPA, and LeJEPA.

This paper proposes compressing instruction prompts into a single activation vector via learned weighted sums of intermediate layer activations, achieving under 2% accuracy drop and revealing insights into LLM activation space structure.

This paper investigates how confidence-related information evolves during LLM answer generation and introduces future confidence distillation, which trains predictors on pre-solution hidden representations using post-solution correctness probes to achieve reliable and sample-efficient confidence estimation.

The paper introduces Riemannian Mean Pooling (RMP), a method that aggregates token embeddings from pre-trained language models using Riemannian geometry via pullback metrics, showing improved performance over Euclidean pooling on sentence classification tasks.

This paper audits the measurement validity of representation metrics in grokking, showing that values at the grokking transition overstate converged circuit complexity and that compression lags generalization. It provides tooling to separate onset from compression and reports negative results on generality.

This controlled study compares geometric algebra (Cl(3,0)) layers against a minimal scalarization baseline for SO(3)-equivariant vector learning, finding that geometric algebra adds no benefit for single-stage tasks but significantly beats scalarization in low-data regimes for deeply composed group operations.