Tag
The paper analytically studies Dense Associative Memories trained on codewords of random expander LDPC codes, identifying a memorization-to-generalization transition reminiscent of grokking. It introduces a thresholding learning rule that achieves sample-efficient learning with only O(log n) training samples and enables the emergence of latent variable representations.
A perspective paper proposing a four-stage research agenda (Ground, Structure, Verify, Act) for interpreting JEPA-based latent world models, arguing interpretability should shift from decoding representations to understanding how latent variables interact, evolve, and causally drive prediction.
DSReg提出通过依赖稀疏性正则化（Structural Diversity条件），在无需重建、解码器或标签的情况下可证明地恢复单个世界隐变量，建立了首个完全可识别的JEPA表示学习方法。
ReLaG is a scalable, modality-agnostic framework that infers groups of related samples via proximity graphs and community detection to produce independent train–test splits, addressing the over-optimistic generalization estimates caused by random splits on data with latent relations. It scales better than existing relation-aware methods and offers a label-free procedure to adapt splitting resolution to production settings, available as pip-installable open-source software.
EmoStance is a method for empathetic response generation that uses emoji weak supervision to model response-side affective orientation, improving contextual specificity and perceived responsiveness in dialogues.
This paper explores how attention mechanisms in language models render latent variables accessible without a selective gate, identifying a demand-specific mid-depth window where attention-mediated gathering occurs.
This paper proposes Missing-Data Flow Matching, a method that treats missing coordinates of training samples as latent variables and averages the flow matching loss over possible values. Theoretical analysis shows the correction is exact and provides design guidance, with experiments validating the approach on tabular data.
This paper investigates the architectural locus of causal emergence in active inference agents, finding that a slow global latent variable (g) serves as the primary site for integrated information (Φr) and that scalar Φr should not be interpreted as a direct measure of learned integration.
Weave of Formal Thought (WoFT) introduces a sound and complete constrained decoder for code generation that guarantees syntactic validity relative to the full Tree-sitter specification, and a fine-tuning method that trains models to interleave grammar symbols using reweighted wake-sleep, improving perplexity on Python code generation.
This paper introduces score-based methods for causal discovery in the presence of latent variables, offering theoretical guarantees of consistency and score equivalence, and unifies several constraint-based approaches.
The paper introduces diverse dictionary learning, showing that key set-theoretic relationships among latent variables can be identified from observational data without strong assumptions, enabling partial or full identifiability with minimal inductive bias.