This paper investigates the grokking transition in neural networks using replica-overlap probes, but reports challenges with the probe's validity and offers post-hoc statistical analysis.
The paper investigates privileged context design in on-policy self-distillation, demonstrating that intermediate levels of abstraction can improve model performance over full solutions while using fewer hint tokens.
This paper provides a mechanistic comparison of knowledge-conflict circuits in LLMs under instruction tuning, finding that tuning gates rather than rewires these circuits across multiple model families, with implications for interpretability.
EMGBlend introduces a self-supervised framework for pretraining on heterogeneous EMG datasets, addressing differences in electrode layouts, frequency support, and data source imbalances to improve gesture recognition and force decoding tasks.
SambaGraph introduces a spatio-temporal graph dataset and benchmark for modeling soccer tactical responses, curated from 2022 FIFA World Cup data to classify actions and retrieve defensive examples.
DefaultGNN is a dual-perspective graph neural network framework for predicting corporate default using buyer-seller transaction networks, achieving improvements over baselines and validating with real-world data.
This paper proposes a training recipe for tabular foundation models using a Joint-Embedding Predictive Architecture (JEPA) to address latent term collapse, and compares its performance to a value-only approach across 147 real datasets.
Hill Sampling is a simple test-time scaling method that repeatedly samples edits to the best verified program using frozen LLMs, achieving state-of-the-art results on algorithmic problems like circle packing and Erdős' minimum-overlap problem.
This paper introduces the first method for continuous gradient descent optimization in machine learning models with p-adic parameters, using the Berkovich affine line to enable effective learning on tasks like modular arithmetic.
This paper proposes a proxy-guided hierarchical reinforcement learning framework to defend against diverse inference attacks on smart meter data by learning battery-based load-shaping policies that disrupt non-intrusive load monitoring patterns.
Proposes Terminal Shrinkage Averaging (TSA) to separate learning-rate schedule from model estimator in LLM pretraining, improving validation quality and potentially accelerating benchmarks.
This paper proposes a practical recipe for semi-supervised federated ASR using online pseudo-labels with server update stabilization, demonstrating significant improvements over prior methods in both in-domain and cross-domain settings.
This paper introduces Mean Velocity Matching (MVM) to parameterize stochastic reverse dynamics in diffusion models using a single learned field, enabling both stochastic and deterministic sampling with competitive generation quality.
This paper introduces PermuFormer, an autoregressive transformer pretrained on multi-task, multi-encoding data for permutation-focused tasks in algebraic combinatorics, demonstrating effective fine-tuning on downstream tasks compared to baselines.
This paper introduces the Lightweight Ranking Heads framework to accelerate multi-task experimentation in production recommender systems by enabling dynamic task injection without retraining backbone models, reducing iteration cycles from weeks to days at YouTube scale.
The paper investigates uncertainty quantification methods for neural surrogates in computer-aided engineering, comparing Gaussian processes, Monte Carlo dropout, and deep ensembles on industrial datasets to evaluate their performance in various engineering contexts.
This paper introduces a graph-based deep reinforcement learning framework for the one-dimensional bin packing problem, reducing optimality gaps compared to existing methods and enabling zero-shot generalization across instance sizes.
This research paper extends FunctionGemma 270M for practical on-device Android workflows by introducing a synthetic dataset and fine-tuning the model, achieving improved accuracy for function calling while balancing performance and coverage.
This paper proposes a causal framework for understanding concept drift in data streams using Structural Causal Models, with a taxonomy and generator for simulating and evaluating drift events in non-stationary environments.
MT-ProtBERT is a multi-task learning model for classifying intrinsically disordered proteins under data scarcity, integrating self-supervised and biochemistry-informed tasks to outperform existing methods like PARROT.