memorization

TRACES is a new AI benchmark by Apodex designed to measure discoverative AI by focusing on the problem-solving process rather than memorized answers, challenging traditional benchmarks.

This paper studies the LSR-Synth benchmark and finds that a fixed vocabulary covers most tasks, while language-model-generated candidates rarely expand the set of solvable instances except when vocabulary coverage is disrupted.

This paper evaluates LLMs on understanding and generating novel Chinese xiehouyu riddles, finding that while models like Gemini 3.1 Pro excel at comprehension due to memorization, they fall short in creative generation compared to humans, highlighting issues of data contamination and limits of reasoning.

Proposes a unified post-hoc detection framework for copyright infringement in AI models, using conditional sensitivity and differential privacy to measure memorization across modalities.

This paper introduces OrthoGrad, a geometric intervention that removes the radial component of weight gradients during optimization, and shows that it reduces memorization of noisy labels in small-data regimes but does not prevent eventual memorization.

This paper introduces analytically tractable Bayesian information restricted diffusion (BIRD) models to study the memorization-generalization phase transition in diffusion models, finding that generation proceeds near the edge of memorization and that information restriction helps circumvent the curse of dimensionality.

This paper investigates the extent to which language models memorize training data, exploring implications for privacy and security.

This paper proposes Native Factorized Weights for transformers, where every linear layer is trained as a product of two low-rank matrices from initialization. Experiments show a corpus-determined optimal rank that minimizes validation loss and a generalization band, outperforming dense baselines with fewer parameters.

This paper investigates parametric memorization in tabular foundation models that use in-context learning, introducing a probing framework (IclMem) to separate context-based predictions from memorization. It finds moderate memorization signals under specific conditions but notes they largely vanish under realistic training scenarios.

This paper uses statistical mechanics to explain the relationship between machine learning and memorization.

This paper systematically studies the damage caused by exact document repetition during language model pretraining, showing that repeating a moderately sized subset a moderate number of times maximally harms performance, and that repetition can waste up to 33% of compute (as measured by compute-equivalent loss).

Cyclic denoising is introduced as a novel extraction attack that reveals ultrastable memorized training images in diffusion models by repeatedly noising and denoising samples. The technique requires no gradients or weight inspection and has implications for privacy auditing.

Presents output vector editing, a constrained-optimization weight edit to mitigate memorization in LLMs by modifying MLP neuron output vectors instead of zeroing activations, achieving up to 87.9% suppression with minimal locality failures.

This paper introduces DiRL, a direction-aware reinforcement learning framework that distinguishes reasoning-driven diversity from memorization-driven diversity in LLM exploration. It extracts an internal reasoning-memorization direction from model representations and shapes rewards to prioritize reasoning-aligned exploration, showing improvements on math and general reasoning benchmarks.

This paper introduces the concept of deployment-time memorization in foundation-model agents, analyzing how memory design choices (summarization aggressiveness, retrieval breadth, deletion mode) affect personalization utility, extraction risk, and deletion fidelity, proposing new metrics such as Personalization Recall, Adversarial Extraction Rate, and Forgetting Residue Score.

PropMe is a propensity-aware framework for evaluating LLM memorization, distinguishing between forced reproduction capabilities and natural propensity using SimpleTrace for deterministic attribution across open models and datasets.

The paper proposes a method to mitigate spurious correlations by disentangling learning dynamics of core and spurious features using a two-stage sample scoring function, achieving state-of-the-art debiasing performance with only 10% of training data.

This paper investigates memorization in diffusion models and finds that they preferentially memorize prototypical examples with common substrings, even after deduplication, and that early stopping leads to an overproduction of common motifs, dubbed 'slop'.

This paper introduces NumLeak, a framework for detecting when foundation models memorize public numeric benchmarks from pretraining rather than demonstrating out-of-sample skill, and shows that top LLMs recall values like Fama-French returns with high fidelity, proposing a simple system-prompt defense.

This paper introduces infilling extraction, a new method for extracting training data from diffusion language models by using arbitrary binary masks, showing that such models are more vulnerable to memorization attacks than previously thought.