Tag
A research paper analyzing what iterated self-feeding probes of language models actually measure, distinguishing model-dependent signals from construction artifacts using a ring of resampled token cells and common random numbers coupling.
This paper introduces a lightweight, offline-trained logit correction method for grammar constrained decoding that leverages internal parser and lexer states to restore the LM's true probability distribution without expensive online sampling, improving output quality while maintaining low inference latency.
Introduces CurveFP, a closed-product codebook family that distributes quantized magnitudes across interleaved logarithmic curves, enabling exact sign XOR and integer-index updates. It achieves FP8-class behavior with seven-bit inference and improves perplexity on 7B-9B models.
RLMOpt is a prompt optimizer that uses a recursive language model to drive the search policy itself, outperforming existing methods like GEPA across multiple benchmarks while using fewer rollouts and producing shorter prompts.
This paper proposes an adaptive supervised anchoring framework for on-policy self-distillation, addressing the problem of rollout-conditioned signal degradation in language model training. The method separates rollout-conditioned distribution matching from canonical-context supervision, improving task acquisition while preserving general capabilities.
This paper compares human and LLM scalar judgments for sentences with focus particles 'even' and 'only' across different response scale configurations, finding stable semantic-driven differences but noting the model's lack of response variability.
This paper uses sparse autoencoders to decompose how language models represent the default Assistant, roleplay personas, and story characters, finding that personas retain an Assistant core while differentiating across layers, and story characters lack that core.
This paper proposes amortizing the high token cost of reasoning-mode LLMs by distilling domain-specific skills from existing trajectories into system prompts, recovering most of the reasoning gap on agentic benchmarks while emitting far fewer tokens.
This paper introduces the workbook time machine, a pipeline that automatically creates benchmarks for evaluating language models on creating derived spreadsheet objects like formulas, charts, pivot tables, and conditional formatting. The authors produce WTM-Corpus and a curated 150-task benchmark WTM-Bench, then evaluate spreadsheet agents across artifact types, step complexity, and instruction granularity.
This paper investigates the 'knowing-saying gap' in language models, showing that linear probes can detect corrupted context with near-perfect accuracy yet fail to predict final answer errors, with implications for deployment monitoring and intervention strategies.
This paper investigates why language models fail at two-hop generalization, showing that models succeed when the second hop follows training distribution but fail when it deviates, and proposes a recurrent-style training strategy to improve out-of-distribution two-hop reasoning.
Introduces Ekphrasis, a 400-task benchmark for measuring visual creative ideation in text-only LLMs, separating usefulness, expressiveness, and novelty. The paper validates it with cross-modal grounding, showing text-level visual ideation ordering survives rendering.
Ask-E is a new benchmark and training environment that evaluates and trains models on generating questions calibrated to specific skill levels, defined by the capabilities of two existing language models. Frontier models score below 50% on calibration, and training on Ask-E improves downstream math benchmarks without new math data or correctness-based rewards.
This paper introduces the Cross-Lingual Comprehension Gap (CLCG) metric to measure how LLM response quality degrades when content is presented in non-English languages. Across 18 languages and multiple models, it finds a significant performance drop, especially for low-resource languages, questioning the assumption of English-centric capability transfer.
A user shares an observation that Qwen and Gemma tokenize code very differently, with Qwen using far fewer tokens for the same HTML/JS input, which may explain differences in coding and language performance. They also note a potential retraining project by LiquidAI using a more efficient tokenizer.
This paper introduces HiLP, a hierarchical representation training method that adds multi-scale self-predictive learning to transformer pretraining, aiming to reduce compounding error and improve long-horizon reasoning and speculative decoding efficiency.
This paper introduces KV-Skill, a design space of external factorized operators that frozen language models read through a lightweight interface, enabling task knowledge to be acquired from text or rewards and deployed independently. Experiments across ten benchmarks show consistent improvements over text skills, prefix tuning, and LoRA, with composable loadable skills.
A systematic empirical study showing that concept directions extracted from one language model can steer other independently trained models when sufficient scale (≥1.7B parameters) is reached, providing functional evidence for the Platonic Representation Hypothesis and highlighting scale thresholds for cross-model interpretability tools.
This paper investigates subliminal learning in language models, showing that biases can transfer from teacher to student via seemingly random synthetic data. The authors find that adding Gaussian noise to weights increases transfer, and that students inherit not just the semantic bias but also the type of intervention used, with implications for training safety and data auditing.
The paper introduces the Ignition Index, a metric for measuring global workspace dynamics in language models, validated across multiple architectures and tasks, showing selective detection of ignition-like representational transitions.