Tag
Stanford researchers retested 50 widely cited findings about LLMs using 20 new models and found that 44% no longer held, 34% had significantly weakened, and only 16% still held, suggesting that LLM research findings may have a “shelf life” of only about 15 months.
The author introduces the concept of Epistemically Isolated Artificial Intelligence (EIAI) — AI systems deliberately limited to a defined knowledge environment — highlighting "Talkie: an LM from 1930," a language model trained only on pre-1931 text, as a fascinating example of temporally bounded epistemic reasoning.
FrontierMat introduces a benchmark of 330 validated frontier materials R&D problems for language models, finding the strongest model scores only 53.7% and that a domain-adaptation method lifts an open-weight model to 55.4%, matching proprietary leaders.
DiRAFT synthesizes financial transcripts by sampling concept-graph plans from aggregate discourse priors rather than individual source passages, and empirically evaluates residual confidentiality risks on two financial corpora, finding stochastic sampling best balances transition fidelity and privacy.
提出了异步流映射语言模型（aFMLM），通过token级时间调度将预测顺序编码进单步生成的隐藏计算中，在Sudoku和GSM8K代码生成任务上刷新了一步生成的最先进性能。
TurnTurk-GPT is a text-based turn-taking predictor for Turkish, created by fine-tuning a Turkish GPT-2 model with the TurnGPT objective, which outperforms multilingual end-of-turn detectors and locates Turkish turn-final cues in verb-final morphology.
This paper introduces bidirectional KV-cache patching as a method to trace how fine-tuned behaviors are instantiated in language models, showing that behaviors like responding in German or emergent misalignment can be transferred between a LoRA fine-tuned model and its base model via KV cache reuse, with effects concentrating on pivotal tokens such as system-prompt and chat-template positions.
HyperRepair is a framework that recovers the general capabilities of fine-tuned language models while retaining domain specialization at inference cost only, using a hypernetwork that generates a Repairing Side Net from activation fingerprints. It is evaluated across two model families, 12 LoRA adaptation datasets, and 15 general benchmarks, achieving 93.4% of base models' general score while keeping 99.5% of fine-tuned domain performance.
This arXiv paper finds that language models exhibit a 'Women-Are-Wonderful' bias in blame attribution, showing both leniency toward women being judged and substantially harsher judgments when conduct affects a woman, based on 11,270 counterfactual conflicts from r/AmItheAsshole across four models.
该论文研究了RAG系统中的从众偏差：当检索到的多数文档是互相一致的投毒文档时，Gemma-3-12b检测中毒声明的能力从25%显著下降至10%，表明语言模型会随附多数证据而忽视事实核查。
This paper shows that zeroing out 729 task-redundant attention heads in GPT-2-XL at inference time reduces verbatim memorisation by 91.5% while slightly improving WikiText-103 perplexity, offering a training-free way to mitigate privacy leakage from LLMs.
Meta Superintelligence Labs researchers formalize 'adaptive personalization under epistemic uncertainty' and introduce a multilingual benchmark revealing a recognition–generation gap in LLMs, where models default to exploit-first behavior despite appropriate strategy identification; targeted training on internal strategy representations narrows the generalization gap.
该论文发现RL后训练中fp32主权重经bf16四舍五入导出时，大多数小于半个bf16间隔的策略更新被静默丢弃，使采样器落后优化器50-150步；作者提出仅需一行代码的Fair Cast（随机舍入/误差反馈），在Qwen3-0.6B到4B的25组配对实验中平均提升6.1分。
The paper introduces EDB-Bio, a 96k-episode benchmark for scientific evidence deliberation across three biological domains, showing that frontier and open-weight models are systematically underconfidence-prone (Brier scores near a trivial base-rate forecaster) and that post-training with proper scoring rules improves calibration, yielding an Ibis0 model 2.5× more efficient than the best frontier model in sequential experiment selection.
Dejan.ai replicates a NeurIPS 2025 study showing that training a tiny fraction of randomly chosen weights can unlock latent skills: Gemma 3 270M went from ~0% to 97% on number sorting after training just 1,000 of its 100M weights, with similar gains for word reversal and clean JSON output.
The paper introduces Seq@K, an evaluation protocol measuring whether a language model can solve a problem within K sequential attempts given feedback after each failure, showing it reveals a materially different model-selection landscape than Pass@K across 7 benchmarks and 16 frontier models.
ThinkJev is a training-free method that builds fast (System 1) and slow (System 2) answering paths from a single hybrid-thinking open model, reading calibrated option probabilities directly or after variable-length thought, so it only pays for thinking where it statistically guarantees a target error rate.
Kyutai Labs 重新审视了 Neural Thickets 方法，指出在输出 logits 上加入随机偏置、其余权重保持不变，即可匹配甚至超越对模型所有权重加噪并保留最佳副本的效果。
OpenEBFM introduces Energy-Based Foundation Models that separate generation into energy steering and token generation, showing better data, parameter, and compute scaling than matched GPT baselines at up to 1B parameters, with full code, checkpoints, and an EBFM chatbot released as open source.
This paper studies auxiliary trajectory preferences (confidence-based and entropy-based metrics) to improve small language model self-improvement and transfer, showing UID-Conj. gains of about 2 points over unmodified TTRL and uniform-weighted SFT on AIME tasks.