weight-updates

Tag

Cards List
#weight-updates

Rank Measures of Fine-Tuning Updates Need a Noise Baseline

arXiv cs.LG ↗ · 2026-10-19 Cached

该论文指出,用秩来衡量微更新ΔW时必须加入噪声基线:方向计数类指标显示微更新仅略低于噪声,而稳定秩则显示远低于噪声,二者共同揭示更新具有少数强方向加长尾的结构。研究还发现指令数据、DPO偏好标签对更新并无显著特殊性,并质疑了'后训练是表面性的'相关论断。

0 favorites 0 likes
#weight-updates

Eight continual-learning tools compared by the layer they operate on, and the column the comparison left out

Reddit r/AI_Agents ↗ · 2026-10-15

作者对比了八种 agent 持续学习工具按其作用层(记忆、提示、技能、权重)的分类,并补充了一个关键缺失维度:权重更新是否能在不停止服务的情况下到达推理引擎。文章重点介绍了能在推理持续进行时热更新权重的 reef 项目。

0 favorites 0 likes
#weight-updates

GRRR: The Geometry of Reshaping, Rotation, and Routing in Decoder LLM post-training

arXiv cs.LG ↗ · 2026-09-22 Cached

This paper analyzes post-training weight updates in LLMs using singular value decomposition, identifying geometric components that drive performance gains, with insights suggesting that reshaping singular values is less critical than rotating and routing changes.

0 favorites 0 likes
#weight-updates

I just solved continual learning

Reddit r/artificial ↗ · 2026-09-16

The article proposes a method for continual learning in AI where the model autonomously decides what to learn and updates its weights in real-time, potentially enabling continuous self-evolution towards AGI.

0 favorites 0 likes
#weight-updates

Can a Language Model Learn Facts Continually in Its Weights?

arXiv cs.CL ↗ · 2026-07-14 Cached

This paper investigates whether language models can learn new facts in their weights through continual learning. Using invented facts and sequential writes into Qwen3 models, it finds that training data breadth determines knowledge type and retention: bare-statement facts are quickly forgotten (1% accuracy after 20 writes), while facts learned from diverse restatements retain 46% accuracy. Forgotten facts are not erased but become behaviorally inaccessible due to later writes redirecting questions, and context remains the reliable channel for fact composition and survival.

0 favorites 0 likes
#weight-updates

@rohanpaul_ai: This paper shows an AI improving itself better when it rewrites its setup and updates its model. The problem is that mo…

X AI KOLs Following ↗ · 2026-06-11 Cached

This paper introduces SIA, a self-improving AI loop that combines scaffold rewriting and weight updates (via LoRA) to enhance task performance. Tested on three diverse tasks, it outperforms setups using only scaffold improvements.

0 favorites 0 likes
#weight-updates

Context Is Software, Weights Are Hardware

Hacker News Top ↗ · 2026-04-19 Cached

Aravind Jayendran argues that while longer context windows improve LLM performance, they cannot fully replace weight updates, framing context as transient software and weights as hardware that fundamentally alters model capabilities.

0 favorites 0 likes
← Back to home

Submit Feedback