Tag
FAIR at Meta 提出 Routed Continual Learning:将过往轨迹按簇划分、每个簇搭配固定记忆库中的一种记忆系统,并由 LLM 路由器决定复用或新建簇,在交错任务流上相对全局 ICL 历史提升 15–30% 归一化奖励并降低 65–72% API 成本。
Meta FAIR 团队正在招聘合同制 Research Engineer,研究方向包括自动化 AI 研递的智能体、递归自我改进、测试时学习算法及对齐问题,并列举了 Self-Improving Pretraining 等多项相关论文成果。
Meta FAIR's latest paper proposes the Autodata method, which uses an intelligent data scientist Agent to autonomously generate and optimize high-quality data, enabling a 4B small model to defeat a 397B large model on legal reasoning tasks. This indicates that data quality can bridge the gap in parameter count, providing new insights for data pipelines and scaling.
A researcher announces their departure from Meta FAIR after two years working on LLM reasoning, reflecting on their experience leading a team.
Introduces Self-Pruned Key-Value Attention (SP-KV), a mechanism that learns to predict future utility of key-value pairs to dynamically prune the KV cache, reducing memory usage and decoding speed by 3-10x with minimal performance degradation. The model and utility predictor are trained end-to-end using next-token prediction.
DAIR AI's weekly roundup highlights top research papers including HeavySkill, which improves model performance via internalized parallel reasoning, and Sakana AI's Conductor, which uses RL to optimize agent orchestration. It also covers Meta FAIR's work on self-improving pretraining.
Deployment of Meta's DINOv3 ViT-L/16 distilled model on Replicate, providing efficient visual embeddings for tasks like similarity search and retrieval.