Tag
An announcement for a talk at PyTorch Conference North America where Ziming will discuss using OpGuard for bitwise debugging in LLM training.
SLCA-GRPO introduces Segment-Locked Credit Assignment to improve reinforcement learning for tool-calling agents by decoupling advantage estimation and using hierarchical rewards, leading to faster convergence and higher accuracy.
The paper proposes a graph-based framework (GRAFT) to estimate step-level advantages in agentic reinforcement learning for large language models, addressing biases in group-based methods like GRPO and showing gains on multi-turn benchmarks.
The article describes a project with experiments on training a local LLM using continued pretraining (CPT) and RAG for domain-specific knowledge, featuring comprehensive evaluations and findings.
Stanford University's CS336 course 'Building Language Models from Scratch' has released 18 complete lecture videos, open-source assignments, and course materials, comprehensively covering the entire workflow of large language model engineering, suitable for self-learners to systematically study LLM development.
This paper proposes CC-OPD, a novel on-policy distillation method for multi-constraint instruction following that uses counterfactual ablations to enhance training signals, achieving superior performance where a 1.5B model surpasses its 7B teacher on benchmarks.
The author theorizes that AI companies are spreading negativity to contaminate the positive online data sources that competitors rely on, resulting in a predominantly negative sentiment in AI models.
The article discusses how automated reinforcement learning drives rapid AI advancements in chess and math, but warns of potential dangers in biology, calling for urgent action to address safety concerns.
A step-by-step repository guiding users to build and train a transformer model from scratch using PyTorch, with comprehensive coverage of data processing, training, and post-training techniques like SFT and RLHF.
This paper proposes a dependency-aware trajectory refinement method for efficient multi-turn agent fine-tuning, improving accuracy and reducing inference costs.
The article speculates that AI companies may be nearing the end of significant LLM advancements, with rising training costs and diminishing returns potentially leading to slower progress to maintain hype.
An expert with experience in training frontier LLMs and engineering viruses argues against the doomer scenario of an AI-created supervirus.
ToolGrad introduces an efficient method for generating tool-use datasets using textual gradients, enabling better LLM training with lower cost and improved performance on out-of-distribution tasks.
The article details the blog author's measures to block old browsers to reduce load from malicious crawlers, likely for LLM training, and addresses specific issues with feed readers like Inoreader and Feedly, as well as browsers like Vivaldi and archive services.
Hugo Vergnes trained a 3.8B parameter LLM to achieve a 0.384 CORE score for $998, showcasing cost-effective model training methods accessible outside major research labs.
DE-Venus is a unified framework for data-efficient Reinforcement Learning with Verifiable Rewards in large language models, reducing annotation and training costs while maintaining performance.
This paper proposes Evidence Sufficiency Boundary Training to enhance selective answering in grounded multi-hop QA systems by learning when to abstain from answering due to insufficient evidence, leading to improved boundary localization and reduced unsupported-answer rates on benchmarks like HotpotQA and MuSiQue.
The paper introduces CREST, a method to improve multi-turn agent training by assigning separate credits to each turn and using a self-teacher to focus learning without overriding rewards, showing performance gains on benchmarks.
TRANSIT is a runtime that makes unified virtual memory practical for large-scale LLM training, reducing GPU usage by up to 50% without code changes, and will be presented at PyTorch Conference North America.
The tweet recommends two resources for learning practical RL post-training techniques, focusing on GRPO++ tricks to enhance reinforcement learning in large language models.