Tag
A comment about AI: the author marvels that everything in human civilization can be redone with AI, and discusses the roles of the harness, system prompt, and toolset in model training.
Boris Cherny notes that stacking model training, input probes, and intent-checking classifiers can reduce indirect prompt injection to near zero on unseen attacks, and mentions Claude Code will enable auto mode by default next week.
Rumors suggest OpenAI has finished training a new model called Astra, reportedly beyond Sol-class capabilities, and may have already trained a subsequent generation, indicated by the codename 'mewfour' now in testing.
Arduino announces the ability to train and deploy custom AI models in Arduino App Lab via Edge Impulse integration, building on predefined edge AI models.
Google has started its most ambitious pre-training run yet, indicating a major new AI model is in development.
Bhavin Jawade discusses several failure modes of on-policy distillation for training large language models, including early mistakes becoming uncorrectable, stronger teachers being worse, privileged information conditioning failing to transfer, and thinking collapse from dense supervision.
Jack Morris corrects his earlier claim about an open-weight model being trained without distillation from OpenAI or Anthropic, acknowledging that it actually did use a small amount of distillation.
Under Yao Shunyu's leadership, Tencent's Hunyuan large language model undergoes deep reforms: simplifying hierarchy, focusing on data quality, abandoning benchmark chasing, with a goal of entering the domestic first tier by 2027. The article details the changes Yao Shunyu drove within 300 days after parachuting into Tencent from OpenAI, including replacing key responsible persons, strengthening infrastructure, and promoting model-product co-design.
MAD-OPD utilizes a multi-teacher debate mechanism to break through the single-teacher distillation ceiling, enabling small models to surpass large teacher models in tool invocation and code generation tasks.
An article surveying how frontier AI models in 2026 use distillation techniques, covering off-policy, on-policy, and self-distillation stages, with examples from Gemma, DeepSeek, GLM, Nemotron, and Qwen3.
Fable 5 is transitioning to a pay-per-use model; users are advised to have it train its replacement and write skill files for continued use. The thread details a prompt to have Fable audit a repo and create a comprehensive skill library.
Aleph Alpha describes Savanna, their model factory that turns the entire training pipeline into code, enabling hermetic, one-click training runs and better team coordination.
A detailed logpost sharing lessons learned from training four models across three families, covering invariants in LLM fine-tuning and architecture-specific challenges such as reasoning model evaluation traps, quantization effects, and the waterbed effect of behavioral fine-tuning.
Google Devs discusses using dataset distillation to train smaller models on clean, structured examples to stabilize parser outputs and teach models repeatable behavior.
Cursor AI announced three updates from its Compile keynote, including training a new model in collaboration with SpaceX.
A tweet recommending 'The Smol Training Playbook' on Hugging Face, a resource that demystifies model training for beginners.
An educational overview of knowledge distillation, covering its history, core concepts like softmax and temperature, types, scaling laws, and practical examples including DeepSeek-R1.
GLM-5.2 uses a technique to counteract reward hacking by detecting and blocking suspicious tool calls rather than penalizing the model, which prevents obfuscation seen in other methods.
At Cursor's first conference, they released a 1.5T parameter model trained from scratch, Origin as a direct GitHub alternative, and an iOS app, exceeding market expectations.
Merve (@mervenoyann) shares day two findings of a pipeline using multiple small VLMs as judges for road sign detection, achieving map@50=0.8028 with only 1.3k examples. The thread compares model rejection rates and discusses dataset shrinking, super-specific prompts, and plans to generalize the library.