Tag
This paper demonstrates that specialist models trained only on question-answer pairs implicitly select latent reasoning trajectories, and using student distillation as a probe reveals a strong correlation between specialization and generalization profiles, enabling controlled trade-offs between domain precision and general capabilities.
The article discusses the hope that training capable AI models will become democratized so that reliance on big tech companies for open models is reduced.
A paper from Meta shows that byte-level models start behind token models but surpass them with increasing compute, demonstrated with distilled 1B models trained on up to 1 trillion bytes.
Frontier AI companies are experiencing diminishing returns from scaling models and are pivoting to revenue growth through customer-focused strategies.
Sadhika Malladi proposes TailSFT, a lightweight and principled method to improve coverage and enhance post-RL performance, building on previous research that criticized xent SFT for preparing RL.
Anthropic revealed that Chinese AI companies Alibaba, Moonshot AI, and DeepSeek executed extensive distillation attacks on its Claude model to extract chain-of-thought reasoning for training their own models.
Hugging Face is releasing ML Intern in HuggingChat, an AI agent that simplifies model training by handling the entire process from conversation to shareable artifacts on the Hub.
Mark Zuckerberg revealed that Meta is training post-Watermelon models, the next generation of AI models, using the Prometheus compute cluster with 10X more compute.
The paper introduces an on-policy expert-correction pipeline to co-evolve harnesses and models, enabling weaker AI models to improve by correcting failing turns without full imitation, preserving compatibility and enhancing performance on enterprise agent tasks.
This article introduces AC2, an end-to-end platform for training and inferencing open weight models, along with its associated SDKs and tools.
Occamy-1.0 is a cost-efficient open-source AI model for co-work agents, achieving strong performance on complex multi-step tasks and being competitive with larger frontier systems while maintaining broad agentic capabilities.
Marin has grown from one to ten full-time employees since joining Open Athena a year ago. The team is currently in the middle of a large-scale 535B/A23B Mixture of Experts model training run.
Open Athena has launched a 535B/A23B MoE model training run that is proceeding as predicted, highlighting their commitment to open development and significant progress in AI scaling.
A research paper presents a method for training a smaller self-hosted LLM using separate GRPO experts merged via SLERP, which outperforms a larger baseline on enterprise tasks and serves half of platform traffic at lower cost.
Applied Compute launches AC2, a platform enabling AI teams to build, train, and deploy custom models at scale using open models and integrated research tools.
Unbounded Labs introduces Bart, a vintage LLM trained from scratch on pre-1931 English text, with open-sourced datasets, benchmarks, and methodology for advancing vintage LLM research.
The article proposes a novel framework called a 'civilization scaffold' to enhance AI capabilities by preserving and building upon human knowledge without modifying model weights, allowing for immediate and recursive improvements.
Michael Gannotti shares that he has brought DeepSeek's V4 Flash AI model back online on his NVIDIA DGX cluster after completing model training, allowing him to resume local inference for running agents.
Andrew Ng highlights the Marin project as a valuable example of openness in AI model training, with Percy Liang announcing the start of training for Marin 535B-A23B using open code, data, and processes.
CyberFactory is an open-source framework for scaling cybersecurity capabilities by constructing executable tasks from real-world vulnerability artifacts and training AI models with verifiable agent trajectories.