Tag
IQuest Q1 is a new coding model from IQuest_research featuring 320B/15B parameters, a custom license, 512K context, and a hybrid attention mechanism for efficient long context processing.
The article discusses the process of utilizing open training data from Hugging Face to train Marin’s 535B model, which involved 25T tokens from 152 datasets with permissible licenses.
Accelerated Understanding is building AI models that simulate and understand physics in 4D to accelerate invention and discovery, providing directional feedback that traditional experiments lack.
Anima Anandkumar announces the launch of startup Accelerated Understanding, which is training large-scale AI models to simulate and understand physics in 4D, aiming to accelerate invention and discovery by pushing context lengths to trillions.
The paper proposes a compute-efficient two-step hyperparameter transfer framework to predict optimal learning rates for large Mixture-of-Experts models, enabling efficient pretraining without costly hyperparameter sweeps.