Tag
The article introduces ActFlow, a continued pre-training scheme that expands the valid design space for flow and diffusion models, enabling out-of-distribution generative modeling and evolvable search spaces in scientific discovery.
This paper presents an end-to-end pipeline for adapting LLMs to low-resource programming languages, using Pharo as a case study, achieving superior code completion with small models suitable for real-time IDE support.
This paper discovers predictable scaling laws for optimal hyperparameters (learning rate, batch size) in LLM continued pre-training, proposing a two-stage framework that reduces hyperparameter search overhead by up to 90% while maintaining performance.
This paper proposes LayerTracer, an interpretable framework for layer allocation in continued pre-training, demonstrating that freezing deep layers while training shallow ones outperforms full-parameter fine-tuning. It offers a low-cost, actionable strategy for resource-constrained teams optimizing Large Language Models.