Tag
LoRA (low-rank adaptation) is the most popular parameter-efficient fine-tuning method for LLMs. This video introduces how LoRA and its variants (LoRA+, QLoRA, VeRA, DoRA) work.
This article delves into the principles of LoRA and its variants (QLoRA, VeRA, DoRA), explaining how low-rank decomposition reduces trainable parameters to enable efficient fine-tuning of large models.
This paper introduces DoRA-RBAC, a framework for composing LLM adapters, and tests whether geometry-aware merging improves multi-domain performance. Results show no consistent advantage over standard averaging, suggesting adapter interference is not primarily driven by parameter-space geometry.
The article discusses deploying AI agents in finance while ensuring compliance with NIS2/DORA regulations, focusing on transparency, guardrails, and accountability for potential data breaches.
This paper benchmarks sub-1B models on mathematical reasoning tasks, revealing that full fine-tuning actively harms performance in models under 300M parameters, while parameter-efficient fine-tuning (PEFT) like LoRA and DoRA provides stability. The authors recommend defaulting to PEFT for all aligned sub-1B models and caution against full FT for architectures smaller than 500M to prevent catastrophic forgetting.
A Reddit post maps major EU regulations affecting AI deployments (AI Act, DORA, NIS2, Data Act, CRA, EHDS) into a single timeline and asks which deadline enterprises are most unprepared for.
A complete guide to fine-tuning NVIDIA's Cosmos Predict 2.5 world model using LoRA/DoRA for generating robot manipulation videos, covering dataset preparation, training, inference, and evaluation.