lora

This paper introduces an adaptive anisotropic learning-rate model for LoRA fine-tuning to address within-module heterogeneity, improving performance and rank capacity utilization across benchmarks.

This article describes a method to train LoRA adapters using AsyncGRPOTrainer and sync them via Storage Buckets across separate Hugging Face Jobs, eliminating the need for NCCL communication.

X-AuT is a progressive framework for compressing audio-encoder layers in speech large language models, reducing inference cost while restoring accuracy via techniques like cross-scale distillation and LoRA adaptation.

A user trained a LoRA model to automate the creation of style-consistent 'bufos' in Slack, opening up inference for public use with a dedicated website.

This repository provides LoRAs for the MiniMax H3 model, designed to run in ComfyUI for video enhancement, such as sharpening videos while maintaining photorealism.

This paper diagnoses intra-adapter contention in MoE+LoRA fine-tuning and introduces SpawnLoRA to dynamically add sub-adapters, reducing negative transfer across domains.

A developer is experimenting with building an AI assistant called Arcon using a 4B local model with LoRA, incorporating persistent memory and personality features, and seeks community feedback.

This paper benchmarks federated LoRA adaptation of BiomedCLIP for chest X-ray classification across four international cohorts, demonstrating improved performance over unadapted models and approaching centralized training results.

TalkFa is a unified benchmark for Farsi dialogue generation and understanding, consisting of three datasets validated through experiments with LLMs and human evaluation.

This paper introduces SpeakPay and a Nepali financial speech dataset, showing that LoRA fine-tuning of Whisper reduces Word Error Rate by 67.2% and improves transaction success rates for low-resource language accessibility.

FedRoRA is a novel framework for personalized federated LoRA fine-tuning that addresses rank heterogeneity and data heterogeneity in federated learning by decoupling adaptation into shared global directions and personalized magnitudes.

This paper introduces RW-LoRA, a decentralized LoRA fine-tuning method using random walks to reduce communication and computation costs while achieving competitive performance on NLP tasks.

RustEAI is a local AI-powered Rust coder for Apple Silicon Macs, based on a LoRA fine-tune of Qwen2.5-Coder-0.5B, providing one-shot code generation without cloud dependency.

Normalized Low-Rank Adaptation (NoRA) stabilizes LoRA training by normalizing down-projection matrices, accelerating convergence and improving performance without extra parameters or inference cost.

This paper investigates failures in a 2B model for dialogue games and introduces a diagnosis-guided post-training recipe using SFT, DPO, and LoRA to boost performance while maintaining general capabilities.

This article explains five parameter-efficient fine-tuning techniques for large language models, such as LoRA and VeRA, detailing how each method adapts model weights with minimal updates.

Alibaba PAI releases LoRA checkpoints for accelerating MiniMax-H3 video generation using Parallel Decoding Distillation, enabling efficient inference in 8 steps.

FCPRAG proposes a fusion-controller framework for parametric retrieval-augmented generation that improves stability and performance in multi-passage LoRA injection, showing consistent gains over baselines in experiments.

The paper evaluates Large Language Models for hate speech detection in Roman Urdu, a low-resource language, demonstrating that Parameter-Efficient Fine-Tuning with LoRA significantly improves classification performance compared to zero-shot inference.

sbgrid is a Krea 2 Turbo LoRA that creates an 8-panel storyboard as a single image from one prompt, hosted as a Hugging Face Space.