parameter-efficient

Tag

Cards List
#parameter-efficient

GRASP: Gradient-Aligned Sequential Parameter Transfer for Memory-Efficient Multi-Source Learning

arXiv cs.LG ↗ · 2026-06-16 Cached

GRASP proposes a method for multi-source transfer learning that sequentially merges source models into a single target model with constant O(1) memory usage, using gradient-based parameter alignment to avoid negative transfer. Experiments show it outperforms ensemble methods while being much more memory-efficient.

0 favorites 0 likes
#parameter-efficient

The Hidden Power of Scaling Factor in LoRA Optimization

arXiv cs.AI ↗ · 2026-06-12 Cached

This paper reveals that the scaling factor α in LoRA optimization is more influential than the learning rate, and proposes LoRA-α, a framework that improves performance and simplifies hyperparameter search by restoring α to its principled regime.

0 favorites 0 likes
#parameter-efficient

Fine-tuning Multi-modal LLMs with ART: Art-based Reinforcement Training

Hugging Face Daily Papers ↗ · 2026-06-10 Cached

ART (Art-based Reinforcement Training) enables parameter-efficient fine-tuning of frozen multimodal LLMs by optimizing raw visual input via gradient backpropagation, achieving performance comparable to LoRA while supporting pre-compiled computational graphs for high-throughput engines like vLLM.

0 favorites 0 likes
#parameter-efficient

The Order Matters: Sequential Fine-Tuning of LLaMA for Coherent Automated Essay Scoring

arXiv cs.CL ↗ · 2026-06-10 Cached

This paper investigates sequential fine-tuning of LLaMA-3.1-8B for automated essay scoring using a curriculum aligned with discourse structure, showing improved coherence and performance compared to independent or randomized training.

0 favorites 0 likes
#parameter-efficient

@FinanceYF5: Aizpurua team at Multiverse Computing, Spain, proposes using small quantum circuits to expand pretrained large models: instead of stacking parameters, they compress complex mathematical relationships into quantum circuits. Adding only about 6,000 parameters to Llama 3.1 8B (less than 0.01% of the original model) reduces perplexity by 1.4%.

X AI KOLs Following ↗ · 2026-06-08 Cached

The Aizpurua team at Multiverse Computing, Spain, proposes expanding pretrained large models with small quantum circuits. Adding just about 6,000 parameters to Llama 3.1 8B reduces perplexity by 1.4%, demonstrating the feasibility of quantum-circuit-assisted large model scaling.

0 favorites 0 likes
#parameter-efficient

Parameter-Efficient Fine-Tuning with Learnable Rank

arXiv cs.CL ↗ · 2026-06-04 Cached

Researchers from Adelaide University introduce LR-LoRA (Learnable Rank LoRA), a parameter-efficient fine-tuning method that dynamically learns the adapter rank for each transformer layer during training rather than using a fixed global rank. LR-LoRA achieves state-of-the-art performance on language understanding and commonsense reasoning benchmarks, outperforming fixed-rank LoRA baselines.

0 favorites 0 likes
#parameter-efficient

LoomVideo: Unifying Multimodal Inputs into Video Generation and Editing

Hugging Face Daily Papers ↗ · 2026-06-04 Cached

LoomVideo introduces a 5B-parameter unified architecture for video generation and editing that reduces computational overhead using novel conditioning mechanisms and multi-modal alignment, achieving competitive performance and faster inference.

0 favorites 0 likes
#parameter-efficient

SALSA: Speech Aware LLM Adaptation via Learned Steering Activation Vectors

arXiv cs.CL ↗ · 2026-06-02 Cached

SALSA introduces a lightweight adaptation method for speech-aware LLMs that learns layer-wise steering vectors via supervised objective, achieving significant improvements (up to 46.8% relative) on out-of-domain speech benchmarks, and shows that steering the encoder layers is more effective than modifying the LLM backbone.

0 favorites 0 likes
#parameter-efficient

CSULoRA: Closest Safe Update Low-Rank Adaptation

arXiv cs.LG ↗ · 2026-06-01 Cached

CSULoRA is a post-hoc method for correcting trained LoRA adapters to preserve safety alignment while maintaining utility, using closest safe update estimation.

0 favorites 0 likes
#parameter-efficient

Tiny but Trusted: Efficient Vision-Language Reasoning for Time-Series Anomaly Detection

Hugging Face Daily Papers ↗ · 2026-05-28 Cached

This paper introduces VisAnomReasoner, a parameter-efficient vision-language model fine-tuned on a novel benchmark (VisAnomBench) with natural-language rationales, achieving over 21pp improvement in precision and F1 for time-series anomaly detection and strong cross-benchmark generalization.

0 favorites 0 likes
#parameter-efficient

Signs Beat Floats: Low-Rank Double-Binary Adaptation for On-Device Fine-Tuning

arXiv cs.LG ↗ · 2026-05-26 Cached

LoRDBA replaces LoRA's floating-point low-rank factors with binary sign carriers and channel-wise scales, enabling efficient on-device fine-tuning with significant footprint reduction and minimal latency overhead, matching fp16 quality.

0 favorites 0 likes
#parameter-efficient

Parameter Efficient Multi-Class Intelligent Scheduling for Multimodal Online Distributed Industrial Anomaly Detection

arXiv cs.LG ↗ · 2026-05-26 Cached

This paper proposes MODIAD, a framework for multimodal online distributed industrial anomaly detection, addressing resource constraints with a Multi-class Intelligent Scheduling problem and a Resource Efficient Class-Wise Low Rank Adaptation (REC-LoRA) strategy. Experiments on MVTec 3D-AD and Eyecandies datasets demonstrate superior performance and efficiency.

0 favorites 0 likes
#parameter-efficient

FuRA: Full-Rank Parameter-Efficient Fine-Tuning with Spectral Preconditioning

arXiv cs.LG ↗ · 2026-05-25 Cached

FuRA introduces a full-rank parameter-efficient fine-tuning method using spectral preconditioning via block tensor-train decomposition, achieving higher accuracy than full fine-tuning with LoRA-level efficiency. It outperforms LoRA and full FT on LLM and VLM tasks.

0 favorites 0 likes
#parameter-efficient

From Parameters to Data: A Task-Parameter-Guided Fine-Tuning Pipeline for Efficient LLM Alignment

arXiv cs.LG ↗ · 2026-05-22 Cached

P2D is a unified framework that leverages task-sensitive attention heads for both data selection and structural pruning, achieving an 8.3 pp performance gain and 7.0× speedup by updating only 10% of heads on 10% of data.

0 favorites 0 likes
#parameter-efficient

@techNmak: Everyone is fine-tuning LLMs. Almost nobody understands what is actually being updated inside the model. Here are 5 tec…

X AI KOLs Timeline ↗ · 2026-05-21

Explains five parameter-efficient fine-tuning techniques: LoRA, LoRA-FA, VeRA, Delta-LoRA, and LoRA+, detailing how each modifies model weights during adaptation.

0 favorites 0 likes
#parameter-efficient

HELLoRA: Hot Experts Layer-Level Low-Rank Adaptation for Mixture-of-Experts Models

arXiv cs.LG ↗ · 2026-05-20 Cached

HELLoRA introduces activation-aware adapter placement for MoE models, attaching LoRA only to hot experts to reduce parameters and FLOPs while improving performance on reasoning, code, and safety tasks.

0 favorites 0 likes
#parameter-efficient

Learning Faster with Better Tokens: Parameter-Efficient Vocabulary Adaptation for Specialized Text Summarization

arXiv cs.CL ↗ · 2026-05-19 Cached

This paper proposes a parameter-efficient vocabulary adaptation method for LLM-based text summarization in specialized domains, augmenting pretrained tokenizers with domain-specific tokens and selectively replacing under-trained ones to reduce training time by 35-55% and parameter counts by up to 37%.

0 favorites 0 likes
#parameter-efficient

@_vmlops: FINE-TUNING A 12B MODEL ON A SINGLE GPU IS REAL NOW most people think you need a massive gpu cluster to fine-tune large…

X AI KOLs Timeline ↗ · 2026-05-17 Cached

Hugging Face's PEFT library enables parameter-efficient fine-tuning of large models on a single GPU, reducing compute and storage costs while maintaining performance.

0 favorites 0 likes
#parameter-efficient

PEML: Parameter-efficient Multi-Task Learning with Optimized Continuous Prompts

arXiv cs.CL ↗ · 2026-05-15 Cached

PEML proposes a parameter-efficient multi-task learning method that co-optimizes continuous prompts and model weights via low-rank adaptation. It achieves up to 6.67% average accuracy improvement on multiple benchmarks.

0 favorites 0 likes
#parameter-efficient

Parameter-Efficient Multi-View Proficiency Estimation: From Discriminative Classification to Generative Feedback

Hugging Face Daily Papers ↗ · 2026-05-05 Cached

This paper introduces three parameter-efficient methods for multi-view proficiency estimation on the Ego-Exo4D dataset, shifting from discriminative classification to generative feedback. The proposed models achieve state-of-the-art accuracy with significantly fewer parameters and training epochs than video-transformer baselines.

0 favorites 0 likes
← Previous
Next →
← Back to home

Submit Feedback