@_akhaliq: Code2LoRA Hypernetwork-Generated Adapters for Code Language Models under Software Evolution
Summary
This paper introduces Code2LoRA, a hypernetwork-based method to generate adapters for code language models, addressing challenges under software evolution.
View Cached Full Text
Cached at: 06/05/26, 05:19 PM
Code2LoRA
Hypernetwork-Generated Adapters for Code Language Models under Software Evolution https://t.co/e4vq1C83mY
Similar Articles
Code2LoRA: Hypernetwork-Generated Adapters for Code Language Models under Software Evolution
Code2LoRA introduces a hypernetwork that generates LoRA adapters from a repository in a single forward pass, allowing frozen code LLMs to adapt to repository context without extra tokens, and supporting evolving codebases efficiently. It also delivers RepoPeftBench, a benchmark for repo-conditioned code modeling.
JumpLoRA: Sparse Adapters for Continual Learning in Large Language Models
JumpLoRA introduces a novel sparse adapter framework for continual learning in LLMs using JumpReLU gating to dynamically isolate task parameters and prevent catastrophic forgetting. The method enhances LoRA-based approaches and outperforms state-of-the-art continual learning methods like ELLA.
Feature Geometry of LoRA Adapters: A Sparse Autoencoder Analysis of Representational Divergence in Fine-Tuned Language Models
This paper uses Sparse Autoencoders to analyze the geometry of LoRA-induced representations in language models, finding that LoRA updates occupy partially distinct feature structures not fully captured by pretrained interpretability dictionaries.
Selective State-Space Adaptation and Retrieval for Language Model Reasoning
Proposes MaLoRA and MaRA, two adapter families that introduce selective state-space recurrence for token-level and context-level adaptation in frozen language models, achieving significant gains on multi-hop reasoning benchmarks like MuSiQue and 2WikiMultihopQA.
ReLoRA: Knowledge-Reusing Adaptation for Fast Rollout of Evolving LLM Services
ReLoRA is a knowledge-reusing adaptation framework that efficiently restores service-ready LoRA adapters for evolving LLM services, reducing time-to-readiness by up to 8.9× and improving accuracy by up to 4.6% through adaptive initialization and scheduled regularization.