Code2LoRA: Hypernetwork-Generated Adapters for Code Language Models under Software Evolution

Hugging Face Daily Papers Papers
code-language-models lora hypernetwork software-evolution repo-conditioned code-generation

Summary

Code2LoRA introduces a hypernetwork that generates LoRA adapters from a repository in a single forward pass, allowing frozen code LLMs to adapt to repository context without extra tokens, and supporting evolving codebases efficiently. It also delivers RepoPeftBench, a benchmark for repo-conditioned code modeling.

Code language models need repository-level context to resolve imports, APIs, and project conventions. Existing methods inject this knowledge as long inputs (retrieved through RAG or dependency analysis) or through per-repository fine-tuning and LoRA -- costly at repository scale and brittle to evolving codebases. We introduce Code2LoRA, a hypernetwork framework that generates repository-specific LoRA adapters, effectively injecting repository knowledge with zero inference-time token overhead. Code2LoRA supports two usage scenarios: Code2LoRA-Static converts a single repository snapshot into an adapter, suitable for comprehension of stable codebases; while Code2LoRA-Evo maintains an adapter backed by a GRU hidden state updated per code diff, suitable for active development of evolving codebases. To evaluate Code2LoRA against parameter-efficient fine-tuning baselines, we build RepoPeftBench, a benchmark of 604 Python repositories with two tracks: a static track with 40K training and 12K test assertion-completion tasks, and an evolution track with 215K commit-derived training and 87K commit-derived test tasks. On the static track, Code2LoRA-Static achieves 63.8% cross-repo and 66.2% in-repo exact match, matching the per-repository LoRA upper bound; on the evolution track, Code2LoRA-Evo achieves 60.3% cross-repo exact match (+5.2 pp over a single shared LoRA). Code2LoRA's code can be found at https://anonymous.4open.science/r/code2lora-6857; the model checkpoints and RepoPeftBench datasets can be found at https://huggingface.co/code2lora.
Original Article
View Cached Full Text

Cached at: 06/05/26, 06:10 PM

Paper page - Code2LoRA: Hypernetwork-Generated Adapters for Code Language Models under Software Evolution

Source: https://huggingface.co/papers/2606.06492 Repository context is the bottleneck for code LLMs: every completion needs to know the project’s imports, APIs, and conventions, and today we pay for that context at every single query - through RAG, dependency analysis, or ever-longer prompts - or we fine-tune a LoRA per repo and watch it go stale on the next commit.

We ask a simple question: what if the repositoryitselfwere the prompt, but only once?Code2LoRAis a hypernetwork that reads a repo (or its stream of commits) and emits a LoRA adapter for a frozen code LLM in a single forward pass. The repo’s knowledge lives in the weights, and inference adds zero extra tokens.

It comes in two flavors.

  • Code2LoRA-Staticsnapshots a repo into an adapter and, with no per-repo training, already matches the per-repository LoRA upper bound on in-repo eval and beats RAG / dependency-resolved-context / FFT+RAG cross-repo (+9.9pp EM).
  • Code2LoRA-Evois the part we’re most excited about: a GRU walks the commit history and refreshes the adapter in O(1) per commit, so the model keeps up with active development instead of fighting it. On a strictly post-cutoff 92-repo OOD holdout — repos the encoder has never seen — Code2LoRA-Evo lifts a Qwen2.5-Coder backbone from 44.6% to 74.1% EM.

To make all of this measurable we also releaseRepoPeftBench: 604 Python repos, 62K static and 400K commit-derived assertion-completion tasks, with in-repo, cross-repo, and temporal-OOD splits. We hope it’s useful as a benchmark for repo-conditioned and evolution-aware code modeling beyond our own setup.

Code:https://anonymous.4open.science/r/code2lora-6857· Data & models: https://huggingface.co/code2lora

Similar Articles

JumpLoRA: Sparse Adapters for Continual Learning in Large Language Models

arXiv cs.CL

JumpLoRA introduces a novel sparse adapter framework for continual learning in LLMs using JumpReLU gating to dynamically isolate task parameters and prevent catastrophic forgetting. The method enhances LoRA-based approaches and outperforms state-of-the-art continual learning methods like ELLA.

Beyond LoRA: Is Sparsity-Induced Adaptation Better?

arXiv cs.LG

This paper proposes sparsity-induced adaptations to LoRA, including Cheap LoRA (cLA) and a chained circulant variant (c³LA), and provides theoretical generalization bounds along with empirical evaluations showing up to 10% training time reduction and 15% peak GPU memory savings while maintaining competitive performance.

Video2LoRA: Parametric Video Internalization for Vision-Language Models

Hugging Face Daily Papers

This paper introduces Video2LoRA, a method that predicts Low-Rank Adaptation (LoRA) weights directly from video representations, enabling efficient video processing in frozen vision-language models. It reduces visual token load by up to 1500x and query TTFT by 6-80x while maintaining performance on video summarization and captioning benchmarks.