hypernetwork

The article introduces FANS, a hypernetwork-based framework for heterogeneous federated learning that learns a shared architecture space and uses parallel training with self-distillation to optimize model selection across diverse devices.

HyperFix proposes a lightweight hypernetwork to predict nonlinear corrections for task vector merging across varying task subsets, reducing per-subset tuning costs and outperforming existing methods.

This paper proposes MoEGen, a parameter-efficient fine-tuning framework that uses mixture-of-experts to generate instance-adaptive LoRA updates via expert codes and a lightweight hypernetwork, improving performance on commonsense reasoning benchmarks without storing separate adapters per expert.

Compliance2LoRA proposes a hypernetwork-based framework that generates policy-compliant LoRA adapters on demand for large reasoning models, enabling adjustable safety alignment across arbitrary policy subsets without retraining separate models.

PRISMR proposes a framework using hypernetworks and LoRA to internalize list structure, overcoming parse collapse in multimodal listwise ranking. It introduces a large-scale benchmark and shows reduced parse collapse and improved ranking performance across domains and backbones.

Introduces Code2LoRA, a hypernetwork that converts an entire code repository into a LoRA adapter for code LLMs, eliminating inference-time token overhead for repository-level knowledge.

This paper introduces Code2LoRA, a hypernetwork-based method to generate adapters for code language models, addressing challenges under software evolution.

LatentSkill converts textual skills into LoRA adapters stored in weight space, reducing context overhead while maintaining modularity and composability for LLM agents, achieving significant improvements on ALFWorld and Search-QA benchmarks.

Code2LoRA introduces a hypernetwork that generates LoRA adapters from a repository in a single forward pass, allowing frozen code LLMs to adapt to repository context without extra tokens, and supporting evolving codebases efficiently. It also delivers RepoPeftBench, a benchmark for repo-conditioned code modeling.

Proposes a 'lift' method for training input-convex neural networks (ICNNs) that uses an unconstrained hypernetwork to emit non-negative inter-layer weights, softening the loss landscape and escaping gradient attenuation, achieving lower test loss than projected gradient descent and softplus reparametrization.