Tag
HypoLoop is a family of native weight-editing methods that revises which parameters to edit and how errors drive fitting based on scientific feedback, validated across 22 scientific tasks with DeepSeek and Claude, yielding notable error reductions on Burgers and DIODE benchmarks.
The paper proposes UUE, a lightweight LLM unlearning framework that reformulates forgetting as controllable model editing by projecting parameter updates into the orthogonal null space of retained knowledge, with a closed-form untargeted objective and pluggable low-rank adapters (UUE-L) that outperform existing methods on TOFU and WMDP benchmarks.
Dejan.ai replicates a NeurIPS 2025 study showing that training a tiny fraction of randomly chosen weights can unlock latent skills: Gemma 3 270M went from ~0% to 97% on number sorting after training just 1,000 of its 100M weights, with similar gains for word reversal and clean JSON output.
本文提出 TF-IG,一种参数高效的交互门控时频学习框架,用于基于 PPG 的心率估计,在 BIDMC PPG 数据集上以仅 84.6K 参数实现 1.227 BPM 的 MAE 和 0.9934 的皮尔逊相关系数。
论文提出DualView,一种利用冻结LLM提取广泛与输出特定表示、通过双分支架构联合预测多个数值目标的参数高效多输出文本回归方法,在五个任务、八个LLM上超越七个监督基线,且训练参数量减少82–1842倍。
提出Harness Compilation (HC)——一种离线调整冻结小视觉语言模型与其外部harness之间分工的流程,利用大教师模型基于学生执行轨迹优化可复用内容与控制,并在七个VQA设置上以≤9B参数的学生模型取得9.9–23.9分的提升。研究还分析了五类运行时决策(调用、选择、参数生成、证据整合、弃答)应如何在模型与harness间分配。
This Samsung Semiconductor paper proposes LINEUP, which decouples personalization from per-user adaptation by learning a bank of reusable low-rank factors and restricting each user to only 8 trainable scalars instead of 4.19M per-user LoRA parameters, outperforming baselines on all 12 metrics across six personalized LLM tasks.
HyperThink 提出一种 text-to-parameter 超网络方法,通过一次前向传播直接预测 LLM 的参数更新,从而免去长思维链解码,在低延迟场景下显著提升数学与通用推理的准确率-延迟权衡。
论文提出 Local Support Learning (LSL),一种在无需访问历史数据的情况下缓解大模型灾难性遗忘的框架,通过 GMM 门控函数使权重适配器只在自身训练分布的输入上激活,从而在最多 7B 参数的 LLM 上同时保留预训练与微调能力。
This paper introduces STAIR (Stale-Token Attention for Inter-query Reuse), a method that stores keys and values from earlier response generations in a fixed bank and trains only 12,288 parameters to direct current queries, improving later-turn accuracy by up to 11.67 percentage points on Qwen models across four benchmarks.
The paper proposes LLMAE, a method to repurpose pre-trained decoder-only LLMs as continuous text autoencoders using a latent bottleneck, achieving high-fidelity reconstruction and enabling downstream tasks like image captioning.
This paper proposes modality-gated deep adapters to extend frozen multimodal embedding models with new modalities while preserving existing outputs bit-for-bit, demonstrated with audio and thermal packs that improve benchmark performance.
This paper proposes a lightweight evolutionary heuristic scheduler to optimize denoising trajectories in diffusion large language models, addressing failure modes like EOS Overflow and Proximal Bias, and outperforming baselines on reasoning and planning benchmarks.
ALPINE introduces an ultra-lightweight spatial-relational architecture for few-shot image classification that achieves accuracy gains with fewer parameters, faster convergence, and better robustness compared to baselines like Prototypical Networks and MAML.
This paper presents a three-stage training pipeline for developing compact dense retrievers, introducing PolDense for Polish and EuroDense for European languages, which achieve strong performance with significantly reduced parameters compared to larger models.
This paper introduces an adaptive anisotropic learning-rate model for LoRA fine-tuning to address within-module heterogeneity, improving performance and rank capacity utilization across benchmarks.
This paper diagnoses intra-adapter contention in MoE+LoRA fine-tuning and introduces SpawnLoRA to dynamically add sub-adapters, reducing negative transfer across domains.
DRET is a parameter-efficient knowledge-transfer method that injects biomedical domain knowledge into smaller models like DistilBERT via embedding transfer, achieving performance competitive with larger specialized models.
This paper explores the trade-off between factual knowledge acquisition and capability retention in language models when using QLoRA, showing that higher-rank QLoRA improves fact learning but may degrade out-of-domain performance.
This article explains five parameter-efficient fine-tuning techniques for large language models, such as LoRA and VeRA, detailing how each method adapts model weights with minimal updates.