Tag
Gefen is a memory-efficient optimizer that reduces AdamW's memory footprint by ~8x by automatically sharing second-moment estimates and quantizing the first moment using a learned codebook, while maintaining AdamW-level performance.
Taylor-Calibrate proposes a principled initialization method for hybrid linear attention models that significantly improves the efficiency of distilling pretrained Transformers into Gated DeltaNet students, achieving up to 88x improvement and reducing training tokens by 4.9x-9.2x.
Flash-GMM introduces a fused Triton kernel for Gaussian Mixture Models that achieves 20x speedup and enables training on datasets 100x larger on a single GPU, making soft clustering a viable drop-in replacement for k-means in approximate nearest neighbor search.
Google's TurboVec is a new open-source tool that reduces memory usage from 31GB to 4GB for AI search data, leveraging TurboQuant for faster search than FAISS, and integrates with LangChain and LlamaIndex while running fully offline.
This paper reveals that zeroth-order fine-tuning of LLMs is dominated by a single decoding layer, which can be identified by activation outliers, and fine-tuning only that layer matches or exceeds full-model fine-tuning with up to 4.52x speedup.
GRZO is a novel zeroth-order optimization method for fine-tuning large language models that reduces variance by using group-relative normalization, achieving better accuracy and memory efficiency compared to MeZO.
Introduces DualOptim+, an optimization framework for LLM unlearning that uses shared base states and decoupled delta states to balance forgetting and retaining objectives, with a quantized variant for reduced memory.
Rosalind is a deterministic genomics engine in Rust designed to run whole-genome pipelines with O(√t) memory, enabling bioinformatics on laptops and edge devices.
The author built TinyHarness, a low-memory-footprint AI harness compatible with Ollama, Llama.cpp, and vllm, aiming to compete with tools like pi and opencode.
AirLLM is an open-source tool that optimizes inference memory usage, enabling 70B LLMs to run on a single 4GB GPU without quantization, and supports 405B models on 8GB VRAM.
Orthrus is a dual-architecture framework that combines autoregressive LLMs with diffusion models for fast parallel token generation while maintaining exact inference fidelity via shared KV caches and consensus mechanisms, achieving up to 7.8x speedup.
Proposes Memory-Efficient Looped Transformer (MELT), a novel recurrent LLM architecture that decouples reasoning depth from memory consumption by sharing a single KV cache across loops and using chunk-wise training with interpolated transition and attention-aligned distillation.
GroupDPO introduces a memory-efficient algorithm for group-wise direct preference optimization that leverages multiple candidate responses per prompt while reducing peak memory usage through decoupled backpropagation. The method demonstrates consistent improvements over standard DPO across offline and online alignment settings.