Follow-up benchmark post showing GCMP, a LoRA replacement, matches LoRA r=32 accuracy on Qwen3.8-27B with 55× fewer adapter parameters (2.9M/11MB vs 159M/320MB), with details on serving memory, training speed trade-offs, and 4-bit base comparisons.
Follow-up to my CQ post from last week. That post covered the 4-bit base model; this one is about the adapter. The LoRA vs GCMP runs on 4-bit bases have now finished. GCMP is a LoRA replacement we built. On Qwen3.8-27B it trains 2.9M parameters (~11 MB), vs ~159M (~320 MB) for LoRA r=32 on the same 7 projections. That's 55× smaller. Accuracy: same as LoRA All adapters were trained on Alpaca for 500 steps with identical settings (batch 2×4, length 512, LR 2e-5, seed 42). Every row was scored with the same lm-evaluation-harness code and questions, greedy decoding, thinking off. Comparisons use paired McNemar tests. Base + adapter Params Answer loss GSM8K SQuAD BoolQ ARC-C MMLU IFEval TruthfulQA BF16, no adapter – 1.678 0.683 0.746 0.874 0.588 0.839 0.805 0.542 BF16 + LoRA r=32 159M 1.480 0.903 0.776 0.896 0.612 0.844 0.825 0.518 BF16 + GCMP 2.9M 1.471 0.890 0.760 0.892 0.618 0.846 0.815 0.514 NF4 + LoRA r=32 159M 1.464 0.793 0.748 0.880 0.608 0.840 0.835 0.528 NF4 + GCMP 2.9M 1.458 0.833 0.756 0.878 0.618 0.837 0.840 0.522 BF16 base (no quantization at all): no significant difference on any benchmark (all p ≥ 0.2). NF4 base: GCMP is +4.0 on GSM8K (p=0.04) and tied on everything else. Loss: GCMP's answer loss is slightly lower on both bases, and its validation loss was lower at all 10 checkpoints on both. One significant win on one task isn't enough to call it better, so the claim is same quality as LoRA at 1/55th the size. Why the size matters For one fine-tune, 320 MB vs 11 MB doesn't matter much. It starts to matter when you have many: Multi-tenant serving: one fine-tune per customer or task on a single base model. In vLLM, GCMP adapters load once at startup and each request picks its adapter by model name. A batch can mix adapters and runs in one kernel call per layer, with no merging or swapping. Resident GPU memory: vLLM keeps each adapter in its compact form and never builds a dense weight update. Memory is about 11 MB per adapter, plus a fixed ~1 MB routing buffer and a shared workspace of ≤ ~64 MB that doesn't grow with adapter count. Storage and loading: 1,000 adapters are ~11 GB instead of ~320 GB. That's faster to ship, cache and hot-load. Edge and small GPUs: an extra fine-tune costs megabytes, not hundreds of MB. Estimated 1 adapter 100 1,000 LoRA r=32 320 MB 32 GB 320 GB GCMP 11 MB 1.1 GB 11 GB These are estimates from adapter size. I haven't measured memory and throughput at 100 or 1,000 adapters yet, and that's the next benchmark. What's not good yet Training speed: GCMP takes ~2× LoRA's step time (BF16 on Unsloth: 2.74 vs 1.37 s/step; NF4: 5.6 vs 3.2). It does use slightly less training memory (21.5 vs 23.8 GiB on NF4). Baseline: r=32 is a generous LoRA baseline. r=4/8/16 comparisons are next, and I'll post them either way. Serving: vLLM serving of GCMP currently needs a CQ base. On full-precision models it runs in Transformers for now. Scope: one model, one dataset, one run per case. The quick benchmark subset (200–1,140 questions per task) detects ~5-point differences, not 1-point ones. Quick update on CQ The CQ base still matches BF16 (no significant difference on any of the 9 benchmarks) while NF4 loses 12 points on GSM8K. After fine-tuning, though, the gap closes: CQ + GCMP vs NF4 + GCMP: tied on GSM8K (83.0 vs 83.3). Both 4-bit stacks vs BF16 + GCMP: both end up ~6 points behind on GSM8K. So CQ's GSM8K lead didn't survive fine-tuning in this test. CQ + LoRA is being retrained, because the adapter from our older trainer turned out to be broken, and I'll add it when it's done. Code and licensing The Python code is open source. The CUDA kernels are compiled with a license check, which is free during early access. Repo, with the commands to reproduce every number: https://github.com/thousandml/training-factory Write-up with the full tables and the serving memory breakdown: https://huggingface.co/blog/thousandml/cq-gcmp-4bit-finetuning If you run multi-tenant fine-tunes, I'd like to know: is adapter memory or count actually a bottleneck for you, or is it something else?
A LoRA adapter was trained on an abliterated Qwen 3.8-27B model to enhance internal codebase recall, demonstrating superior performance over Claude models on private-repo-specific tasks in evaluations.
Release of a ~320MB LoRA adapter RL-trained on Qwen3.8-27B inside Hermes Agent, achieving thinking-level accuracy with thinking disabled, 3.5–4.4× fewer tokens, and strong SWE-bench Pro / Terminal-Bench results, with GGUF weights available for local use.
An update on the LoRA over GGUF series showing that Qwen3.8-Flash-Next (125B-A6B + 51B engram) can now be LoRA-trained in 40 GiB VRAM without CPU offloading, achieving 200 token/s on Strix Halo, with initial GGUF support merged in transformers 5.18 and further kernel optimizations (GGTensile) on the horizon.
LoRA Speedrun is a public wall-clock leaderboard for fine-tuning techniques, measuring how fast LoRA adapters can achieve target accuracy on GSM8K and SQuAD tasks using Qwen2.5-1.5B and SmolLM2-1.7B on a single L40S GPU, with automated verification.
A personal release of Jeff-Qwen3.5-0.8B plus 9 task-specific LoRA adapters that acts as a fast, calibrated "System 1" classifier in front of larger models, claiming 38× faster decisions and +8.7 accuracy points over Qwen3.8-27B while using under 2 GB extra memory.