Up to 3.2x Faster Inference with LFM2.5-DSpark

Source: https://huggingface.co/blog/LiquidAI/lfm25-dspark Back to Articles

• How does DSpark work
• Training and Architecture
• Quality parity
• Inference Speed Up on CPU and GPU
• How to use LFM2.5-DSpark
• Get Started
• Citation

Today, we releaseDSpark draft model checkpointsfor three models from our LFM2.5 family: LFM2.5-1.2B-Instruct, LFM2.5-2.6B, and LFM2.5-8B-A1B. These add a speculative decoding path that trades a minimal memory increase for a large decoding speedup without changing output quality:

• Faster inference: up to 3.18 throughput improvement on a GPU and up to 2.87x on-device.
• Toward on-device agentic inference: cuts function-calling latency by 57% on average for LFM2.5-2.6B
• Day-one support for llama.cpp and SGLang: LFM-compatible DSpark integration is open-sourced upstream

https://huggingface.co/blog/LiquidAI/lfm25-dspark#how-does-dspark-workHow does DSpark work

The decode phase in LLM inference is traditionally memory-bound. Most latency comes from streaming weights from DRAM into SRAM, not from intense computation. Speculative decoding addresses this by using a lightweight draft model to produce candidate tokens, then having the target model verify them all in a single forward pass, sharing the cost of loading the weights across all tokens we verify.

Over the years, multiple approaches of speculation have been proposed, with the most prominent beingEAGLE-3,DFlash, and, most recently,DSpark, which combines three components:

• DFlash-style parallel backboneconditioned on the target model’s context features, producing hidden states for all draft tokens in a single forward pass.
• A lightweight sequential head, modeled as a Markov chain between neighboring tokens, that adds inter-token dependency, raising the acceptance rate at later positions.
• A confidence-scheduled verifierthat predicts each token’s survival probability and prunes low-confidence suffixes when verification would cost more than it saves.

https://huggingface.co/blog/LiquidAI/lfm25-dspark#training-and-architectureTraining and Architecture

We follow the DSpark recipe with a larger and more diverse data mix covering SFT, chat, code, and function-calling data. Based on our ablations, the first versions of the draft models are simplified attention-only draft models, with 5 layers and a block of 9. For each draft model, we ran 15 epochs on the entire dataset and selected the epoch with the highest acceptance rate rather than the lowest loss.

The resulting draft models are relatively small, with each around ~300M parameters.

ComponentLFM2.5-1.2B-InstructLFM2.5-8B-A1BLFM2.5-2.6BDecoder stack (5 layers)241.2M241.2M241.2MHidden-state projection21.0M21.0M21.0MMarkov head33.6M65.5M65.5MNorms + confidence head27.5k27.5k27.5kTotal295.7M327.7M****327.7M

https://huggingface.co/blog/LiquidAI/lfm25-dspark#quality-parityQuality parity

Under greedy decoding, a draft token is only accepted if it matches the target model’s distribution. On rejection, the target model’s own token takes its place. The emitted sequence is thereforeidentical to baseline greedyby construction, so benchmark accuracy (pass@1 or exact match) is unchanged.

https://huggingface.co/blog/LiquidAI/lfm25-dspark#inference-speed-up-on-cpu-and-gpuInference Speed Up on CPU and GPU

Our DSpark draft models for LFM2.5 ship with day-one support forllama.cpp(implementationbuilds on top of the official codebase, which we run withexperimental metal kernels) and **SGLang (**implementationbuilds on the official SGLang implementation of DSpark).

We measure on-device throughput with llama.cpp and Metal on an M4 Max MacBook Pro using FP16 GGUF weights and up to 256 output tokens. We measure GPU throughput with SGLang on a single H100 80 GB in BF16. Both configurations use a DSpark block size of 9, a batch size of 1, and a temperature of 0. We evaluate them on five benchmark datasets.

All three drafter models deliver noticeable throughput improvements on both the large-scale accelerator (H100) and the edge deployment (M4 Max MacBook).

For**LFM2.5-2.6B,**speedup on the MacBook is especially noticeable, as it pushes the interactivity level a user can enjoy far beyond the throughput offered by most proprietary cloud models (around ~140 tok/s, depending on the dataset).

DatasetAcceptance (of 10)Speedup on H100Speedup on M4 MaxMATH5005.423.06x326 → 1000 tok/s2.25x61 → 137 tok/sHumanEval4.542.56x326 → 835 tok/s2.63x61 → 161 tok/sMBPP4.712.64x326 → 861 tok/s2.11x62 → 132 tok/sGSM8K4.322.22x312 → 693 tok/s2.36x60 → 143 tok/sMT-Bench5.072.87x325 → 933 tok/s1.99x62 → 123 tok/sMean4.812.67x323 → 864 tok/s2.27x61 → 139 tok/s Across various multi-tool scenarios, DSpark reduces the latency by 57% on average for LFM2.5-2.6B.

ForLFM2.5-1.2B-Instruct, we see much more variance in dataset acceptance rates, so speedup varies by as much as 52% depending on the underlying text distribution.

DatasetAcceptance (of 10)Speedup on H100Speedup on M4 MaxMATH5006.022.56x668 → 1712 tok/s2.62x140 → 366 tok/sHumanEval5.312.26x664 → 1499 tok/s2.87x136 → 389 tok/sMBPP5.522.37x667 → 1578 tok/s2.74x137 → 375 tok/sGSM8K4.341.67x624 → 1041 tok/s2.73x140 → 381 tok/sMT-Bench3.901.66x657 → 1091 tok/s1.72x137 → 237 tok/sMean5.022.10x656 → 1384 tok/s2.54x138 → 350 tok/s ForLFM2.5-8B-A1B, the acceptance rate increases compared to two dense models, yet on-device we get only an 18% improvement on average. This gap is due to the current MoE implementation in llama.cpp’s Metal backend, and to the fact that verifying k tokens activates more experts and thus more weight traffic than a single decode step.

DatasetAcceptance (of 10)Speedup on H100Speedup on M4 MaxMATH5008.273.18x428 → 1362 tok/s1.21x93 → 112 tok/sHumanEval7.022.58x426 → 1100 tok/s1.12x91 → 101 tok/sMBPP6.932.64x426 → 1122 tok/s1.09x89 → 97 tok/sGSM8K4.021.29x385 → 496 tok/s1.44x90 → 129 tok/sMT-Bench8.523.02x426 → 1288 tok/s1.04x87 → 90 tok/sMean6.952.54x418 → 1074 tok/s1.18x90 → 106 tok/s

https://huggingface.co/blog/LiquidAI/lfm25-dspark#how-to-use-lfm25-dsparkHow to use LFM2.5-DSpark

Running the DSpark draft models withSGLangrequires an SGLang build with DSpark support for LFM2 targets (PR #31041). Launch the target with the draft attached:

python -m sglang.launch_server \
  --model-path LiquidAI/LFM2.5-2.6B \
  --speculative-algorithm DSPARK \
  --speculative-draft-model-path LiquidAI/LFM2.5-2.6B-DSpark \
  --speculative-draft-attention-backend flashinfer \
  --disable-radix-cache --mem-fraction-static 0.75 --port 30000

Then query the OpenAI-compatible endpoint athttp://localhost:30000/v1. The block size is read from the draft’sconfig\.json; the baseline is the same command without the three\-\-speculative\-\*flags.

Running them withllama.cpprequires the respective llama.cpp build (PR#27383).

llama-server -m LFM2.5-2.6B-F16.gguf \
  -md LFM2.5-2.6B-DSpark-F16.gguf \
  --spec-type draft-dspark --spec-draft-n-max 10 --spec-draft-n-min 0 \
  -fa on -ngl 99

The block size is read from the sidecar metadata (n-max is clamped to it). Speculative decoding isexact: the target verifies every proposed token, so greedy output equals the target alone; per-responsetimingsreportdraft\_n/draft\_n\_accepted.

https://huggingface.co/blog/LiquidAI/lfm25-dspark#get-startedGet Started

The DSpark draft model checkpoints are available on Hugging Face as Safetensors and in GGUF format:

• Safetensors:LFM2.5-2.6B-DSpark,LFM2.5-1.2B-Instruct-DSpark, andLFM2.5-8B-A1B-DSpark
• GGUF:LFM2.5-2.6B-DSpark-GGUF,LFM2.5-1.2B-Instruct-DSpark-GGUF,LFM2.5-8B-A1B-DSpark-GGUF

We can’t wait to see what you build.

https://huggingface.co/blog/LiquidAI/lfm25-dspark#citationCitation

For citations, please use the following reference or BibTeX:

Liquid AI, “LFM2.5-DSpark: Up to 3.2x Faster Inference from H100 to MacBook”, Liquid AI Blog, Aug 2026.

@article{liquidAI2026dspark,
  author = {Liquid AI},
  title = {LFM2.5-DSpark: Up to 3.2x Faster Inference from H100 to MacBook},
  journal = {Liquid AI Blog},
  year = {2026},
  note = {www.liquid.ai/blog/lfm2.5-dspark},
}