novita/kimi-k2.6-dspark · Hugging Face

Source: https://huggingface.co/novita/kimi-k2.6-dspark

https://huggingface.co/novita/kimi-k2.6-dspark#kimi-k26-dspark-speculatorKimi-K2.6 DSpark speculator

https://huggingface.co/novita/kimi-k2.6-dspark#overviewOverview

A DSpark speculator model for theKimi\-K2\.6base model, enabling faster inference through speculative decoding. DSpark extends the DFlash parallel draft backbone with two lightweight heads: aMarkov logit-bias head(low-rank intra-block token dependency) and aper-position confidence head(accept-rate prediction). Trained with a vendored fork of thespeculatorslibrary through the Camelot-Ray online pipeline (draft consumes hidden states streamed from a live Kimi-K2.6 vLLM server).

https://huggingface.co/novita/kimi-k2.6-dspark#model-specificationsModel Specifications

• Base Model: Kimi-K2.6
• Format: Safetensors (single-file bf16, 6.3 GB, 44 tensors)
• Draft: 3 layers (Qwen3-style GQA), hidden 7168, 56 heads / 8 KV heads, head_dim 128, FFN 18432, rope_theta 50000,block\_size=8
• Vocabulary: pruned draft vocab 32,000 (d2t/t2d remap tables shipped in the weights), target vocab 163,840; mappings built from training-distribution assistant-turn token frequencies
• DSpark heads: Markov rank 256 (vanilla), confidence head (with-markov),mask\_token\_id=163608
• Aux hidden-state layers: [1, 29, 57]
• Trained context: seq 20000

https://huggingface.co/novita/kimi-k2.6-dspark#evaluation-resultsEvaluation Results

Offline teacher-forced (training distribution, frozen final checkpoint):

metricvaluemean accepted length (tau, /8)****3.97greedy acceptance (pos 1-7)0.618 Per-position acceptance (positions 1-7):0\.816 / 0\.720 / 0\.652 / 0\.598 / 0\.553 / 0\.512 / 0\.474

Online (vLLM nightly spec-decode, greedy,num\_speculative\_tokens=7), per-benchmark mean accepted length:

benchmarkonline accept_lenoffline tau /8gsm8k4.455.13math5004.255.00aime3.814.44humaneval3.724.60mtbench2.823.41mmstar2.532.87ceval1.551.99 Cross-model transfer: serving Kimi-K2.7-Code with this K2.6-trained speculator loses only ~11% accepted length (same architecture family).

https://huggingface.co/novita/kimi-k2.6-dspark#serving-with-vllmServing with vLLM

Requires vLLM nightly (DSpark support):

uv pip install vllm --extra-index-url https://wheels.vllm.ai/nightly

vllm serve <path-or-id-of-Kimi-K2.6> \
    --tensor-parallel-size 8 \
    --max-model-len 20000 \
    --trust-remote-code \
    --speculative-config '{
        "model": "novita/kimi-k2.6-dspark",
        "num_speculative_tokens": 7,
        "method": "dspark"
    }'

Known vLLM-nightly (0.23.1rc1.dev786) caveats, with workarounds:

1. Draft-side FA3 AOT scheduling crashes withscheduler\_metadata must have shape \(metadata\_size\)— the GPU-worker spec-decode path missesfast\_build=Truewhen building draft attention metadata. Patchvllm/v1/worker/gpu/spec\_decode/speculator\.py/vllm/v1/worker/gpu/attn\_utils\.pyto passfast\_build=True(mirrorsbuild\_for\_drafting\(\)on the legacy proposer path).
2. CUDA-graph capture fails with a flashinfer allreduce workspace-size error under spec-decode token expansion; disable the fusion:\-\-compilation\-config='\{"pass\_config": \{"fuse\_allreduce\_rms": false\}\}'.

https://huggingface.co/novita/kimi-k2.6-dspark#training-detailsTraining Details

• Data: Regenerated open-perfectblend dataset — the open-perfectblend instruction mix with all assistant turns regenerated by Kimi-K2.6 itself (raw hidden states streamed from the live verifier); seq 20000
• Steps: 20000 (16.0h, zero restarts); loss 1.656 → 0.363 (1000-step window)
• Schedule: lr 3e-4 cosine, warmup 300, global batch 8, accumulation 2
• Loss: 0.1·CE + 0.9·TV over block-diffusion anchors, decay_gamma 4.0, max_anchors 3072
• Semantics: post-norm last hidden captured at rollout (apply\_verifier\_norm=False), hidden_states = concat of aux layers [1, 29, 57]