@karminski3: DeepSeek truly excels in both cost-effectiveness and technology... Some classmates don't understand what DSpark is, so here's a quick tutorial. Speculative decoding is a technique to improve the output speed of large models. The essence is to let a small model generate text for the large model to check. Because currently...

DeepSeek really delivers a deadly combo of cost-effectiveness and performance…

Some of you may be wondering what DSpark is — let me give you a quick tutorial.

Speculative decoding (also called draft decoding) is a technique to speed up the inference of large language models. The basic idea is to let a small model predict the next tokens for the large model, and then the large model verifies whether those predictions are correct. Since current models are typically bottlenecked by memory bandwidth while GPU compute is plentiful, the large model’s prefill speed (reading tokens) is much faster than its decode speed (generating tokens one by one). So by having the small model draft a continuation along the same reasoning path, the large model only needs to verify (which is essentially just prefill). If the small model’s guesses are correct, you leverage the fast prefill speed, achieving multi‑token generation and significantly higher throughput.

But here’s the catch: an external draft model also needs to prefill, consumes VRAM, and competes for memory bandwidth. Is there a smarter way? That’s where DSpark comes in.

Take a look at the diagram below (left side is the DeepSeek‑V4 architecture from @rasbt). DSpark is inserted right after the Final RMSNorm layer. Instead of a full‑blown draft model, it’s a tiny 3‑layer MTP (Multi‑Token Prediction) transformer stack.

After the large model has computed through its 60+ layers, it pushes the “high‑level concept” (feature vector / hidden state) of the current token into the Final RMSNorm exit. Before this vector gets translated into concrete text, DSpark intercepts it:

First, it does semi‑autoregressive fast drafting (MTP + Markov Head). DSpark has its own small set of parameters, instantly predicts 5 tokens (as feature vectors) in parallel, then uses its own internal serial network to smooth out the logic. (Note: parallel first, then serial — this avoids the logical incoherence that pure parallel generation would cause.)

Then, it applies a confidence prediction head to estimate how accurate its own guesses are. For example, if the last 2 out of 5 are uncertain, it simply cuts them off to avoid wasting the large model’s compute on verification.

Finally, the remaining 3 tokens are fed back through the vocabulary projection layer, translating the vectors into actual token IDs. That’s all DSpark does.

After that, the large model scans DSpark’s output in one go (using only prefill, no decode). If the draft is correct, it directly accepts all tokens — instead of generating one token per step, it now outputs 3 tokens at once!

The key point: speculative decoding does not reduce reasoning quality, and it can boost speed by 60‑85%. Previously you’d hire a small model to write the draft; now it’s like implanting a chip directly into the brain.

As of now, SGLang already has a PR for this feature (29538), and DeepSeek has just released a bunch of DSpark‑modified small models on their HuggingFace page. I’ll go out on a limb and predict that future models will come with DSpark built‑in.

#dspark #deepseek #speculative_decoding #draft_decoding