We just added recipes for DeepSeek-V4-Flash-Vision & DeepSeek-V4-Flash-0731 on 2x DGX Spark.

http://docs.sglang.io/cookbook/autoregressive/DeepSeek/DeepSeek-V4…

• Flash-Vision comes with native vision support
• Flash-0731 supports both fp4 and @NVIDIAAI official nvfp4
• Both run on the b12x kernel library

Enjoy!

DeepSeek-V4 - SGLang Documentation

Source: https://docs.sglang.io/cookbook/autoregressive/DeepSeek/DeepSeek-V4

Deployment

Install SGLang

For all methods and hardware platforms, see theofficial SGLang installation guide. The two paths below match thePython / Dockertoggle in the command panel.

• Python (pip / uv)
• Docker

Then run thePythonoutput of the command panel below in that environment.

For how to launch the image, seeInstall → Method 3: Using Docker. A minimal example (substitute the innersglang serve \.\.\.with whatever the command generator below produces):NVIDIA GPUsA single image —lmsysorg/sglang:latest— covers thedatacenter GPUsin this cookbook (B200 / B300 / GB200 / GB300 / H100 / H200 / RTX PRO 6000). The one exception isFlash Vision (Exp), whose support has not shipped in a release yet: its cells use the preview imagelmsysorg/sglang:dev\-dsv4\-flash\-vision(the command panel picks it automatically — see theFlash Vision notes).DGX Sparkis the other exception: its three cells (Flash Official FP4, Flash Official NVFP4, Flash Vision FP4) use the DGX Spark–only preview imagelmsysorg/sglang:dev\-v4f\-2dgx\-v2(the command panel picks it automatically — see theDGX Spark notes); do not use that image on any other hardware.

**AMD GPUs (ROCm)**AMD uses the daily-updatedlmsysorg/sglang\-rocmimages. You can find the latest images onDocker Hub. We recommend the ROCm 7.2 version.For example:

• MI355X→lmsysorg/sglang\-rocm:v0\.5\.18\-rocm720\-mi35x\-20260829
• MI300X→lmsysorg/sglang\-rocm:v0\.5\.18\-rocm720\-mi30x\-20260829

Pick your hardware + recipe to generate the launch command. The three serving strategies cover the common operating points:

• Low-Latency— fastest reply for a single user. Pick for chat.
• Balanced— good speed with several users at once. Use for typical multi-user serving.
• High-Throughput— most tokens per second across many users. Best for batch jobs.

Panel controls(top of the command box):

• Python / Docker— baresglang serve …for an existing SGLang env, or adocker run … sglang serve …wrap against the per-hardware image from theInstall SGLangpanel above.
• ⧉ Copy— copies the current command (with whichever framing is active) to your clipboard.
• $ cURL— a sample request againstlocalhost:30000to confirm the server is up.
• ⚙ Env— edits the placeholders (HOST\_IP,PORT,HF\_TOKEN,NODE\_RANK,NODE0\_IP) the command and cURL share. Persists in localStorage across cookbooks.
• Verified / Not Verifiedbadge — green when the\(hw, variant, quant, strategy, nodes\)combo has been run end-to-end on real hardware; yellow when auto-derived from a neighbor and not yet re-checked.

Playground

The Playground is where you experiment withSGLang features beyond the verified matrix. The Deploy panel above only emits combinations the SGLang team has signed off on; the Playground lets you turn on additional knobs on top of whichever cell the Deploy panel is currently showing. The base is read live from your Deploy selection — only your overrides change.The knobs come in two flavors:

• Built-in SGLang features— parallelism overrides (TP / CP / DP-Attention — DP-Attention’s value is the DP degree, withoffto disable), MoE backend + EP, reasoning / tool-call parsers, speculative-decoding presets, prefill/decode disaggregation, HiCache tiers, and HiSparse hierarchical sparse attention (decode-role only — the card appears once PD-Disagg mode is set to decode).
• DeepSeek-V4 specific features— MegaMoE W4A8 / W4A4 fused kernel (Blackwell only; Hopper SM90 uses a separate all-FP8 MegaMoE path — see Configuration Tips below).

Lines highlightedgreenare added by your overrides; lines withred strikethroughwere in the verified base but stripped by an override. When no override differs from the base cell, the playground inherits the base’sVerifiedbadge; any actual change flips it toNot Verifieduntil the new configuration is run end-to-end and submitted back.

Panel controlsreusePython / Docker·⧉ Copy·$ cURL·⚙ Envfrom the Deploy panel, plus one extra:

• Submit ↗— opens a pre-filled GitHub issue so you can land your override combo as a new verified cookbook cell. Shown only while the badge saysNot Verified; click it once you’ve actually run the command on your hardware and confirmed it works.

1. Model Introduction

DeepSeek-V4is the next-generation Mixture-of-Experts model from DeepSeek, released 2026-04-24 under anMIT License. The 0731 Flash and 0813 Pro refreshes add checkpoints with a bundled DSpark draft head, and the experimental Flash Vision checkpoint builds image understanding on top of the 0731 Flash base:

VariantTotal paramsActive (MoE)Use**DeepSeek-V4-Flash****284B13Bsingle-node serving on B200 / B300 / GB200 / GB300 / H200 (TP=4); RTX PRO 6000 (TP=2); H100 (TP=8)DeepSeek-V4-Flash-0731****30413BFlash Official (0731), with a bundled DSpark draft head; verified on 8×B200, 4×GB300, and 4×H200DeepSeek-V4-Flash-Vision-Exp****305B13BFlash Vision (Exp) — experimental multimodal (image-text-to-text): the 0731 Flash base + vision encoder & aligner; verified on 4×B200 (TP=4), requires thepreview buildDeepSeek-V4-Pro****1.6T49Bhigh-capacity: B200 / B300 (TP=8) · GB300 (TP=4) · H200 FP4 (TP=8) · GB200 (2-node, TP=8) · H200 FP8 (2-node, TP=16) · H100 (2-node, TP=16)DeepSeek-V4-Pro-0813**1.65T49BPro Official (0813), with a bundled DSpark draft head; verified on 4×GB300 (TP=4) · B200 / B300 / H200 FP4 (TP=8) · GB200 (2-node, TP=8) · H100 (2-node, TP=16) · MI355XThe Instruct checkpoints ship asFP4 MoE experts + FP8 attention / dense(one mixed-precision checkpoint covers every FP4-capable GPU). Matching\*\-Baserepos ship pure FP8 mixed and are for further pre-training only — not for chat or tool calling.**Highlights:**hybrid CSA + HCA attention (~27% inference FLOPs / ~10% KV cache vs DSv3.2 at 1M context), manifold-constrained hyper-connections (mHC), Muon optimizer,1M-token context(32T+ pre-training tokens), three reasoning modes (Non-think/Think High/Think Max— use ≥ 384K context for Think Max), and a dedicatedencoding\_dsv4\.encode\_messagesPython encoder + DSML tool-call grammar.Recommended generation:temperature=1\.0,top\_p=1\.0.**Resources:**HuggingFace ·Flash Official (0731)·Flash·Flash Vision (Exp)·Pro·Pro Official (0813)· ModelScope ·Flash·Pro.

2. Configuration Tips

Concurrency & DeepEP dispatch bufferMust hold:max\-running\-requests × MTP\_draft\_tokens ≤ SGLANG\_DEEPEP\_NUM\_MAX\_DISPATCH\_TOKENS\_PER\_RANK. Violating it blows DeepEP’s dispatch buffer at steady-state load (deep\_ep\.cpp:1105). When tuning, move\-\-cuda\-graph\-max\-bs\-decode,\-\-max\-running\-requests, and the env together.The generator currently picks values on theconservativeside (mirroring an internal stress-test matrix). They run safely out of the box but likely leave throughput on the table — please tune them up toward your actual workload’s peak concurrency and report findings back so the defaults can be revised.Speculative decodingThe original Flash and Pro recipes use EAGLE. Flash Official (0731) and Pro Official (0813) use the bundled DSpark draft head; seeDSparkfor its launch and tuning notes.

For the original Flash and Pro checkpoints:

• low\-latency: steps=3, draft-tokens=4 → largest win at bs=1.
• balanced: steps=1, draft-tokens=2 → gentler MTP, reduces throughput hit at higher batch.
• high\-throughput: MTP disabled — at saturation the verify step costs more than it saves.
• MTP runs on the v2 speculative path.

DGX Spark (2x GB10): Flash Official FP4 / NVFP4, Flash Vision FP4TheDGX Sparkrow has three recipes, allBalanced · Multi-Nodes:Flash Official (0731) · FP4,Flash Official (0731) · NVFP4, andFlash Vision (Exp) · FP4. None of these checkpoints fits one 128GB GB10, so every recipe runs TP=2 across two DGX Sparks connected over ConnectX-7 (RoCE). Every other DGX Spark combination is greyed out on purpose.

• Docker image— all three cells uselmsysorg/sglang:dev\-v4f\-2dgx\-v2, a preview build madeonly for DGX Spark(branchb12x\-vision@452239a74f): it bakes in the SM12xb12xMoE (W4A8) and compressed-MLA attention kernels (#34878,#35899,#34018), the Flash Vision model support (#37253), the b12x image-prefill fix that lets Flash Vision serve images on SM12x, the NVFP4 MTP-layer dispatch fix, and the CuTeDSL and NCCL pins the GB10 pair needs. Do not use it on other hardware, and use the panel’s Docker mode — the bare Python command needs theb12xkernel package this image ships.
• Run the same command on both Sparkswith\-\-node\-rank 0/\-\-node\-rank 1and\-\-dist\-init\-addrpointing at node 0 over the ConnectX-7 link. Thedocker runflags the panel emits (\-\-network host \-\-ulimit memlock=\-1:\-1 \-\-cap\-add IPC\_LOCK \-\-device /dev/infiniband) are what let NCCL use RDMA; without them NCCL silently falls back to TCP and decode slows by roughly 40%.
• Env knobsin the cells are part of the recipe:SGLANG\_SM120\_FLASHMLA\_BACKEND=b12xselects the b12x attention path,SGLANG\_B12X\_MAX\_TOKENSmust equal\-\-chunked\-prefill\-size, andPYTORCH\_CUDA\_ALLOC\_CONF=expandable\_segments:Trueavoids unified-memory fragmentation OOMs on GB10.
• NVFP4 (nvidia/DeepSeek\-V4\-Flash\-0731\-NVFP4)— only the routed experts are NVFP4; attention, shared experts and the DSpark MTP layer stay in the checkpoint’s native formats. On SM12x that means three extra flags:\-\-moe\-runner\-backend flashinfer\_cutlass(b12x’s MoE is MXFP4-only and trtllm-gen kernels are sm100-only),\-\-speculative\-moe\-runner\-backend b12x(the DSpark draft’s MTP experts are MXFP4 and run on b12x), and\-\-disable\-shared\-experts\-fusion(HashTopK rejects fused shared experts under the cutlass runner). Throughput and DSpark acceptance match the FP4 cell within noise.
• Flash Vision— images are served natively on the b12x recipe with the same flags as Flash Official (sendimage\_urlcontent on/v1/chat/completions, seeVision); text-only requests work unchanged. Expect roughly 15–20% lower text throughput than Flash Official on this checkpoint — its bundled DSpark head accepts fewer drafts (~3.2 vs ~3.9) — with text accuracy intact.

DeepSeek-V4-Flash-Vision-Exp (Experimental)deepseek\-ai/DeepSeek\-V4\-Flash\-Vision\-Expis DeepSeek’s first experimental multimodal V4 checkpoint: the 0731 Flash base plus a vision encoder and aligner, served through the samesglang serveflow with OpenAI-styleimage\_urlinputs (seeVisionbelow). Select theFlash Visionvariant in the Deploy panel for its recipes.

• Preview build required— support lands viasgl-project/sglang#37253and has not shipped in a release. Docker mode on the Flash Vision cells already emits the preview imagelmsysorg/sglang:dev\-dsv4\-flash\-vision; for a Python environment, install SGLang from that PR’s branch. The DGX Spark Flash Vision cell is the exception: it uses the DGX Spark imagelmsysorg/sglang:dev\-v4f\-2dgx\-v2(see theDGX Spark notes).
• Verified matrix— MMMU-Pro via sgl-eval attemperature 1\.0,top\-p 0\.95,\-\-reasoning\-effort max.
• Engine auto-configuration— the engine picks theflashinfer\_mxfp4MoE runner and auto-disables shared-experts fusion for this checkpoint (its HashTopK routing rejects fused shared experts); don’t pass\-\-enforce\-shared\-experts\-fusion.
• Chunked prefill & radix cache stay enabled— the scheduler keeps image spans consistent automatically: chunked-prefill truncation points are span-aligned (an image span always prefills within a single extend, overshooting the chunk budget by at most one span), and a radix-cache prefix match ending deep inside an image span is re-issued from the span start.
• Speculative decoding— the checkpoint bundles a DSpark head, and the low-latency recipes enable it with\-\-speculative\-algorithm DSPARK(verified with image batches on B200 via the MMMU-Pro round; the other hardware rows are pending verification). The balanced and high-throughput recipes run target-only: they use DP Attention, which DSpark is incompatible with on current releases. As on the 0731/0813 checkpoints, do not pass the EAGLE flags.

**Shared experts fusion (Blackwell, flashinfer_mxfp4)**On the Blackwell fp4 recipes (\-\-moe\-runner\-backend flashinfer\_mxfp4), the shared expert runs as a separate FP8 MLP on an alternate stream by default. Adding:

routes it as one extra MXFP4 expert through the same trtllm-gen MoE kernel, so the whole MoE runs on a single stream (~4 fewer kernel launches and 2 fewer stream syncs per MoE layer). The shared expert is requantized from FP8 to MXFP4 at load time. Measured on GB200 tp4: gsm8k and AIME25 accuracy on par with the unfused baseline; Mean TTFT -13% to -21% and P99 ITL -15% to -53% at QPS 1-8 with neutral throughput.Only for deployments without expert parallelism (e.g. the single-node low-latency recipes): withmoe\_ep\_size \> 1the flag is rejected at startup, unless the DeepEP/MegaMOE per-rank shared-slot path is in use. Not applicable toFlash Vision (Exp)— the engine auto-disables the fusion on that checkpoint.Compressed attention state dtypeDeepSeek-V4 uses hybrid compressed attention for long-context efficiency.SGLANG\_DSV4\_COMPRESS\_STATE\_DTYPEcontrols the dtype of the C4 / C128 compressed attention state pools. Supported values arefloat32/fp32(default:float32) andbfloat16/bf16. For BF16 on the offline compression path:

This BF16 setting applies only to the compressed attention state pools and reduces the GPU memory footprint of each compressed-state slot. It does not change model weight precision or the main KV cache dtype. With automatic pool sizing and no explicit capacity cap, the same memory budget holds more slots, and the startup log shows largerc4\_stateandc128\_statepool sizes. Keep the defaultfloat32setting for the most conservative behavior.**EPLB + Waterfill (Experimental)**For recorded/static EPLB reproduction, first record an expert-distribution file by followingCapture expert selection distribution in MoE models. For reproduction runs, use the generatedexpert\_distribution\_recorder\_\*\.ptas the initial expert location.**Please checkout to latest main branch for this feature.**For non-PD reproduction, use:

For PD-Disagg reproduction, usenormalmode on the prefill server andlow\_latencymode on the decode server. Add the same\-\-init\-expert\-locationflag to both commands:

You can also add\-\-ep\-num\-redundant\-expertsand\-\-eplb\-algorithmto customize EPLB placement.Waterfill also supports MegaMOE. Use\-\-moe\-a2a\-backend megamoe \-\-enable\-waterfillto keep the MegaMOE backend while applying Waterfill to the fused shared expert slot.**FP4 Indexer (Experimental)**DeepSeek-V4 uses the default indexer path unless\-\-enable\-deepseek\-v4\-fp4\-indexeris set. Enable this flag to use the experimental FP4 C4 indexer on SM100 GPUs with DeepGEMM FP4 indexer support. This path is intended for decode-heavy long-context workloads where reducing indexer cache bandwidth is beneficial.

NVFP4 Hybrid CheckpointsThenvidia/DeepSeek\-V4\-Pro\-NVFP4andnvidia/DeepSeek\-V4\-Flash\-NVFP4checkpoints quantize MoE experts toNVFP4while keeping attention and dense layers inFP8. The official releases have matching NVFP4 checkpoints atnvidia/DeepSeek\-V4\-Flash\-0731\-NVFP4andnvidia/DeepSeek\-V4\-Pro\-0813\-NVFP4. All of them require\-\-moe\-runner\-backend flashinfer\_trtllm\_routedwhich will be automatically selected if not provided.

or

Requires Blackwell (SM100+). The MTP layer in this checkpoint stays MXFP4-packed and is routed through theMxfp4FlashinferTrtllmMoEMethodpath automatically.The official (0731 / 0813) NVFP4 checkpoints preserve the bundled DSpark draft head, so their low-latency recipes use\-\-speculative\-algorithm DSPARKinstead of the EAGLE/MTP shape flags — same as the corresponding original-precision official checkpoints.Hopper (H100 / H200) noteTwo options are available for running DeepSeek-V4 on Hopper:

• Original FP4 checkpoints— run the MoE experts with W4A16 kernels (Marlin or the FlashInfer SM90 CUTLASS runner) as the command generator picks for Hopper cells. With FlashInfer >= 0.6.18 you can instead select theW4A8path — MXFP4 weights with FP8 activations via FlashInfer’s Humming kernels — by adding\-\-flashinfer\-mxfp4\-moe\-precision fp8to\-\-moe\-runner\-backend flashinfer\_mxfp4; the low-latency Hopper cells now generate this form. Both work on H100 and H200; FP4 is the only option for H100 (no FP8 path). It is TP-only; on H200 the Pro variant fits on a single 8-GPU node, while H100 Pro needs 2 nodes (TP=16).
• Converted FP8 checkpoints(H100 and H200 only) — pre-repackaged FP8 weights atsgl\-project/DeepSeek\-V4\-Flash\-FP8andsgl\-project/DeepSeek\-V4\-Pro\-FP8unlock DP-attention + DeepEP and richer parallelism (e.g. Pro TP=16 across 2 nodes).

On these FP8 checkpoints you can additionally enable the all-FP8MegaMoEpath on SM90 for higher long-context / large-decode throughput — see theSM90 (Hopper) FP8 MegaMoEnote in Configuration Tips below.PD-Disagg recipes on H200 may requiredocker run \-\-privileged \-\-ulimit memlock=\-1(or\-\-device /dev/infiniband:/dev/infiniband \-\-cap\-add IPC\_LOCK) so mooncake can discover the IB HCAs; without IB exposure mooncake silently falls back to TCP, which can lead to garbled KV transfer on large checkpoints.RTX PRO 6000 (SM120 / Blackwell Desktop) noteRTX PRO 6000 (96 GB) runsFlash onlywith the FlashInfer MXFP4 MoE runner. V4-Pro doesn’t fit on 8× 96 GB; the Deploy panel greys out unsupported recipes. HiCache and MegaMoE arenotsupported on RTX PRO 6000.AMD (MI300X / MI355X) note

• Model checkpoints— for correct accuracy, the FP4 model uses the stockdeepseek\-ai/DeepSeek\-V4\-\{Flash,Pro\}, and the FP8 model uses the repackagedsgl\-project/DeepSeek\-V4\-\{Flash,Pro\}\-FP8.
• Supported models—MI300Xsupports DeepSeek-V4-Flash in FP8;MI355Xsupports DeepSeek-V4-Flash / Pro in both FP4 and FP8. All recipes run single-node.
• TP / DP setting— both TP=4 and TP=8 are supported. At low concurrency we recommendTP-only; at high concurrency useTP + DP, which additionally needs\-\-dp 8 \-\-enable\-dp\-attention \-\-enable\-prefill\-delayer \-\-prefill\-delayer\-max\-delay\-ms 5000.
• MTP— speculative decoding is supported; add\-\-speculative\-algorithm EAGLE \-\-speculative\-num\-steps 3 \-\-speculative\-eagle\-topk 1 \-\-speculative\-num\-draft\-tokens 4.
• Kernels— uses the Unified KV attention and the flydsl MoE.

MoRI EP (AMD expert parallelism)On AMD, expert parallelism uses theMoRIall-to-all backend (\-\-moe\-a2a\-backend mori), not DeepEP. Add the flags below on top of the verified recipe when sharding experts across GPUs; set\-\-ep\-sizeto the EP degree (typically the GPU count on one node).Two optional env vars improve MoRI throughput (both off by default):

• FP4: enable both env vars.
• FP8: useSGLANG\_MORI\_RECV\_BOUND=1only; omitSGLANG\_MORI\_DISPATCH\_DTYPE=mxfp8.

MegaMoEMegaMoE fuses expert dispatch + GEMM into a single kernel for higher throughput on MoE layers. To enable it, use theMegaMoEchip in the Playground below — the playground will swap\-\-moe\-a2a\-backend deepepfor\-\-moe\-a2a\-backend megamoeand add the relevant launch settings automatically.Two variants are exposed:

• W4A8— default MegaMoE kernel (FP4 weights, FP8 activations).
• W4A4— adds\-\-enable\-w4a4\-mxfp4\-megamoeto run the custom W4A4 kernel (FP4 activations). The flag configures the required DeepGEMM settings. Higher throughput with negligible accuracy drop (~89.5 GPQA on Pro).

Notes:

• The W4A8 / W4A4 variants above areBlackwell-only(B200 / B300 / GB200 / GB300). On**Hopper (SM90, H100 / H200)**use the all-FP8 MegaMoE path described below instead.
• MegaMoE isonly wired into thehigh\-throughputrecipeon Blackwell (persgl-project/sglang#26451). The chip is hidden onlow\-latencyandbalanced— switch tohigh\-throughputto expose it.
• When running MegaMoE, don’t set\-\-moe\-runner\-backendmanually.
• AdjustSGLANG\_OPT\_DEEPGEMM\_MEGA\_MOE\_NUM\_MAX\_TOKENS\_PER\_RANKbased on your workload and memory usage. Setting higher number of tokens for MegaMoE requires more HBM space (recommended: 8320 for high-throughput).

SM90 (Hopper) FP8 MegaMoE (Experimental)On SM90 (Hopper, H100 / H200), the all-FP8 MegaMoE path routes MoE through the DeepGEMMmega\_moerunner for higher long-context / large-decode throughput on the FP8 checkpoints. Unlike the Blackwell W4A8 / W4A4 variants above, experts stay inFP8— keepSGLANG\_DSV4\_FP4\_EXPERTS=0. It requires asgl\-deep\-gemmbuild with SM90 FP8 MegaMoE support.**Please use the latest image for this feature.**Enable the MegaMoE path with\-\-moe\-a2a\-backend megamoe

SGLANG\_OPT\_DEEPGEMM\_MEGA\_MOE\_NUM\_MAX\_TOKENS\_PER\_RANKcaps the number of tokens the MegaMoE path processes per rank (i.e. per GPU); the MegaMoE path is only used for batches at or below this cap. The right value depends on your parallelism / token-split scheme, and larger values reserve more HBM.GB300 PD-Disagg cross-pod MNNVLOn some GB300 clusters with cross-pod KV transfer over NVLink, mooncake may fail withnvlink\_transport\.cpp:497 Requested address \.\.\. not found\!. If this happens, prependMC\_FORCE\_MNNVL=1 NCCL\_MNNVL\_ENABLE=1 NCCL\_CUMEM\_ENABLE=1to both prefill and decodesglang servecommands.

3. Advanced Usage

3.1 Reasoning

Enable thedeepseek\-v4reasoning parser (toggleReasoning Parserin theParserscard of thePlayground above) to separate thinking from the final answer intoreasoning\_contentvscontent.

Streaming with Thinking Process (Python)

Example Output

3.2 Tool Calling

Enable thedeepseekv4tool-call parser (toggleTool Call Parserin theParserscard of thePlayground above) to surface structured tool calls viamessage\.tool\_calls.

Python Example with Thinking Process

Example Output

3.3 HiCache (Hierarchical KV Caching)

HiCache enables multi-tier KV cache offloading (GPU → CPU → Storage), significantly expanding effective context capacity for long-context and multi-turn scenarios. Combined with UnifiedRadixTree, it provides intelligent prefix caching across all tiers.To enable HiCache, open theHiCachecard in thePlayground aboveand flipEnable:

• L2 (GPU + CPU)— leave Storage onauto(default). Cold KV pages spill to CPU pinned memory only.
• L3 (GPU + CPU + Storage)— pick a Storage backend (file/mooncake/hf3fs/nixl); the Playground emits the canonicalpage\_first\_directmem-layout +directIO backend +wait\_completeprefetch policy, matching theHiCache best-practices recipe.

For AMD devices,

• L2 (GPU + CPU)— leave Storage onauto(default). Cold KV pages spill to CPU pinned memory only. UsedirectIO backend +page\_first\_directorlayer\-firstmem-layout.
• L3 (GPU + CPU + Storage)— pick a Storage backend (file); the Playground emits the canonicalpage\_first\_directmem-layout +directIO backend +wait\_completeprefetch policy, matching theHiCache best-practices recipe.

The Write policy knob defaults towrite\_through(the upstream default); switch towrite\_back/write\_through\_selectiveto trade durability for write speed when the storage tier is slow.For more details, see theHiCache documentation.

3.4 DSpark (Speculative Decoding)

Flash Official (0731) and Pro Official (0813) bundle a DSpark draft head indeepseek\-ai/DeepSeek\-V4\-Flash\-0731anddeepseek\-ai/DeepSeek\-V4\-Pro\-0813. The target and draft weights therefore come from the same checkpoint: enable DSpark with\-\-speculative\-algorithm DSPARKand do not set a separate\-\-speculative\-draft\-model\-path.The experimentalFlash Vision checkpointalso bundles a DSpark head, enabled the same way: the Flash Vision low-latency recipes ship with\-\-speculative\-algorithm DSPARK(verified with image batches on B200 via the MMMU-Pro round; other hardware rows pending). The balanced and high-throughput Flash Vision recipes stay target-only because they run DP Attention.Unlike the EAGLE recipes for the original Flash and Pro checkpoints, this recipe omits\-\-speculative\-num\-steps,\-\-speculative\-eagle\-topk, and\-\-speculative\-num\-draft\-tokens. SGLang reads the DSpark shape from the checkpoint.

The verified 4×GB300 FP4 low-latency command is:

Keep\-\-mem\-fraction\-static 0\.90on this topology to leave enough headroom for the batch-256 verify graph. The first cold start can take 10–15 minutes while FlashInfer autotunes and SGLang captures the draft and verify graphs; later starts reuse the cache. This path is verified end-to-end on 4×GB300 with SGLang v0.5.16.Tune proposed draft tokens.\-\-speculative\-dspark\-block\-size Nasks DSpark to proposeNtokens per step; the target verifies a window ofN \+ 1. If the flag is omitted, SGLang reads the value from the checkpoint. Both the 0731 and 0813 checkpoints resolve to five proposed tokens (the startup log reportsgamma=5, verify\_num\_draft\_tokens=6), which is the verified default. Use theDSpark Proposed Draft Tokensslider in thePlaygroundto sweep one through five.Larger blocks can improve decode latency when acceptance stays high, but they also increase verification work and graph memory. Start from the checkpoint default, then sweep downward under the real prompt-length and concurrency distribution. The gain is usually largest for short interactive traffic and narrows as prefill dominates. Track P50/P99 TTFT and TPOT, total throughput, accepted length, GPU memory, and stop rate rather than choosing from acceptance alone.For every candidate, compare with the same recipe without\-\-speculative\-algorithm DSPARK. Restart the server between the DSpark and non-speculative legs, keep the request corpus, sampling, concurrency, and warmup identical, and give eachbench\_servingleg its own\-\-flush\-cache. Leave\-\-speculative\-draft\-attention\-backendunset unless a separate profiling run justifies an override.DSpark currently requires CUDA,pp\_size == 1, and DP Attention disabled. It is not compatible with PD disaggregation on current SGLang releases; selecting a prefill or decode role in the Playground automatically removes the inherited DSpark flags. The DP-Attention and MI355X Flash Official recipes therefore run target-only. If a larger draft block or concurrency causes graph-capture OOM, lower\-\-mem\-fraction\-static, the draft block size, or the configured maximum running requests, then rerun both performance and accuracy gates.

3.5 Vision (Image Inputs)

The experimentalDeepSeek\-V4\-Flash\-Vision\-Expcheckpoint (theFlash Visionvariant in the Deploy panel — see theconfiguration notes) takes images via the OpenAI-compatibleimage\_urlcontent type, as public URLs or base64data:URIs; text and images mix freely in one message. Vision input works with the same server the Deploy panel produces — no extra model-specific flags needed.

Image Understanding (Python)

Example Output

SGLang (@sgl_project): Congrats @deepseek_ai on releasing the weights for DeepSeek-V4-Flash-Vision-Exp!

HF link: https://t.co/1966Yy9I3t

We’re working on support now, with an updated cookbook coming ASAP!

Welcome to a new era of native vision from DeepSeek 🐋👀