garnermccloud/Qwen3.8-Flash-Next-NVFP4-SSD-Stream · Hugging Face

Source: https://huggingface.co/garnermccloud/Qwen3.8-Flash-Next-NVFP4-SSD-Stream

https://huggingface.co/garnermccloud/Qwen3.8-Flash-Next-NVFP4-SSD-Stream#qwen38-flash-next-nvfp4—ssd-streamQwen3.8 Flash-Next NVFP4 - SSD Stream

Run Qwen3.8 Flash-Next with 48 GB less RAM. Keep the speed.

This isRadixArk/Qwen3\.8\-Flash\-Next\-NVFP4, prepared forsglang\-ssd\-stream. RadixArk produced the NVFP4 checkpoint from Qwen3.8 Flash-Next; this layout loads its 48 GB lookup table from a local SSD instead of occupying system RAM. The required data is fetched while the GPU is already working.

https://huggingface.co/garnermccloud/Qwen3.8-Flash-Next-NVFP4-SSD-Stream#what-this-model-isWhat this model is

The source checkpoint uses ModelOpt NVFP4 W4A4 for the routed experts. Its attention, shared experts, embeddings, LM head, vision components, and MTP weights remain BF16; the predictive lookup table is FP8.

This prepared checkpoint does not retrain, fine-tune, or requantize the model. It makes one serving-oriented layout change:

1. The 128 FP8 lookup-table shards are removed from the normal model load.
2. Their rows are stored in one read-only 51,200,245,760-byte SSD sidecar.
3. ssd\-stream\.jsonrecords its exact shape, source revision, and SHA-256.
4. The remaining model and native MTP weights are preserved from the source.

Sidecar SHA-256:b070f9644adf93794d8a1030584ab705809387e64396a9327a68fa3a3a6666b3

https://huggingface.co/garnermccloud/Qwen3.8-Flash-Next-NVFP4-SSD-Stream#performancePerformance

https://huggingface.co/garnermccloud/Qwen3.8-Flash-Next-NVFP4-SSD-Stream#rtx-pro-6000-blackwellRTX PRO 6000 Blackwell

Matched tests on one RTX PRO 6000 Blackwell used native MTP, CUDA graphs, and a 1,024-token completion:

RTX PRO 6000 resultValueGPU memory96 GBNormal lookup-table RAM47.68 GiBSSD Stream working RAMAbout 64 MiBRAM returnedAbout 47.6 GiBNormal RAM-loaded speed148.5-156.2 tok/sSSD Stream speed164.7 tok/s The measured result is the point of SSD Stream: removing the table from RAM did not make it the decode bottleneck. Content-heavy requests dominated by unseen, random lookup rows measured 126-137 tok/s.

The RTX acceptance suite also passed structured tool calls, unrelated images, a 120,043-token retrieval request, native MTP, CUDA graph replay, restart reuse, and a sustained alternating workload without swap, OOM, restart, or host-memory growth.

https://huggingface.co/garnermccloud/Qwen3.8-Flash-Next-NVFP4-SSD-Stream#dgx-sparkDGX Spark

DGX Spark resultValueUnified memory128 GBNormal lookup-table memory47.68 GiBSSD Stream working memoryAbout 64 MiBUnified memory returnedAbout 47.6 GiBNormal RAM-loaded speedPending hardware runSSD Stream speedPending hardware run The native reader supports Linux aarch64. After the Spark benchmark fills in these speed rows, the same CLI will detect DGX Spark and select its tested memory and context profile automatically.

https://huggingface.co/garnermccloud/Qwen3.8-Flash-Next-NVFP4-SSD-Stream#run-itRun it

Use Linux with a Blackwell GPU and a local SSD:

curl -LsSf https://raw.githubusercontent.com/garnermccloud/sglang-ssd-stream/main/install.sh | sh
~/.local/bin/sglang-ssd-stream serve

The first launch downloads and verifies this snapshot, then starts SGLang’s OpenAI-compatible API athttp://127\.0\.0\.1:30000/v1.

curl -s http://127.0.0.1:30000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "Qwen3.8-Flash-Next-NVFP4-SSD-Stream",
    "messages": [{"role": "user", "content": "Reply with: SSD Stream works"}],
    "max_tokens": 32
  }'

The validated automatic profile is for a single RTX PRO 6000 Blackwell.

https://huggingface.co/garnermccloud/Qwen3.8-Flash-Next-NVFP4-SSD-Stream#model-capabilitiesModel capabilities

• Text, image, and video input with text output
• Reasoning and structured tool calling
• Native MTP speculative decoding
• Up to 262K context in the source architecture
• OpenAI-compatible chat-completions API through SGLang

https://huggingface.co/garnermccloud/Qwen3.8-Flash-Next-NVFP4-SSD-Stream#why-hugging-face-shows-68b-parameters-and-8-bit-precisionWhy Hugging Face shows 68B parameters and 8-bit precision

Those automatic labels describe visible storage tensors, not this model’s logical architecture or its primary weight precision.

• The 68B figure counts about 60.4B packeduint8storage elements plus 8.0B BF16 elements in the Safetensors files. Each packed byte can hold two NVFP4 values.
• The 51.2B FP8 lookup-table elements live in the SSD sidecar, so Hugging Face’s Safetensors scanner does not count them.
• The8\-bitlabel comes from those storage dtypes. The routed-expert weights are NVFP4 W4A4; selected model components remain BF16; the streamed table is FP8.

The source model describes the architecture as approximately 180B logical parameters in total. The 68B header is therefore not a comparable parameter count.

https://huggingface.co/garnermccloud/Qwen3.8-Flash-Next-NVFP4-SSD-Stream#technical-detailsTechnical details

The model performs 16 deterministic table lookups per generated token. Each FP8 row is 160 bytes. SSD Stream starts those reads before the model reaches the block that consumes them, allowing SSD I/O to overlap GPU computation.

The extension:

1. copies row IDs asynchronously into pinned host memory;
2. deduplicates the exact 4 KiB filesystem pages required by those rows;
3. submits concurrent reads through a Rustio\_uringengine;
4. restores rows in their requested order;
5. converts staged FP8 data on a separate CUDA stream; and
6. synchronizes only when a read outlasts the overlapping GPU work.

It reserves a 32 MiB registered page pool and two 16 MiB pinned staging buffers. It does not reserve a multi-gigabyte private cache. Native MTP, multimodal input, structured tools, and CUDA graph replay remain enabled.

sglang\-ssd\-streamis an independent extension for SGLang. It is not an official SGLang component.

https://huggingface.co/garnermccloud/Qwen3.8-Flash-Next-NVFP4-SSD-Stream#license-and-attributionLicense and attribution

The model weights retain the source model’s license and terms. See:

• Qwen/Qwen3\.8\-Flash\-Next
• RadixArk/Qwen3\.8\-Flash\-Next\-NVFP4
• sglang\-ssd\-stream, Apache-2.0