vmlinux/Qwen3.5-122B-A10B-Heretic-ROCmFP4-iMatrix-GGUF · Hugging Face

Source: https://huggingface.co/vmlinux/Qwen3.5-122B-A10B-Heretic-ROCmFP4-iMatrix-GGUF

https://huggingface.co/vmlinux/Qwen3.5-122B-A10B-Heretic-ROCmFP4-iMatrix-GGUF#qwen35-122b-a10b-heretic-%C2%B7-rocmfp4-imatrixQwen3.5 122B-A10B Heretic · ROCmFP4 iMatrix

https://huggingface.co/vmlinux/Qwen3.5-122B-A10B-Heretic-ROCmFP4-iMatrix-GGUF#a-compact-importance-calibrated-rocmfp4-build-for-high-memory-amd-systemsA compact, importance-calibrated ROCmFP4 build for high-memory AMD systems

122B total · 10B active · 60.70 GiB · 28.45 tok/s native · 33.68 tok/s MTP · BF16 KLD 0.100716 · PP 353.3 W/MTP 328.6 t/s

This GGUF uses custom ROCmFP4 tensor types. It requiresROCmFPXor a runtime with equivalent support; stockllama\.cpp, Ollama, LM Studio, and similar stock runtimes cannot load it.

https://huggingface.co/vmlinux/Qwen3.5-122B-A10B-Heretic-ROCmFP4-iMatrix-GGUF#downloadsDownloads

Browse every repository file →

https://huggingface.co/vmlinux/Qwen3.5-122B-A10B-Heretic-ROCmFP4-iMatrix-GGUF#what-it-isWhat it is

This is an iMatrix-calibrated quantization of the full-weight Qwen3.5 122B-A10B Heretic checkpoint. It was built with ROCmFPX’s compactQ4\_0\_ROCMFP4\_STRIX\_LEANrecipe, but the hardware-oriented preset name has been left out of the public filename.

ResultBF16 mean KLD0.100716 ± 0.003407Two-sigma KLD upper bound0.107530Improvement from iMatrix14.01% lower KLDGreedy decode28.448 tok/sGreedy decode with MTP33.677 tok/s4,277-token prefill****350.248 tok/s The main model is complete and runs independently. The 23-tensor, one-layer MTP file is an optional external companion; no MTP layer is embedded in the main GGUF. Runtimes call this role adraft model, butDraftis omitted from the public filename because it does not mean unfinished.

https://huggingface.co/vmlinux/Qwen3.5-122B-A10B-Heretic-ROCmFP4-iMatrix-GGUF#qualityQuality

The calibrated model and its uncalibrated control were replayed against the same saved distributions from the exact BF16 parent on ROCm0.

BuildMean KLDPPL ratioSame topRMS ΔpiMatrix0.100716 ± 0.0034071.080217 ± 0.010198****87.629% ± 0.622%****10.881%Uncalibrated0.117119 ± 0.0041831.109251 ± 0.01129986.774% ± 0.640%12.077% iMatrix calibration reduced mean KLD by14.01%, reduced the PPL ratio by 0.029034, and improved same-top agreement by 0.855 percentage points. The two-sigma KLD upper bound is0.107530, clearing the selected0.126performance-oriented target.

The paired replay covers 11 complete 512-token chunks and 2,805 evaluated next-token distributions. These were recovered at an exact probability-block boundary from an interrupted 32-chunk BF16 run. This is useful preliminary evidence, not a completed 32-chunk acceptance gate.

https://huggingface.co/vmlinux/Qwen3.5-122B-A10B-Heretic-ROCmFP4-iMatrix-GGUF#performancePerformance

Measured on a 128 GB AMD Strix Halo system with Vulkan/RADV, 131,072 context, parallel 1, batch/ubatch 2048/1024, Q8_0 KV cache, flash attention, and MTP off. Values are medians from repeated runs.

WorkloadRepeatsDecodePrompt processingGreedy, 256 generated tokens528.448 tok/s71.929 tok/sSampled, 256 generated tokens328.418 tok/s71.294 tok/s4,277-token prefill + 128 generated328.041 tok/s****350.248 tok/s Relative to the otherwise matched uncalibrated build, decode improved by 2.43%, 5.23%, and 2.84% respectively. Long-prompt prefill changed by -1.03%. These are single-system measurements, not general performance guarantees.

https://huggingface.co/vmlinux/Qwen3.5-122B-A10B-Heretic-ROCmFP4-iMatrix-GGUF#optional-mtp-accelerationOptional MTP acceleration

The external MTP companion was tested against this exact iMatrix trunk in a matched MTP-off/on A/B. The model, runtime, context, cache, batch geometry, prompts, and companion identity were held constant; only MTP execution changed.

WorkloadNative decodeMTP decodeDecode gainAcceptanceEnd-to-end gainGreedy, 256 tokens28.328 tok/s**33.677 tok/s****+18.89%85.03%+16.44%Sampled, 256 tokens28.035 tok/s30.953 tok/s****+10.41%80.00%+8.76%4,277-token prefill + 12827.585 tok/s33.752 tok/s****+22.36%**90.73%-0.45% MTP reduced prompt-processing throughput by about 7% because the companion must catch up on the prompt. It therefore helps most when generation is long enough to repay that fixed cost. The exact identity probe was unchanged, and both the MTP-off and MTP-on lanes passed technical prose, executable code semantics, strict JSON, tool-calling, balanced-analysis, and defensive-safety smokes.

https://huggingface.co/vmlinux/Qwen3.5-122B-A10B-Heretic-ROCmFP4-iMatrix-GGUF#quantizationQuantization

The model was quantized once from the validated BF16 GGUF using a 339-chunk code/technical importance matrix and theQ4\_0\_ROCMFP4\_STRIX\_LEANpreset.

Tensor typeCountQ4\_0\_ROCMFP4\_FAST457Q4\_0\_ROCMFP460F32361Q5\_K1 In this preset, attention K/V tensors retain dual-scale ROCmFP4 protection, most transformer weights use the compact FAST layout, and token embeddings/output use Q5_K. The artifact contains 879 tensors across 48 blocks and no embedded MTP layer.

https://huggingface.co/vmlinux/Qwen3.5-122B-A10B-Heretic-ROCmFP4-iMatrix-GGUF#runRun

llama-server \
  --model Qwen3.5-122B-A10B-Heretic-ROCmFP4-iMatrix.gguf \
  --host 127.0.0.1 --port 8080 \
  -dev Vulkan0 --n-gpu-layers 999 \
  --ctx-size 131072 --parallel 1 \
  --flash-attn on --batch-size 2048 --ubatch-size 1024 \
  --cache-type-k q8_0 --cache-type-v q8_0 \
  --jinja --reasoning-format deepseek

Adjust context and cache settings for your memory budget. The tested runtime wascharlie12345/ROCmFPXcommita6a93765f7ce9779c13f9881164a65f7a9f31198, built in Release mode forgfx1151with Vulkan and HIP enabled, plus a local Qwen3.5 MoE graph correction. Results with newer ROCmFPX revisions have not yet been established.

To enable the optional companion, append:

--spec-type draft-mtp \
  --spec-draft-model Qwen3.5-122B-A10B-Heretic-ROCmFP4-MTP.gguf \
  --spec-draft-device Vulkan0 --spec-draft-ngl 999 \
  --spec-draft-type-k f16 --spec-draft-type-v f16 \
  --spec-draft-n-max 2 --spec-draft-p-min 0.6 \
  --spec-draft-p-split 0.10 --spec-draft-backend-sampling

https://huggingface.co/vmlinux/Qwen3.5-122B-A10B-Heretic-ROCmFP4-iMatrix-GGUF#tested-systemTested system

PlatformAMD Strix Halo, 128 GB unified memoryGPU targetRadeon 8060S /gfx1151BackendVulkan / RADVKernelLinux 6.17.0-1028-oemMesa25.2.8 No vision projector is included; treat this release as text-only.

https://huggingface.co/vmlinux/Qwen3.5-122B-A10B-Heretic-ROCmFP4-iMatrix-GGUF#lineage-and-creditsLineage and credits

This repository exists because several people published substantial work openly. Their roles are kept distinct so downstream users can find the original sources.

• Official base model:Qwen/Qwen3.5-122B-A10B, Apache 2.0.
• Full-weight Heretic source:trohrbaugh/Qwen3.5-122B-A10B-heretic, accessed through the pinnedCCSSNE mirroratfd6d6bc89e9be47577736c952dd4d1d7c830f09f.
• MTP tensor source:OptimizeLLM/Qwen3.5-122B-A10B-heretic-MTP-NVFP4at07b7c2101c33f23a84fd3c618d00f17ea12269c3; its model card reports that the MTP tensors were restored from the official Qwen checkpoint.
• Heretic software and method:p-e-w/heretic, created by Philipp Emanuel Weidmann. The source checkpoint reports Heretic v1.2.0.
• Related full-weight Heretic reference:scrunter/Qwen3-VL-235B-A22B-Thinking-heretic. This related 235B model is not the tensor source for this release.
• ROCmFP4 implementation, quantizer, and runtime:charlie12345/ROCmFPX.
• Strix Halo deployment stack:hec-ovi/llama-vulkan-strix.
• Release inspiration:plunderstruck/Qwen3.6-27B-MTP-ROCmFP4-GGUF.
• Conversion, validation, benchmarking, and packaging:vmlinux.

Please preserve this lineage, the Apache 2.0 license, and a description of your changes when redistributing derivatives.

https://huggingface.co/vmlinux/Qwen3.5-122B-A10B-Heretic-ROCmFP4-iMatrix-GGUF#licenseLicense

The distributed model derivative is provided under the upstream Apache License 2.0. The includedLICENSEis copied from the pinned full-weight source. Runtime and tooling repositories retain their own licenses: Heretic software is AGPL-3.0, while ROCmFPX and the deployment tooling are separately licensed. No runtime source code is bundled in this model repository.