pottokao/Qwen-Image-2.1-Text-Encoder-Heretic-GGUF · Hugging Face

Source: https://huggingface.co/pottokao/Qwen-Image-2.1-Text-Encoder-Heretic-GGUF

https://huggingface.co/pottokao/Qwen-Image-2.1-Text-Encoder-Heretic-GGUF#qwen-image-21-text-encoder-heretic–gguf-%C2%B7-fp8-%C2%B7-bf16Qwen-Image-2.1 Text Encoder (Heretic) — GGUF · FP8 · bf16

Gotgot input of size \[1, 512, 12288\]with the GGUF? It’s fixed.The GGUF files were fine all along — ComfyUI-GGUF just didn’t load the Qwen3-VL vision tower. Install the add-on nodeComfyUI-GGUF-Qwen3VL-TE, make sure themmprojfile sits next to the GGUF, restart ComfyUI. No need to re-download anything. The same patch also fixesUnknown model architecture\!for Qwen-Image-2.1DiTGGUFs — so the whole pipeline can run on GGUF.

https://huggingface.co/pottokao/Qwen-Image-2.1-Text-Encoder-Heretic-GGUF#made-with-a-simple-nodeMade with a simple node

The sample / showcase images in this repo were generated with the technique behind**ComfyUI-QwenImage-PhotoStyles**— a small ComfyUI node (17 photographic styles; one short prompt becomes a full styled prompt via the PE-T2I rewriter). If you’re interested in how the images are made, the node and its recipe are there.

https://huggingface.co/pottokao/Qwen-Image-2.1-Text-Encoder-Heretic-GGUF#%F0%9F%93%A5-which-file-should-i-use📥 Which file should I use?

Your setupFile(s)SizeLoaderSmallest(GGUF)qwen3vl\_8b\_heretic\-Q4\_K\_M\.gguf+mmproj\-qwen3vl\_8b\_heretic\-f16\.gguf5.0 + 1.2 GBCLIPLoaderGGUF+add-on nodeNVIDIA GPU, no extra nodesqwen3vl\_8b\_fp8\_heretic\.safetensors9.3 GBstockCLIPLoaderMac / non-CUDA, no extra nodesqwen3vl\_8b\_bf16\_heretic\.safetensors17.5 GBstockCLIPLoader All three are inthis repo. Every loader uses**typeqwen\_image**and feedsTextEncodeQwenImage21.

https://huggingface.co/pottokao/Qwen-Image-2.1-Text-Encoder-Heretic-GGUF#%E2%9A%99%EF%B8%8F-recommended-sampler-settings⚙️ Recommended sampler settings

Qwen-Image-2.1 is a full (non-distilled) model.Use 25 steps— more steps don’t make text sharper and start to burn the highlights.

• **No text in the image:**plainKSampler,cfg 1.0(fastest, softest look; the negative prompt is ignored at cfg 1).
• Text in the image (signs, labels, posters):switch cfgpartway through— the first ½–⅔ of the steps atcfg 1lock in composition and materials, the rest atcfg 3with a negative prompt redraws the lettering. You get cfg 1’s lookandcfg 3’s readable text, in one pass with one seed.

In ComfyUI that’s two chainedKSamplerAdvancednodes (same model, seed and steps):

add\_noise``steps``cfg``start\_at\_step``end\_at\_step``return\_with\_leftover\_noise1stenable251.0012–17enable2nddisable253.012–1725disable Split later (17) keeps more of the cfg 1 look; split earlier (12) gives crisper text. Example negative prompt:oversaturated, overexposed, gibberish text.

https://huggingface.co/pottokao/Qwen-Image-2.1-Text-Encoder-Heretic-GGUF#%F0%9F%8D%B1-all-gguf-qwen-image-21🍱 All-GGUF Qwen-Image-2.1

PartWhereLoaderPrompt rewriter*(optional)*PE-T2I Heretic GGUFllama.cpp / LLM nodeText encoderthis repo— Q4_K_M + mmprojCLIPLoaderGGUF+patch nodeDiTQwen-Image-2.1 DiT GGUF— Q8_0 / Q6_K / Q4_K_MUnetLoaderGGUF—stock, no patchVAEofficialqwen\_image\_2\.1\_vae\_bf16\.safetensors``VAELoader

https://huggingface.co/pottokao/Qwen-Image-2.1-Text-Encoder-Heretic-GGUF#%F0%9F%96%BC%EF%B8%8F-showcase🖼️ Showcase

Juan I-Jong style Henri Cartier-Bresson style William Eggleston style Daido Moriyama style Fan Ho style Gregory Crewdson style Thomas Struth style Richard Avedon style Irving Penn style Peter Lindbergh style Saul Leiter style Shōmei Tōmatsu style Ko Si-chi style Chen Man style Lee Friedlander style Hiroh Kikai style Masahisa Fukase style Tim Walker style Helmut Newton style Helmut Newton style Chen Man style Tim Walker style Chen Man style Richard Avedon styleThis is what Q4_K_M across the board looks like— every image above was made with:

AI-generated. Style homage only — not affiliated with or endorsed by the photographers.

https://huggingface.co/pottokao/Qwen-Image-2.1-Text-Encoder-Heretic-GGUF#gguf-in-comfyui–3-stepsGGUF in ComfyUI — 3 steps

1. Install two custom nodesintoComfyUI/custom\_nodes/: cd ComfyUI/custom_nodes git clone https://github.com/pottokao-dotcom/ComfyUI-GGUF-Qwen3VL-TE No git? On the GitHub page clickCode → Download ZIP, unzip it intoComfyUI/custom\_nodes/. New to custom nodes:official guide. The patch fixes both errors: the text encoder’s\[1, 512, 12288\], andUnknown model architecture\!when a Qwen-Image-2.1DiTGGUF has no architecture metadata (e.g.unsloth/leejetbuilds).
2. Put both files inComfyUI/models/text\_encoders/ - qwen3vl\_8b\_heretic\-Q4\_K\_M\.gguf - mmproj\-qwen3vl\_8b\_heretic\-f16\.gguf← the vision tower (required— also used for image editing) Don’t rename either file— the vision tower is matched by name. If it’s missing, the add-on stops with aMissing vision towererror telling you which file to download.
3. Restart ComfyUI, thenCLIPLoaderGGUF→ type**qwen\_image**→TextEncodeQwenImage21→ the rest of the official workflow. The console should show\[GGUF\-Qwen3VL\-TE\] added 351 Qwen3\-VL vision tensors from mmproj\.

Verified 2026-09-23 on ComfyUI 0.36.0 + ComfyUI-GGUF6ea2651(NVIDIA GPU):text-to-image and reference-image editingboth work and match the bf16 encoder’s output for the same seed up to Q4 quantization noise. Not tested on a Mac.

What the patch changes, and why: seethe bottom of this page.

https://huggingface.co/pottokao/Qwen-Image-2.1-Text-Encoder-Heretic-GGUF#files-in-this-repoFiles in this repo

FileSizeUse forqwen3vl\_8b\_heretic\-Q4\_K\_M\.gguf5.03 GBComfyUI viaCLIPLoaderGGUF+ add-on · llama.cppmmproj\-qwen3vl\_8b\_heretic\-f16\.gguf1.16 GBvision tower —requirednext to the GGUFqwen3vl\_8b\_fp8\_heretic\.safetensors9.34 GBComfyUI via stockCLIPLoader(NVIDIA GPU)qwen3vl\_8b\_bf16\_heretic\.safetensors17.53 GBComfyUI via stockCLIPLoader(any device, incl. Mac) — full precision

https://huggingface.co/pottokao/Qwen-Image-2.1-Text-Encoder-Heretic-GGUF#other-formats-of-the-same-modelOther formats of the same model

RepoFormatSizeNote…\-Hereticbf1617.5 GBHFtransformersshards (+ the same bf16 single file)…\-int8\-convrotINT8 convrot9.35 GBofficial template default format, CUDA…\-W4A8W4A86.31 GBCUDA…\-NVFP4NVFP46.31 GBBlackwell GPUs

https://huggingface.co/pottokao/Qwen-Image-2.1-Text-Encoder-Heretic-GGUF#ablation-inherited-from-the-bf16-sourceAblation (inherited from the bf16 source)

RefusalsKL divergenceStock Qwen-Image-2.1 text encoder100/1000*(by definition)This family5/100***0.0220** Produced withHereticdirectional ablation (o\_proj+down\_proj), 200 trials / 60 startup trials, knee point of the Pareto front. Independently re-checked on the bf16 source:0/20refusals,4/4benign questions answered correctly. Full methodology in thebf16 repo.

https://huggingface.co/pottokao/Qwen-Image-2.1-Text-Encoder-Heretic-GGUF#format-detailsFormat details

**GGUF:**Q4_K_M language model, vision tower as a separate f16mmproj.

**FP8:**self-quantizedfloat8\_e4m3fn(Comfy-Org doesn’t ship an FP8 encoder):

LayersCountPrecisionFFN + attention + embed/lm_head (all 2-D weights)254FP8e4m3fnVision tower351 tensorsbf16 — untouchednorms / biases—bf16 Remapped to the ComfyUI key layout (model\.layers\.…, nolanguage\_model\.prefix).

Requires a ComfyUI build withQwenImage21support (0.36.0 or newer).

https://huggingface.co/pottokao/Qwen-Image-2.1-Text-Encoder-Heretic-GGUF#%F0%9F%8E%A8-also-for-qwen-image-21-the-prompt-rewriters-heretic🎨 Also for Qwen-Image-2.1: the prompt rewriters (Heretic)

Qwen-Image-2.1 ships twoprompt rewriters— small LLMs that expand a short request into the detailed prompt the model was trained on. Both are available refusal-ablated, and both haveGGUF builds that run anywhere llama.cpp runs (Mac included):

RewriterWhat it doesBuildsPE-T2I(text-to-image)one line in any language → detailed English prompt + aspect ratioGGUF·NVFP4·bf16PE-I2I(image editing)vague edit instruction + input image(s) → precise editing promptGGUF + mmproj A short request goes in; one JSON line comes out (real output of the PE-T2I GGUF, trimmed):

in:  雨夜的東京小巷，一隻流浪狗，森山大道風格
out: {"rewritten_prompt": "A vertical nighttime street photograph in the style of Daido Moriya shows a
      narrow, rain-soaked urban alley with a stray dog standing in the lower foreground. The scene is
      composed with strong depth: dark building walls and utility poles frame the left and right sides,
      while the wet pavement stretches into a softly blurred background. … (2,137 characters)",
      "wh_ratio": "2:3"}

Showcase— generated with theNVFP4builds through the full pipeline (PE-T2I rewriter → Qwen-Image-2.1 text encoder → DiT), each from a one-line prompt, one seed per image, no retouching. Also shown in thePE-T2I NVFP4 repo.

## https://huggingface.co/pottokao/Qwen-Image-2.1-Text-Encoder-Heretic-GGUF#what-the-patch-handlesWhat the patch handles

ComfyUI-GGUF-Qwen3VL-TEchanges nothing on disk — it only adjusts how ComfyUI-GGUF loads Qwen-Image-2.1 files, in memory, at load time.

1. Text encoder —\[1, 512, 12288\]

RuntimeError: Given normalized_shape=[4096], expected input with shape [*, 4096],
but got input of size [1, 512, 12288]

ComfyUI-GGUF loads themmprojvision tower only forqwen2vl, notqwen3vl. Without it, ComfyUI doesn’t recognise the encoder as Qwen3-VL, builds the wrong model, and gets 12288-wide hidden states instead of 4096. The patch loads the matchingmmproj\-\*\.gguffrom the same folder and renames its tensors to ComfyUI’s Qwen3-VL layout (model\.visual\.\*). The vision tower is then really used — reference images for editing go through it.

2. DiT —Unknown model architecture\!

ValueError: This model is not currently supported - (Unknown model architecture!)

A DiT GGUF withoutgeneral\.architecturemetadata (stable-diffusion.cpp convention, e.g. unsloth / leejet) is identified by its tensor names, and ComfyUI-GGUF’s list has no Qwen-Image entry. The patch recognises Qwen-Image-2.1 byimg\_in,txt\_in\.in\_layerandtxt\_in\.text\_normand loads it asqwen\_image. DiT GGUFs that already carry the metadata (e.g. Abiray’s) are unaffected.

Once ComfyUI-GGUF handles either case itself, that part of the patch detects it and does nothing.

Not affiliated with, or endorsed by, Alibaba / Qwen.Community derivative (refusal-ablated) ofQwen/Qwen3\-VL\-8B\-Instruct— the model Qwen-Image-2.1 uses, unmodified, as its text encoder. Qwen releases that model underApache-2.0, so this derivative is redistributed under Apache-2.0 (seeLICENSEandNOTICE).