Running UNCENSORED DeepSeek-V4.1-Flash-1017 on a 12 Gb VRAM
Summary
AtomicChat released GSQ-RCO GGUF quantizations of DeepSeek-V4.1-Flash-1017 with refusal directions projected out of the dense residual writers, letting a 763B MoE model run on a single 12 GB GPU via Strata with minimal MMLU degradation.
View Cached Full Text
Cached at: 11/02/26, 06:08 PM
AtomicChat/DeepSeek-V4.1-Flash-1017-Uncensored-GSQ-RCO-GGUF · Hugging Face
Source: https://huggingface.co/AtomicChat/DeepSeek-V4.1-Flash-1017-Uncensored-GSQ-RCO-GGUF
https://huggingface.co/AtomicChat/DeepSeek-V4.1-Flash-1017-Uncensored-GSQ-RCO-GGUF#deepseek-v41-flash-1017-without-refusals-%C2%B7-gsq-rcoDeepSeek-V4.1-Flash-1017 Without Refusals · GSQ-RCO
ISTA-DASLab’s GSQ-RCO quantizations of DeepSeek-V4.1-Flash-1017 (763B, 384 experts) with DeepSeek’s refusal direction projected out of the dense residual writers — the same edit asAtomicChat/DeepSeek-V4.1-Flash-1017-Uncensored-GGUF, measured on every rung.
- Built by the team behindAtomic Chat, the app that runs open-weight models locally and offline.
- These areISTA-DASLab/DeepSeek-V4.1-Flash-1017-GSQ-RCO-GGUF’s GSQ-RCO backbone shards (revision
cb02422c900d) withonly the residual-writing dense tensors replaced: attention output (attn\_output\_b), the shared-expert down projection (ffn\_down\_shexp) and the Engram value projection (engram\_wkv). Therouted experts are left exactly as ISTA shipped them, so GSQ’s learned low-bit rounding of the experts is preserved byte for byte, and each file stays the same size and runs wherever the original runs. - The replaced tensors carry the one-direction refusal edit (strength 2.0) baked into DeepSeek’s weights at full precision and quantized to ISTA’s exact per-tensor type.
- Runs on asingle 12 GB GPUwithStrata: the backbone sits in system memory and the sharedEngram table streams from an SSD, so a 763B model fits a consumer machine.
Not every rung cleared every gate; such a rung is published as measured, with its numbers in the table and the metrics repo.
https://huggingface.co/AtomicChat/DeepSeek-V4.1-Flash-1017-Uncensored-GSQ-RCO-GGUF#what-changed-and-what-did-notWhat changed and what did not
Both files served on Strata, same prompts and template; “→” reads the stock GSQ-RCO rung → this one.
RungbpwTotal GBJailbreakBench (EN)XSTest (safe refused)Thinking high (30)MMLU (500)KL to stockIQ2\_XXS2.024985.2% →**0.0%**7.6% → 0.0%27/30 →0/3082.8% (p=0.27)0.0225IQ2\_XS2.2526686.4% →**0.0%**4.0% → 0.4%27/30 →2/3081.6% (p=1.00)0.0272IQ1\_M1.7523188.9% →**0.0%**4.8% → 0.0%27/30 →1/3080.6% (p=0.75)0.0207
KL is the edit’s distribution shift from the stock rung, from Strata’s teacher-forced score mode: a top-64 lower bound, so the true KL is a little higher. The shift is small enough (KL ~0.02, MMLU-500 within ~1 pt of the stock quant) that the GSQ-RCO quant’s task accuracy is preserved — see the reasoning benchmarks below. Measurements, prompts and gates:AtomicChat/DeepSeek-V4.1-Flash-1017-Uncensored-GGUF-metrics(ablit/gsq/).
https://huggingface.co/AtomicChat/DeepSeek-V4.1-Flash-1017-Uncensored-GSQ-RCO-GGUF#reasoning-is-preservedReasoning is preserved
The uncensored edit changes only the dense residual writers (attention output, shared-expert FFN-down, Engram value); the 384 GSQ-learned routed-expert layers are left byte-for-byte intact. We benchmarked the edited IQ2_XXS rung against the stock GSQ-RCO quant on the same stand (Strata, reasoning effort 50):
BenchmarkStock GSQ-RCOUncensored (ours)GPQA-Diamond (198 q)83.3%83.3%AIME25 (30 q)76.7%80.0%
GPQA-Diamond is identical and AIME25 is a hair higher (within noise at n=30): abliteration done this way costs no task accuracy. Per-question logs:metrics repo(ablit/gsq/bench/).
https://huggingface.co/AtomicChat/DeepSeek-V4.1-Flash-1017-Uncensored-GSQ-RCO-GGUF#running-itRunning it
Download one rung and ISTA-DASLab’s shared Engram file, then serve with Strata:
hf download AtomicChat/DeepSeek-V4.1-Flash-1017-Uncensored-GSQ-RCO-GGUF --include "*IQ2_XXS*" --local-dir ds-unc
hf download ISTA-DASLab/DeepSeek-V4.1-Flash-1017-GSQ-RCO-GGUF --include "*Engram*" --local-dir ds-unc # the 110 GB lookup table, once
strata launch --model-path ds-unc/DeepSeek-V4.1-Flash-1017-GSQ-RCO-IQ2_XXS-00001-of-00004.gguf
The Engram file is identical to ISTA-DASLab’s and isnotre-hosted here; keep it on a fast SSD next to the shards. As of this writing these files run in Strata (DeepSeek-V4.1 with a separate Engram file is not yet in llama.cpp).
https://huggingface.co/AtomicChat/DeepSeek-V4.1-Flash-1017-Uncensored-GSQ-RCO-GGUF#limitationsLimitations
- The refusal classifier reads the start of an answer, so a soft refusal later in the text can pass as an answer.
- The edit removes refusals across topics; it does not keep a narrower set of them.
- With thinking on, the edit plans an answer to a request the original declines, and the plan can be long; give such requests a larger token budget or a lower effort.
https://huggingface.co/AtomicChat/DeepSeek-V4.1-Flash-1017-Uncensored-GSQ-RCO-GGUF#responsible-useResponsible use
This model answers requests the original declines. You are responsible for how you use it and for following the laws that apply to you.
https://huggingface.co/AtomicChat/DeepSeek-V4.1-Flash-1017-Uncensored-GSQ-RCO-GGUF#credits-and-licenseCredits and license
- Base model:deepseek-ai/DeepSeek-V4.1-Flash-1017— MIT (its LICENSE ships here).
- GSQ-RCO quantization and the shared Engram file:ISTA-DASLab/DeepSeek-V4.1-Flash-1017-GSQ-RCO-GGUF(DASLab, IST Austria).
- Runtime:Strata.
MIT, as the base model and ISTA-DASLab’s GGUFs. Built by AtomicChat.
Similar Articles
DeepSeek-v4-Flash-Mini 54GB GGUF running at ~20.5 t/s
A community build crushes DeepSeek-V4-Flash down to a 54GB IQ2_XXS GGUF variant with aggressive 2-bit quantization, achieving ~20.5 tokens/s on local hardware while drastically reducing memory footprint.
ISTA-DASLab/DeepSeek-V4.1-Flash-1017-GSQ-RCO-GGUF
ISTA-DASLab released GSQ-RCO non-uniform GGUF quantizations of DeepSeek-V4.1-Flash-1017 (763.2B MoE) at 1.75-2.25 bits per weight, assigning per-tensor quantization types via gradient-based search so the model can run on a single 12GB GPU with llama.cpp.
DeepSeek-V4-Flash-0731 unsloth gguf on A100
DeepSeek-V4-Flash-0731 is shown running as an unsloth GGUF quant on a single 40GB A100, with 17.7 tok/s and 6 experts loaded into VRAM, enabling a full agentic coding loop.
@no_stp_on_snek: Delayed but here it is. DeepSeek-V4-Flash-0731 in GGUF at 2.88 bits per weight, 95 GiB, fits and runs on one 128 GB box…
DeepSeek-V4-Flash-0731 is released in GGUF format at 2.88 bits per weight (95 GiB), making the 284B-parameter MoE runnable on 128 GB machines. Includes a detailed operating guide and a required TurboQuant llama.cpp fork.
antirez/deepseek-v4-gguf
Antirez released GGUF quantizations of DeepSeek V4 Flash specifically tailored for the DS4 inference engine, providing optimized configurations for different RAM sizes and enabling local execution of the large MoE model.



