Quants here https://huggingface.co/GestaltLabs/Ornstein-3.5-9B-V1.5-GGUF…

GestaltLabs/Ornstein-3.5-9B-V1.5-GGUF · Hugging Face

Source: https://huggingface.co/GestaltLabs/Ornstein-3.5-9B-V1.5-GGUF

GGUF quantizations ofGestaltLabs/Ornstein-3.5-9B-V1.5— a reasoning-focused fine-tune of Qwen 3.5 9B for AI-research and technical problem-solving, and an intermediate (V1.5) release between V1 and the upcoming V2 (a much more rigorous post-training run involving reinforcement-learning methods).

These builds keep the model’s nativemulti-token-prediction (MTP) head(for speculative decoding in compatible runtimes) and ship a separatevision projector(mmproj) so the multimodal tower can be used for image/video input.

https://huggingface.co/GestaltLabs/Ornstein-3.5-9B-V1.5-GGUF#filesFiles

FileBitsSizeNotesOrnstein\-3\.5\-9B\-V1\.5\-Q8\_0\.gguf89.8 GBnear-losslessOrnstein\-3\.5\-9B\-V1\.5\-Q6\_K\.gguf67.6 GBvery high qualityOrnstein\-3\.5\-9B\-V1\.5\-Q5\_K\_M\.gguf56.6 GBhigh qualityOrnstein\-3\.5\-9B\-V1\.5\-Q4\_K\_M\.gguf45.8 GBrecommended defaultOrnstein\-3\.5\-9B\-V1\.5\-Q3\_K\_M\.gguf34.7 GBsmallestmmproj\-Ornstein\-3\.5\-9B\-V1\.5\-f16\.gguff160.9 GBvision encoder — pair with any quant for image/video

https://huggingface.co/GestaltLabs/Ornstein-3.5-9B-V1.5-GGUF#usage-llamacppUsage (llama.cpp)

Text:

llama-cli -hf GestaltLabs/Ornstein-3.5-9B-V1.5-GGUF:Q4_K_M -p "Derive the variance of a sum of two correlated random variables."

Multimodal (image/video) — add the vision projector:

llama-mtmd-cli -m Ornstein-3.5-9B-V1.5-Q4_K_M.gguf \
  --mmproj mmproj-Ornstein-3.5-9B-V1.5-f16.gguf \
  --image picture.jpg -p "Describe this image."

Quality and speed scale with quant size;Q4_K_Mis a strong default for ~8 GB of VRAM/RAM. The MTP head enables speculative decoding where supported.

https://huggingface.co/GestaltLabs/Ornstein-3.5-9B-V1.5-GGUF#highlights-full-precision-modelHighlights (full-precision model)

• Strong gains in multi-step reasoning over the base, with the largest improvements on hard, graduate-level scientific reasoning.
• General conversational ability preserved.

https://huggingface.co/GestaltLabs/Ornstein-3.5-9B-V1.5-GGUF#support-this-workSupport This Work

I’m a PhD student in visual neuroscience at the University of Toronto who also happens to spend way too much time fine-tuning, merging, and quantizing open-weight models on rented H100s and a local DGX Spark. All training compute is self-funded — balancing GPU costs against a student budget. If my uploads have been useful to you, consider buying a PhD student a coffee. It goes a long way toward keeping these experiments running.

Support on Ko-fi

https://huggingface.co/GestaltLabs/Ornstein-3.5-9B-V1.5-GGUF#licenseLicense

Apache 2.0 — inherited from the Qwen 3.5 9B base release.