IFM/K2-Horizon-MoVA-36B-A4B-GGUF · Hugging Face

Source: https://huggingface.co/IFM/K2-Horizon-MoVA-36B-A4B-GGUF

This repository contains GGUF versions of theIFM/K2-Horizon-MoVA-36B-A4Bfor use withllama\.cpp. The model tensors are stored in their original BF16 precision. The GGUF files include the tokenizer metadata and allama\.cpp-compatible chat template. **Compatibility:**These models require a version ofllama\.cppcontaining K2 Horizon architecture support. PR to llama.cpp is in progress. MBZUAI-IFM fork of llama.cpp is inhttps://github.com/MBZUAI-IFM/llama.cpp/tree/model/K2Horizon

K2-Horizon-MoVA-36B-A4B is the sparse member of the K2-Horizon family: a Mixture-of-Experts model with Mixture-of-Values attention (MoVA) that stores 36B parameters and runs 4B per token. We have released the final checkpoint; intermediate checkpoints, along with the data and the training code, will be released.

https://huggingface.co/IFM/K2-Horizon-MoVA-36B-A4B-GGUF#k2-horizon-mova-36b-a4b-highlightsK2-Horizon-MoVA-36B-A4B Highlights

• **Frontier-class results at 4B active parameters.**On agentic and reasoning benchmarks it outscores open weight dense (approximately 30B model size) and MoE models up to 15× its size; and also performs competitively against closed frontier models (seeBenchmark Results).
• **512K context.**Native 524,288-token context from the midtraining stages onward.
• **Intermediate checkpoints.**Intermediate checkpoints will be released so capability changes can be studied across training rather than at a single checkpoint.
• **Fully open.**Training data/recipe and the training code will be made public.

https://huggingface.co/IFM/K2-Horizon-MoVA-36B-A4B-GGUF#benchmark-resultsBenchmark Results

Open-weight modelsK2-Horizon-MoVA-36B-A4BNemotron 3 UltraNemotron 3 SuperG9v3-39A5BQwen3.6-35B-A3BMuse Glimmer-30BGemma 4 31B-it# Params36B550B120B39B35B30B31B# Activated params4B55B12B5B3B30B31BArchitectureMoEMoEMoEMoEMoEDenseDenseAgentstau3-Banking

Agentic tool use

26.814.210.322.19.323.514.8CodingTerminal-Bench 2.1

Agentic terminal use

58.653.938.632.644.951.743.4SciCode

Scientific coding

38.939.936.034.035.843.643.4Scientific ReasoningHumanity’s Last Exam (without tools)

Expert-level reasoning

25.228.420.817.522.222.023.6GPQA Diamond

Graduate-level science QA

80.886.780.080.584.183.585.7CritPt

Frontier physics reasoning

2.13.1****3.10.30.32.61.4GeneralAA-LCR

Long-context reasoning

66.371.060.362.066.780.068.3AA-Omniscience Accuracy

Factual accuracy

18.822.624.314.918.827.020.0AA-Omniscience Non-Hallucination

Non-hallucination rate

69.270.313.087.049.518.115.0 Scores in %. Bold marks the best score in each row. Sections follow theArtificial Analysis Intelligence Indexcategories. Baseline scores are from Artificial Analysis; Muse Glimmer-30B at high reasoning effort, all other open models in their reasoning mode.

https://huggingface.co/IFM/K2-Horizon-MoVA-36B-A4B-GGUF#quickstartQuickstart

https://huggingface.co/IFM/K2-Horizon-MoVA-36B-A4B-GGUF#servingServing

vLLM, recipe atrecipes.vllm.ai/IFM:

vllm serve IFM/K2-Horizon-MoVA-36B-A4B \
  --revision main \
  --tensor-parallel-size 2 \
  --enable-expert-parallel \
  --trust-remote-code \
  --dtype bfloat16 \
  --max-model-len 131072 \
  --reasoning-parser k2_horizon \
  --tool-call-parser k2_horizon \
  --enable-auto-tool-choice

SGLang recipe validated on 2× H200 in theSGLang K2 Horizon cookbook:

python3 -m sglang.launch_server \
  --model-path IFM/K2-Horizon-MoVA-36B-A4B \
  --revision main \
  --tp 2 \
  --ep 2 \
  --dtype bfloat16 \
  --attention-backend fa3 \
  --json-model-override-args '{"xllm_source_router_gemm_partitions":2}' \
  --reasoning-parser k2_horizon \
  --tool-call-parser k2_horizon \
  --host 0.0.0.0 --port 30000

https://huggingface.co/IFM/K2-Horizon-MoVA-36B-A4B-GGUF#api-usageAPI Usage

Recommended settings:reasoning\_effort="high",temperature=1\.0,top\_p=0\.95. Reasoning depth is selected per request throughchat\_template\_kwargs. Thinking is returned inreasoning\_contentand the answer incontent.

from openai import OpenAI

client = OpenAI(base_url="http://localhost:30000/v1", api_key="EMPTY")
response = client.chat.completions.create(
    model="IFM/K2-Horizon-MoVA-36B-A4B",
    messages=[{"role": "user", "content": "Explain the result step by step."}],
    temperature=1.0,
    top_p=0.95,
    max_tokens=32768,
    extra_body={"chat_template_kwargs": {"reasoning_effort": "high"}},
)
message = response.choices[0].message
print("Reasoning:", getattr(message, "reasoning_content", None))
print("Answer:", message.content)

https://huggingface.co/IFM/K2-Horizon-MoVA-36B-A4B-GGUF#transformersTransformers

Validated with Transformers 5.15.0, PyTorch 2.13.0, Safetensors 0.8.0.

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "IFM/K2-Horizon-MoVA-36B-A4B"
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id, device_map="auto", dtype="bfloat16", low_cpu_mem_usage=True, trust_remote_code=True
)

inputs = tokenizer("Explain why long-context evaluation is difficult.", return_tensors="pt").to(model.device)
inputs.pop("token_type_ids", None)
outputs = model.generate(**inputs, max_new_tokens=32768, temperature=1.0, top_p=0.95, do_sample=True)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

https://huggingface.co/IFM/K2-Horizon-MoVA-36B-A4B-GGUF#best-practicesBest Practices

1. **Reasoning effort: alwayshigh.**All reported results use high reasoning effort. Pass\{"chat\_template\_kwargs": \{"reasoning\_effort": "high"\}\}on every request.
2. Sampling parameters.temperature=1\.0,top\_p=0\.95.
3. **Serving.**Use the validated SGLang recipe above: BF16, TP=2, FlashAttention-3, and thexllm\_source\_router\_gemm\_partitionsoverride, which preserves the checkpoint’s router numerics. Full recipes for every K2-Horizon size, with measured H200 latency and throughput, are in theSGLang cookbookand thevLLM recipe.
4. **Parsers.**Enable thek2\_horizonreasoning parser for chat, and add thek2\_horizontool-call parser for agent use. Leave both off for plain completion-style generation.

https://huggingface.co/IFM/K2-Horizon-MoVA-36B-A4B-GGUF#citationCitation

@misc{k2horizon2026,
  title  = {Introducing K2 Horizon: Frontier Performance, Radically Open},
  author = {{IFM Team}},
  year   = {2026},
  url    = {https://ifm.ai/blog/k2/},
}