anoooother one!

Introducing Grug-30b-a3b!

Grug is an open-source experimental model build on top of Qwen-3.6-30b-a3b. Designed to replicate the efficiency of GPT-5.5 by cutting down thinking into only-necessary chunks.

Grug outputs on average ~69.8% less thinking tokens, while preforming within a ~2% total margin of base Qwen-3.6

This lead to significantly faster generation and less context filling, leading to exponential decreases in generation speed.

Download it here: https://huggingface.co/kai-os/Grug-35B-A3B…

kai-os/Grug-35B-A3B · Hugging Face

Source: https://huggingface.co/kai-os/Grug-35B-A3B

Grug 35B A3B is a compact-reasoning fine-tune ofQwen/Qwen3\.6\-35B\-A3B. It keeps the Qwen MoE/A3B architecture: 35B total parameters with roughly 3B activated per token, 40 text layers, 256 experts, and 8 routed experts per token plus the shared expert path.

This repository is published as merged Transformers/safetensors model weights. It was trained with QLoRA/PEFT LoRA, then merged into the base model before upload. You do not need a separate adapter to load this model.

https://huggingface.co/kai-os/Grug-35B-A3B#what-changedWhat Changed

The training target is a terse internal-reasoning style: short high-density notes, fewer filler phrases, and stronger preservation of constraints, equations, checks, bug causes, decisive branches, edge cases, and final-answer validation.

The goal is lower reasoning-token usage relative to the base model while preserving answer quality. It is not meant to hide uncertainty or remove needed reasoning.

https://huggingface.co/kai-os/Grug-35B-A3B#architectureArchitecture

The merged release is a Qwen MoE text-generation model:

• Base model:Qwen/Qwen3\.6\-35B\-A3B.
• Released architecture:Qwen3\_5MoeForCausalLM.
• Model type:qwen3\_5\_moe\_text.
• Hidden size: 2048.
• Text layers: 40.
• Experts: 256.
• Activated experts: 8 routed experts plus shared expert path.
• Native context length from the base model family: 262,144 tokens.

https://huggingface.co/kai-os/Grug-35B-A3B#training-dataTraining Data

The data pipeline started from a recent, filtered reasoning pool and converted verbose traces into compact traces before SFT packing.

Source gate:

• Run date: June 30, 2026.
• Default freshness cutoff: 45 days. Sources older than May 16, 2026 were rejected unless manually allowed.
• Allowed train licenses: MIT, Apache-2.0, CC-BY-4.0, CC0-1.0.
• Hard reject terms included OpenAI, ChatGPT, GPT-5, Claude, Anthropic, Opus, Sonnet, and Gemini.
• Soft-risk sources marked as synthetic/distill were manually reviewed or rejected depending on provenance and license.

Final verified source mix:

SourceLicenseDomainVerified rowshotdogs/uka\-glm\-5\.2MITagent code1,617Scale\-or\-Reason/general\-reasoning\-ift\-pairsMITgeneral reasoning1,305samcheng0/lumia\-reasoning\-sft\-v1Apache-2.0code reasoning1,103HSH\-Intelligence/verified\-math\-reasoning\-3kApache-2.0math672kd13/CodeDebug\-Instruct\-v2\-ReasoningMITcode debug600Madarabr/cortex\-adaptive\-thinkingApache-2.0adaptive reasoning300CL\-From\-Nothing/code\_rose\_initial\_1\_7B\_SFT\_10K\_rollouts\_Qwen3\-4B\-Thinking\-2507\_k12\_t0\.7\_maxtok12288Apache-2.0code reasoning143 Row counts:

• Normalized recent reasoning pool: 8,680 rows.
• Selected verbose reasoning set: 6,144 rows.
• Compact raw transform output: 6,144 rows.
• Clean packed SFT split: 4,517 train / 245 validation / 247 test.
• Qwen training rows accepted after length filter: 4,385.
• Qwen training rows skipped by length filter: 132.
• Evaluation rows used during training: 64.

The compact reasoning transform was generated withcyankiwi/Qwen3\.6\-35B\-A3B\-AWQ\-4bitserved by vLLM. Rows were checked for compression ratio, answer preservation, malformed tags, repetition, fixed reasoning labels, tone issues, and obvious loss of critical information before training.

https://huggingface.co/kai-os/Grug-35B-A3B#training-procedureTraining Procedure

Training was completion-only SFT: prompt tokens were masked with\-100, and only the assistant completion was trained.

Core settings:

• Base model:Qwen/Qwen3\.6\-35B\-A3B.
• Method: QLoRA / PEFT LoRA, merged into full model weights for upload.
• Quantization during training: 4-bit NF4 with BF16 compute.
• Hardware: 8x NVIDIA Tesla V100-SXM2 16GB.
• Max sequence length: 1,792.
• LoRA rank: 4.
• LoRA alpha: 8.
• Batch size: 1.
• Gradient accumulation: 4.
• Learning rate: 8e-5.
• Max steps: 100.
• Eval steps: 25.
• Save steps: 25.
• Train runtime: about 1 hour 41 minutes 44 seconds.
• Train samples per second: 0.066.
• Train steps per second: 0.016.
• Train loss: 1.051.

Validation loss:

StepEval loss251.1510501.0793751.04871001.0399

https://huggingface.co/kai-os/Grug-35B-A3B#local-smoke-testLocal Smoke Test

Final checkpoint smoke outputs were compact and did not use fixedGoal/Rule/Logic/Edgelabels.

Example:

<think>
Shirt 80, 25% off. 25% of 80 = 20. 80 - 20 = 60. Check: 60/80 = 0.75. Correct. Answer 60.
</think>

The sale price is $60.

Bug-fix smoke output correctly identifiedlen\(x\)as the issue and changed it tolen\(xs\).

This is a smoke test, not a broad benchmark. Run your own evals before relying on the model in sensitive or production settings.

https://huggingface.co/kai-os/Grug-35B-A3B#usageUsage

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "kai-os/Grug-35B-A3B"

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
)
model.eval()

messages = [
    {"role": "user", "content": "If a shirt is $80 and goes 25% off, what is the sale price?"}
]
inputs = tokenizer.apply_chat_template(
    messages,
    return_tensors="pt",
    add_generation_prompt=True,
).to(model.device)

with torch.no_grad():
    output = model.generate(inputs, do_sample=False, max_new_tokens=512)

print(tokenizer.decode(output[0], skip_special_tokens=True))

For token-efficiency tests, compare against the base model with the same prompt and decoding settings. Do not use an artificial generation cap for benchmark claims unless the deployment itself requires one.

https://huggingface.co/kai-os/Grug-35B-A3B#limitationsLimitations

• This is an experimental compact-reasoning fine-tune.
• The training run was intentionally small and should be treated as a first Qwen/A3B checkpoint, not a fully benchmarked production model.
• It may over-compress reasoning on tasks that need longer derivations.
• It inherits the base model’s limitations and safety behavior.
• The reported evaluation is local and limited.
• The dataset includes synthetic and distilled reasoning traces from the listed open datasets; review source licenses and provenance before using this in commercial or sensitive settings.

https://huggingface.co/kai-os/Grug-35B-A3B#acknowledgementsAcknowledgements

Thanks toLambda, the inference provider, for compute credits that supported the dataset work, training, and evaluation.

kaios (@kaiostephens): Introducing ⭐Grug-12b!⭐

Grug is an open-source experimental model build on top of Gemma-4-12b. Designed to replicate the efficiency of GPT-5.5 by cutting down thinking into only-necessary chunks.

Grug outputs on average ~69.8% less thinking tokens, while preforming within a