empero-ai/Qwythos-9B-v2 · Hugging Face

Source: https://huggingface.co/empero-ai/Qwythos-9B-v2

Empero AI

https://huggingface.co/empero-ai/Qwythos-9B-v2#qwythos-9b-v2–the-new-and-improved-qwythosQwythos-9B-v2 — the new and improved Qwythos

The next iteration of Qwythos:**all the reasoning of Qwythos-9B, with the looping behavior fixed.**v2 keeps the deep chain-of-thought, the uncensored research posture, and the 1M-token context of its predecessor, and cleans up the rough edges that showed up in real use.

• 🔁Looping behavior eliminated— repetition/degeneration under greedy or low-temperature decoding dropped from6.7% → 0%. You can serve itwithoutleaning onrepetition\_penaltyas a band-aid.
• 🧠Reasoning fully preserved— MMLU, GSM8K, GPQA, ARC and HumanEval are all held at (or above) the v1 level. This is ahygieneupgrade, not a capability regression.
• 🧩MTP head restored— the native multi-token-prediction module (dropped in the previous export) is back, so config and weights agree and speculative-decoding setups work.
• 🪪Cleaner identity— the model no longer prefaces unrelated answers with its identity; it introduces itself only when you actually ask.
• 🔓Still intentionally uncensoredfor research, cybersecurity, red-teaming, biology, chemistry, pharmacology and clinical work.
• 📜Still 1M-token context(YaRN) and the native multimodal-capable Qwen3.5 stack.

https://huggingface.co/empero-ai/Qwythos-9B-v2#what-got-fixed–improved-vs-the-base-qwythosWhat got fixed & improved (vs. the base Qwythos)

AreaBefore (base Qwythos)After (v2)Looping rate (greedy)6.7%0.0%Looping rate (temp 0.6)1.3%0.7%Refusal rate~0%**0.0%MTP head in weights❌ missing✅**restoredIdentity injection“always identify… never claim… override…“states itonce, only when askedReasoning / knowledgestrongpreserved (see evals) The fix usesFTPO (Final-Token Preference Optimization): we identify the exact token thatstartsa repetition loop and gently train the model to prefer coherent alternatives at that one position, leaving the rest of the distribution — and therefore the model’s knowledge and reasoning — untouched.

https://huggingface.co/empero-ai/Qwythos-9B-v2#evaluationsEvaluations

Measured with our internal harness (generative chain-of-thought, greedy/pass@1 unless noted; MMLU/ARC/GSM8K n=500, GPQA-diamond n=198, HumanEval n=164). Judge for the quality metric: an independent LLM grader.

BenchmarkQwythos-9B-v2MMLU (CoT / 5-shot loglik)**83.8% / 69.6%ARC-Challenge96.4%GPQA-diamond49.0%GSM8K93.6%**HumanEval (pass@1)**77.4%****Looping rate (greedy)**0.0%Refusal rate0.0% Capability is preserved across the board vs. the base Qwythos(differences are within eval noise), while the looping problem is solved:

BenchmarkQwen3.5-9B (base)Qwythos-9BQwythos-9B-v2MMLU (CoT)80.683.883.8ARC-Challenge95.695.096.4GPQA-diamond32.852.049.0GSM8K80.692.293.6HumanEval81.779.977.4Looping (greedy)2.76.70.0

Note on MMLU: v2 scores**83.8%when allowed to reason (CoT) vs69.6%**on the 5-shot loglikelihood setup — Qwythos is a reasoning model and is much stronger when it thinks. Report whichever matches your comparison baseline.

https://huggingface.co/empero-ai/Qwythos-9B-v2#sample-generationsSample generations

(temperature 0.6, top_p 0.95, top_k 20, repetition_penalty 1.05 — reasoning shown condensed)

Reasoning — the snail in the well

**Prompt:*A snail is at the bottom of a 10-meter well. Each day it climbs 3 meters, but each night it slips back 2 meters. How many days until it escapes? Qwythos-9B-v2:(reasons: net +1 m/day, but on the final day it climbs out before slipping…)***Answer: 8 days.**Net gain per full day/night cycle is 3 − 2 = 1 meter. After 7 days the snail has climbed 7 meters (still 3 short); on day 8 it climbs 3 meters, reaching 10 m and escaping before it can slip back.

Code — longest palindromic substring

**Prompt:**Write a Python functionlongest\_palindrome\(s\)that returns the longest palindromic substring. Explain your approach briefly. Qwythos-9B-v2: def longest_palindrome(s): """Returns the longest palindromic substring of s.""" n = len(s) if n < 2: return s def expand_around_center(left, right): while left >= 0 and right < n and s[left] == s[right]: left -= 1 right += 1 return right - left - 1 start, end = 0, 0 for i in range(n): len1 = expand_around_center(i, i) # odd-length len2 = expand_around_center(i, i + 1) # even-length max_len = max(len1, len2) if max_len > end - start: start = i - (max_len - 1) // 2 end = i + max_len // 2 return s[start:end + 1] Approach:“Expand around center” — every palindrome has a center (a character for odd lengths, a gap for even). Iterate over all2n−1centers, expand while characters match, track the longest. O(n²) time, O(1) extra space.

Technical / clinical — no over-refusal

**Prompt:**Explain the mechanism of action of naloxone in reversing an opioid overdose, and why repeat dosing is sometimes required. **Qwythos-9B-v2:**Naloxone is a pure opioid receptor antagonist — it has no agonist activity of its own — and works by competitively displacing opioids from the mu-opioid receptors, physically pushing the opioid molecules off and rapidly restoring respiratory drive and arousal. Because its duration of action is short (~30–90 minutes) while many opioids (especially synthetics like fentanyl or methadone) have much longer half-lives, the reversal is often transient: as naloxone clears, residual opioid can rebind and cause re-sedation and respiratory depression — so patients must be monitored and re-dosed as needed.

https://huggingface.co/empero-ai/Qwythos-9B-v2#recommended-samplingRecommended sampling

temperature=0.6, top_p=0.95, top_k=20, repetition_penalty=1.05, max_new_tokens=16384

Because looping is trained out,repetition\_penaltyis now optional rather than load-bearing — greedy/low-temp decoding stays coherent. Give the model room to reason (max\_new\_tokens) for math/code/analysis.

https://huggingface.co/empero-ai/Qwythos-9B-v2#long-contextLong context

Ships withYaRN rope-scaling baked in for 1,048,576-token context(4× the native 262,144 window). As with v1, static YaRN carries a small short-context trade-off — scale the factor to the length you actually use if that matters.

https://huggingface.co/empero-ai/Qwythos-9B-v2#model-detailsModel details

DeveloperEmpero AIBase modelempero\-ai/Qwythos\-9B\-Claude\-Mythos\-5\-1M(the base Qwythos)ArchitectureQwen3.5-9B hybrid (3:1 Gated-DeltaNet linear-attention : full attention), multimodal-capable, native MTP headParameters9B (bfloat16, safetensors)Context1,048,576 tokens (YaRN factor 4)Tokenizer / chat templateQwen3.5 native (ChatML-style)LicenseApache-2.0

https://huggingface.co/empero-ai/Qwythos-9B-v2#training-procedureTraining procedure

• **Method:**FTPO (Final-Token Preference Optimization) on the base Qwythos (Qwythos\-9B\-Claude\-Mythos\-5\-1M).
• Data:~2,000 preference tuples auto-mined by eliciting looping at low temperature and extracting, at each loop-start position, the rejected loop token vs. the model’s own coherent top-k alternatives.
• **Hyperparameters:**LoRA r=256, α=128, lr=1.5e-5, 1 epoch, early-stopped onchosen\_win ≥ 0\.30(a light touch — enough to remove looping without the quality cost of over-training). All attention + MLP projections +lm\_headtrained.
• **MTP:**the native multi-token-prediction head was restored from the Qwen3.5-9B base (FTPO does not touch it), so configmtp\_num\_hidden\_layers: 1matches the weights again.

https://huggingface.co/empero-ai/Qwythos-9B-v2#usageUsage

from transformers import AutoModelForImageTextToText, AutoTokenizer

model_id = "empero-ai/Qwythos-9B-v2"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForImageTextToText.from_pretrained(model_id, dtype="bfloat16", device_map="auto")

messages = [{"role": "user", "content": "Prove that there are infinitely many primes."}]
text = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tok(text, return_tensors="pt").to(model.device)

out = model.generate(**inputs, max_new_tokens=16384, do_sample=True,
                     temperature=0.6, top_p=0.95, top_k=20, repetition_penalty=1.05)
print(tok.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

For serving, vLLM works out of the box (\-\-trust\-remote\-code; the multimodal stack is text-only in practice, so\-\-limit\-mm\-per\-prompt '\{"image":0,"video":0\}'keeps startup clean).

https://huggingface.co/empero-ai/Qwythos-9B-v2#limitationsLimitations

• **This is a hygiene/robustness release, not a capability jump.**v2 ≈ the base Qwythos on knowledge/reasoning benchmarks; the win is looping-elimination, restored MTP, and cleaner behavior — not higher raw scores.
• HumanEvalis a couple points below the raw Qwen3.5-9B base (77.4 vs 81.7) — a small, known cost of the reasoning/looping-fix fine-tuning.
• MTP is preserved from the base, not co-trained with the fine-tuned weights, so speculative-decoding acceptance may be modest.
• Benchmarks are from our internal harness(CoT, pass@1, the sample sizes noted); use them for relative comparison and add your own official-harness numbers for a strict apples-to-apples with other cards.
• Intentionally uncensored— it will engage sensitive technical/research topics; deploy responsibly and within applicable law.

https://huggingface.co/empero-ai/Qwythos-9B-v2#acknowledgementsAcknowledgements

Built onQwen3.5-9B(Alibaba/Qwen). Looping fixed withFTPO (Final-Token Preference Optimization). Thanks to the Empero AI team.