orcarouter/OrcaSAQ-2-27B · Hugging Face

Source: https://huggingface.co/orcarouter/OrcaSAQ-2-27B

OrcaSAQ2 27B

High-fidelity 3-bit Qwen3.8 for long-horizon agents.

54 GB → 12.3 GB · +0.02% PPL · 93.2% Top-1 Agreement · 0.031 KLD · 262K Context

OrcaRouter AI Gateway·X·Discord·GitHub·All Models

https://huggingface.co/orcarouter/OrcaSAQ-2-27B#27b-reasoning-123-gb27B reasoning. 12.3 GB. OrcaSAQ2 27Bcompresses Qwen3.8-27B from a54 GB BF16 checkpoint to 12.3 GBwhile preserving extremely high fidelity to the original model. Built for:long-horizon agents · coding · tool use · reasoning · stateful execution

OrcaSAQ2 is a proprietarysensitivity-aware mixed-precision quantization systemdeveloped by OrcaRouter and its research team behind.

It is optimized around one goal:Preserve as much useful model behavior as possible inside a practical GPU memory envelope.

The resulting checkpoint provides:

• 77.2% smaller storage footprint
• only**+0.02% perplexity**versus BF16
• 93.2% token-level Top-1 agreement
• 0.031 mean KLD
• 262K context
• thinking mode
• tool calling
• MTP speculative decoding
• production serving throughvLLM

https://huggingface.co/orcarouter/OrcaSAQ-2-27B#at-a-glanceAt a glance

MetricBF16OrcaSAQ2Checkpoint54 GB12.3 GBRelative size100%**22.8%Storage reduction—77.2%Decoder precision16-bit3.21 bpw avg.Perplexity5.64685.6482PPL delta—+0.02%**Top-1 agreement100%93.2%Mean KLD—0.031Context262K262K 4.4× smaller. +0.02% perplexity.

The point is not 3-bit.

The point is what survives at 3-bit.

https://huggingface.co/orcarouter/OrcaSAQ-2-27B#bf16-fidelityBF16 fidelity

All numbers below are measured usingthese exact OrcaSAQ2 weightsagainst the BF16 reference through the same evaluation path.

https://huggingface.co/orcarouter/OrcaSAQ-2-27B#wikitext-2WikiText-2

16,376 predicted tokens

BuildSizeDecoder BitsMean KLD ↓Top-1 Agreement ↑PPL ↓Qwen3.8-27B BF1654 GB16—100%5.6468**OrcaSAQ2 27B12.3 GB3.210.03193.2%**5.6482

https://huggingface.co/orcarouter/OrcaSAQ-2-27B#perplexityPerplexity

BF16       5.6468  ████████████████████████████████████████
OrcaSAQ2   5.6482  ████████████████████████████████████████

Delta: +0.02%

https://huggingface.co/orcarouter/OrcaSAQ-2-27B#top-1-agreementTop-1 agreement

OrcaSAQ2 vs BF16

███████████████████████████████████████████████░░░  93.2%

https://huggingface.co/orcarouter/OrcaSAQ-2-27B#model-footprintModel footprint

Qwen3.8-27B BF16

██████████████████████████████████████████████████  54.0 GB

OrcaSAQ2

███████████                                         12.3 GB

77.2% smaller.

https://huggingface.co/orcarouter/OrcaSAQ-2-27B#long-horizon-agentsLong-horizon agents

https://huggingface.co/orcarouter/OrcaSAQ-2-27B#low-bit-fidelity-matters-more-as-the-horizon-growsLow-bit fidelity matters more as the horizon grows.

Short benchmarks can hide small degradation.

Agents cannot.

A small model error can change a tool call.

That changes the environment state.

The changed state affects every decision that follows.

Plan
  ↓
Act
  ↓
Observe
  ↓
Decide
  ↓
Recover
  ↓
Repeat
  ↓
...
  ↓
Task Success

Across long trajectories, small errors can compound into large behavioral differences.

That makes long-horizon execution an especially useful stress test for compressed reasoning models.

OrcaSAQ2 performsstrongly on long-horizon workloads relative to models in its deployment and parameter class, despite operating from a12.3 GB checkpoint.

This makes it particularly suitable for:

• coding agents
• terminal agents
• browser agents
• computer-use agents
• security agents
• repository-scale tasks
• multi-tool workflows
• failure recovery
• long-running stateful execution

https://huggingface.co/orcarouter/OrcaSAQ-2-27B#why-this-mattersWhy this matters

Perplexity asks:

How similar is the next-token distribution?

Long-horizon evaluation asks:

Can the model still finish the job after many decisions?

For agent models, both matter.

https://huggingface.co/orcarouter/OrcaSAQ-2-27B#long-horizon-performanceLong-horizon performance

Agent benchmarks depend heavily on the surrounding scaffold, tools, reasoning budget, timeouts and execution environment. The results below are therefore shown aspublic reference points, not direct apples-to-apples comparisons.

https://huggingface.co/orcarouter/OrcaSAQ-2-27B#swe-bench-verifiedSWE-bench Verified

ModelReported scoreClaude Sonnet 4.679.6Claude Sonnet 4.577.2Gemini 376.2OrcaSAQ2 27B****70.0Qwen3-Coder-480B-A35B69.6Gemini 2.5 Pro63.8GPT-4.154.6 70.0% SWE-bench Verified from a 12.06 GB 27B checkpoint.

https://huggingface.co/orcarouter/OrcaSAQ-2-27B#terminal-bench-21Terminal-Bench 2.1

Model / AgentReported scoreGemini 3.1 Pro / Terminus 270.7Claude Opus 4.6 / Claude Code70.1Claude Opus 4.6 / Terminus 263.8Claude Sonnet 4.6 / Claude Code58.5OrcaSAQ2 27B****58.4Gemini 3 Flash / Gemini CLI56.9GPT-5.4 / Terminus 254.8Claude Sonnet 4.6 / Terminus 251.5 58.4% Terminal-Bench 2.1 while fitting in ~12 GB of checkpoint storage.

Public scores use different agent stacks and should not be interpreted as a strict model-only ranking.

https://huggingface.co/orcarouter/OrcaSAQ-2-27B#architectureArchitecture

Base modelQwen/Qwen3\.8\-27BArchitectureQwen3\_5ForCausalLMLayers64Hidden size5120Hybrid attention48 Gated DeltaNet + 16 full-attention layersContext262,144 tokensVocabulary248,320MTP headIncludedThinkingSupportedTool callingSupportedCheckpoint12.3 GBDecoder average3.21 bpwServingvLLMVisionNot includedLicenseApache-2.0

https://huggingface.co/orcarouter/OrcaSAQ-2-27B#production-servingProduction serving

https://huggingface.co/orcarouter/OrcaSAQ-2-27B#up-to-901-toks-single-stream-on-a-16-gb-gpuUp to 90.1 tok/s single-stream on a 16 GB GPU

Measured under a15.7 GiB GPU memory cap.

Configuration1 Stream8 Streams16 StreamsKV PoolvLLM · MTP off65.3 tok/s332 tok/s****333 tok/s29,354 tokvLLM · MTP on****90.1 tok/s220 tok/s219 tok/s14,563 tok

https://huggingface.co/orcarouter/OrcaSAQ-2-27B#mtp-speculative-decodingMTP speculative decoding

Single-stream decode

MTP off    █████████████████████████████       65.3 tok/s

MTP on     ████████████████████████████████████████
                                                90.1 tok/s

+38% single-stream decode throughput

MTP trades additional compute and KV capacity for stronger interactive decode performance.

It is particularly useful for:

• coding assistants
• interactive agents
• terminal agents
• tool-heavy applications
• low-concurrency inference

For highly batched workloads, benchmark both configurations.

https://huggingface.co/orcarouter/OrcaSAQ-2-27B#27b-on-a-16-gb-gpu27B on a 16 GB GPU

OrcaSAQ2’s checkpoint is12.3 GB.

That makes deployment possible on hardware that cannot hold the original 54 GB BF16 checkpoint.

16 GB GPU
┌───────────────────────────────────────────┐
│                                           │
│   OrcaSAQ2 weights          12.3 GB       │
│   ███████████████████████████████████     │
│                                           │
│   Remaining               ~3.7 GB         │
│   ██████████                              │
│                                           │
└───────────────────────────────────────────┘

Actual usable memory depends on:

• vLLM overhead
• KV-cache configuration
• MTP
• batch size
• context length
• CUDA graph configuration

A practical starting point for a 16 GB GPU is approximately32K interactive context, then tune based on the workload.

The model architecture supports up to262K context.

https://huggingface.co/orcarouter/OrcaSAQ-2-27B#built-for-agentsBuilt for agents

https://huggingface.co/orcarouter/OrcaSAQ-2-27B#long-horizon-executionLong-horizon execution

plan → act → observe → recover → repeat

https://huggingface.co/orcarouter/OrcaSAQ-2-27B#codingCoding

Repository-scale generation, editing, testing and debugging.

https://huggingface.co/orcarouter/OrcaSAQ-2-27B#tool-useTool use

Structured workflows where action-selection quality matters.

https://huggingface.co/orcarouter/OrcaSAQ-2-27B#reasoningReasoning

Preserving the capabilities of the 27B base model under an aggressive deployment constraint.

https://huggingface.co/orcarouter/OrcaSAQ-2-27B#single-gpu-deploymentSingle-GPU deployment

A12.3 GBcheckpoint designed around practical inference hardware.

https://huggingface.co/orcarouter/OrcaSAQ-2-27B#production-serving-1Production serving

vLLM + MTP + OpenAI-compatible APIs.

https://huggingface.co/orcarouter/OrcaSAQ-2-27B#generation-samplesGeneration samples

One prompt each, first attempt.

https://huggingface.co/orcarouter/OrcaSAQ-2-27B#pelican-on-a-bicyclePelican on a bicycle

The standard SVG test, asked for as an animation.

Chain over the chainring, cranks 180° out of phase, parallax background. Pure SMIL, no JavaScript. Used as generated.

https://huggingface.co/orcarouter/OrcaSAQ-2-27B#low-poly-statue-of-libertyLow-poly Statue of Liberty

Create a html low-poly 3D models of the Statue of Liberty

A single self-contained HTML file: Three.js scene, orbit controls, procedural geometry.

https://huggingface.co/orcarouter/OrcaSAQ-2-27B#quickstartQuickstart

https://huggingface.co/orcarouter/OrcaSAQ-2-27B#installInstall

pip install -U vllm huggingface_hub

pip install git+https://github.com/Continuum-AI-Corp/OrcaSAQ2-kernel

https://huggingface.co/orcarouter/OrcaSAQ-2-27B#downloadDownload

hf download orcarouter/OrcaSAQ2-27B \
  --local-dir ./OrcaSAQ2-27B

https://huggingface.co/orcarouter/OrcaSAQ-2-27B#serve-with-vllmServe with vLLM

vllm serve ./OrcaSAQ2-27B \
  --served-model-name OrcaSAQ2-27B \
  --max-model-len 262144 \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_xml \
  --speculative-config '{"method":"qwen3_next_mtp","num_speculative_tokens":2}'

https://huggingface.co/orcarouter/OrcaSAQ-2-27B#openai-compatible-apiOpenAI-compatible API

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8000/v1",
    api_key="not-needed",
)

response = client.chat.completions.create(
    model="OrcaSAQ2-27B",
    messages=[
        {
            "role": "user",
            "content": "Analyze this repository and plan the next five actions."
        }
    ],
)

print(response.choices[0].message.content)

https://huggingface.co/orcarouter/OrcaSAQ-2-27B#recommended-samplingRecommended sampling

temperature = 1.0
top_p       = 0.95
top_k       = 20

Thinking mode is enabled by default.

For agent deployments, benchmark against the actual tool schema, context distribution and reasoning budget used in production.

https://huggingface.co/orcarouter/OrcaSAQ-2-27B#evaluation-philosophyEvaluation philosophy

A low-bit reasoning model should not be judged by checkpoint size alone.

We look at the intersection of:

Footprint × BF16 Fidelity × Capability × Long-Horizon Stability × Serving Performance

A useful low-bit model must remain useful after compression.

https://huggingface.co/orcarouter/OrcaSAQ-2-27B#why-perplexity-alone-is-not-enoughWhy perplexity alone is not enough

Perplexity is useful and reproducible.

It is not a complete measure of agentic capability.

Quantization can affect:

reasoning
   ↓
planning
   ↓
tool selection
   ↓
state tracking
   ↓
recovery
   ↓
task completion

That is why OrcaSAQ2 reports BF16 fidelity metrics alongside downstream and long-horizon evaluation.

https://huggingface.co/orcarouter/OrcaSAQ-2-27B#methodMethod

OrcaSAQ2 uses a proprietarysensitivity-aware mixed-precision quantization systemdeveloped by OrcaRouter.

The implementation is optimized to preserve model quality under a strict deployment-memory target.

Detailed quantization methodology, calibration strategy, precision allocation and packing techniques are not currently disclosed.

https://huggingface.co/orcarouter/OrcaSAQ-2-27B#limitationsLimitations

• OrcaSAQ2 inherits the capabilities, biases and limitations of Qwen3.8-27B.
• Quantization is not mathematically lossless.
• 93.2% Top-1 agreementmeans some token decisions differ from BF16.
• +0.02% PPLis a model-fidelity measurement and does not guarantee identical downstream performance.
• Long-horizon comparisons should use a controlled same-harness evaluation.
• This checkpoint istext-only.
• The vision tower is not included.
• OrcaSAQ2 requires the OrcaSAQ2 vLLM integration.
• Maximum architectural context does not imply that the full context fits into every GPU memory envelope.

https://huggingface.co/orcarouter/OrcaSAQ-2-27B#open-source-from-orcarouterOpen source from OrcaRouter

https://huggingface.co/orcarouter/OrcaSAQ-2-27B#orcacode-reviewOrcaCode Review

Open multi-model code review.

https://huggingface.co/orcarouter/OrcaSAQ-2-27B#orcareplayOrcaReplay

Record, replay, fork and debug AI-agent runs.

https://huggingface.co/orcarouter/OrcaSAQ-2-27B#orcarouter-liteOrcaRouter Lite

Self-hosted multi-model AI infrastructure.

Open model. Open harness. Open bill.

https://huggingface.co/orcarouter/OrcaSAQ-2-27B#citationCitation

@misc{qwen38,
    title = {Qwen3.8-Max: A New Bar for Coding and Cowork},
    author = {{Qwen Team}},
    year = {2026},
    month = {August},
    url = {https://qwen.ai/blog?id=qwen3.8}
}

https://huggingface.co/orcarouter/OrcaSAQ-2-27B#licenseLicense

Apache-2.0

Inherited from:

Qwen/Qwen3\.8\-27B

Quantization does not change the underlying license obligations.

One Gateway. Every Model.

Route Smarter · Ship Safer · Spend Less

Website·Models·GitHub·Discord·X