一个 295B 参数的旗舰大模型，现在单张 96GB 推理显卡就能跑，而且解码速度还提升了 50%。

腾讯混元团队为 Hy3（295B 参数）推出量化版本。1bit 版本（IQ1_M）把权重从 598GB 压缩到 85.5GB，缩小 6.7 倍，单张 96GB 推理显卡即可部署；4bit 版本（Q4_K_M）体积 169.9GB，两张显卡可承载，性能接近满血模型。配合 MTP 投机解码，1bit 版本解码速度提升约 50%，4bit 版本提升近 60%。

这件事的意义不只是「更便宜了」。之前 295B 参数级别的模型，推理动辄需要多机多卡，只有大型云厂商和顶级实验室能负担。现在单张 96GB 显卡就能跑，意味着更多中小机构和独立研究者可以在本地接触这个能力层级，这样不用通过 API 租用，可以直接持有和控制模型权重。

Agent、多语言代码、工具调用、长文理解等任务上，量化版表现接近满血。全部版本已开源，GGUF 格式，兼容 llama.cpp 生态。

大模型量化的核心趋势不再是「性能损耗可以接受」，而变成了「性能损耗本身越来越小」，当 1bit 版本还能保留接近满血能力，量化不再是妥协，而是另一种部署策略。

现在同等能力下是否可以实现最小的部署门槛已经变成了接下来大模型落地最重要的竞争维度。

Hy3 模型：https://huggingface.co/tencent/Hy3 低比特GGUF模型：https://huggingface.co/AngelSlim/Hy3-GGUF… GPTQ Int4 模型：https://huggingface.co/AngelSlim/Hy3-GPTQ-Int4…

tencent/Hy3 · Hugging Face

Source: https://huggingface.co/tencent/Hy3 中文｜ English

🖥️Official Website| 💬GitHub

https://huggingface.co/tencent/Hy3#table-of-contentsTable of Contents

• Model Introduction
• Stronger Agent Capabilities
• More Reliable Product Experiences
• Benchmark Appendix
• News
• Model Links
• Quickstart
• Deployment- vLLM - SGLang
• Finetuning
• RL Post-training
• Quantization
• License
• Contact Us

https://huggingface.co/tencent/Hy3#model-introductionModel Introduction

Hy3is a 295B-parameter Mixture-of-Experts (MoE) model with 21B active parameters and 3.8B MTP layer parameters, developed by the Tencent Hy Team. Following the Hy3 Preview launch in late April, we gathered feedback from 50+ products and scaled up post-training with higher quality data. Today, we introduce Hy3, which outperforms similar-size models and rivals flagship open-source models with 2-5x parameters. It also shows significant gains in utility across various products and productivity tasks.

PropertyValueArchitectureMixture-of-Experts (MoE)Total Parameters295BActivated Parameters21BMTP Layer Parameters3.8BNumber of Layers (excluding MTP layer)80Number of MTP Layers1Attention Heads64 (GQA, 8 KV heads, head dim 128)Hidden Size4096Intermediate Size13312Context Length256KVocabulary Size120832Number of Experts192 experts, top-8 activatedSupported PrecisionsBF16

https://huggingface.co/tencent/Hy3#stronger-agent-capabilitiesStronger Agent Capabilities

Building on Hy3 Preview, we further improved the quality and diversity of post-training data while scaling up RL training. Hy3 shows solid gains across reasoning, agentic, and long-context tasks, competitive with much larger flagship models.

In productivity scenarios such as coding, office work, financial modeling, frontend design, and game development, Hy3 has made remarkable progress and can now serve as a reliable, cost-effective model option.

We don’t think public benchmark scores tell the full story. So we ran a blind evaluation with 270 experts using tasks from their work, and Hy3 scored 2.67/4, outperforming GLM-5.1 at 2.51/4. The advantage was most substantial in frontend development, data & storage, and CI/CD tasks.

https://huggingface.co/tencent/Hy3#more-reliable-product-experiencesMore Reliable Product Experiences

Model usefulness is not fully captured by benchmarks. Based on extensive product feedback, we identified and fixed the following issues, receiving consistently positive feedback from product teams.

Stability of tool calls and output formats: We fixed multiple baseline reliability issues, bringing the model to production-grade standards across tool configurations and output constraints. Tool-call error recovery and overall efficiency improved. Hy3 also generalizes across different agent scaffoldings. On SWE-Bench Verified, accuracy variance across scaffoldings like CodeBuddy, Cline, and KiloCode remains within 4%.

Knowledge and anti-hallucination: Guided by the ideal of “answer when grounded, state when evidence is missing, do not conflate sources or fabricate data,” we implemented fine-grained data cleaning and training constraints. In internal evaluations based on real-world scenarios, Hy3’s hallucination rate dropped from 12.5% to 5.4%, and commonsense error rates fell from 25.4% to 12.7%. These improvements materially reduce fact conflation, fabrication, and logical contradiction.

Complex context retention and multi-turn intent tracking: Through joint optimization of SFT and RL, Hy3 improved on operational pain points like coreference resolution, ellipsis recovery, and multi-turn constraint inheritance. On internal comprehensive multi-turn tests, the issue rate dropped from 17.4% to 7.9%. Hy3 also improved markedly on long-dialogue evals like MRCR. Its outputs are more concise while ensuring complex intents do not decay or drift over long-horizon interactions.

https://huggingface.co/tencent/Hy3#benchmark-appendixBenchmark Appendix

https://huggingface.co/tencent/Hy3#newsNews

• 🔥 We open-sourceHy3andHy3-FP8model weights onHugging Face,ModelScope,GitCode, andCNB.

https://huggingface.co/tencent/Hy3#model-linksModel Links

https://huggingface.co/tencent/Hy3#quickstartQuickstart

Deploy Hy3 withvLLMorSGLangfirst, then call the OpenAI-compatible API:

from openai import OpenAI

client = OpenAI(base_url="http://127.0.0.1:8000/v1", api_key="EMPTY")

response = client.chat.completions.create(
    model="hy3",
    messages=[
        {"role": "user", "content": "Hello! Can you briefly introduce yourself?"},
    ],
    temperature=0.9,
    top_p=1.0,
    # reasoning_effort: "no_think" (default, direct response), "low", "high" (deep chain-of-thought)
    extra_body={"chat_template_kwargs": {"reasoning_effort": "no_think"}},
)
print(response.choices[0].message.content)

Recommended parameters:temperature=0\.9,top\_p=1\.0. Reasoning mode: Setreasoning\_effortto"high"for complex tasks (math, coding, reasoning) or"no\_think"for direct responses.

See theDeploymentsection below for how to start the API server.

https://huggingface.co/tencent/Hy3#deploymentDeployment

Hy3 has 295B parameters in total. To serve it on 8 GPUs, we recommend using H20-3e or other GPUs with larger memory capacity.

For production serving, we recommend using vLLM or SGLang, both of which provide dedicated recipes for Hy3:

• vLLM- seevLLM recipes
• SGLang- seeSGLang cookbook

https://huggingface.co/tencent/Hy3#vllmvLLM

Build vLLM from source:

uv venv --python 3.12 --seed --managed-python
source .venv/bin/activate
git clone https://github.com/vllm-project/vllm.git
cd vllm
uv pip install --editable . --torch-backend=auto

Start the vLLM server with MTP enabled:

# Switch to trtllm backend to work-around mnnvl workspace size issue.
export VLLM_FLASHINFER_ALLREDUCE_BACKEND=trtllm
vllm serve tencent/Hy3 \
  --tensor-parallel-size 8 \
  --speculative-config.method mtp \
  --speculative-config.num_speculative_tokens 2 \
  --tool-call-parser hy_v3 \
  --reasoning-parser hy_v3 \
  --enable-auto-tool-choice \
  --port 8000 \
  --served-model-name hy3

https://huggingface.co/tencent/Hy3#sglangSGLang

Build SGLang from source:

git clone https://github.com/sgl-project/sglang
cd sglang
pip3 install pip --upgrade
pip3 install "transformers>=5.6.0"
pip3 install -e "python"

Launch SGLang server with MTP enabled:

python3 -m sglang.launch_server \
  --model tencent/Hy3 \
  --tp-size 8 \
  --tool-call-parser hunyuan \
  --reasoning-parser hunyuan \
  --speculative-num-steps 2 \
  --speculative-eagle-topk 1 \
  --speculative-num-draft-tokens 3 \
  --speculative-algorithm EAGLE \
  --port 8000 \
  --served-model-name hy3

https://huggingface.co/tencent/Hy3#finetuningFinetuning

Hy3 provides a complete model finetuning pipeline. For detailed documentation, please refer to:Finetuning Guide

https://huggingface.co/tencent/Hy3#rl-post-trainingRL Post-training

Hy3 supports GRPO reinforcement learning training withverl, training on Megatron-LM (model conversion via NVIDIA Megatron-Bridge) with vLLM rollout. For detailed documentation, please refer to:RL Training Guide

https://huggingface.co/tencent/Hy3#quantizationQuantization

We provideAngelSlim, a more accessible, comprehensive, and efficient toolkit for large model compression. AngelSlim supports a comprehensive suite of compression tools for large-scale multimodal models, including common quantization algorithms, low-bit quantization, and speculative sampling.

https://huggingface.co/tencent/Hy3#licenseLicense

Hy3 is released under theApache License 2.0. SeeLICENSEfor details.

https://huggingface.co/tencent/Hy3#contact-usContact Us

If you would like to leave a message for our R&D and product teams, welcome to contact us. You can also reach us via email:

📧[email protected]

Hy3 is developed by the Tencent Hy Team.