Hy3 1Bit 89-93 GB
Summary
Announcement of Hy3 1-bit quantized model with 89-93 GB memory footprint.
Similar Articles
@Xudong07452910: A flagship large model with 295B parameters can now run on a single 96GB inference GPU, with 50% faster decoding. Tencent Hunyuan team releases quantized versions for Hy3 (295B parameters). The 1-bit version (IQ1_M) compresses weights from 598GB to 85.5GB, a 6…
Tencent Hunyuan team releases quantized versions for the 295B-parameter Hy3 large model. The 1-bit version compresses weights to 85.5GB, enabling deployment on a single 96GB inference GPU with ~50% faster decoding. The open-source GGUF format is compatible with the llama.cpp ecosystem.
@atomic_chat_hq: 1-bit Hy3 running locally is 2.2x faster than its API at the same quality! We gave both models the same task and compar…
Tencent's Hy3 295B model now available in 1-bit and 4-bit GGUF formats, achieving 2.2x faster local inference compared to cloud API while maintaining quality, as demonstrated by running on 4x RTX 5090 with 128GB VRAM.
Hy3 (295B MoE) and NVIDIA Nemotron-Labs-Audex-30B-A3B (audio-capable 30B MoE) GGUF quants
GGUF quantizations of Tencent's Hy3 295B MoE model and NVIDIA's audio-capable 30B MoE model are released with detailed benchmarks, imatrix calibration, and full reproducibility data.
@0xSero: Just added 2 new model compressions: Hy3-FP8 & NVFP4 I recommend trying this model it's very strong and fits on 256gb o…
0xSero has released new FP8 and NVFP4 quantized versions of the Tencent Hy3-preview model, enabling it to run on 256GB VRAM with full context.
llama.cpp: Hy3 PR + GGUFs
The Hy3 model is now supported in llama.cpp via a pull request, with GGUF quantizations available. Early testing shows coherent output from Q2_K at 10-11 t/s on high-end hardware.