Tencent compressed Hy4-preview from 1.5TB to about 200GB GGUF and kept about 98% performance.
Summary
Tencent has compressed the Hy4-preview model from 1.5TB to about 200GB in GGUF format while maintaining approximately 98% of its performance.
Similar Articles
Tencent Packs Open Source 770B-Parameter Hy4 Preview into 214 GiB
腾讯混元团队成员公开了 Hy4 Preview(770B 参数 MoE)的极致量化方案,通过 STQ1_0 编码与混合精度分配将约 1.5TB 权重压缩至 214 GiB,并在 llama.cpp 中实现接近 IQ1_M 的推理速度。
Tencent Hy4 Preview full benchmarks
Tencent has previewed full benchmarks for its Hy4 model, providing detailed performance assessments and insights into its capabilities.
Hy4-flash quants are down from 70 to 49 GB, run on a 64Gb vRAM
AtomicChat released GGUF quantizations of Tencent's 220B Hy4-flash MoE that shrink in-memory weights to 49 GB so the model runs on 64 GB VRAM machines, with the Engram n-gram table split into a separate disk-backed file and quality benchmarks against BF16 and Tencent's own quants.
Tencent Hy4-flash (85 GB GGUF) on an RTX 3060 12 GB + 32 GB RAM, with a 20,000-token prompt
A developer ran Tencent's Hy4-flash (85 GB GGUF quant) on an RTX 3060 12 GB with 32 GB RAM using RustingStrata, their Rust GGUF engine, benchmarking 20,000-token prompt handling via CUDA/Vulkan and explaining the expert-weight offloading design that makes it possible.
Tencent/Hy4-preview 770B-A49B weight dropped
Tencent releases Hy4-preview, a new Mixture-of-Experts flagship AI model with 770B total parameters and 49B activated per token, featuring advanced techniques like Gated DeepSeek Sparse Attention and identity Hyper-Connections.