Tencent compressed Hy4-preview from 1.5TB to about 200GB GGUF and kept about 98% performance.

Reddit r/LocalLLaMA Models

Summary

Tencent has compressed the Hy4-preview model from 1.5TB to about 200GB in GGUF format while maintaining approximately 98% of its performance.

No content available
Original Article

Similar Articles

Tencent Packs Open Source 770B-Parameter Hy4 Preview into 214 GiB

Reddit r/ArtificialInteligence

腾讯混元团队成员公开了 Hy4 Preview(770B 参数 MoE)的极致量化方案,通过 STQ1_0 编码与混合精度分配将约 1.5TB 权重压缩至 214 GiB,并在 llama.cpp 中实现接近 IQ1_M 的推理速度。

Tencent Hy4 Preview full benchmarks

Reddit r/singularity

Tencent has previewed full benchmarks for its Hy4 model, providing detailed performance assessments and insights into its capabilities.

Hy4-flash quants are down from 70 to 49 GB, run on a 64Gb vRAM

Reddit r/LocalLLaMA

AtomicChat released GGUF quantizations of Tencent's 220B Hy4-flash MoE that shrink in-memory weights to 49 GB so the model runs on 64 GB VRAM machines, with the Engram n-gram table split into a separate disk-backed file and quality benchmarks against BF16 and Tencent's own quants.

Tencent/Hy4-preview 770B-A49B weight dropped

Reddit r/LocalLLaMA

Tencent releases Hy4-preview, a new Mixture-of-Experts flagship AI model with 770B total parameters and 49B activated per token, featuring advanced techniques like Gated DeepSeek Sparse Attention and identity Hyper-Connections.