@julien_c: transformers x @ggml_org collab
Summary
Transformers and ggml_org collaborate to integrate GGML kernels into the transformers library, enabling GGUF file loading at performance levels comparable to llama.cpp.
View Cached Full Text
Cached at: 09/23/26, 12:05 PM
transformers x @ggml_org collab 🐐
Lysandre (@LysandreJik): Transformers has supported loading GGUF files for a few years now, by unquantizing them.
Thanks to @_marcsun, we’re now using GGML kernels through the
kernelslibrary to run at the same performance as llama.cppHuge kudos to the entire @ggml_org for making these kernels!
Similar Articles
Transformers now runs llama.cpp quants
Hugging Face's transformers library now supports GGUF models from llama.cpp, enabling efficient local inference on consumer hardware through familiar APIs.
@ggerganov: Run GGUF models directly with transformers. This work brings ggml's Metal kernels to the transformers ecosystem, increa…
This work integrates ggml's Metal kernels into the transformers library, allowing GGUF models to run directly with improved compatibility and performance for fast local inference on Mac.
GGUFs in transformers natively!
Hugging Face announces native support for GGUF files in the transformers library, allowing easier use of quantized models with PyTorch tooling and performance comparable to llama.cpp.
GGML and llama.cpp join HF to ensure the long-term progress of Local AI
GGML and llama.cpp have joined Hugging Face to ensure long-term sustainability of local AI development. Georgi Gerganov's team will maintain full autonomy over the projects while receiving resources to scale community support and improve integration between llama.cpp inference and transformers model definitions.
huihui-ai/Huihui-GLM-5.2-abliterated-GGUF
A quantized GGUF version of the abliterated GLM-5.2 model is released on Hugging Face, enabling local inference with various tools like Transformers, llama.cpp, and vLLM.