llama.cpp updates - granite-speech-4.1-2b, LFM2.5-ColBERT/Embedding-350M, Vulkan backend related changes & Misc items

Reddit r/LocalLLaMA Tools
llama-cpp vulkan model-support backend-updates open-source ui-improvements

Summary

llama.cpp adds support for granite-speech-4.1-2b and LFM embedding models, introduces multiple Vulkan backend enhancements (e.g., CONV_3D, spec constants, overflow fixes), and includes UI/UX improvements and miscellaneous fixes.

Supported Models: granite-speech-4.1-2b-plus by 24818 LFM2.5-ColBERT-350M & LFM2.5-Embedding-350M by 24913 Vulkan: vulkan: link ggml-cpu when GGML_VULKAN_CHECK_RESULTS / RUN_TESTS are enabled #24444 vulkan: make mul_mm ALIGNED a spec constant #24689 vulkan: support CONV_3D #24612 vulkan: Support GET_ROWS_BACK #24883 vulkan: support all backend tests for SQR/SQRT/SIN/COS/CLAMP/LEAKY_RELU/NORM #24582 vulkan: Apply bias before softmax in FA, to avoid overflow #24909 Misc: ui: New Logo + Navigation cleanup & Mobile UI/UX improvements #24897 And other fixes, etc., Hope that Vulkan list gives some boost on pp/tg(Experts could let us know about that). Don't want to post multiple threads(for those models) so including all other items in this single thread.
Original Article

Similar Articles

llama.cpp milestone

Reddit r/LocalLLaMA

A milestone release of llama.cpp, the open-source library for running large language models locally on consumer hardware, bringing improvements in performance or new features.

ggml-org/llama.cpp

GitHub Trending (daily)

llama.cpp is an open-source C/C++ library for efficient LLM inference on local hardware, supporting various quantization methods and multiple backends (CPU, GPU, etc.).

llama.cpp B9387 Significant AMD/ROCm PP Update

Reddit r/LocalLLaMA

llama.cpp version b9387 introduces MFMA support for AMD CDNA architecture (MI100, MI200, MI300 series), improving processing pipeline performance on datacenter AMD GPUs.