granite

IBM has launched Granite 4.2, a family of open-weight large language models with variants up to 30B parameters, featuring a 128,000-token context window and agentic reinforcement learning for enhanced capabilities.

An independent benchmark of PrismML's 1-bit Bonsai-8B against IBM's Granite and other models on CPU tool calling shows that with grammar-constrained decoding, Bonsai-8B achieves a 92% pass rate, outperforming larger models, but fails without constraints. Granite is the best raw model at 72%.

A new contrastive ablation operator called apostate is introduced that reduces model refusal from 96% to 5% while preserving harmless behavior with only 0.081 KL divergence, tested on Granite 3.3-8B.

This pull request adds support for the Granite4 Vision model to llama.cpp, an open-source LLM inference engine.