Blockway released Agens Volundr 32B Preview, an Apache-2.0 32B dense model on a hybrid architecture where only 18 of 72 layers keep a KV cache, aiming for efficient long-context inference on consumer/local GPUs. The team reports competitive benchmarks versus Qwen3.8-27B on coding and math but weaker agentic performance, with a custom sglang build required for now.
Hi r/LocalLLaMA. I'm on the team at Blockway, a small team in Hong Kong (disclosure: this is our model). Today we released Agens Volundr 32B Preview, the first model built on our own hybrid architecture. We trained it on limited compute, it isn't perfect, and we'd rather tell you where it falls short up front. WHY WE BUILT IT Our customers run models on their own machines. At long context, the KV cache, not the weights, decides what fits. So we designed a model where most layers don't keep one. ARCHITECTURE (72 layers, dense ~32B, every layer runs on every token) 54 KDA (Kimi Delta Attention) layers: linear attention with a fixed-size recurrent state, no KV cache 17 BCSA layers (our compressed-sparse attention): exact window over the last 4,096 tokens; older context pooled 4:1 into blocks, and a learned indexer reads the top 512 blocks 1 full-attention layer (layer 72) Engram: a hashed n-gram memory held in host RAM, attached at 2 of the 72 layers mHC: 4 residual streams instead of 1 So only 18 of 72 layers keep a KV cache. Context window: 262K. SPEED (single user, our sglang build) BF16 on two 48 GB GPUs, decode: 25.1 tok/s at 1K, 24.1 at 8K, 24.1 at 32K, 24.0 at 64K, 23.9 at 128K BF16 prefill: 2,122 / 2,180 / 1,916 / 1,679 / 1,297 tok/s (1K to 128K) INT4 (31.7 GiB) on one 48 GB GPU, decode: 31.0 tok/s at 1K, 29.3 at 8K, 29.1 at 32K Aggregate throughput: 127 tok/s at 8 users, 130 at 16 users (BF16); 117 at 8 users (INT4) DFlash2 drafter (separate repo), single user, same server with it on vs off: up to 3.6x on JSON/tool output, 2.0x on code, about 1.6x in thinking mode. Not worth it above roughly 8 concurrent users. BENCHMARKS (all run by us on one harness with the same settings, including the comparison models; full table and footnote on the model card) Ahead of Qwen3.8-27B on LiveCodeBench v6 (+4.2), HumanEval (+4.3), AIME 2025 (+2.9), MATH-500 (+1.6) Roughly level on MMLU-Pro, IFEval, GPQA Diamond Behind on agent tasks: tau2-bench 74.2 vs 79-80, SWE-bench Verified (50-task subset) 44 vs 58-64. Closing that gap is the main focus of the full v1, which continues pre-training to about 10B tokens and adds training on long agentic sessions. KNOWN LIMITATIONS (please read before trying) Needs our sglang build. Stock sglang and vLLM can't load it yet. GGUF / llama.cpp is planned, not available today. Long agentic sessions are its weakest area in this Preview. It's still training; treat this as a preview, not a final model. RUN IT docker pull ghcr.io/blockwayz/agens-sglang:preview-sm89 (48 GB Ada GPUs) docker pull ghcr.io/blockwayz/agens-sglang:preview-sm90 (H100 / H200) The full launch command is in the model card. LINKS BF16: https://huggingface.co/Blockway/Agens-Volundr-32B-Preview INT4: https://huggingface.co/Blockway/Agens-Volundr-32B-Preview-INT4 DFlash2 drafter: https://huggingface.co/Blockway/Agens-Volundr-32B-Preview-DFlash2 Project page: https://github.com/BlockWayz/Agens-Volundr Serving code: https://github.com/BlockWayz/agens-sglang Apache-2.0. We're a small team, and the most useful thing you can do is try it and tell us where it breaks: an issue, a failing prompt, a benchmark you'd like us to run. We'll be in the comments.
Ternary Bonsai 27B, a large language model, is demonstrated running locally on an NVIDIA RTX 5090 GPU, requiring under 6GB of memory and enabling end-to-end agentic workflows on consumer hardware.
A community developer released Ninfer 4080, an optimized local inference runtime that runs the Qwen 3.8 27B GSQ model at 100k context on a 16GB RTX 4080, achieving up to ~2720 tok/s prefill and 262 tok/s decode via DFlash2 speculative decoding — significantly faster than general-purpose engines like llama.cpp or vLLM.
The article describes a practical setup for running local agentic coding with a quantized Qwen 3.8 27B model on a 12GB VRAM GPU, achieving productive performance using tools like OpenCode and Magic Context for context management.