@TheAhmadOsman: HOLYYYY 27B model under 6GBs and 4GBs Local AI will be the default P.S. We are gonna get this optimized in ODS by @Osma…
Summary
Ternary Bonsai 27B, a large language model, is demonstrated running locally on an NVIDIA RTX 5090 GPU, requiring under 6GB of memory and enabling end-to-end agentic workflows on consumer hardware.
View Cached Full Text
Cached at: 07/14/26, 10:33 PM
HOLYYYY
27B model under 6GBs and 4GBs
Local AI will be the default
P.S. We are gonna get this optimized in ODS by @OsmanticAI ASAP https://t.co/cUJ2fIbQeB
PrismML (@PrismML): Here is Ternary Bonsai 27B running an end-to-end agentic workflow locally with Hermes on an NVIDIA GeForce RTX 5090 GPU.
The model reasons, calls tools, reads outputs, modifies files, and surfaces insights - all on consumer hardware, while all private files, intermediate states,
Similar Articles
@witcheer: can’t believe gpt-oss-20b perfs on 8GB vRAM 21B total params, 3.6B active (MoE). OpenAI, Apache 2.0. uses only 1.8 GB V…
A new open-source MoE model, gpt-oss-20b (21B total, 3.6B active), runs on only 1.8GB VRAM and achieves perfect scores on agentic coding tasks, outperforming other local models like Gemma and Qwen.
@UnslothAI: GLM-5.2 can now be run locally! The 2-bit model retains ~82% accuracy after we shrunk it from 1.51TB to 238GB (-84% siz…
UnslothAI announces GLM-5.2, Z.ai's strongest open model with 744B parameters, now runnable locally via dynamic GGUF quantization reducing size by ~84% to 239GB while retaining ~82% accuracy. It fits on 256GB Macs and supports long-context, reasoning, and agentic tasks.
Bonsai 27B: 1-bit dense LLM running locally in your browser using custom WebGPU kernels
Bonsai 27B is a 1-bit dense large language model that can run locally in a browser using custom WebGPU kernels, enabling efficient on-device inference.
@TheAhmadOsman: Been playing with @PrismML's new model that turned Qwen 3.5 27B into a sub-4GB and sub-6GB weights and I am impressed C…
PrismML released a compressed version of Qwen 3.5 27B that fits in sub-4GB and sub-6GB memory, enabling impressive local AI performance.
@sudoingX: this is a laptop running a 31b parameter model at 99% gpu autonomously through hermes agent, 15 tok/s sustained, 22.8 o…
A 31B parameter model runs locally on a laptop via Hermes agent at 15 tok/s, using 22.8 GB VRAM and 94 W power, highlighting fully autonomous, private AI inference without cloud dependencies.