@no_stp_on_snek: In progress
Summary
Promoting Atlas Inference, an open-source inference serving tool that achieved 200+ tok/s on a Qwen3.6-35B-A3B benchmark.
View Cached Full Text
Cached at: 05/24/26, 08:18 AM
In progress https://t.co/DFkWLU43lH
Azeez (@AtlasInference): Try Atlas Inference. You’ll be ready to serve in <2 mins. https://t.co/vxZLwBJMub ⚡️
Works with sparkrun out the box, happy to share Docker commands as well but all are on the website.
Open source too, most recently achieved 200+ tok/s on a Qwen3.6-35B-A3B benchmark!
Similar Articles
@no_stp_on_snek: https://x.com/no_stp_on_snek/status/2052833502475833384
An open-source stack using Qwen2.5-32B-Instruct with longctx and vllm-turboquant on a single AMD MI300X achieves competitive results (0.601-0.688) versus SubQ's closed model (0.659) on the MRCR v2 1M-context benchmark, demonstrating open-weights approaches are within striking distance.
@no_stp_on_snek: Check out Buun's work, he cookin.
A user highlights Buun's work on optimizing AI models, achieving high-speed inference of Qwen 3.6 on a single 3090 GPU and developing DFlash2 for Qwen 3.8.
@no_stp_on_snek: Very nice. Huge for team 3090. And TurboQuant+ is already implemented in a bunch of inference engines.
A reply celebrates Unsloth AI's upcoming Qwen3.8-27B model, which will run on 17GB RAM/VRAM setups, and notes TurboQuant+ is already integrated into many inference engines — great news for RTX 3090 users.
@bastani_behnam: We just published how we unlocked +50% inference capacity on a 27B model — no new GPUs, no new nodes, at a fraction of …
OpenInfer demonstrates "vertical disaggregation" that boosts Qwen 3.5 27B throughput by ~50% by co-executing quantized layers across a single node’s AMD EPYC CPU and Nvidia L40S GPU with a custom SLA-aware scheduler.
@no_stp_on_snek: btw this was my loop. as you can see i didn't put much thought into it (typos and all), just a side thing to assess the…
Release of Qwopus3.6-27B-v2-MTP, a fine-tuned multi-token prediction reasoning model based on Qwen3.6-27B, optimized for coding, DevOps, and math tasks with improved generation speed.