@no_stp_on_snek: Development time on inference engines I’m working on

@no_stp_on_snek: In progress

X AI KOLs Following

Promoting Atlas Inference, an open-source inference serving tool that achieved 200+ tok/s on a Qwen3.6-35B-A3B benchmark.

@no_stp_on_snek: can the inference engine itself change model behavior? ran two quant and speculative-decode stacks of the same base mod…

X AI KOLs Following

A developer compares two inference stacks (production build vs SignalNine's q27) on the same Qwen model and finds they produce different honesty under pressure, with one fabricating progress and the other refusing appropriately, suggesting inference engines can affect model behavior beyond speed and quality metrics.

@mohitwt_: Day 20/30 of Inference Engineering building a speculative decoding runtime that drafts multiple tokens ahead with a sma…

X AI KOLs Following

A developer is building Octane, a speculative decoding runtime for local LLM inference on consumer hardware, aiming for 2-3x speedup with exact output quality. Currently in active development with paged KV cache, continuous batching, and batched attention implemented.

@TheAhmadOsman: LLM Inference Engine Stack Breakdown and Workload/Bottlenecks Cheatsheet From the upcoming Inference Engine Comprehensi…

X AI KOLs Timeline

Ahmad Osman shares a cheatsheet breaking down the LLM inference engine stack and common workload bottlenecks ahead of a comprehensive article.

@jino_rohit: over the last 6-8 months, ive been trying to move towards the ml systems and ai infra space. these are some of my favor…

X AI KOLs Timeline

The author shares their work over 6-8 months in ML systems and AI infrastructure, including a lightweight Python LLM inference engine (tachyon) that achieves 600+ tokens/s on consumer hardware with continuous batching and prefix caching, alongside blog posts on CUDA/CUTE DSL and collective communication, and contributions to SGLang and vLLM.