Tag
A highly recommended 4.5-hour GPU programming lesson on CUDA and ThunderKittens by Ben Spector, offering an in-depth, behind-the-scenes look at kernel optimization.
OpenAI's voice AI architecture uses WebRTC with a split relay-transceiver design to handle low-latency audio for 900 million weekly active users.
Google released Nano Banana 2 Lite, a faster and cheaper AI image generator, priced at $0.034 per 1,000 images with 4-second generation times. The model is optimized for high-volume workflows and is available via Google AI Studio and Gemini API.
TileRT is a tile-based runtime achieving ultra-low-latency LLM inference, with recent milestones including 1000+ tokens/s on a 1-trillion-parameter model. It supports models like DeepSeek-V3.2 and GLM-5, and is available as open-source on GitHub.
MaineCoon is a 22B real-time text-to-audio-video model that achieves up to 47.5 FPS on a single H100 GPU, enabling low-cost, long-duration streaming with synchronized speech and visuals for live AI characters.
Wan-Streamer is a unified end-to-end multimodal model for real-time audio-visual interaction using causal attention and integrated processing of visual, audio, and text modalities, achieving sub-second latency.
This paper presents a quantized, integer-only transformer implementation for jet tagging on AMD Versal AI Engines, including a reusable open-source framework that maps transformer layers to AIE tiles for low-latency trigger systems at CERN LHC.
Discusses leveraging Gemma 4 12B's encoder-free architecture for native voice input, seeking out-of-the-box solutions for low-latency streaming audio ingestion.
This paper presents a low-latency real-time audio game commentary system that uses LLM-based parallel text generation to reduce inter-utterance silence from 9.6 to 0.3 seconds, significantly improving perceived speaking rhythm compared to sequential baselines.
DeepMind researcher Brendan O'Donoghue provides an in-depth introduction to text diffusion models, which generate text through iterative denoising. Compared to autoregressive models, they offer lower latency but limited throughput, and demonstrate unique advantages such as self-correction and dynamic computation.
JetBrains launches Mellum, a fast LLM designed for low-latency and high-performance workflows.
Discord migrated over 80% of its voice and video traffic to Cloudflare's edge network spanning 300+ cities, significantly reducing latency and packet loss globally, with improvements like 34% lower ping in Frankfurt.
Resonate is a low-latency, low-memory algorithm for perceptually relevant spectral analysis of audio signals, using resonator models with exponentially weighted moving averages.
The article shares practical lessons for building low-latency, high-throughput AI agents, including workload estimation, token reduction, parallelism, microservices, and handling LLM failures.
An open-weights 8B parameter voice model achieves only 110ms latency, faster than average human conversation latency of 200-250ms. It can be run locally and is freely available via a GitHub repository.
Miso One is an open-source 8B parameter text-to-speech model with real emotional range and 110ms latency, designed for voiceover work.
LuMay Voice Agent is gaining attention for its business workflow focus, built-in CRM and automation, enterprise compliance, and reliable real-world calling capabilities.
Google DeepMind released Magenta RealTime 2, an open music generation model for on-device streaming with low-latency control via text, audio examples, and MIDI.
VisualThink-VLA introduces a visual intermediate reasoning framework for vision-language-action policies that preserves spatial precision and dramatically reduces latency compared to text-based reasoning, achieving sub-second inference and state-of-the-art success rates on robot manipulation benchmarks.
This paper presents a stateful inference architecture for multi-agent tool calling that reuses KV cache across turns and employs speculative decoding, achieving 2.1x-4.2x speedup over vLLM and SGLang on agentic workflows.