ray

At PyTorchCon 2026, Ion Stoica traces Ray's evolution from reinforcement learning into the distributed compute layer behind modern AI workloads, highlighting topology-aware scheduling for TPU pods and Blackwell GPUs, NIXL GPU transfers, TorchFT integration, and future plans to rewrite core components in Rust and deepen Kubernetes integration.

Nscale, a full-stack AI cloud platform, has joined the PyTorch Foundation as a Platinum Member, with Nick Jones joining the Governing Board and Eyal Lantzman joining the Technical Advisory Council, following its planned acquisition of Anyscale, the company behind Ray.

Chef Robotics 训练其食品组装机器人背后的 Vision-Language-Action 模型速度提升 10 倍，在 16 块 A100 上实现 92% 的多 GPU 扩展效率，依托 Anyscale 与 Ray 平台加速分布式训练。

Robert Nishihara announces a webinar on large-scale LLM inference, covering engine optimizations like KV cache, paged attention, prefix caching, and continuous batching, plus Ray, vLLM, SGLang integration, agent deployment, and GPU/token cost comparison.

PyTorch blog publishes a Ray-focused guide to PyTorch Conference North America 2026 in San Jose, highlighting keynotes and sessions on co-evolving Ray and Kubernetes, elastic training stacks, and scalable RL from speakers at Anyscale, Google, LinkedIn, Pinterest, and Uber.

Benchmarking distributed LLM inference on Kubernetes reveals unauthenticated control-plane ports and security exposures, with mitigations like NetworkPolicy and WireGuard impacting performance.

Lila Sciences presented at Ray Summit about their internal self-service AI research platform, which uses Flight, Ray, and GitOps to support automated scientific discovery through optimized build-test-learn loops.

NVIDIA BioNeMo Inference Runtime accelerates biomolecular structure-prediction models using optimized kernels, CUDA Graphs, and Ray for high throughput while maintaining PyTorch workflow integration.

RayOrch introduces a programming model and distributed execution engine for lineage-controlled dataflows in foundation-model data preparation, achieving significant speedups on GPUs compared to existing systems like RayData and Daft.

Pinterest Engineering shares a detailed investigation into CPU bottlenecks and network driver issues causing Ray-based training jobs to crash on their Kubernetes platform, offering lessons in profiling performance problems.

British AI neocloud Nscale acquires software startup Anyscale for $1.65 billion to strengthen its AI compute stack by adding workload management and scaling capabilities.

Molt is an open-source, PyTorch-native agentic RL framework from NVIDIA NeMo, designed for scalable, fully-async training of large multimodal agent models using Ray, vLLM, and NVIDIA AutoModel.

Ray 2.56 is released with stability improvements for Ray Data and a re-architecture of Ray Serve for better LLM serving performance.

Ray 2.56 has been released with improvements to Ray Data, Ray Serve for LLMs, GPU-domain-aware placement groups, and Kubernetes integration.

Robert Nishihara highlights a paper on disaggregating RL workloads, showing that using compute-optimized H800s for prefill and bandwidth-optimized H20s for decode can cut rollout times by 21-51% and 47% respectively, emphasizing that no single hardware type fits all stages.

Ray Serve LLM achieves up to 4x higher throughput on prefill-heavy workloads and 24x on decode-heavy workloads in Ray 2.56, matching rust-based routing frameworks like vllm-router in production benchmarks, announced in partnership with Google Cloud GKE team.

This blog post from Anyscale explains the intuition behind Prefill-Decode (PD) disaggregation for LLM serving, showing how separating prefill and decode phases onto dedicated GPUs can achieve up to 2.7x better goodput and 67% cost savings when using Ray and vLLM on AMD MI325X, while also discussing when PD disaggregation does not help.

Anyscale on Azure is now in public preview. Daniel Arrizza and Paul Yu will host a working session on building and deploying production AI workloads within an Azure tenant, integrating with existing Azure services.

Microsoft AI announces MAI-Thinking-1, a 35B active/1T total MoE reasoning model competitive on STEM and coding tasks, developed using Ray for distributed training and orchestration.

Snowflake now supports job-based batch inference powered by Ray, enabling distributed GPU execution for scaling model inference over millions of unstructured datapoints with a single API call.