Tag
The PyTorch Foundation project Ray announced a Special Interest Group (SIG) for Ray on Accelerators, uniting hardware vendors, cloud providers, and the Ray community around a vendor-neutral roadmap covering hardware-aware testing, topology-aware scheduling, and Ray Direct Transport.
CrowdGPT v2.6 推出了重大更新，通过众包方式在普通 PC 上协作训练 10 亿参数 LLM，加入拜占庭交叉验证、加密签名与协作数据集，并在 6 天内新增训练 4.5B tokens。
HyScale 提出了一种面向长上下文训练的学习型稀疏注意力系统，通过 HyTopK（免分数表的精确 top-k 选择）和选择重放（selection replay）技术，在 1M token 序列上将 GLM-5.1 训练速度提升至 Megatron-LM 的 2.07 倍。
论文提出一种对抗鲁棒的广义编码计算（GCC）扩展框架，通过平滑正则化限制恶意worker输出的影响，并给出随对抗worker比例衰减的非渐近误差界，适用于分布式系统中的近似编码计算。
At PyTorchCon 2026, Ion Stoica traces Ray's evolution from reinforcement learning into the distributed compute layer behind modern AI workloads, highlighting topology-aware scheduling for TPU pods and Blackwell GPUs, NIXL GPU transfers, TorchFT integration, and future plans to rewrite core components in Rust and deepen Kubernetes integration.
苹果发布新版 MLX 框架，展示如何在单台 Mac Studio 上运行 Muse Spark Open（近万亿参数）等前沿模型，并通过 RDMA over Thunderbolt 驱动在多台 Mac 间实现分布式张量/流水线并行推理，在四台 M5 Ultra 集群上达到每秒 36 tokens 以上的解码速度。
Apple researchers present Minions, a system that aggregates idle Apple Silicon GPUs across macOS devices into a dynamic, fault-tolerant distributed virtual GPU cloud for large-scale ML inference, offering a cost-effective alternative to centralized GPU infrastructure.
FlashMask-3 introduces a Hopper-optimized mask-aware sparse attention kernel and zero-communication distributed mask localization for long-context training, achieving major TFLOPs and throughput gains over FlashMask, FlexAttention, MagiAttention, and Megatron-Core baselines, including 7.3-11.0% end-to-end speedup on a 118B MoE trained on 128 H100 GPUs at 128K context.
Unison Cloud が MIT ライセンスでオープンソース化されました。Nimbus ワーカーノード、API サーバー（Haskell 製）、UI など4つのプロジェクトが公開され、任意のノードプールを分散コンピュータとして扱えるようになります。
Rgpu 是一个开源工具，可通过 TCP 让 PyTorch 张量和现有 CUDA 程序在远程 NVIDIA GPU 上执行计算，客户端应用保持在本地运行，提供 PyTorch 设备接口和 CUDA shim 两条路径。
PyTorch blog publishes a Ray-focused guide to PyTorch Conference North America 2026 in San Jose, highlighting keynotes and sessions on co-evolving Ray and Kubernetes, elastic training stacks, and scalable RL from speakers at Anyscale, Google, LinkedIn, Pinterest, and Uber.
This article demonstrates how to pool RAM from old devices to run a larger local AI model and build a private chatbot using custom knowledge from PDFs, ensuring all data remains local.
The article explores the idea of building a community-owned AI to compete with corporate frontiers, referencing open-source software models and suggesting technologies like Petals for distributed GPU computation, while highlighting organizational over technical hurdles.
DeepSeek Elastic Compute (DSec) is a sandbox infrastructure for effective agentic training of large language models at scale, featuring elastic execution with unified SDK and integration with reinforcement learning frameworks.
The article explores the concept of decentralized AI as an alternative to centralized systems, reviewing research on federated learning and distributed computing that could enable collective ownership of AI.
PyTorch 2.14 release introduces fault tolerance as a first-class c10d concept with in-place process-group reconfiguration, alongside features like NVGEMM kernels, a new nccl2 backend, and native linear algebra for Apple Silicon.
NVIDIA announced PAIR at IFA, a tool that automatically links local network systems to distribute inference requests for efficient AI agent operation.
Nvidia has launched PAIR, an open-source tool that links idle home computers to create a personal AI data center for distributed local AI inference tasks, supporting agentic workflows.
This paper proposes a manifold-aware encoding strategy for general coded computing that exploits the intrinsic low-dimensional geometry of high-dimensional data to improve straggler resilience in distributed systems. Experiments show reduced mean squared recovery error in neural network inference and polynomial evaluation tasks.
PyTorch 2.14 introduces updates to Inductor, PyTorch Distributed, dynamic shapes, Apple Silicon support, and more, with a live Q&A and PyTorchCon announcement.