Tag
Nvidia participated in a $14 million Series B round for Indian startup Simplismart, which builds software to run AI models more efficiently on existing GPU infrastructure, valuing the company at about $100 million.
DeepSeek V4.1 Flash 的大规模推理部署文章，介绍 DeepInfer 结合分布式推理、缓存、负载均衡与投机解码提升效率，DeepSeek 单集群日处理 155 万亿 token，每 GPU 小时收入 38 美元，并开源了大部分推理框架。
该论文研究在数据有限但算力充裕场景下的重复数据预训练，发现小批量通过梯度噪声提供隐式正则化，并在固定时间预算下小批量+少GPU可取得更低验证损失，挑战了常见地增大数据并行度以最大化GPU利用率的做法。
A 1.4M-parameter U-Net student model, distilled from FLUX.2 klein, achieves real-time 30-40 FPS neural weather restyling (snow, wet, night) inside a Minecraft Fabric mod on a budget GTX 1650, using a PatchGAN fine-tune to fix washed-out outputs from pure pixel losses.
A paper introduces galahad-kv, a memory layer that persists LLM key-value states to encrypted local NVMe disk and reloads them byte-exact instead of recomputing, demonstrating a 50-million-token window on a single H100 with Gemma 4 models that is 2.8–4.3x faster and 8.8–12.3x less energy-hungry than recompute.
PyTorch Foundation Ambassador Abdulsalam Bande will present a poster on ERRC (Entropy-Reinvested Residual Correction) at PyTorch Conference North America 2026 in San Jose, compressing inter-GPU communication to boost multi-GPU LLM inference efficiency, alongside the broader conference program covering the open-source AI stack.
dotwave.ai shares technical write-ups showing their inference engine can serve 56 concurrent real-time sessions on a single H100 (vs 1 for NVIDIA's reference stack) by keeping the model resident on the GPU and batching all sessions per tick, cutting per-conversation GPU cost by ~98% with zero missed 160ms audio deadlines.
QwenGyre is an elastic reinforcement learning framework designed to address challenges in training extreme long-horizon LLM agents, improving GPU utilization and reducing trajectory redundancy for enhanced training efficiency.
Demonstrates running the quantized Qwen3.6-35B model on a low-end laptop to create a Zelda-like RPG in 24 minutes, showing that GPU-poor scenarios are manageable in 2026.
TRANSIT is a runtime that makes unified virtual memory practical for large-scale LLM training, reducing GPU usage by up to 50% without code changes, and will be presented at PyTorch Conference North America.
A tweet discusses the overlooked issue of GPU efficiency post-deployment and references a startup raising $13M to address GPU idle time through virtualization.
ModeSwitch-LLM is a lightweight controller that routes LLM inference requests to appropriate fixed modes (e.g., FP16, quantization, speculative decoding) on a single GPU, achieving up to 2.10× latency speedup and 51.7% energy reduction without retraining the model.
The author speculates that loading only active parameters of MoE models onto GPUs could drastically improve efficiency and allow running large models like Kimi locally, though acknowledges this is currently impractical.
Unsloth open-source tool boosts large-model fine-tuning speed 2-5× and slashes VRAM by 80%, letting a single RTX 4090 finish in hours what once needed an A100 cluster.