Tag
CacheScout is an agent-aware KV-cache runtime layer for multi-agent LLM serving that learns agent execution semantics online to guide cache eviction and prefetching, improving cache hit rate and reducing latency.
This paper introduces a contract-grade verifier of twelve adversarial gates for checking LLM-generated GPU kernels, finding that 39.5% of kernels accepted by standard loose tests are broken. It also presents the first native Blackwell training backward kernel for the GDN (gated-linear-recurrence) family.
Hawk is a training-free framework that uses hardware-aware knowledge to improve NPU kernel generation via LLMs, raising generation accuracy from 49.4% to 80.0% and achieving up to 2.2× execution speedup over state-of-the-art baselines.
Junchen Jiang delivered three keynotes arguing that KV cache is an underappreciated asset for LLM inference, enabling cost savings, latency reduction, and quality improvements, and should be treated as a core data layer in future inference infrastructure.
This article introduces ExecuTorch, a unified PyTorch-native deployment framework designed to run AI models on diverse edge devices without requiring model conversion or reimplementation.