openvino

The article presents research on distributed LLM inference for Intel PC fleets, focusing on pipeline-parallel sharded inference using OpenVINO with performance optimizations for heterogeneous hardware.

Gemma 4 E2B achieves 1.3x faster prefill and 2.8x better performance-per-watt on Intel AI PCs using OpenVINO with LiteRT NPU support, enabling efficient background LLM tasks.