Tag
This paper introduces HYMELL, a hybrid analytical-machine-learning framework for estimating LLM inference latency and energy across prefill and decode phases, validated on NVIDIA H100 with under 5% error for LLaMA 3 8B.
This paper presents an analytically structured, empirically calibrated methodology for estimating LLM inference energy on NVIDIA H100 GPUs without direct measurement, separating prefill and decoding phases and decomposing energy into compute, parameter-access, KV-cache write, and attention-read components.
This paper introduces WattLayer, a task-independent layer-wise energy estimation model for neural networks, evaluated on over 100,000 layers across 295 architectures, achieving a median error of 19.6% and outperforming state-of-the-art methods.