on-device-llm

This paper empirically studies how prompt wording affects energy consumption for on-device LLMs, showing that keyword choices can significantly impact decoding length and total energy, suggesting prompt engineering as a lightweight energy optimization lever.

Quant.npu introduces a fully static quantization framework for mobile NPUs, using learnable parameters and rotation matrices to enable efficient low-bit LLM inference without runtime re-computation, achieving up to 15.1% latency reduction.