inference-scheduling

The paper introduces Decode-Latency Feedback Prefill (DLFP), a model-free proportional controller implemented in vLLM that adapts chunked prefill sizing based on observed decode latency, reducing P99 inter-token latency by ~28% on Qwen3-0.6B on an A100, while also documenting a clear generalization failure on larger models and multi-GPU setups.