Tag
FluidPD introduces SLO-aware in-place elasticity for prefill-decode disaggregated LLM serving, using FluidToken to offload prefill work during transient imbalance and FluidRole to reassign workers between phases during sustained shifts, improving SLO attainment over static SGLang by up to 94.6 percentage points on Azure production traces.
FleetSieve introduces a decision-critical profiling method for SLO-aware LLM fleet configuration that optimizes resource allocation by reducing unnecessary measurements, achieving efficiency gains over uniform profiling.