learned-scheduler

Tag

Cards List
#learned-scheduler

Calibrate, Then Route: A Measured Study of Learned Request Routing for Disaggregated LLM Serving

arXiv cs.AI · 3d ago Cached

This paper presents a measured study on learned request routing for disaggregated LLM serving, evaluating a calibrated router that uses admission-time features to assign requests, achieving higher mean goodput compared to traditional methods on a real vLLM/NIXL cluster.

0 favorites 0 likes
← Back to home

Submit Feedback