expert-placement

This paper introduces Director, a distributed MoE serving system that minimizes end-to-end latency using prediction-driven, online proactive expert placement. It employs a lightweight predictor and a relaxation-based optimizer to achieve up to 55% latency reduction for models like Mistral, DeepSeek, and Qwen.