Tag
The paper introduces inference networks, a graph-based framework for optimizing the use of multiple LLMs in inference, with optimal activation policies that minimize cost while meeting performance targets.