compound-ai

CAFE is an open-source platform that applies design-of-experiments principles to evaluate compound AI systems, attributing answer quality variance to components and their interactions using factorial designs and mixed-effects models.

Introduces BOHM, a zero-cost hierarchical attribution method for compound AI systems that extracts attribution from routing weights, outperforming Shapley-based methods in many real-world deployments.

Maxime Rivest argues that compound AI systems for images are undervalued and suggests leveraging optimization frameworks like DSPy and GEPA to automate pipeline creation involving SAM and classifiers.