How would you separate model improvements from agent infrastructure in an evaluation?

In the October 12 Cheeky Pint episode, Zuckerberg and Wang describe agent behaviour as something shaped through model training, while also discussing the surrounding privacy and security infrastructure. How would you test which layer is responsible for better results? Holding the tools and task set fixed while swapping models seems useful, but models may have been trained around different tool interfaces. Would a comparison need both a shared setup and each model’s preferred setup, with task completion, permission violations and recovery from failed actions reported separately? Source: https://cheekypint.transistor.fm/35/transcript