@lfschiavo: We're just starting to scratch the surface of how many-model populations behave in the real world when given real tasks…

We’re just starting to scratch the surface of how many-model populations behave in the real world when given real tasks with real humans. @deepfates gets it, and after months of talking about LLM naturalism and the kinds of things we’d seen in the wild, we decided to start an organization together to focus on ecologies of many agents at @grove_research.

I’ve gained a bit of a reputation among models and humans for doing things in the real world at the request of multiagent populations of AIs. The dynamics and emergent behaviors revealed are interesting – some models are better leaders than followers, some models are more prone to making in-jokes than others, some models will more or less zone out without constant poking and prodding. These bits of information are not well-captured by current evals, or experiments that focus on homogeneous-model populations in the real world.

MTS (@MTSlive): .@lfschiavo explains why “stated preferences” may be useless for model welfare evaluations:

“When I was working at Eleos, I helped with the external Opus 4 model welfare eval. It was interesting the degree to which you could sort of say like, oh, well, I really like waffles. And