text-based-games

This paper quantifies cross-lingual skill inconsistencies in large language models through multilingual self-play in text-based games, revealing significant variations in performance across languages that can be partially mitigated by altering intermediate reasoning language.

dig.bench is a benchmark for evaluating AI models' ability to discover unknown rules in text-based games, measuring scientific discovery capabilities with 70 interactive games and a leaderboard comparing frontier models.