tropical-semiring

This paper proposes Tropical Reinforcement Learning, which replaces the expected-return sum with a max over verified solution log-probabilities (the tropical semiring) to enable compositional reasoning in LLMs, introducing the Tropic training algorithm that outperforms on-policy baselines by up to 16 points on tasks like Sokoban, Countdown, FrozenLake, and WebShop.