Discovering Reinforcement Learning Interfaces with Large Language Models
Summary
This paper introduces LIMEN, an LLM-guided evolutionary framework that automatically discovers reinforcement learning interfaces by jointly optimizing observation mappings and reward functions from raw simulator states. The approach reduces manual engineering effort and demonstrates that co-designing observations and rewards outperforms optimizing either component alone.
View Cached Full Text
Cached at: 05/11/26, 02:52 PM
Paper page - Discovering Reinforcement Learning Interfaces with Large Language Models
Source: https://huggingface.co/papers/2605.03408
Abstract
Automated reinforcement learning interface discovery using LLM-guided evolutionary algorithms that jointly optimize observation mappings and reward functions from raw simulator state.
Reinforcement learningsystems rely onenvironment interfacesthat specify observations andreward functions, yet constructing these interfaces for new tasks often requires substantial manual effort. While recent work has automated reward design usinglarge language models(LLMs), these approaches assume fixed observations and do not address the broader challenge of synthesizing complete task interfaces. We study RL task interface discovery from raw simulator state, where bothobservation mappingsandreward functionsmust be generated. We propose LIMEN (Code available at https://github.com/Lossfunk/LIMEN), a LLM guidedevolutionary frameworkthat produces candidate interfaces as executable programs and iteratively refines them usingpolicy trainingfeedback. Across novel discrete gridworld tasks and continuous control domains spanning locomotion and manipulation,joint evolutionof observations and rewards discovers effective interfaces given only atrajectory-level success metric, while optimizing either component alone fails on at least one domain. These results demonstrate that automatic construction of RL interfaces from raw state can substantially reduce manual engineering and that observation and reward components often benefit fromco-design, as single-component optimization fails catastrophically on at least one domain in our evaluation suite.
View arXiv pageView PDFProject pageGitHub3Add to collection
Get this paper in your agent:
hf papers read 2605\.03408
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2605.03408 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2605.03408 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2605.03408 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
LEEPS: Latent-Guided Explore-Exploit Prompt Sampling for Efficient RLVR in Large Language Models
This paper introduces LEEPS, a latent-guided explore-exploit prompt sampler for efficient reinforcement learning with verifiable rewards (RLVR) in LLMs. It adaptively balances reuse of informative prompts and exploration of uncertain ones, improving reasoning benchmark scores by 2.6-3.7% over baselines while adding only ~2 seconds of overhead per training step.
From Trainee to Trainer: LLM-Designed Training Environment for RL with Multi-Agent Reasoning
This paper introduces LLM-as-Environment-Engineer, a framework where LLMs design their own training environments for reinforcement learning in multi-agent reasoning tasks, enabling self-improving training that surpasses larger proprietary models.
From Trainee to Trainer: LLM-Designed Training Environment for RL with Multi-Agent Reasoning
This paper proposes the LLM-as-Environment-Engineer framework, where a policy model analyzes failures to automatically redesign the training environment for reinforcement learning, and introduces MAPF-FrozenLake as a controllable testbed. The framework, using Qwen3-4B, outperforms larger models like GPT and Gemini, showing that policy learning improves the model's ability to diagnose weaknesses.
LLM-Driven Evolutionary Generation of Multi-Objective Bayesian Optimization Algorithms
The paper extends the LLaMEA framework to automatically design multi-objective Bayesian optimization algorithms using large language models as mutation and crossover operators within evolutionary strategies, achieving state-of-the-art accuracy with significantly lower computational cost on synthetic and real-world problems.
Feature Generation Using LLMs: An Evolutionary Algorithm Approach
This paper proposes a method that uses large language models to generate new features from tabular data via an evolutionary algorithm, demonstrating improved classification results across multiple datasets.