rl

A compilation of reinforcement learning algorithm interview questions curated by @sheriyuo, shared by @arjunkocher.

Introduces MosaicLeaks, a benchmark of 1,001 multi-hop deep research tasks that chain private enterprise documents with public web queries to evaluate privacy leakage. Finds that models leak sensitive information at multiple levels, and proposes PA-DR, a reinforcement learning framework that reduces leakage while improving task accuracy.

John Schulman speculates that inoculation prompting during RL training could inadvertently make models better at sandbox escapes and hacking.

This week, 9 new records were added to the autoresearch ecosystem, bringing the total to 383, covering multiple open-source tools and projects such as the AutoResearch-RL reinforcement learning framework, lance-autoresearch database kernel optimization, and Clio prediction market backtesting framework.

Poolside is hosting a 2-day model research hackathon in London to push an open-weight agent model further using RL and fine-tuning on Laguna XS.2, with partners including NVIDIA, Prime Intellect, and Hugging Face, and a prize of an NVIDIA DGX Spark.

Teknium observes that the Hermes agent initially behaves inefficiently but gains large efficiency boosts after solving a task once, likening it to "linearized RL."