Hindsight Experience Replay

# Hindsight Experience Replay Source: [https://openai.com/index/hindsight-experience-replay/](https://openai.com/index/hindsight-experience-replay/) ## Abstract Dealing with sparse rewards is one of the biggest challenges in Reinforcement Learning \(RL\)\. We present a novel technique called Hindsight Experience Replay which allows sample\-efficient learning from rewards which are sparse and binary and therefore avoid the need for complicated reward engineering\. It can be combined with an arbitrary off\-policy RL algorithm and may be seen as a form of implicit curriculum\. We demonstrate our approach on the task of manipulating objects with a robotic arm\. In particular, we run experiments on three different tasks: pushing, sliding, and pick\-and\-place, in each case using only binary rewards indicating whether or not the task is completed\. Our ablation studies show that Hindsight Experience Replay is a crucial ingredient which makes training possible in these challenging environments\. We show that our policies trained on a physics simulation can be deployed on a physical robot and successfully complete the task\.