Enforcing Action Similarity in Reinforcement Learning

Résumé: How to make a robot learn to move? In this presentation, I show the challenges of designing a good exploration approach for reinforcement learning environments representing physical systems, such as robots. Specifically, I show that certain properties of these systems, for example, their inertia or control frequency, may adversely affect the learning. Using two reinforcement learning algorithms, Actor-Critic with Experience Replay and Autocorrelated aCtions (ACERAC) and Actor Critic with Exeprience Replay and Sustained Actions (SusACER), I present two possible solutions, based on the premise of making subsequent actions similar to each other: the autocorrelation of action noise and the sustaining of actions over several time steps.