de.mpg.escidoc.pubman.appbase.FacesBean
English
 
Help Guide Disclaimer Contact us Login
  Advanced SearchBrowse

Item

ITEM ACTIONSEXPORT

Released

Journal Article

Parameter-exploring policy gradients

MPS-Authors
http://pubman.mpdl.mpg.de/cone/persons/resource/persons84135

Osendorfer C, Rückstiess T, Graves A, Peters,  J
Department Empirical Inference, Max Planck Institute for Biological Cybernetics, Max Planck Society;
Dept. Empirical Inference, Max Planck Institute for Intelligent Systems, Max Planck Society;

Locator
There are no locators available
Fulltext (public)
There are no public fulltexts available
Supplementary Material (public)
There is no public supplementary material available
Citation

Sehnke, F., Osendorfer C, Rückstiess T, Graves A, Peters, J., & Schmidhuber, J. (2010). Parameter-exploring policy gradients. Neural Networks, 21(4), 551-559. doi:10.1016/j.neunet.2009.12.004.


Cite as: http://hdl.handle.net/11858/00-001M-0000-0013-C026-2
Abstract
We present a model-free reinforcement learning method for partially observable Markov decision problems. Our method estimates a likelihood gradient by sampling directly in parameter space, which leads to lower variance gradient estimates than obtained by regular policy gradient methods. We show that for several complex control tasks, including robust standing with a humanoid robot, this method outperforms well-known algorithms from the fields of standard policy gradients, finite difference methods and population based heuristics. We also show that the improvement is largest when the parameter samples are drawn symmetrically. Lastly we analyse the importance of the individual components of our method by incrementally incorporating them into the other algorithms, and measuring the gain in performance after each step.