Approximate Dynamic Programming and Reinforcement Learning

Approximate Dynamic Programming and Reinforcement Learning
Dozent: Hao Shen
Assistent: Martin Gottwald
Zielgruppe: Master
Umfang: Ergänzungsvorlesung
(2/2/1 SWS Vorlesung/Integrierte Übung/Fragestunde)
Turnus: Wintersemester
Anmeldung: 10.09.2019 (16:00) via TUMonline
Zeit & Ort: Vorlesung: Donnerstags 13.15 - 14.45
Übung: Montags 15.00 - 16.30
Raum (beide): 0502.EG.220
Beginn Vorlesung: 17.10.2019
Beginn Übung: 28.10.2019


Approximate Dynamic Programming (ADP) and Reinforcement Learning (RL) are two closely related paradigms for solving sequential decision making problems. ADP methods tackle the problems by developing optimal control methods that adapt to uncertain systems over time, while RL algorithms take the perspective of an agent that optimizes its behavior by interacting with its environment and learning from the feedback received. Both technologies have succeeded in applications of operation research, robotics, game playing, network management, and computational intelligence.

We will cover the following topics (not exclusively):

  • Markov decision processes
  • Dynamic programming
  • Approximate dynamic programming
  • Reinforcement learning
  • Policy gradient algorithms
  • (Partially observable Markov decision processes)

Target Audience and Registration

The course communication will be handled through the moodle page.

On completion of this course, students are able to:

  • describe classic scenarios in sequential decision making problems
  • explain basic models of ADP/RL methods
  • derive ADP/RL algorithms that are covered in the course
  • characterize convergence properties of the ADP/RL algorithms covered in the course
  • compare performance of the ADP/RL algorithms that are covered in the course, both theoretically and practically
  • select proper ADP/RL algorithms in accordance with specific applications
  • construct and implement ADP/RL algorithms to solve simple decision making problems