Advanced Topics in Learning and Decision Making:

Interactions Between Learning and Control

**Instructor:** Ben Recht

**Time:** M 2:00-5:00 PM

**Location:** 344 Evans Hall

**Office Hours:** TBA

**Location:** 572 Cory Hall

**Description:** This course will study machine learning systems that are able to act upon the physical world, investigating how to learn models of dynamical systems, how to use these models to achieve objectives in a timely fashion, how to optimally acquire new information to improve performance, and how to balance model specification against system controllability. The primary technical focus will be on the mathematical tools at the interface of statistical learning and dynamical systems. Topics will include

dynamic programming and optimal control

linear systems, LQR, and Kalman filtering

dynamical system identification

active learning

bandit problems

reinforcement learning

model invalidation and robust control

receding horizon control

**Required background:** Students should have knowledge of machine learning, optimization and control. A reasonable background would include Stat 210a or equivalent, EECS 227B/C or equivalent, and should have taken an advanced undergraduate course on control theory, time series, or signal processing.

**Format:** The format will be a seminar which will review both classic and cutting edge work on these topics. The first portion of each meeting will be a lecture, and the last hour will be reserved for a focused discussion of either a paper or a particular open problem. The discussion will be lead by one of the course participants, and we will decide on topics during the first course meeting.

**Grading:** Grading is based on participation and a course project. There are three ways that students can participate in the course, scribing, discussion leading, and lecturing.

**Scribing:** Scribing consists of writing up the notes for a lecture in LaTeX. Notes will be due one week after the scribed lecture.

**Course project:** The course project will involve independent work on a topic of the student's own choosing. Course projects will be presented in an informal poster session at the end of semester, and the work will be summarized in a write-up. The poster presentations will be during R&R week.

**Texts**

*Dynamic Programming and Optimal Control.*Dimitri P. Bertsekas. Athena Scientific, Belmont, MA, 2005. (3rd edition)

*Predictive Control for Linear and Hybrid Systems.*F. Borrelli, A. Bemporad, and M. Morari. 2015. pdf.

**January 23, 2017:**Introduction. Dynamic Programming with full and incomplete information. Filtering and control. Readings: Bertsekas Chapter 1, Borelli et al, Chapter 8.