University of California at Berkeley

Dept of Electrical Enginring & Computer Sciences

Fall 2012 offering (reasonably similar to current year's offering)

Fall 2011 offering (fairly similar to current year's offering)

Fall 2009 offering (not particularly closely matched to current year's offering)

Professor: Pieter Abbeel

TAs: Sandy Huang and Zoe McCarthy

Pieter: Mondays 1:15-2:15pm (and by email arrangement) in 746 Sutardja Dai Hall

Sandy: Wednesdays 4-5pm in 730 Sutardja Dai Hall

Zoe: Fridays noon-1pm in 411 Soda Hall

- Announcements
- Assignments
- Prerequisites
- Class goals
- Grading
- Assignment policy
- Syllabus and materials
- Related materials

- Please sign up on Piazza for CS287 Advanced Robotics for all future announcements.
- Welcome to the Fall 2015 edition of CS287!

- Due: Wed 9/9, Problem Set 1: Markov Decision Processes, Value Iteration, Linear Programming ps1.pdf, starter code v2
- Due: Wed 9/23, Problem set 2: Function Approximation and LQR ps2.pdf, starter code
- Due: Wed 10/7, Problem Set 3: Convex Optimization, Sequential Convex Programming, Optimization-based Motion Planning and Control ps3.pdf, starter code
- Due: Wed 11/4, Problem Set 4: Multivariate Gaussians, Kalman Filtering, Maximum Likelihood, EM ps4.pdf, starter code
- Due: Fri 12/11, Optional Extra Credit Problem Set 1: Extended Kalman Filtering and Gaussian Belief Space Planning through Sequential Convex Programming EC1-EKF-GBSP.pdf, starter code
- Due: Fri 12/11, Optional Extra Credit Problem Set 2: Learning from Demonstrations through Trajectory Generalization with Thin-Plate Splines EC2-LfD.pdf, starter code
- Due: Fri 12/11, Optional Extra Credit Problem Set 3: Guided Policy Search EC3-GPS.pdf, starter code
- Due: Wed 12/16, Optional Extra Credit Problem Set 4: Policy Search and Locomotion EC4-PS-v2.pdf, diff-v1-v2.pdf, starter code v2

- Collaboration: Students may discuss assignments. However, each student must code up and write up their solutions independently.
- Late assignments: Recognizing that students may face unusual circumstances and require some flexibility in the course of the semester, each student will have a total of seven free late (calendar) days to use as s/he sees fit. Late days are counted at the granularity of days: e.g., 3 hours late is one late day. If an assignment is submitted beyond the late-day budget, you will lose 20 (out of 100) points per day over budget (but you cannot go below zero).

- An algorithmic or theoretical contribution that extends the current state of the art.
- An implementation of a state-of-the-art algorithm using real-world data and/or robots.

- 1 or 2 students per project.
- Oct 30th:
*Approved by instructor*abstracts due: 1 page description of project + goals for milestone. Make sure to meet with Pieter before then! - Nov 13th: 3 page milestone due. You are not graded on the milestone. Think of it as a sanity check for yourself that you indeed have started to make progress on the project and an opportunity to get feedback on your progress thus far, as well as on any revisions you might have made to your project goals.
- December 7th: Project presentations.
- December 14th: Final paper due. This should be a 6 page paper, structured like a conference paper. I.e., focus on the problem setting, why it matters and what's interesting/novel about it, your approach, your results, analysis of results, limitations, future directions. Cite and briefly survey prior work as appropriate but don't re-write prior work when not directly relevant to understand your approach.
- Late days cannot be used for the final project.

- Familiarity with mathematical proofs, probability, algorithms, linear algebra; ability to implement algorithmic ideas in code.
- If your probability is rusty, you might want to handpick some homework/section exercises from past CS188 offerings, located here and similar url's replacing sp12 with fa11, sp11, fa10, sp10, fa09, etc.
- Consent of instructor required for undergraduate students. Come see instructor after lecture or during office hours.

- Learn the math and algorithms underneath state-of-the-art robotic systems. The majority of these techniques are heavily based on probabilistic reasoning and optimization---two areas with wide applicability in modern Artificial Intelligence. An intended side-effect of the course is to generally strengthen your expertise in these two areas.
- Implement, and experiment with, these algorithms.
- Be able to understand research papers in the field of robotics:
- Main conferences: ICRA, IROS, RSS, ISER, ISRR.
- Main journals: IJRR, T-RO, Autonomous Robots.

- Try out some ideas/extensions of your own.
- Note: the focus of the course is on math and algorithms. We will not study mechanical or electrical design of robots.

- Open-ended final project (30%)
- 1/5 quality of presentation, 1/5 quality of writing of the final paper, 3/5 quality of the results themselves.
- Assignments (70%)
- Each regular assignment gets re-scaled out of 100 and equally contributes 70/4 of the points.
- Each extra credit assignment gets re-scaled out of 50, and each can contribute up to 0.5 * 70/4 points.
- Thrun, Burgard, Fox, Probabilistic Robotics
- If you want to brush up your linear algebra background, I suggest working through this course (video lectures and homeworks available online) at your own pace: Stephen Boyd's EE263: Introduction to Linear Dynamical Systems.
- If you want to learn more about the linear systems aspects (Kalman filtering, LQR), I recommend Stephen Boyd's EE363: Linear Dynamical Systems.
- If you want go deeper into the theory of linear systems, I recommend: Claire Tomlin's EE221a: Linear System Theory
- If you want to learn more about convex optimization, I recommend: Stephen Boyd's EE364a: Convex Optimization I and Stephen Boyd's EE364b: Convex Optimization II. Both of them have all course materials, including lecture videos, available online.
- For (although draft-status) more about optimal control and motion planning, Russ Tedrake's class: Underactuated Robotics: Learning Planning, and Control for Efficient Agile Machines could give you a somewhat different angle, some complementary ideas, and more examples.
- A more traditional book on control theory: Astrom and Murray, Feedback Systems
- A more traditional book on nonlinear control: Slotine and Li, Applied Nonlinear Control.
- A great introductory text on reinforcement learning: Sutton and Barto, Reinforcement Learning
- A more mathematically oriented text on reinforcement learning: Bertsekas and Tsitsiklis, Neuro-dynamic programming
- Earlier offerings of my graduate class had more emphasis on reinforcement learning / approximate dynamic programming than the current offering: CS294-40, Fall 2008 and CS287, Fall 2009

In the Fall 2011 edition a couple of students volunteered to record and post lecture videos. They posted them here. While the syllabus doesn't match exactly, some of them might be still of interest this year, too.

*Tentative* schedule (edits in progress):