CS287 Fall 2012

University of California at Berkeley
Dept of Electrical Engineering & Computer Sciences

CS 287: Advanced Robotics, Fall 2012

Fall 2011 offering (reasonably similar to current year's offering)
Fall 2009 offering (not particularly closely matched to current year's offering)

Instructor: Pieter Abbeel
Lectures: Tuesdays and Thursdays, 3:30pm-5:00pm, 310 Soda Hall
Office Hours: Wednesdays 4:00-5:00pm (and by email arrangement) in 746 Sutardja Dai Hall
Communication: Piazza is intended for general questions about the course, clarifications about assignments, student questions to each other, discussions about material, and so on. I strongly encourage all students to participate in discussion, ask, and answer questions through this site. I will monitor discussions closely. To sign up, go to the Piazza website and sign up with "UC Berkeley" and "CS287" for your school and class.

Announcements
Assignments
Prerequisites
Class goals
Grading
Assignment policy
Syllabus and materials
Related materials

Announcements

Please sign up on Piazza for CS287 Advanced Robotics for all future announcements.
Welcome to the Fall 2012 edition of CS287!

Assignments

Problem Set 3: ps3.pdf, starter code. Due: Monday November 12th at 23:59pm.
Problem Set 2: ps2.pdf, starter code. Due: Monday October 29th at 23:59pm.
Problem Set 1 (updated 2012/9/23, 00:16): ps1-v2.pdf, starter code v3, Due: Monday September 24th at 23:59pm.
Final project guidelines

Prerequisites

Familiarity with mathematical proofs, probability, algorithms, linear algebra; ability to implement algorithmic ideas in code.
Consent of instructor required for undergraduate students. Come see instructor after lecture or during office hours.

Class Goals

Master the math and algorithms underneath state-of-the-art robotic systems. The majority of these techniques are heavily based on probabilistic reasoning and optimization---two areas with wide applicability in modern Artificial Intelligence. An intended side-effect of the course is to generally strengthen your expertise in these two areas.
Implement and experiment with these algorithms.
Be able to understand research papers in the field of robotics:
- Main conferences: ICRA, IROS, RSS, ISER, ISRR.
- Main journals: IJRR, T-RO, Autonomous Robots.
Try out some ideas/extensions of your own.
Note: the focus of the course is on math and algorithms. We will not study mechanical or electrical design of robots.

Grading

Open-ended final project (40%)
Assignments (60%)

Assignment policy

Collaboration: Students may discuss assignments. However, each student must code up their solutions independently and write down their answers independently.
Late assignments: Recognizing that students may face unusual circumstances and require some flexibility in the course of the semester, each student will have a total of seven free late (calendar) days to use as s/he sees fit. Late days are counted at the granularity of days: e.g., 3 hours late is one late day. If an assignment is submitted beyond the late-day budget, you will lose 20 (out of 100) points per day over budget (but you cannot go below zero).
Late days cannot be used for the final project.
Final project guidelines

Syllabus and materials

All slides are made available here as the semester progresses.
In the Fall 2011 edition A couple of students have volunteered to record and post lecture videos. They posted them here. They might be of interest this year, too.
If your probability is rusty, you might want to handpick some homework/section exercises from past CS188 offerings, located here and similar url's replacing sp12 with fa11, sp11, fa10, sp10, fa09, etc.
Tentative schedule (edits in progress):

Lecture	Topic	Readings	Optional/Additional Readings
Th Aug 23	Course Introduction
Tu Aug 28	MDP's, Exact Methods: Value Iteration, Policy Iteration, Linear Programming, LP notes	Sutton and Barto, Reinforcement Learning, Chapters 3 and 4
Th Aug 30	Discretization of Continuous State Space MDPs, Function Approximation / Feature-based Representations		Moore and Atkeson, 1993, Munos and Moore, MLJ 2001, Chow and Tsitsiklis, 1991, Gordon, 1995, Tsitsiklis and Van Roy, 1996, Kushner and Dupuis, 1992/2001,
Tu Sep 4	LQR, iterative LQR / Differential Dynamic Programming
Th Sep 6	No Lecture
Tu Sep 11	Unconstrained Nonlinear Optimization, Constrained Nonlinear Optimization, Convex Optimization Problems		Boyd and Vandenberghe, Chapters 9-11
Th Sep 13	No Lecture
Tu Sep 18	Unconstrained Nonlinear Optimization, Constrained Nonlinear Optimization, Convex Optimization Problems (Same set of slides as for previous lecture.)	cvx_example.m	Boyd and Vandenberghe, Chapters 9 -11
Th Sep 20	Locally Optimal Control through Nonlinear Optimization: Shooting Methods, Collocation Methods, Model Predictive Control (MPC)	code examples
Tu Sep 25	Autononomous Driving
Th Sep 27	Motion Planning: PRM, RRT + variants	Steven M. Lavalle, Motion Planning, Chapters 5, 14, RRT*, Karaman and Frazzoli, LQR trees, Tedrake, code example
Tu Oct 2	Motion Planning: A* + variants	Likhachev slides, Steven M. Lavalle, Motion Planning, Chapters 5, 14	Videos: Urban Challenge Parking, Quadruped, Mobile Manipulation; Papers: Likhachev, Gordon, Thrun, ARA: Anytime A with Provable Bounds on Sub-Optimality, van den Berg, Shah, Huang, Goldberg, Anytime Nonparametric A, Koenig, Likhachev, Furcy, Lifelong Planning A
Th Oct 4	Probability Review, Bayes Filters	Intro: PR 1; Probability Review and Bayes Filters: PR 2
Tu Oct 9	Multivariate Gaussians	PR 3
Th Oct 11	Kalman Filtering	PR 3	From Gauss to Kalman
Tu Oct 16	EKF, UKF	PR 3	Julier and Uhlmann, the UKF
Th Oct 18	Smoother, MAP
Tu Oct 23	Maximum Likelihood, EM
Th Oct 25	Inverse Optimal Control
Tu Oct 30	Optimization-based Motion Planning in the Presence of Obstacles; Learning Trajectories from Demonstrations --- Guest Lecturer: John Schulman
Th Nov 1	POMDPs
Tu Nov 6	Guest Lecture: Steve Cousins, CEO Willow Garage
Th Nov 8	SEIF, EnKF, EKF-SLAM, Motion Models, Beam Sensor Models, Scan Matching	Motion Models: PR 5.1, 5.2, 5.3; Beam Sensor: PR 6.1, 6.2, 6.3;
Tu Nov 13	Optimization for Estimation, GraphSLAM, Particle Filters and Localization	Particle Filters: PR 4, Localization: PR 8.1, 8.3	Doucet, Godsill, Andrieu, 1998, Arulampalam et al., 2002
Th Nov 15	Mapping with Known Poses, Rao-Blackwellized Particle Filters, gMapping	Mapping with Known Poses: PR 9	Doucet, de Freitas, Murphy, Russell, Grisetti, Stachniss, Burgard T-RO 2006
Tu Nov 20	Grasping: Grasp Quality Metrics, Object Instance Detection		Optimizing Grasp Quality--Schulman, Goldberg, Abbeel ISRR 2011, Murray, Li, Sastry, Chapter 5, Instance Recognition Challenge Winner--Tang, Miller, Singh, Abbeel ICRA 2012
Th Nov 22	Happy Thanksgiving!
Tu Nov 27	Autonomous Helicopters and Course Wrap-Up		Abbeel, Coates, Ng, IJRR 2010
Th Nov 29	No Lecture --- Lecture moved to 12/7
Tu Dec 4	No Lecture --- Extended lecture on 12/6 instead
Th Dec 6	Project Presentations: Schedule 12:30pm-5pm, 310 Soda, there will be food
Fr Dec 7	Object Recognition and Detection: Feature Learning --- Guest Lecturer: Adam Coates, 11am-noon, 250 Sutardja Dai Hall	Hands-On Unsupervised Feature Learning Tutorial

Related materials

Most relevant book for estimation:

Thrun, Burgard, Fox, Probabilistic Robotics

There is no direct match for other lectures, but here are some related resources:

If you want to brush up your linear algebra background, I suggest working through this course (video lectures and homeworks available online) at your own pace: Stephen Boyd's EE263: Introduction to Linear Dynamical Systems.
If you want to learn more about the linear systems aspects (Kalman filtering, LQR), I recommend Stephen Boyd's EE363: Linear Dynamical Systems.
If you want go deeper into the theory of linear systems, I recommend: Claire Tomlin's EE221a: Linear System Theory
If you want to learn more about convex optimization, I recommend: Stephen Boyd's EE364a: Convex Optimization I and Stephen Boyd's EE364b: Convex Optimization II. Both of them have all course materials, including lecture videos, available online.
For (although draft-status) more about optimal control and motion planning, Russ Tedrake's class: Underactuated Robotics: Learning Planning, and Control for Efficient Agile Machines could give you a somewhat different angle, some complementary ideas, and more examples.
A more traditional book on control theory: Astrom and Murray, Feedback Systems
A more traditional book on nonlinear control: Slotine and Li, Applied Nonlinear Control.
A great introductory text on reinforcement learning: Sutton and Barto, Reinforcement Learning
A more mathematically oriented text on reinforcement learning: Bertsekas and Tsitsiklis, Neuro-dynamic programming
The current (=Fall 2012) offering will have some pretty strong overlap with the Fall 2011 offering.
Earlier offerings of my graduate class had more emphasis on reinforcement learning / approximate dynamic programming than the current offering: CS294-40, Fall 2008 and CS287, Fall 2009