Announcements

• PS3 out: ps3.pdf due Friday Dec 4.
• In class presentations:
  Guidelines:
  • We will start at 12:30 sharp.
  • Time: for 8 minutes it's entirely your show (going a bit shorter is ok, but unfortunately, given the time constraints, we cannot go longer and you will get cut off at the 8 minute mark; it's up to you to which extent to entertain questions throughout the presentation)
  • Presentation carries 10 out of 45 points of the final project.
  • Feel free to sync up with me if you would like to discuss the contents of your presentation beforehand
  Schedule:
  
  • Tuesday December 1: Barron, Smith, Liu, Kolev, Lin, Strausser, Chang-Siu, Moldovan, Soerensen
  • Thursday December 3: Hoburg, Weekly, Song, Swift, Hunter, Javdani, Berg Kirkpatrick, Singh+Tang, Maitin-Shepard
  On your presentation and the delivery:
  
  • Jonathan Shewchuk has a nice webpage on giving an academic talk
  • While this is not a course on public speaking, you might as well take advantage of the opportunity and practice your delivery skills. In case you have never taken a public speaking class before, here are a few (of course somewhat subjective) pointers from a first lecture in such a class: pdf. It can be fun to pay attention to a small selection of them.
  Looking forward to your presentations!
• Latest update: 2009/11/24 Problem set 2 faq
• 2009/11/07 Problem set 2 is out. Due Friday November 20, 23:59 ps2.pdf, starter code:.tar.gz, .zip
• 2009/10/25 PS 1: Q&A page. Includes installer+instructions for helicopter graphics!
• 2009/10/07 Project due-dates: Oct 9: abstract, Nov 9: milestone, Dec 3: presentation, Dec 11: final paper
• 2009/10/07 Project guidelines pdf
• 2009/10/07 PS 1 is out: PS 1, starter code: .tar.gz .zip Due: Monday October 26, 23:59pm.
  If you don't have access to matlab, let me know so we can find a solution.

• Announcements
• Assignments
• Course description
• Prerequisites
• Grading
• Assignment policy
• Syllabus and materials
• Related materials

Assignments

• ps3.pdf
• ps2.pdf, starter code:.tar.gz, .zip
• ps1.pdf, starter code:.tar.gz,.zip

Course description

• Control: underactuation, controllability, Lyapunov, dynamic programming, LQR, feedback linearization, MPC
• Estimation: Bayes filters, KF, EKF, UKF, particle filter, occupancy grid mapping, EKF slam, GraphSLAM, SEIF, FastSLAM
• Manipulation and grasping: force closure, grasp point selection, visual servo-ing, more sub-topics tbd
• Reinforcement learning: value iteration, policy iteration, linear programming, Q learning, TD, value function approximation, Sarsa, LSTD, LSPI, policy gradient, inverse reinforcement learning, reward shaping, hierarchical reinforcement learning, inference based methods, exploration vs. exploitation
• Brief coverage of: system identification, simulation, pomdps, k-armed bandits, separation principle
• Case studies: autonomous helicopter, Darpa Grand/Urban Challenge, walking, mobile manipulation.

Prerequisites

Grading

• Open-ended final project (45%)
• Assignments (2 times 25% [first 2 assignments], 1 time 5% [last assignment])

Assignment policy

• Collaboration: Students may discuss assignments. However, each student must code up their solutions independently and write down their answers independently.
• Late assignments: Recognizing that students may face unusual circumstances and require some flexibility in the course of the semester, each student will have a total of seven free late (calendar) days to use as s/he sees fit. Late days are counted at the granularity of days: e.g., 3 hours late is one late day. If an assignment is submitted beyond the late-day budget, you will lose 20 (out of 100) points per day over budget (but you cannot go below zero).
  Late days cannot be used for the final project.

Syllabus and materials

1. AM: Astrom and Murray, Feedback Systems, pdf
2. T: Tedrake, Underactuated Robotics, Course Notes for MIT 6.832. August 2009 snapshot (referenced on this page and in the slides) [Here is a link to the current working draft ]
3. SB: Sutton and Barto, Reinforcement Learning, html
4. BT: Bertsekas and Tsitsiklis, Neuro-dynamic programming
5. TBF: Thrun, Burgard, Fox, Probabilistic Robotics
6. [optional readings] Slotine and Li, Applied Nonlinear Control --- great read on the topic

Related courses

• Ben Van Roy's class: Approximate Dynamic Programming
• Stephen Boyd's class: Linear Dynamical Systems
• Russ Tedrake's class: Underactuated Robotics: Learning Planning, and Control for Efficient Agile Machines
• Claire Tomlin's class: Linear system theory