Welcome to CS 287H Algorithmic Foundations of Human-Robot (and Human-AI) Interaction, Spring 2021!

Instructor: Anca Dragan (anca at berkeley dot edu)

GSI: Andreea Bobu (abobu at berkeley dot edu)

Lectures: TuTh, 3:30-5:00, Zoom (link is on Piazza; email Andreea if you're not added to Piazza)


As robot autonomy advances, it becomes more and more important to develop algorithms that are not solely functional, but also mindful of the end-user. How should the robot move differently when it's moving in the presence of a human? How should it learn from user feedback? How should it assist the user in accomplishing day to day tasks? These are the questions we will investigate in this course.

We will contrast existing algorithms in robotics with studies in human-robot interaction, discussing how to tackle interaction challenges in an algorithmic way, with the goal of enabling generalization across robots and tasks. We will also sharpen research skills: giving good talks, experimental design, statistical analysis, literature surveys.

Format: This course combines lectures with paper presentations by the students, encouraging both fundamental knowledge acquisition as well as open-ended discussions. Each student will also carry out an individual research project OR an in-depth literature survey.

Learning objectives: At the end of this course, you will have gained both knowledge/abilities related to human-robot interaction, as well as to research and presentation skills:

    [Human-robot interaction abilities]
  • articulate the challenges of developing algorithms that support HRI
  • apply optimization techniques to generate motion for HRI
  • contrast and relate model-based and model-free learning from demonstration
  • apply Bayesian inference and learning techniques to enhance coordination in collaborative tasks
  • develop a basic understanding of verbal and non-verbal communication
  • ground algorithmic HRI in the relvant psychology background
    [Research skills]
  • communicate scientific content to a peer audience
  • analyze and diagram the literature related to a particular topic
  • critique a scientific paper's experimental design and analysis

Prerequisites: There are no official prerequisites but a knowledge of probability and multivariate calculus is expected.


  • Student Presentations (30%): Each student will get the opportunity to present. You will be graded based on your level of insight into the material (including how well you answer questions from us and the rest of the class), how well you relate the paper to other papers and lecture material, as well as how well you present the material to the class and lead a discussionn . There will be 2 presenters for each paper. We will have 2-3 papers per lecture.
  • Quizzes (15%): We will have a set of (pandemic update: take-home) quizzes (10 min each), to test the understanding of the material. These quizzes are not meant to be onerous, but are meant to ensure that you read carefully through the papers and revisit the lecture notes. They are graded with a check, a minus, or a plus for particularly good answers.
  • Homework (15%): New this year, responding to feedback from previous years, we will develop a few (2-3) homework assignments. We will grade these lightly, and mainly take the opportunity to get feedback from you and refine them.
  • Final Project (30%): You have a choice between a research project, and an in-depth literature survey (~50 relevant papers, organized by different features, identifying gaps in the state of the art). You will have a final presentation, and submit a proposal along the way (1 page) and a report (up to 5 pages) at the end.
  • Participation (10%): Be engaged! Ask and answer questions!
  • Important fine print: Despite these percentages, you will not pass this class if you don't submit a proposal and a final report for your project, if you don't present your final project, or if you don't show up for class regularly (even if your computed final score is above passing).
  • Expectations: You can expect me to start and end class on time, devise quizzes that adequately cover the material, and grade your quizzes and send you feedback on your presentations in a timely manner. In turn, I can expect you to come to class (pandemic update: Zoom) on time, be attentive and engaged in class, and refrain from using laptops (pandemic update: using laptops to do other tasks outside of attending lecture), cell phones and other electronic devices during class. Please take notes, and ask questions when something is not clear. I also expect you to spend an adequate amount of time on the readings each week (~3 hours), and spend ~60 hours on your final project.

    Important dates:

  • 1 April - 1 Page Proposal (by this time you should have a clear idea of what you want to do and what your key insight is in the case of a project, and how you will select and classify papers in the case of a literature survey); Please discuss project ideas with the GSI and also feel free to email Anca.
  • 27 & 29 April - Final Presentations
  • 12 May - Final Reports (ideally start gearing up for a paper submission)
  • Project Proposal Instructions: They should be 1 page (+ references).
    If you are doing a research project:

    • motivate the problem
    • briefly describe how state of the art tackles it and what is missing
    • state your key insight clearly
    • scope your project to be what you would do for 1 paper; you don't have to have all results for the project, you can have just preliminary result at the end of the semester, but choose a problem that is not trivial and not an entire PhD thesis either

    If you are doing a literature survey:
    • describe the topic
    • describe how you will find papers -- what proceedings will you search, what keywords on google scholar, what starting papers;
    • what are the inclusion criteria -- how are you going to decide whether to include a paper or not
    • provide 5-10 initial papers
    • provide the groups/axes/independent variables you want to use to categorize the field

    Possible venues for projects:

  • Conference Papers: NeurIPS, CoRL, WAFR, ICRA, HRI, RSS
  • Short Papers: HRI Late Breaking Report
  • Schedule: Find a tentative schedule below. This is subject to change.

    # Date Topic Reading Comments Notes
    1 Jan 19 The What, Why, and How of Algorithmic HRI no mandatory reading
    2 Jan 21 Motion Planning 1 (lecture) no mandatory reading, but read these after Motion Planning is done if interested:
  • Lozano-Perez "Spatial Planning: A Configuration Space Approach"
  • Kavraki "Analysis of Probabilistic Roadmaps for Path Planning"
  • Lavalle "Randomized Kinodynamic Planning"
  • Hsu "Path Planning in Expansive Configuration Spaces"
  • notes (previous year) and visualization
    3 Jan 26 Motion Planning 2 (lecture) more on graph search here and here
    more on randomzied sampling:
  • RRT-Connect (Kuffner and LaValle)
  • On the probabilistic foundations of probabilistic roadmap planning(Epsilon,Alpha,Beta)-Expansiveness
  • constraints
  • 4 Jan 28 Trajectory Optimization 1 (lecture) no mandatory reading notes (previous year)
    5 Feb 2 Trajectory Optimization 2 (lecture) no reading
    6 Feb 4 Trajectory Optimization 3 (lecture) no mandatory reading,
  • Zucker "CHOMP: Covariant Hamiltonian Optimization for Motion Planning"
  • about presentations
    7 Feb 9 Optimal Motion Algorithms (papers)
  • "Elastic Bands: Connecting Path Planning and Control" (1993) pdf
  • "Finding Locally Optimal, Collision Free Trajectories with Sequentional Qadratic Programming" (2014) pdf
  • ILQR pdf
  • 8 Feb 11 Optimal Motion in HRI (papers)
  • "Planning human-aware motions using a sampling-based costmap planner" (2011) link
  • "Generating Human-Like Motion for Robots" (2013) pdf
  • [Tools of HRI]
    9 Feb 16 Experimental Design 1 (lecture) no reading
    10 Feb 18 Experimental Design 2 (lecture) no reading
    11 Feb 23 Experimental Design 3 (lecture) no mandatory reading
  • "Evaluating Fluency in Human-Robot Collaboration" (2013) pdf
  • notes (previous year)
    12 Feb 25 Refresher on MDPs, POMDPs, Dec-POMDPs, and Games More at CS188
    [Learning from Demonstration]
    13 March 2 Inverse Reinforcement Learning 1 (lecture) notes (previous year)
    14 March 4 Inverse Reinforcement Learning 2 (lecture) no mandatory reading
  • "Maximum Margin Planning" (2006) link
  • "Maximum Entropy IRL" (2010) pdf
  • "Bayesian IRL" (2007) pdf
  • "Learning Attractor Landscapes for Learning Motor Primitives" (2003) pdf
  • "Movement Primitives via Optimization" (2015) pdf
  • 15 March 9 LfD in Psychology (papers)
  • "Understanding Intentions of Others" (1995) pdf/li>
  • 2 short papers: "Infant Imitation After a 1-Week Delay" (1988) pdf1 ; "Rational Imitation in Preverbal Infants" (2002) pdf2
  • 16 March 11 LfD Algorithms (papers)
  • "Socially Compliant Navigation via IRL" pdf
  • "Predicting Human Reaching Motion" (2015) pdf
  • 17 March 16 LfD in HRI (papers)
  • "Trajectories and Keyframes for Kinesthetic Teaching" (2012) pdf
  • "Designing Robot Learners that Ask Good Questions" (2012)pdf
  • further reading:
  • "Using Perspective Taking to Learn from Ambiguous Demonstrations" (2006) pdf
  • 18 March 18 LfD beyond demonstrations (papers) Learning from physical corrections and robustness to misspecification.
  • "Learning robot objectives from physical human interaction. pdf
  • "Batch Active Preference-Based Learning for Reward Functions" pdf
  • "Preferences implicit in the state of the world. link
  • slides 1 slides 2
    19 March 23 spring break
    20 March 25 spring break
    21 March 30 Intent Inference (lecture) no mandatory reading
  • "Planning Based Prediction for Pedestrians" (2009)pdf
  • "Goal Inference as Invese Planning" (2007) pdf
  • notes (previous year)
    22 April 1 Intent Expression (lecture) no mandatory reading<
  • "Obsessed with Goals" (2007) pdf
  • "Legibility and predictability of robot motion" (2013) pdf
  • notes (previous year)
    23 April 6 Intent Inference Algorithms (papers)
  • "Shared Autonomy via Hidsight Optimization" (2015) link
  • "Intention-Aware Motion Planning" (2013) pdf
  • 24 April 8 Intent Expression Algorithms (papers)
  • "Anticipation in Robot Motion" (2011) pdf
  • "Expressing Robot Incapability" (2018) pdf
  • 25 April 13 Intent Expression in HRI (papers)
  • "Improving Robot Readability" (2011) pdf
  • "Communication of Intent in Assistive Free Flyers" (2014) pdf
  • [Coordination]
    26 April 15 Human-Robot Collaborative Learning (papers)
  • Cooperative IRL (2016) link
  • "CrossTraining" (2013) pdf
  • 27 April 20 Robot Influence on Human Actions (papers)
  • Influence-aware planning (2016) link
  • "Mutual Adaptation" (2016) pdf
  • 28 April 22 Interaction as a Game (lecture) no madatory reading
  • formalism and examples
  • notes (previous year)
    29 April 27 Project Presentations
    30 April 29 Project Presentations

    For more readings, check out a few other class websites (this is by no means a comprehensive list):

  • Cooperative Machines (MIT)
  • Human-Robot Interaction (GaTech)
  • Human-Robot Interaction (UW)
  • Manipulation Algorithms (CMU)