News

Research
My research is at the intersection of machine learning, perception, and control for robotics. In particular, I'm interested in how learning algorithms can enable robots to autonomously acquire complex sensorimotor skills.

All Papers

Generalizing Skills with SemiSupervised Reinforcement Learning
Chelsea Finn, Tianhe Yu, Justin Fu,
Pieter Abbeel,
Sergey Levine
International Conference on Learning Representations (ICLR), 2017
arXiv
/
video results
/
code
We formalize the problem of semisupervised reinforcement learning (SSRL), motivated by realworld scenarios where reward information
is only available in a limited set of scenarios such as when a human supervisor is present, or in a controlled laboratory setting.
We develop a simple algorithm for SSRL based on inverse reinforcement learning and show that it can improve performance by using
'unlabeled' experience.


Deep Visual Foresight for Planning Robot Motion
Chelsea Finn, Sergey Levine
International Conference on Robotics and Automation (ICRA), 2017
arXiv
/
video
We combine an actionconditioned predictive model of images, "visual foresight," with modelpredictive control for planning how
to push objects. The method is entirely selfsupervised, requiring minimal human involvement.


ResetFree Guided Policy Search: Efficient Deep Reinforcement Learning with Stochastic Initial States
William Montgomery*,
Anurag Ajay*,
Chelsea Finn,
Pieter Abbeel,
Sergey Levine
International Conference on Robotics and Automation (ICRA), 2017
arXiv
/
video
/
code
We present a new guided policy search algorithm that allows the method to be used in domains where the initial conditions are stochastic, which makes the method
more applicable to general reinforcement learning problems and improves generalization performance in our robotic manipulation experiments.


A Connection Between Generative Adversarial Networks, Inverse Reinforcement Learning, and EnergyBased Models
Chelsea Finn*, Paul Christiano*,
Pieter Abbeel,
Sergey Levine
NIPS Workshop on Adversarial Training, 2016
arXiv
We show that a samplebased algorithm for maximum entropy inverse reinforcement learning (MaxEnt IRL) corresponds to a generative adversarial network (GAN) with a particular choice of discriminator.
Since MaxEnt IRL is simply an energybased model (EBM) for behavior, we further show that GANs optimize EBMs with the corresponding discriminator,
pointing to a simple and scalable EBM training procedure using GANs.


Unsupervised Learning for Physical Interaction through Video Prediction
Chelsea Finn, Ian Goodfellow, Sergey Levine
Neural Information Processing Systems (NIPS), 2016
arXiv
/
videos
/
data
/
code
Our video prediction method predicts a distribution over pixel flow to apply to the previous image, rather than pixels values directly. We also introduce
a dataset of 50,000 robotic pushing sequences, consisting of over 1 million frames.


Adapting Deep Visuomotor Representations with Weak Pairwise Constraints
Eric Tzeng,
Coline Devin,
Judy Hoffman,
Chelsea Finn,
Pieter Abbeel,
Sergey Levine,
Kate Saenko,
Trevor Darrell
Workshop on the Algorithmic Foundations of Robotics (WAFR), 2016
arXiv
Collecting realworld robotic experience for learning an initial visual representation can be expensive. Instead, we show that it is possible to learn
a suitably good initial representation using data collected largely in simulation.


Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimization
Chelsea Finn, Sergey Levine, Pieter Abbeel
International Conference on Machine Learning (ICML), 2016
Oral presentation at the NIPS 2016 Deep Learning Symposium
arXiv /
video / talk
We propose an method for Inverse Reinforcement Learning (IRL) that can handle unknown dynamics and scale to flexible, nonlinear cost functions. We evaluate our algorithm on a series of simulated tasks and realworld robotic manipulation problems, including pouring and inserting dishes into a rack.


EndtoEnd Training of Deep Visuomotor Policies
Sergey Levine*,
Chelsea Finn*, Trevor Darrell,
Pieter Abbeel
CCC Blue Sky Ideas Award
Journal of Machine Learning Research (JMLR), 2016
arXiv /
video /
project page /
code
We demonstrate a deep neural network trained endtoend, from perception to controls, for robotic manipulation tasks.


Deep Spatial Autoencoders for Visuomotor Learning
Chelsea Finn, Xin Yu Tan,
Yan Duan, Trevor Darrell, Sergey Levine,
Pieter Abbeel
International Conference on Robotics and Automation (ICRA), 2016
arXiv /
video
We learn a lower dimensional visual statespace without supervision using deep spatial autoencoders, and use it to learn nonprehensile manipulation
tasks, such as pushing a lego block and scooping a bag into a bowl.


Learning Deep Neural Network Policies with Continuous Memory States
Marvin Zhang, Zoe McCarthy,
Chelsea Finn, Sergey Levine,
Pieter Abbeel
International Conference on Robotics and Automation (ICRA), 2016
arXiv /
video
We propose a method for learning recurrent neural network policies using continuous memory states. The method learns to store information in and use the memory states
using trajectory optimization. Our method outperforms vanilla RNN and LSTM baselines.


Bridging text spotting and SLAM with junction features.
HsuehCheng Wang,
Chelsea Finn,
Liam Paull,
Michael Kaess,
Ruth Rosenholtz,
Seth Teller,
John Leonard
International Conference on Intelligent Robots and Systems (IROS), 2015
We develop a method that integrates textspotting with simultaneous localization and mapping (SLAM), that determines loop closures using text in the environment.


Beyond LowestWarping Cost Action Selection in Trajectory Transfer
Dylan HadfieldMenell,
Alex X. Lee,
Chelsea Finn
Eric Tzeng,
Sandy Huang,
Pieter Abbeel,
International Conference on Robotics and Automation (ICRA), 2015
We consider the problem of selecting which demonstration to transfer to the current test scenario.
We frame the problem as an options Markov decision process (MDP) and develop an approach to learn a Qfunction from expert demonstrations.
Our results show significant improvement over nearestneighbor selection.

