Sergey Levine


Assistant Professor, UC Berkeley, EECS
Address:
754 Sutardja Dai Hall
UC Berkeley
Berkeley, CA 94720-1758
Email:

I am an Assistant Professor in the Department of Electrical Engineering and Computer Sciences at UC Berkeley. In my research, I focus on the intersection between control and machine learning, with the aim of developing algorithms and techniques that can endow machines with the ability to autonomously acquire the skills for executing complex tasks. In particular, I am interested in how learning can be used to acquire complex behavioral skills, in order to endow machines with greater autonomy and intelligence. To see a more formal biography, click here.

News and Announcements

August 24, 2016 My colleagues at Google have released the grasping and pushing data used for Levine et al. '16 (ISER) and Finn et al. '16 (NIPS): Google Brain Robotics Data.
August 12, 2016 Two papers accepted for oral presentation at NIPS 2016, and three accepted for poster presentation! Congratulations to my coauthors.
July 1, 2016 Two papers accepted to IROS 2016, and one to ISER 2016! Congratulations to my coauthors.
June 15, 2016 Three new preprints on deep robotic learning posted!
May 22, 2016 The paper "Optimal Control with Learned Local Models: Application to Dexterous Manipulation" wins the ICRA 2016 Best Manipulation Paper Award! Congratulations to my coauthors, Vikash and Emo!
April 24, 2016 Two papers accepted into ICML 2016 and one into JMLR! Congratulations to my co-authors.
April 22, 2016 I'll be co-organizing the Workshop on Action and Anticipation for Visual Learning at ECCV 2016!
April 15, 2016 Two new preprints on deep robotic learning posted!
March 7, 2016 New preprint on large-scale robotic deep learning posted!
March 3, 2016 Four new preprints on deep reinforcement learning and control posted!
March 1, 2016 Public implementation of guided policy search now available on GitHub!
[Show/hide older news]

Recent Talk

This is a recent talk summarizing some of my work on deep learning for robotic control.

Representative Publications

These recent papers provide an overview of my research, including: large scale robotic learning, deep reinforcement learning algorithms, inverse optimal control, and deep learning of robotic sensorimotor skills.

Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collection.
Sergey Levine, Peter Pastor, Alex Krizhevsky, Deirdre Quillen. ISER 2016.
[PDF (extended)] [PDF (ISER)] [Video] [arXiv (extended)] [Google Research Blog] [Data]
This paper presents an approach for learning grasping with continuous servoing by using large-scale data collection on a cluster of up to 14 individual robots. We collected about 800,000 grasp attempts, which we used to train a large convolutional neural network to predict grasp success given an image and a candidate grasp vector. We then construct a continuous servoing mechanism that uses this network to continuously make decisions about the optimal motor command to maximize the probability of grasp success. We evaluate our approach by grasping objects that were not seen at training time, and compare to an open-loop variant that does not perform continuous feedback control.
Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimization.
Chelsea Finn, Sergey Levine, Pieter Abbeel. ICML 2016.
[PDF] [Video] [arXiv]
In this paper, we explore optimal control methods that can be used to train deep neural network cost functions. We formulate an efficient sample-based approximation for MaxEnt IOC, and evaluate our method on a series of simulated tasks and real-world robotic manipulation problems, including pouring and inserting dishes into a rack.
Continuous Deep Q-Learning with Model-based Acceleration.
Shixiang Gu, Timothy Lillicrap, Ilya Sutskever, Sergey Levine. ICML 2016.
[PDF] [arXiv]
In this paper, we explore algorithms and representations to reduce the sample complexity of deep reinforcement learning for continuous control tasks. We propose two complementary techniques for improving the efficiency of such algorithms. First, we derive a continuous variant of the Q-learning algorithm, which we call normalized adantage functions (NAF). To further improve the efficiency of our approach, we explore the use of learned models for accelerating model-free reinforcement learning, and show that iteratively refitted local linear models are especially effective for this.
End-to-End Training of Deep Visuomotor Policies.
Sergey Levine*, Chelsea Finn*, Trevor Darrell, Pieter Abbeel. JMLR 17, 2016.
[PDF] [Video] [arXiv]
This paper presents a method for training visuomotor policies that perform both vision and control for robotic manipulation tasks. The policies are represented by deep convolutional neural networks with about 92,000 parameters. By learning to perform vision and control together, the vision system can adapt to the goals of the task, essentially performing goal-driven perception. Experimental results on a PR2 robot show that this method achieves substantial improvements in the accuracy of the final policy.

Recent Preprints

PLATO: Policy Learning using Adaptive Trajectory Optimization.
Gregory Kahn, Tianhao Zhang, Sergey Levine, Pieter Abbeel. arXiv 1603.00622.
[Overview] [PDF] [Video] [arXiv]
Towards Adapting Deep Visuomotor Representations from Simulated to Real Environments
Eric Tzeng, Coline Devin, Judy Hoffman, Chelsea Finn, Xingchao Peng, Sergey Levine, Kate Saenko, Trevor Darrell. arXiv 1511.07111.
[Overview] [PDF] [arXiv]

All Papers and Articles

2016

Guided Policy Search as Approximate Mirror Descent.
William Montgomery, Sergey Levine. NIPS 2016.
[Overview] [PDF] [arXiv]
Learning to Poke by Poking: Experiential Learning of Intuitive Physics.
Pulkit Agrawal, Ashvin Nair, Pieter Abbeel, Jitendra Malik, Sergey Levine. NIPS 2016.
[Overview] [PDF] [Video] [arXiv]
Unsupervised Learning for Physical Interaction through Video Prediction.
Chelsea Finn, Ian Goodfellow, Sergey Levine. NIPS 2016.
[Overview] [PDF] [Video] [arXiv] [Data]
Backprop KF: Learning Discriminative Deterministic State Estimators.
Tuomas Haarnoja, Anurag Ajay, Sergey Levine, Pieter Abbeel. NIPS 2016.
[Overview] [PDF] [arXiv]
Value Iteration Networks.
Aviv Tamar, Sergey Levine, Pieter Abbeel. NIPS 2016.
[Overview] [PDF] [arXiv]
Learning Dexterous Manipulation for a Soft Robotic Hand from Human Demonstration.
Abhishek Gupta, Clemens Eppner, Sergey Levine, Pieter Abbeel. IROS 2016.
[Overview] [PDF] [Video] [arXiv]
One-Shot Learning of Manipulation Skills with Online Dynamics Adaptation and Neural Network Priors.
Justin Fu, Sergey Levine, Pieter Abbeel. IROS 2016.
[Overview] [PDF] [Video] [arXiv]
Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collection.
Sergey Levine, Peter Pastor, Alex Krizhevsky, Deirdre Quillen. ISER 2016.
[Overview] [PDF (extended)] [PDF (ISER)] [Video] [arXiv (extended)] [Google Research Blog] [Data]
End-to-End Training of Deep Visuomotor Policies.
Sergey Levine*, Chelsea Finn*, Trevor Darrell, Pieter Abbeel. JMLR 17, 2016.
[Overview] [PDF] [Video] [arXiv]
Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimization.
Chelsea Finn, Sergey Levine, Pieter Abbeel. ICML 2016.
[Overview] [PDF] [Video] [arXiv]
Continuous Deep Q-Learning with Model-based Acceleration.
Shixiang Gu, Timothy Lillicrap, Ilya Sutskever, Sergey Levine. ICML 2016.
[Overview] [PDF] [arXiv]
MuProp: Unbiased Backpropagation for Stochastic Neural Networks.
Shixiang Gu, Sergey Levine, Ilya Sutskever, Andriy Mnih. ICLR 2016.
[Overview] [PDF] [arXiv]
Learning Visual Predictive Models of Physics for Playing Billiards.
Katerina Fragkiadaki*, Pulkit Agrawal*, Sergey Levine, Jitendra Malik. ICLR 2016.
[Overview] [PDF] [arXiv]
High-Dimensional Continuous Control Using Generalized Advantage Estimation.
John Schulman, Philipp Moritz, Sergey Levine, Michael I. Jordan, Pieter Abbeel. ICLR 2016.
[Overview] [PDF] [arXiv]
Deep Spatial Autoencoders for Visuomotor Learning.
Chelsea Finn, Xin Yu Tan, Yan Duan, Trevor Darrell, Sergey Levine, Pieter Abbeel. ICRA 2016.
[Overview] [PDF] [Video] [arXiv]
Learning Deep Control Policies for Autonomous Aerial Vehicles with MPC-Guided Policy Search.
Tianhao Zhang, Gregory Kahn, Sergey Levine, Pieter Abbeel. ICRA 2016.
[Overview] [PDF] [Video] [arXiv]
Optimal Control with Learned Local Models: Application to Dexterous Manipulation.
Vikash Kumar, Emanuel Todorov, Sergey Levine. ICRA 2016.
[Overview] [PDF] [Video]
Model-Based Reinforcement Learning with Parametrized Physical Models and Optimism-Driven Exploration.
Christopher Xie, Sachin Patil, Teodor Moldovan, Sergey Levine, Pieter Abbeel. ICRA 2016.
[Overview] [PDF] [arXiv]
Learning Deep Neural Network Policies with Continuous Memory States.
Marvin Zhang, Zoe McCarthy, Chelsea Finn, Sergey Levine, Pieter Abbeel. ICRA 2016.
[Overview] [PDF] [arXiv]

2015

Recurrent Network Models for Human Dynamics.
Katerina Fragkiadaki, Sergey Levine, Panna Felsen, Jitendra Malik. ICCV 2015.
[Overview] [PDF] [Video] [arXiv]
Incentivizing Exploration In Reinforcement Learning With Deep Predictive Models.
Bradly C. Stadie, Sergey Levine, Pieter Abbeel. arXiv 1507.00814. 2015.
[Overview] [PDF] [arXiv]
Learning Compound Multi-Step Controllers under Unknown Dynamics.
Weiqiao Han, Sergey Levine, Pieter Abbeel. IROS 2015.
[Overview] [PDF] [Video]
Learning from Multiple Demonstrations using Trajectory-Aware Non-Rigid Registration with Applications to Deformable Object Manipulation.
Alex X. Lee, Abhishek Gupta, Henry Lu, Sergey Levine, Pieter Abbeel. IROS 2015.
[Overview] [PDF] [Video]
Trust Region Policy Optimization.
John Schulman, Sergey Levine, Philipp Moritz, Michael I. Jordan, Pieter Abbeel. ICML 2015.
[Overview] [PDF] [Video] [arXiv]
Learning Contact-Rich Manipulation Skills with Guided Policy Search.
Sergey Levine, Nolan Wagener, Pieter Abbeel. ICRA 2015.
[Overview] [PDF] [Video]
Learning Force-Based Manipulation of Deformable Objects from Multiple Demonstrations.
Alex X. Lee, Henry Lu, Abhishek Gupta, Sergey Levine, Pieter Abbeel. ICRA 2015.
[Overview] [PDF] [Video]
Optimism-Driven Exploration for Nonlinear Systems.
Teodor Mihai Moldovan, Sergey Levine, Michael I. Jordan, Pieter Abbeel. ICRA 2015.
[Overview] [PDF]

2014

Learning Neural Network Policies with Guided Policy Search under Unknown Dynamics.
Sergey Levine, Pieter Abbeel. NIPS 2014.
[Overview] [PDF] [Video]
Learning Complex Neural Network Policies with Trajectory Optimization.
Sergey Levine, Vladlen Koltun. ICML 2014.
[Overview] [PDF] [Video]
Motor Skill Learning with Local Trajectory Methods.
Sergey Levine. Ph.D. thesis, Stanford University, 2014.
[Overview] [PDF]
Offline Policy Evaluation Across Representations with Applications to Educational Games.
Travis Mandel, Yun-En Liu, Sergey Levine, Emma Brunskill, Zoran Popović. AAMAS 2014.
[Overview] [PDF] [Website]

2013

Exploring Deep and Recurrent Architectures for Optimal Control.
Sergey Levine. NIPS Workshop on Deep Learning 2013.
[Overview] [PDF]
Variational Policy Search via Trajectory Optimization.
Sergey Levine, Vladlen Koltun. NIPS 2013.
[Overview] [PDF]
Inverse Optimal Control for Humanoid Locomotion.
Taesung Park, Sergey Levine. RSS Workshop on Inverse Optimal Control & Robotic Learning from Demonstration, 2013.
[Overview] [PDF]
Guided Policy Search.
Sergey Levine, Vladlen Koltun. ICML 2013.
[Overview] [PDF] [Video]

2012

Continuous Inverse Optimal Control with Locally Optimal Examples.
Sergey Levine, Vladlen Koltun. ICML 2012.
[Overview] [PDF] [Video/Code]
Continuous Character Control with Low-Dimensional Embeddings.
Sergey Levine, Jack M. Wang, Alexis Haraux, Zoran Popović, Vladlen Koltun. ACM SIGGRAPH 2012.
[Overview] [PDF] [Video/Code]
Physically Plausible Simulation for Character Animation.
Sergey Levine, Jovan Popović. SCA 2012.
[Overview] [PDF] [Video]

2011

Nonlinear Inverse Reinforcement Learning with Gaussian Processes.
Sergey Levine, Zoran Popović, Vladlen Koltun. NIPS 2011.
[Overview] [PDF] [Poster] [Video/Code]
Space-Time Planning with Parameterized Locomotion Controllers.
Sergey Levine, Yongjoon Lee, Vladlen Koltun, Zoran Popović. ACM Transactions on Graphics 30 (3). 2011.
[Overview] [PDF] [Video]

2010

Feature Construction for Inverse Reinforcement Learning.
Sergey Levine, Zoran Popović, Vladlen Koltun. NIPS 2010.
[Overview] [PDF] [Poster] [Website]
Gesture Controllers.
Sergey Levine, Philipp Krähenbühl, Sebastian Thrun, Vladlen Koltun. ACM SIGGRAPH 2010.
[Overview] [PDF] [Video]

2009

Real-Time Prosody-Driven Synthesis of Body Language.
Sergey Levine, Christian Theobalt, Vladlen Koltun. ACM SIGGRAPH Asia 2009.
[Overview] [PDF] [Video]
Modeling Body Language from Speech in Natural Conversation.
Sergey Levine. Master's research report, Stanford University, 2009.
[Overview] [PDF] [Video]
Body Language Animation Synthesis from Prosody.
Sergey Levine. Undergraduate thesis, Stanford University, 2009.
[Overview] [PDF] [Video]

Research Support

Google, 2016 - present
NVIDIA, 2016 - present
National Science Foundation, 2016 - present
Office of Naval Research, 2016 - present
© 2009-2016 Sergey Levine.