Pulkit Agrawal

I am an Assistant Professor in the department of Electrical Engineering and Computer Science (EECS) at MIT. My lab is a part of the Computer Science and Artificial Intelligence Lab (CSAIL), is affiliated with the Laboratory for Information and Decision Systems (LIDS) and involved with NSF AI Institute for Artificial Intelligence and Fundamental Interactions ( IAIFI ).

I completed my Ph.D. at UC Berkeley; undergraduate studies from IIT Kanpur. Co-founded SafelyYou Inc. that builds fall prevention technology. Advisor to Tutor Intelligence, Common Sense Machines, and AI Foundry.

 /  LinkedIn  /  Email  /  CV  /  Biography  /  Google Scholar

profile photo
Research

The overarching research interest is to build machines that have similar manipulation and locomotion abilities as humans. These machines will automatically and continuously learn about their environment and exhibit both common sense and physical intuition. I refer to this line of work as "computational sensorimotor learning". It encompasses problems in peception, control, hardware design, robotics, reinforcement learning, and other learning approaches to control. My past work has also drawn inspiration from cognitive science, and neuroscience.

Ph.D. Thesis (Computational Sensorimotor Learning)  /  Thesis Talk  /  Bibtex

TEDxMIT Talk: Why machines can play chess but can't open doors? (i.e., why is robotics hard?)

Recent Awards

Gabe Margolis wins the 2022 Ernst A. Guillemin Thesis Award in Artificial Intelligence and Decision Making.
Best Paper Award at Conference on Robot Learning (CoRL) 2021 to our work on in-hand object re-orientation.
Joshua Gruenstien wins the 2021 Jeremy Gerstle Undergraduate Research Award.
Amazon Research Award in Robotics, 2020.
Amazon Machine Learning Research Award , 2019.
Salesforce Research Deep Learning Grant, 2019.

Research Group

The lab is an unsual collection of folks working on something that is unconceivable/unthinkable, but not impossible in our lifetime: General Artificial Intelligence. Life is short, do what you must do :-) I like to call my group: Improbable AI Lab.

Graduate Students
Tao Chen
Zhang-wei Hong
Anurag Ajay
Ruben Castro
Jacob Huh (co-advised with Phillip Isola)
Anthony Simeonov (co-advised with Alberto Rodriguez)
Aviv Netanyahu
Richard Li
Gabe Margolis
Idan Shen
Xiang Fu (co-advised with Tommi Jaakkola)

Masters of Engineering (MEng. Students) Undergraduate Researchers (UROPs)
Meenal Parakh, Marcel Torne, Alisha Fong, Alina Sarmiento, Srinath Mahakali, Andrew Jenkins, Abhaya Ravikumar, Alex Hu, Isabel Sperandino

Visiting Researchers
Tifanny Portela, Yandong Ji , Steven Li , Seungwook Han

Collaborators
Brian Cheung , Andi Peng

Openings
We have openings for Ph.D. Students, PostDocs, and MIT UROPs/SuperUROPs. If you would like to apply for the Ph.D. program, please apply directly to MIT EECS admissions. For all other positions, send me an e-mail with your resume.

Recent Talks

Forum for Artificial Intelligence, UT Austin, March 2023.
GRASP Seminar, University of Pennysylvania, March 2023.
Fun with Robots and Machine Learning , Robotics Colloqium, University of Washington, Nov 2022.
Navigating Through Contacts , RSS 2022 Workshop in The Science of Bumping into Things.
Coming of Age of Robot learning , Technion Robotics Seminar (April 14 2022) / MIT Robotics Seminar (March 2022).
Rethinking Robot Learning , Learning to Learn: Robotics Workshop, ICRA'21.
Self-Supervised Robot Learning, Robotics Seminar, Robot Learning Seminar, MILA.
Challenges in Real-World Reinforcement Learning, IAIFI Seminar, MIT.
The Task Specification Problem, Embodied Intelligence Seminar, MIT.

Teaching

Courses
Computational Sensorimotor Learning
Graduate Machine Learning: FA'20, FA'21
Intelligent Robot Manipulation: FA'19

Professional Education
These courses are intended for industry professionals and not MIT students.
Advanced Reinforcement Learning: Summer'23
Reinforcement Learning: Summer'23

Pre-Prints
sym Human-Assisted Continual Robot Learning with Foundation Models
Meenal Parakh*, Alisha Fong*, Anthony Simeonov, Abhishek Gupta, Tao Chen, Pulkit Agrawal
(*equal contribution) arXiv, 2023

paper / project page / bibtex

An LLM-based task planner that can learn new skills opens doors for continual learning.

Publications
Visual Dexterity: In-Hand Reorientation of Novel and Complex Object Shapes
Tao Chen, Megha Tippur, Siyang Wu, Vikash Kumar, Edward Adelson, Pulkit Agrawal
Science Robotics, 2023

paper / project page / bibtex

A real-time controller that dynamically reorients complex and novel objects by any amount using a single depth camera.

sym Compositional Foundation Models for Hierarchical Planning
Anurag Ajay*, Seungwook Han*, Yilun Du*, Shuang Li, Abhi Gupta, Tommi Jaakkola, Josh Tenenbaum, Leslie Kaelbling, Akash Srivastava, Pulkit Agrawal
(* equal contribution)
NeurIPS, 2023

paper / project page / bibtex

Composing existing foundation models operating on different modalities to solve long-horizon tasks.

sym Breadcrumbs to the Goal: Goal-Conditioned Exploration from Human-in-the-Loop Feedback
Marcel Torne, Max Balsells, Zihan Wang, Samedh Desai, Tao Chen, Pulkit Agrawal, Abhishek Gupta
NeurIPS, 2023

paper / project page / code / bibtex

Method for guiding goal-directed exploration with asynchronous human feedback.

sym Beyond Uniform Sampling: Offline Reinforcement Learning with Imbalanced Datasets
Zhang-Wei Hong, Aviral Kumar, Sathwik Karnik, Abhishek Bhandwaldar, Akash Srivastava, Joni Pajarinen, Romain Laroche, Abhishek Gupta, Pulkit Agrawal
NeurIPS, 2023

paper / bibtex / code

Optimizing the sampling distribution enables offline RL to learn a good policy in skewed datasets primarily composed of sub-optimal trajectories.

sym Shelving, Stacking, Hanging: Relational Pose Diffusion for Multi-modal Rearrangement
Anthony Simeonov, Ankit Goyal*, Lucas Manuelli*, Lin Yen-Chen, Alina Sarmiento,Alberto Rodriguez, Pulkit Agrawal**, Dieter Fox**
(*equal contribution, **equal advising) CoRL, 2023

paper / project page / code / bibtex

Relational rearrangement with multi-modal placing and generalization over scene layouts via diffusion and local scene conditioning.

sym Learning to See Physical Properties with Active Sensing Motor Policies
Gabriel B. Margolis, Xiang Fu, Yandong Ji, Pulkit Agrawal
Conference on Robot Learning (CoRL), 2023

paper / project page / bibtex

Learn to perceive physical properties of terrains in front of the robot (i.e., a digital twin).

Visual Pre-training for Navigation: What Can We Learn from Noise?
Yanwei Wang, Ching-Yun Ko, Pulkit Agrawal
IROS 2023, NeurIPS 2022 Workshop

paper / code / project page / bibtex

Learning to navigate by moving the camera across random images.

sym Autonomous Robotic Reinforcement Learning with Asynchronous Human Feedback
Max Balsells*, Marcel Torne*, Zihan Wang, Samedh Desai, Pulkit Agrawal, Abhishek Gupta
CoRL, 2023

paper / bibtex

Leveraging crowdsourced non-expert human feedback to guide exploration in robot policy learning.

sym TGRL: An Algorithm for Teacher Guided Reinforcement Learning
Idan Shenfeld, Zhang-Wei Hong, Aviv Tamar, Pulkit Agrawal
ICML, 2023

paper / code / project page / bibtex

An algorithm for automatically balancing learning from teacher's guidance and task reward.

sym Straightening Out the Straight-Through Estimator: Overcoming Optimization Challenges in Vector Quantized Networks
Minyoung Huh, Brian Cheung, Pulkit Agrawal, Phillip Isola
International Conference on Machine Learning (ICML), 2023

paper / website / code / bibtex

A set of suggestions that simplifies training of vector quantization layers.

sym Parallel Q-Learning: Scaling Off-policy Reinforcement Learning under Massively Parallel Simulation
Zechu Li*, Tao Chen*, Zhang-Wei Hong, Anurag Ajay, Pulkit Agrawal
(* indicates equal contribution)
ICML, 2023
paper / code / bibtex

Scaling Q-learning algorithms to 10K+ workers.

sym Diagnosis, Feedback, Adaptation: A Human-in-the-Loop Framework for Test-Time Policy Adaptation
Andi Peng, Aviv Netanyahu, Mark Ho, Tianmin Shu, Andreea Bobu,
Julie Shah, Pulkit Agrawal
ICML, 2023

paper / project page / bibtex

A step towards using counterfactuals for improving policy adaptation.

sym Statistical Learning under Heterogenous Distribution Shift
Max Simchowitz*, Anurag Ajay*, Pulkit Agrawal, Akshay Krishnamurthy
(* equal contribution)
ICML, 2023

paper / bibtex

In-distribution error for certain features predicts their out-of-distribution sensitivity.

sym DribbleBot: Dynamic Legged Manipulation in the Wild
Yandong Ji*, Gabriel B. Margolis*, Pulkit Agrawal (*equal contribution)
International Conference on Robotics and Automation (ICRA), 2023

paper / project page / bibtex
Press: TechCrunch, IEEE Spectrum, NBC Boston, Insider, Yahoo!News, MIT News

Dynamic legged object manipulation on diverse terrains with onboard compute and sensing.

TactoFind: A Tactile Only System for Object Retrieval
Sameer Pai*, Tao Chen*, Megha Tippur*, Edward Adelson, Abhishek Gupta, Pulkit Agrawal
(*equal contribution, † equal advising)
International Conference on Robotics and Automation (ICRA), 2023

paper / project page / bibtex

Localize, identify, and fetch a target object in the dark with tactile sensors.

sym Is Conditional Generative Modeling all you need for Decision Making?
Anurag Ajay*, Yilun Du*, Abhi Gupta*, Josh Tenenbaum, Tommi Jaakkola, Pulkit Agrawal
(*equal contribution)
ICLR, 2023 (Oral)

paper / project page / bibtex

Return conditioned generative models offer a powerful alternative to temporal-difference learning for offline decision making and reasoning with constraints.

sym Learning to Extrapolate: A Transductive Approach
Aviv Netanyahu*, Abhishek Gupta*, Max Simchowitz, Kaiqing Zhang, Pulkit Agrawal
(*equal contribution)
ICLR, 2023

paper / bibtex

Transductive reparameterization converts out-of-support generalization problem into out-of-combination generalization which is possible under low-rank style conditions.

“harness” Harnessing Mixed Offline Reinforcement Learning Datasets via Trajectory Weighting
Zhang-Wei Hong, Pulkit Agrawal, Remi Tachet des Combes, Romain Laroche
ICLR, 2023

paper / bibtex

Return reweighted sampling of trajectories enables offline RL algorithms to work with skewed datasets.

sym The Low-Rank Simplicity Bias in Deep Networks
Minyoung Huh, Hossein Mobahi, Richard Zhang, Brian Cheung, Pulkit Agrawal, Phillip Isola
Transactions of Machine Learning Research (TMLR), 2023

paper / website / bibtex

Deeper Networks find simpler solutions! Also learn why ResNets overcome the challenges associated with very deep networks.

sym Redeeming Intrinsic Rewards via Constrained Optimization
Eric Chen*, Zhang-Wei Hong*, Joni Pajarinen, Pulkit Agrawal
(*equal contribution)
NeurIPS, 2022

paper / project page / bibtex
Press: MIT News

Method that automatically balances exploration bonus or curiosity against task rewards leading to consistent performance improvement.

SE(3)-Equivariant Relational Rearrangement with Neural Descriptor Fields
Anthony Simeonov*, Yilun Du*, Lin Yen-Chen, Alberto Rodriguez, Leslie P. Kaelbling,
Tomás Lozano-Peréz, Pulkit Agrawal (*equal contribution)
CoRL, 2022

paper / project page / code / bibtex

Learning relational tasks with a few demonstrations in a way that generalizes to new configurations of objects.

sym Walk These Ways: Tuning Robot Control for Generalization with Multiplicity of Behavior
Gabriel B. Margolis, Pulkit Agrawal
CoRL, 2022 (Oral)

paper / code / project page / bibtex

One learned policy embodies many dynamic behaviors useful for different tasks.

sym Distributionally Adaptive Meta Reinforcement Learning
Anurag Ajay*, Abhishek Gupta*, Dibya Ghosh, Sergey Levine, Pulkit Agrawal
(*equal contribution)
NeurIPS, 2022

paper / project page / bibtex

Being adaptive instead of being robust results in faster adaption to out-of-distribution tasks.

sym Efficient Tactile Simulation with Differentiability for Robotic Manipulation
Jie Xu, Sangwoon Kim, Tao Chen, Alberto Rodriguez, Pulkit Agrawal, Wojciech Matusik, Shinjiro Sueda
CoRL, 2022

paper / Code coming soon / project page / bibtex

Tactile Simulator for complex shapes training on which transfers to real-world.

sym Rapid Locomotion via Reinforcement Learning
Gabriel Margolis*, Ge Yang*, Kartik Paigwar, Tao Chen, Pulkit Agrawal
RSS, 2022

paper / project page / bibtex
Press: Wired, Popular Science, TechCrunch, BBC , MIT News

High-speed running and spinning on diverse terrains with a RL based controller.

sym Stubborn: A Strong Baseline for Indoor Object Navigation
Haokuan Luo, Albert Yue, Zhang-Wei Hong, Pulkit Agrawal
IROS, 2022

paper / code / bibtex

State-of-the-art Performance on Habitat Navigation Challenge without any machine learning for navigation.

sym Neural Descriptor Fields: SE(3)-Equivariant Object Representations for Manipulation
Anthony Simeonov*, Yilun Du*, Andrea Tagliasacchi, Joshua B. Tenenbaum, Alberto Rodriguez, Pulkit Agrawal**, Vincent Sitzmann**
(*equal contribution, order determined by coin flip. **equal advising)
ICRA, 2022

paper / website and code / bibtex

An SE(3) Equivariant method for specifiying and finding correspondences which enables data efficient object manipulation.

sym An Integrated Design Pipeline for Tactile Sensing Robotic Manipulators
Lara Zlokapa, Yiyue Luo, Jie Xu, Michael Foshey, Kui Wu, Pulkit Agrawal, Wojciech Matusik
ICRA, 2022

paper / website / bibtex

A method for users to easily design a variety of robotic manipulators with integrated tactile sensors.

sym Stable Object Reorientation using Contact Plane Registration
Richard Li, Carlos Esteves, Ameesh Makadia, Pulkit Agrawal
ICRA, 2022

paper / bibtex

Predicting contact points with a CVAE and plane segmentation improves object generalization and handles multimodality.

sym Discovering Generalizable Spatial Goal Representations via Graph-based Active Reward Learning
Aviv Netanyahu*, Tianmin Shu*, Joshua B. Tenenbaum, Pulkit Agrawal
ICML, 2022

paper / bibtex

Graph-based one-shot reward learning via active learning for object rearrangement tasks.

sym Offline RL Policies Should be Trained to be Adaptive
Dibya Ghosh, Anurag Ajay, Pulkit Agrawal, Sergey Levine
ICML, 2022

paper / bibtex

Online adaptation of offline RL policies using evaluation data improves performance.

“ter” Topological Experience Replay
Zhang-Wei Hong, Tao Chen, Yen-Chen Lin, Joni Pajarinen, Pulkit Agrawal
ICLR, 2022

paper / bibtex

Sampling data from the replay buffer informed by topological structure of the state space improves performance.

“ter” Bilinear Value Networks for Multi-goal Reinforcement Learning
Zhang-Wei Hong*, Ge Yang*, Pulkit Agrawal (*equal contribution)
ICLR, 2022

paper / bibtex

Bilinear decomposition of the Q-value function improves generalization and data efficiency.

sym Equivariant Contrastive Learning
Rumen Dangovski, Li Jing, Charlotte Loh, Seungwook Han, Akash Srivastava, Brian Cheung, Pulkit Agrawal, Marin Soljacic
ICLR , 2022

paper / bibtex

Study revealing complementarity of invariance and equivariance in contrastive learning.

sym Overcoming The Spectral Bias of Neural Value Approximation
Ge Yang*, Anurag Ajay*, Pulkit Agrawal (*equal contribution)
ICLR, 2022

paper / bibtex

Fourier features improve value estimation and consequently data efficiency.

sym A System for General In-Hand Object Re-Orientation
Tao Chen, Jie Xu, Pulkit Agrawal
CoRL, 2021 (Best Paper Award)

paper / bibtex / project page
Press: MIT News

A framework for general in-hand object reorientation.

sym Learning to Jump from Pixels
Gabriel Margolis, Tao Chen, Kartik Paigwar, Xiang Fu,
Donghyun Kim, Sangbae Kim, Pulkit Agrawal
CoRL, 2021

paper / bibtex / project page
Press: MIT News

A hierarchical control framework for dynamic vision-aware locomotion.

sym 3D Neural Scene Representations for Visuomotor Control
Yunzhu Li*, Shuang Li*, Vincent Sitzmann,Pulkit Agrawal, Antonio Torralba (*equal contribution)
CoRL, 2021 (Oral)

paper / website / bibtex

Extreme viewpoint generalization via 3D representations based on Neural Radiance Fields.

sym An End-to-End Differentiable Framework for Contact-Aware Robot Design
Jie Xu, Tao Chen, Lara Zlokapa, Michael Foshey, Wojciech Matusik,
Shinjiro Sueda, Pulkit Agrawal
RSS, 2021

paper / website / bibtex / video /
Press: MIT News

Computational method for design task-specific robotic hands.

sym Learning Task Informed Abstractions
Xiang Fu, Ge Yang, Pulkit Agrawal, Tommi Jaakkola
ICML, 2021

paper / website / bibtex

A MDP formulation that dissociates task relevant and irrelevant information.

sym Residual Model Learning for Microrobot Control
Joshua Gruenstein, Tao Chen, Neel Doshi, Pulkit Agrawal
ICRA, 2021

paper / bibtex

Data efficient learning method for controlling microrobots.

sym OPAL: Offline Primitive Discovery for Accelerating Offline Reinforcement Learning
Anurag Ajay, Aviral Kumar, Pulkit Agrawal, Sergey Levine, Ofir Nachum
ICLR, 2021

paper / website / bibtex

Learning action primitives for data efficient online and offline RL.

sym A Long Horizon Planning Framework for Manipulating Rigid Pointcloud Objects
Anthony Simeonov, Yilun Du, Beomjoon Kim, Francois Hogan, Joshua Tenenbaum, Pulkit Agrawal, Alberto Rodriguez
CoRL, 2020

paper / website / bibtex

A framework that achieves the best of TAMP and robot-learning for manipulating rigid objects.

sym Towards Practical Multi-object Manipulation using Relational Reinforcement Learning
Richard Li, Allan Jabri, Trevor Darrell, Pulkit Agrawal
ICRA, 2020

paper / website / code / bibtex

Combining graph neural networks with curriculum learning for solve long horizon multi-object manipulation tasks.

sym Superposition of Many Models into One
Brian Cheung, Alex Terekhov, Yubei Chen, Pulkit Agrawal, Bruno Olshausen,
NeurIPS, 2019

arxiv / video tutorial / code / bibtex

A method for storing multiple neural network models for different tasks into a single neural network.

sym Real-time Video Detection of Falls in Dementia Care Facility and Reduced Emergency Care
Glen L Xiong, Eleonore Bayen, Shirley Nickels, Raghav Subramaniam, Pulkit Agrawal, Julien Jacquemot, Alexandre M Bayen, Bruce Miller, George Netscher
American Journal of Managed Care , 2019

paper / SafelyYou / bibtex

Computer Vision based Fall Detection system reduces number of falls and emergency room visits in people with Dementia.

sym Zero Shot Visual Imitation
Deepak Pathak*, Parsa Mahmoudieh*, Michael Luo, Pulkit Agrawal*,
Evan Shelhamer, Alexei A. Efros, Trevor Darrell (* equal contribution)
ICLR, 2018   (Oral)

paper / website / code / slides / bibtex

Self-supervised learning of skills helps an agent imitate the task presented as a sequence of images. Forward consistency loss overcomes key challenges of inverse and forward models.

sym Investigating Human Priors for Playing Video Games
Rachit Dubey, Pulkit Agrawal, Deepak Pathak, Alexei A. Efros, Tom Griffiths
ICML, 2018

paper / website / youtube cover / media / bibtex

An empirical study of various kinds of prior information used by humans to solve video games. Such priors make them significantly more sample efficient as compared to Deep Reinforcement Learning algorithms.

sym Learning Instance Segmentation by Interaction
Deepak Pathak*, Yide Shentu*, Dian Chen*, Pulkit Agrawal*, Trevor Darrell, Sergey Levine, Jitendra Malik   (*equal contribution)
CVPR Workshop, 2018

paper / website bibtex

A self-supervised method for learning to segment objects by interacting with them.

sym Fully Automated Echocardiogram Interpretation in Clinical Practice: Feasibility and Diagnostic Accuracy
Jeffrey Zhang, Sravani Gajjala, Pulkit Agrawal, Geoffrey H Tison, Laura A Hallock, Lauren Beussink-Nelson, Mats H Lassen, Eugene Fan, Mandar A Aras, ChaRandle Jordan, Kirsten E Fleischmann, Michelle Melisko, Atif Qasim, Sanjiv J Shah, Ruzena Bajcsy, Rahul C Deo
Circulation, 2018

paper / arxiv / bibtex

Computer vision method for building fully automated and scalable analysis pipeline for echocardiogram interpretation.

sym Curiosity Driven Exploration by Self-Supervised Prediction
Deepak Pathak, Pulkit Agrawal, Alexei A. Efros, Trevor Darrell
ICML, 2017

arxiv / video / talk / code / project website / bibtex

Intrinsic curiosity of agents enables them to learn useful and generalizable skills without any rewards from the environment.

sym What Will Happen Next?: Forecasting Player Moves in Sports Videos
Panna Felsen, Pulkit Agrawal, Jitendra Malik
ICCV, 2017

paper / bibtex

Feature learning by making use of an agent's knowledge of its motion.

sym Combining Self-Supervised Learning and Imitation for Vision-based Rope Manipulation
Ashvin Nair*, Dian Chen*, Pulkit Agrawal*, Phillip Isola, Pieter Abbeel, Jitendra Malik, Sergey Levine
(*equal contribution)
ICRA, 2017

arxiv / website / video / bibtex

Self-supervised learning of low-level skills enables a robot to follow a high-level plan specified by a single video demonstration. The code for the paper Zero Shot Visual Imitation subsumes this project's code release.

sym Learning to Perform Physics Experiments via Deep Reinforcement Learning
Misha Denil, Pulkit Agrawal*, Tejas D Kulkarni, Tom Erez, Peter Battaglia, Nando de Freitas
ICLR, 2017

arxiv / media / video / bibtex

Deep reinforcement learning can equip an agent with the ability to perform experiments for inferring physical quanities of interest.

sym Reduction in Fall Rate in Dementia Managed Care through Video Incident Review: Pilot Study
Eleonore Bayen, Julien Jacquemot, George Netscher, Pulkit Agrawal, Lynn Tabb Noyce, Alexandre Bayen
Journal of Medical Internet Research, 2017

paper / bibtex

Analysis how continuous video monitoring and review of falls of individuals with dementia can support better quality of care.

sym Human Pose Estimation with Iterative Error Feedback
Joao Carreira, Pulkit Agrawal, Katerina Fragkiadaki, Jitendra Malik
CVPR, 2016   (Spotlight)

arxiv / code / bibtex

Iterative Error Feedback (IEF) is a self-correcting model that progressively changes an initial solution by feeding back error predictions. In contrast to feedforward CNNs that only capture structure in inputs, IEF captures structure in both the space of inputs and outputs.

sym Learning to Poke by Poking: Experiential Learning of Intuitive Physics
Pulkit Agrawal*, Ashvin Nair*, Pieter Abbeel, Jitendra Malik, Sergey Levine
(*equal contribution)
NIPS, 2016, (Oral)

arxiv / talk / project website / data / bibtex

Robot learns how to push objects to target locations by conducting a large number of pushing experiments. The code for the paper Zero Shot Visual Imitation subsumes this project's code release.

sym What makes Imagenet Good for Transfer Learning?
Jacob Huh , Pulkit Agrawal, Alexei A. Efros
NIPS LSCVS Workshop, 2016,   (Oral)

arxiv / project website / code / bibtex

An empirical investigation into various factors related to the statistics of Imagenet dataset that result in transferrable features.

sym Learning Visual Predictive Models of Physics for Playing Billiards
Katerina Fragkiadaki*, Pulkit Agrawal*, Sergey Levine, Jitendra Malik
(*equal contribution)
ICLR, 2016

arxiv / code / bibtex

This work explores how an agent can be equipped with an internal model of the dynamics of the external world, and how it can use this model to plan novel actions by running multiple internal simulations (“visual imagination”).

sym Generic 3d Representation via Pose Estimation and Matching
Amir R. Zamir, Tilman Wekel, Pulkit Agrawal, Colin Weil, Jitendra Malik, Silvio Savarese
ECCV, 2016

arxiv / website / dataset / code / bibtex

Large-scale study of feature learning using agent's knowledge of its motion. This paper extends our ICCV 2015 paper.

sym Learning to See by Moving
Pulkit Agrawal, Joao Carreira, Jitendra Malik
ICCV, 2015

arxiv / code / bibtex

Feature learning by making use of an agent's knowledge of its motion.

sym Analyzing the Performance of Multilayer Neural Networks for Object Recognition
Pulkit Agrawal, Ross Girshick, Jitendra Malik
ECCV, 2014

arxiv / bibtex

A detailed study of how to finetune neural networks and the nature of the learned representations.

sym Pixels to Voxels: Modeling Visual Representation in the Human Brain
Pulkit Agrawal, Dustin Stansbury, Jitendra Malik, Jack Gallant
(*equal contribution)
arXiv, 2014

arxiv / unpublished results / bibtex

Comparing the representations learnt by a Deep Neural Network optimized for object recognition against the human brain.

sym The Automatic Assessment of Knowledge Integration Processes in Project Teams
Gahgene Gweon, Pulkit Agrawal, Mikesh Udani, Bhiksha Raj, Carolyn Rose
Computer Supported Collaborative Learning , 2011   (Best Student Paper Award)

arxiv / bibtex

Method for identifying important parts of a group conversation directly from speech data.

Patents
System and Method for Detecting, Recording and Communicating Events in the Care and Treatment of Cognitively Impaired Persons
George Netscher, Julien Jacquemot, Pulkit Agrawal, Alexandre Bayen
US Patent: US20190287376A1, 2019
Invariant Object Representation of Images Using Spiking Neural Networks
Pulkit Agrawal, Somdeb Majumdar, Vikram Gupta
US Patent: US20150278628A1, 2015
Invariant Object Representation of Images Using Spiking Neural Networks
Pulkit Agrawal, Somdeb Majumdar
US Patent: US20150278641A1, 2015

Service
Area Chair, ICML, 2021
Area Chair, ICLR, 2021
Area Chair, NeurIPS, 2020
Area Chair, CoRL, 2020, 2019
Reviewer for CVPR, ICCV, ECCV, NeurIPS, ICML, ICLR, RSS, ICRA, IJRR, IJCV, IEEE RA-L, TPAMI etc.

Lab Alumni
Abhishek Gupta, PostDoc, now Faculty at University of Washington.
Lara Zlokapa, MEng, 2022
Haokuan Luo, MEng, 2022 (now at Hudson River)
Albert Yue, MEng, 2022 (now at Hudson River)
Matthew Stallone, MEng, 2022
Eric Chen, MEng, 2021 (now at Aurora)
Joshua Gruenstein, 2021 (now CEO Tutor Intelligence)
Alon Z. Kosowsky-Sachs, 2021 (now CTO Tutor Intelligence)
Avery Lamp (now at stealth startup) Sanja Simonkovj, 2021 (Masters Student)
Oran Luzon, 2021 (Undergraduate Researcher)
Blake Tickell, 2020 (Visiting Researcher)
Ishani Thakur, 2020 (Undergraduate Researcher)

template / accessibility