Prof. Trevor Darrell
CS Division, University of California, Berkeley
Director, Berkeley Deep Drive (BDD)
Co-Director, Berkeley Artificial Intelligence Research (BAIR)
Faculty Director, California PATH
Prof. Darrell is on the faculty of the CS Division of the EECS Department at UC Berkeley. He leads Berkeley’s DeepDrive Industrial Consortia, is co-Director of the Berkeley Artificial Intelligence Research (BAIR) lab, and is Faculty Director of PATH at UC Berkeley. Darrell’s group develops algorithms for large-scale perceptual learning, including object and activity recognition and detection, for a variety of applications including multimodal interaction with robots and mobile devices. His interests include computer vision, machine learning, natural language processing, and perception-based human computer interfaces. Prof. Darrell previously led the vision group at the International Computer Science Institute in Berkeley, and was on the faculty of the MIT EECS department from 1999-2008, where he directed the Vision Interface Group. He was a member of the research staff at Interval Research Corporation from 1996-1999, and received the S.M., and PhD. degrees from MIT in 1992 and 1996, respectively. He obtained the B.S.E. degree from the University of Pennsylvania in 1988.
Current Teaching
CS294-43: Object and Activity Recognition Seminar
CS294-131: Deep Learning Seminar (Fall, Spring)
CS280: Computer Vision (Spring 2016)
Recent Presentations
Jan 2014 NSF Workshop Talk (pdf)
Sept 2012 BAVM Invited Talk (pdf) (pptx)
Software
CAFFE - Open Source Deep Learning; Yangqing Jia, Evan Shelhamer, Jeff Donahue, Sergey Karayev, Jonathan Long, Ross Girshick, Sergio Guadarrama, Trevor Darrell
LRCN - Long-term Recurrent Convolutional Networks; Jeff Donahue, Lisa Anne Hendricks, Sergio Guadarrama, Marcus Rohrbach, Subhashini Venugopalan, Kate Saenko, Trevor Darrell
LSDA - Large Scale Detector Adaptation; Judy Hoffman, Sergio Guadarrama, Eric Tzeng, Jeff Donahue, Ross Girshick, Trevor Darrell, Kate Saenko
Recent Publications (2017-2015)
Please see [Google Scholar Page sorted by date]
Current Group
Postdocs:
Dr. Marcus Rohrbach
Fisher Yu (Feb 2017)
Zeynep Akata
(Visiting)
Jeff Donahue
Evan Shelhammer
Eric Tzeng
Lisa Hendricks
Yang Gao
Chelsea Finn (w/
Abbeel, Levine)
Deepak Pathak
Samaneh Azadi
Coline Devin (w/ Abbeel, Levine)
Ronghang Hu
Huazhe Xu
Dequan Wang
Erin Grant (w/ Griffths)
2014 Publications
ICML
DECAF: A deep convolutional activation feature for generic visual recognition; Jeff Donahue, Yangqing Jia, Oriol Vinyals, Judy Hoffman, Ning Zhang, Eric Tzeng, Trevor Darrell
On learning to localize objects with minimal supervision; Hyun Oh Song, Ross Girshick, Stefanie Jegelka, Julien Mairal, Zaid Harchaoui, Trevor Darrell
CVPR
PANDA: Pose Aligned Networks for Deep Attribute Modeling; Ning Zhang, Manohar Paluri, Marc'Aurelio Ranzato, Trevor Darrell, Lubomir Bourdev
Rich feature hierarchies for accurate object detection and semantic segmentation; Ross Girshick, Jeff Donahue, Trevor Darrell, Jitendra Malik
Anytime Recognition of Objects and Scenes; Sergey Karayev, Mario Fritz, Trevor Darrell
Learning Scalable Discriminative Dictionary with Sample Relatedness; Jiashi Feng, Stefanie Jegelka, Shuicheng Yan, Trevor Darrell
Continuous Manifold Based Adaptation for Evolving Visual Domains; Judy Hoffman, Trevor Darrell, Kate Saenko
ICRA
Interactive Adaptation of Real-Time Object Detectors; Daniel Ghring and Judy Hoffman and Erik Rodner and Kate Saenko, and Trevor Darrell
RSS
Open-vocabulary Object Retrieval; Sergio Guadarrama, Erik Rodner, Kate Saenko, Ning Zhang, Ryan Farrell, Jeff Donahue, Trevor Darrell
BMVC
Recognizing Image Style; Sergey Karayev, Aaron Hertzmann, Holger Winnemoeller, Aseem Agarwala, Trevor Darrell
ECCV
Part-based R-CNNs for Fine-grained Category Detection, Ning Zhang, Jeff Donahue, Ross Girshick, Trevor Darrell, UC Berkeley
MM
CAFFE: Convolutional Architecture for Fast Feature Embedding; Yangqing Jia, Evan Shelhamer, Jeff Donahue, Sergey Karayev, Jonathan Long, Ross Girshick, Sergio Guadarrama, Trevor Darrell
PAMI
Modeling Radiometric Uncertainty for Vision with Tone-mapped Color Images; Ying Xiong, Daniel Scharstein, Ayan Chakrabarti, Trevor Darrell, Baochen Sun, Kate Saenko, Todd Zickler
NIPS
Large Scale Detector Adaptation; Judy Hoffman, Sergio Guadarrama, Eric Tzeng, Jeff Donahue, Ross Girshick, Trevor Darrell, Kate Saenko
Weakly-supervised Discovery of Visual Pattern Configurations; Hyun Oh Song, Yong Jae Lee, Stefanie Jegelka, Trevor Darrell
Do Convnets Learn Correspondence? Jon Long, Ning Zhang, Trevor Darrell
arXiv
DenseNet: Implementing Efficient ConvNet Descriptor Pyramids; Forrest Iandola, Matt Moskewicz, Sergey Karayev, Ross Girshick, Trevor Darrell, Kurt Keutzer
One-Shot Adaptation of Supervised Deep Convolutional Models; Judy Hoffman, Eric Tzeng, Jeff Donahue, Yangqing Jia, Kate Saenko, Trevor Darrell
Publications 2008-2013:
S. Guadarrama, N. Krishnamoorthy, G. Malkarnenkar, S. Venugopalan, R. Mooney, T. Darrell, K. Saenko, YouTube2Text: Recognizing and Describing Arbitrary Activities Using Semantic Hierarchies and Zero-shot Recognition, ICCV 2013
Y. Jia and T. Darrell, Latent Task Adaptation with Large-Scale Hierarchies, ICCV 2013
N. Zhang, R. Farrell, F. Iandola, T. Darrell, Deformable part descriptors for fine-grained recognition and attribute prediction, ICCV 2013
Y. Jia, J T Abbott, J. Austerweil, T. Griffiths, T. Darrell, Visual Concept Learning: Combining Machine Vision and Bayesian Generalization on Concept Hierarchies, NIPS 2013
Y. Jia, O. Vinyals, and T. Darrell. On Compact Codes for Spatially Pooled Features. ICML 2013
H. O. Song, R. Girshick, and T. Darrell, Discriminatively Activated Sparselets, ICML 2013
J. Donahue, J. Hoffman, E. Rodner, K. Saenko, and T. Darrell, Semi-Supervised Domain Adaptation with Instance Constraints, CVPR 2013
V. Chu, I. McMahon, L. Riano, C. G. McDonald, Q. He, J. Perez-Tejada, M. Arrigo, N. Fitter, J. Nappo, T. Darrell, and K. J. Kuchenbecker. Using
robotic exploratory procedures to learn the meaning of haptic adjectives. ICRA 2013. Best Cognitive Robotics Paper Award
J. Hoffman, E. Rodner, J. Donahue, T. Darrell, K. Saenko, Efficient Learning of Domain-invariant Image Representations, ICLR 2013 Conference
O. Vinyals, Y. Jia, T. Darrell, Why Size Matters: Feature Coding as Nystrom Sampling, ICLR 2013 Workshop
S. Karayev, M. Fritz, T. Darrell, Timely Object Recognition, NIPS 2012
O. Vinyals, Y. Jia, L. Deng, T. Darrell, Learning with Recursive Perceptual Representations, NIPS 2012
S. Chung, C. Christoudias, T. Darrell, S. Ziniel, L. Kalish, A Novel Image Based Tool to Reunite Children with Their Families after Disasters, AEMJ
T. Althoff, H. O. Song, T. Darrell, Detection Bank: An Object Detection Based Video Representation for Multimedia Event Recognition, ACM MM 2012
J. Hoffman, B. Kulis, T. Darrell, K. Saenko, Discovering Latent Domains for Multisource Domain Adaptation, ECCV 2012
H. O. Song, S. Zickler, T. Althoff, R. Girshick, M. Fritz, C. Geyer, P. Felzenszwalb, T. Darrell, Sparselet Models for Efficient Multiclass Object Detection, ECCV 2012
S. Miller, J. Van Den Berg, M. Fritz, T. Darrell, K. Goldberg, P. Abbeel, A geometric approach to robotic laundry folding, IJRR 2012
S. Virtanen, Y. Jia, A. Klami, T. Darrell. Factorized Multi-modal Topic Model. UAI 2012
Y. Xiong, K. Saenko, T. Zickler, T. Darrell, From Pixels to Physics: Probabilistic Color De-rendering, CVPR 2012.
N. Zhang, R. Farrell, T. Darrell. Pose Pooling Kernels for Sub-category Recognition. CVPR 2012
Y. Jia, C. Huang, T. Darrell. Beyond Spatial Pyramids: Receptive Field Learning for Pooled Image Features, CVPR 2012
Y. Jia and T. Darrell, Heavy-tailed Distances for Gradient Based Image Descriptors, NIPS 2011
R. Farrell, O. Oza, N. Zhang, V. Morariu, T. Darrell, and L. Davis, Birdlets: Subordinate Categorization Using Volumetric Primitives and Pose-Normalized Appearance, ICCV 2011.
T. Tuytelaars, M. Fritz, K. Saenko, and T. Darrell, The NBNN Kernel, ICCV 2011.
Y. Jia, M. Salzmann, and T. Darrell, Learning Cross-modality Similarity for Multinomial Data, ICCV 2011.
S. Karayev, A. Janoch, Y. Jia, J. Barron, M. Fritz, K. Saenko, and T. Darrell, A Category-level 3-D Database: Putting the Kinect to Work, in ICCV 2011 Workshop on Consumer Depth Cameras for Computer Vision, 2011.
H. O. Song, M. Fritz, C. Gu and T. Darrell, Visual Grasp Affordances From Appearance-Based Cues, ICCV Workshop on Challenges and Opportunities in Robot Perception, 2011
P. Wang, S. Miller, M. Fritz, T. Darrell, and P. Abbeel, Perception for the Manipulation of Socks, IROS 2011.
K. Saenko, Y. Jia, M. Fritz, J. Long, A. Janoch, A. Shyr, S. Karayev and T. Darrell, Practical 3-D Object Detection Using Category and Instance-Level Appearance Models, IROS 2011
S. Miller, M. Fritz, T. Darrell, and P. Abbeel, Parametrized Shape Models for Clothing, ICRA 2011.
S. Karayev, M. Fritz, S. Fidler, and T. Darrell, A Probabilistic Model for Recursive Factorized Image Features, CVPR 2011.
B. Kulis, K. Saenko, and T. Darrell, What You Saw is Not What You Get: Domain Adaptation Using Asymmetric Kernel Transforms, CVPR 2011.
A. Shyr, T. Darrell, M. Jordan, and R. Urtasun, Supervised Hierarchical Pitman-Yor Process for Natural Scene Segmentation, CVPR 2011.
T. Owens, K. Saenko, A. Chakrabarti, Y. Xiong, T. Zickler, and T. Darrell. Learning Object Color Models from Multi-view Constraints, CVPR 2011.
A. Eden, M. Christoudias, and T. Darrell, Finding Lost Children, POV 2011.
Y. Jia, M. Salzmann, and T. Darrell, Factorized Latent Spaces with Structured Sparsity, NIPS 2010.
M. Fritz, K. Saenko, and T. Darrell, Size Matters: Metric Visual Search Constraints from Monocular Metadata, NIPS 2010.
G. Friedland, O. Vinyals, and T. Darrell, Multimodal Location Estimation. ACM Multimedia 2010.
K. Saenko, B. Kulis, M. Fritz, and T. Darrell, Adapting Visual Category Models to New Domains, ECCV 2010.
M. Christoudias, R. Urtasun, M. Salzmann, and T. Darrell, Learning to Recognize Objects from Unseen Modalities, ECCV 2010.
A. Kapoor, K. Grauman, R. Urtasun, and T. Darrell, Gaussian Processes for Object Categorization, IJCV, 2010.
M. Salzmann, C. H. Ek, R. Urtasun, and T. Darrell. Factorized orthogonal latent spaces. UAI 2010.
M. Fritz, M. Black, G. Bradski, and T. Darrell, An Additive Latent Feature Model for Transparent Object Recognition, NIPS 2009.
Z. Stone, T. Zickler, and T. Darrell, Toward Large-Scale Face Recognition Using Social Network Context, Proc. of the IEEE 2010
B. Kulis and T. Darrell, Learning to Hash with Binary Reconstructive Embeddings, NIPS 2009.
K. Saenko and T. Darrell, Filtering Abstract Senses From Image Search Results, NIPS 2009.
A. Quattoni, X. Carreras, M. Collins, and T. Darrell, An Efficient Projection for L-1/L-Infinity Regularization, ICML 2009.
T. Yeh and T. Darrell, Fast Concurrent Object Localization and Recognition, CVPR 2009.
M. Christoudias, R. Urtasun, A. Kapoor and T. Darrell, Co-training with Noisy Perceptual Observations, CVPR 2009.
A. Geiger, R. Urtasun, and T. Darrell, Rank Priors for Continuous Non-Linear Dimensionality Reduction, CVPR 2009.
M. Frampton, R. Fernandez, P. Ehlen, M. Christoudias, T. Darrell, and S. Peters (2009) Who is ``You? Combining Linguistic and Gaze Features to Resolve Second-Person References in Dialogue, EACL 2009.
K. Saenko, K. Livescu, J. Glass, and T. Darrell, Multistream Articulatory Feature-Based Models for Visual Speech Recognition, TPAMI 2009
K. Saenko, and T. Darrell, Unsupervised Learning of Visual Sense Models for Polysemous Word, NIPS 2008.
T. Yeh, J. Lee, and T. Darrell, Photo-based Question Answering, ACM Multimedia 2008.
T. Yeh, and T. Darrell, Multimodal Question Answering for Mobile Devices, IUI 2008.
M. Christoudias, R. Urtasun and T. Darrell, Multi-View Learning in the Presence of View Disagreement, UAI 2008.
R. Urtasun, D. J. Fleet, A. Geiger, J. Popovic, T. Darrell and N. D. Lawrence, Topologically-Constrained Latent Variable Models, ICML 2008.
M. Christoudias, R. Urtasun and T. Darrell, Unsupervised Feature Selection via Distributed Coding for Multi-view Object Recognition, CVPR 2008.
A. Quattoni, M. Collins, and T. Darrell, Transfer Learning for Image Classification with Sparse Prototype Representations, CVPR 2008.
T. Yeh, J. Lee, and T. Darrell, Dynamic Visual Category Learning, CVPR 2008.
R. Urtasun, and T. Darrell, Sparse probabilistic regression for activity-independent human pose inference, CVPR 2008.
T. Yeh, J. Lee, and T. Darrell, Scalable classifiers for Internet vision tasks, IEEE Workshop on Internet Vision 2008.
Z. Stone, T. Zickler, and T. Darrell, Autotagging Facebook: Social Network Context Improves Photo Annotation, IEEE Workshop on Internet Vision 2008.
Publications 1987-2007:
See list of MIT and Interval publications: http://www.eecs.berkeley.edu/~trevor/Publications.htm
Previous Postdocs
(excluding former students)
Dr. Stefanie Jegelka
Dr. Thomas
Mensink (visiting)
Dr. Sergio
Guardarrama
Dr. Jiashi Feng
Dr. Eric Rodner
Dr. Ryan Farrell
Dr. Lorenzo Riano
Dr.
Daniel Goehring
Dr. Brian Kulis
Dr. David
Demirdjian
Dr. Raquel Urtasun
Dr. Mario Fritz
Dr. Matthieu
Salzmann
Dr. Sanja Fidler
(visiting)
Dr. Nicholas Cebron
Dr. Carl Ek
Dr. Peer Stelldinger
Graduated
Students:
Masters (S.M, MIT.):
Morency,
Louis-Philippe, “Stereo-based head post tracking using Iterative Closest Point
and normal flow constraint,” 2002
Checka,
Neal, “Probabilistic framework for multi-modal multiple person tracking,” 2002
Grauman,
Grauman, L., “A statistical image-based model for
visual hull reconstruction and 3D structure inference,” 2003
Yeh,
Pei-Hsiu (Tom), “IDeixis: image-based ideixis for recognizing locations,” 2004
Christoudias,
C., Mario, “Light field appearance manifolds,” 2004
Saenko,
Ekaterina - joint with Glass, James, “Articulatory features for robust speech
recognition,” 2004
Siracusa,
Michael - joint with Fisher, John, W., “Statistical modeling and analysis of
audio-visual association in speech,” 2004
Ariadna
Quattoni - joint with M. Collins, “Object Recognition
with a Latent Conditional Random Field”, 2005
Masters (M.Eng., MIT)
Smith-Michelson, Jared,
“Design and Application of a head detection and tracking system”, 2000
Villoria,
John, A., “Optimizing clustering algorithms for computer vision,” 2001
Rania, Kalaf Y., “Multi-person tracking using dynamic
programming,” 2001
Rangarajan,
Vibav, S., “Interfacing speech recognition and
vision-guided microphone array technologies,” 2003
Bentley, Frank, H., “A
widget-based architecture for perceptive presence,” 2003
Ko,
Theresa, H., “Untethered human motion recognition for a multi-model interface,”
2003
Sundberg,
Patrik, P., “Pose estimation using cascade trees,”
2004
Ross, Benjamin, “Comparision of nearest neighbor methods,” 2004
Goela,
Naveen, “Matching and compressing sequences of visual hulls,” 2004
Dunagan,
Brian, “Tracking with constraints in a web of sensors,” 2004
You, Shuang,
”Fast pedestrian detection with efficient thermal features”, 2005
Lee, John, “Efficient
Object Recognition and Image Retrieval for Large-Scale Applications” (won
Johnson award for outstanding CS M.Eng. thesis)
Ph.D (MIT).
Shakhnarovich,
Gregory, “Learning Features for Visual Classification”, Oct. 2005 [Assistant
Professor, TTI-Chicago]
Rahimi,
Ali, “Learning to Transform Time Series with a Few Examples”, Oct. 2005 [Intel
Research, Berkeley]
Wilson, Kevin, “Learning
Uncertainty Models for Audiovisual Speech Source Localization in Real-World
Environments”, Aug 2006 [Research Scientist, Mitsubishi Electric Research Labs,
Cambridge (MERL)]
Taycher,
Leonid, “Statistical methods for dynamic visual processing”, Aug 2006 [Google
Boston]
Grauman,
Kristen, “Matching sets of features for efficient retrieval and recognition”,
Aug. 2006 [Assistant Professor, CS, University of Texas, Austin]
Morency,
Louis-Philippe, “Dialogue Context and Visual Gesture Recognition”, Oct 2006
[Research Assistant Professor, University of Southern California ICT]
Wang, Sy Bor, “Detecting Communication
Errors from visual cues during the system's conversational turn”, Aug 2008.
[Trimble Navigations, Silicon Valley]
Yeh,
Tom, “Situated Mobile Media Search”, May 2009. [Postdoc, UMD; Assistant Prof.
UC Boulder]
Quattoni,
Ariadna, joint with M. Collins, “Transfer Learning
Algorithms for Image Classification”, May 2009, [Postdoc, U. Barcelona]
Christoudas,
Mario, “Co-training for Multimodal Gesture Recognition”, June 2009. [Postdoc,
MIT & EPFL]
Saenko,
Kate, “Image Sense Disambiguation: A Multimodal
Approach”. Doctoral Thesis”, August 2009 [Postdoc, MIT & Harvard; Assistant
Prof., BU]
Ph.D
(UCB).
Ashley Eden, “Finding
Lost Children”, December 2010 [Dreamworks]
Alex Shyr,
“Incorporating Supervision in Visual Recognition and Segmentation”, May 2011
[Start-up]
Hyun Oh Song “learning
with Parsimony for Large Scale object Detection and Discovery’ May 2014
[Google]
Yangqing
Jia,”Learning Semantic Image Representations at a
Large Scale”, May 2014 [Google]
Sergey Karayev, “Anytime Recognition of Objects and Scenes”, May
2014 [Start-up]
Ning Zhang, “Visual
Representations for Fine-grained Categorization’, May 2015 [Snapchat]
Jonathan Long,
“Understanding and Designing Convolutional Networks for Local Recognition
Problems”, May 2016 [Postdoc]