 |
|
Jun-Yan Zhu Postdoctoral researcher Computer Science and Artificial Intelligence Laboratory Department of EECS Massachusetts Institute of Technology |
||||
|
||||
|
CV | GitHub | Google Scholar | Arxiv |
||||
|
|
||||||||||||||
| I am currently a postdoctoral researcher at MIT CSAIL, working with Prof. Antonio Torralba, Prof. William T. Freeman, and Prof. Josh Tenenbaum. I obtained my Ph.D. from UC Berkeley after spending five wonderful years at CMU and Berkeley with Prof. Alexei A. Efros. I study computer vision, computer graphics, and machine learning with a goal of building machines capable of understanding and recreating our visual world. My Ph.D. work was supported by a Facebook Fellowship. I received my B.E in Computer Sciences from Tsinghua University in 2012, where I worked with Prof. Zhuowen Tu and Dr. Eric Chang at Microsoft Research Asia. I was also a member of Tsinghua's Graphics Group led by Prof. Shi-Min Hu. |
Cat Papers
If you like cats and love reading cool vision, learning, and graphics papers, check out GitHub | Webpage.
Events
CVPR 2018 Tutorial on Generative Adversarial Networks.
ICCV 2017 Tutorial on Generative Adversarial Networks.
ICML 2017 Workshop on Visualization for Deep Learning.
SIGGRAPH Asia 2014 invited Course on Data-Driven Visual Computing.
Publications
 |
High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs Ting-Chun Wang, Ming-Yu Liu, Jun-Yan Zhu, Andrew Tao, Jan Kautz, Bryan Catanzaro
In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018
|
 |
Spatially Transformed Adversarial Examples Chaowei Xiao*, Jun-Yan Zhu*, Bo Li, Mingyan Liu, and Dawn Song In International Conference on Learning Representations (ICLR), 2018
|
 |
Generating Adversarial Examples with Adversarial Networks Chaowei Xiao, Bo Li, Jun-Yan Zhu, Mingyan Liu, and Dawn Song In International Joint Conference on Artificial Intelligence (IJCAI), 2018
|
 |
Toward Multimodal Image-to-Image Translation Jun-Yan Zhu, Richard Zhang, Deepak Pathak, Trevor Darrell, Alexei A. Efros, Oliver Wang, and Eli Shechtman In Advances in Neural Information Processing Systems (NIPS), 2017
|
 |
Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks Jun-Yan Zhu*, Taesung Park*, Phillip Isola, and Alexei A. Efros In IEEE International Conference on Computer Vision (ICCV), 2017
|
 |
Image-to-Image Translation with Conditional Adversarial Nets Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A. Efros In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017 See neat uses of #pix2pix on Twitter. |
 |
Real-Time User-Guided Image Colorization with Learned Deep Priors Richard Zhang*, Jun-Yan Zhu*, Phillip Isola, Xinyang Geng, Angela S. Lin, Tianhe Yu, and Alexei A. Efros In ACM Transactions on Graphics (SIGGRAPH), 2017
Project |
GitHub |
Youtube |
Video |
Fastforward |
 |
Light Field Video Capture Using a Learning-Based Hybrid Imaging System Ting-Chun Wang, Jun-Yan Zhu, Nima Khademi Kalantari, Alexei A. Efros, and Ravi Ramamoorthi In ACM Transactions on Graphics (SIGGRAPH), 2017
Project |
GitHub |
Youtube |
Training code |
 |
Generative Visual Manipulation on the Natural Image Manifold Jun-Yan Zhu, Philipp Krähenbühl, Eli Shechtman, and Alexei A. Efros In European Conference on Computer Vision (ECCV), 2016 See article in California Magazine
|
 |
A 4D Light-Field Dataset and CNN Architectures for Material Recognition Ting-Chun Wang, Jun-Yan Zhu, Ebi Hiroaki, Manmohan Chandraker, Alexei A. Efros, and Ravi Ramamoorthi In European Conference on Computer Vision (ECCV), 2016
Paper |
Data (thumbnail) |
Full data (15.9G) |
 |
Learning a Discriminative Model for the Perception of Realism in Composite Images Jun-Yan Zhu, Philipp Krähenbühl, Eli Shechtman, and Alexei A. Efros In IEEE International Conference on Computer Vision (ICCV), 2015
|
 |
Mirror Mirror: Crowdsourcing Better Portraits Jun-Yan Zhu, Aseem Agarwala, Alexei A. Efros, Eli Shechtman, and Jue Wang In ACM Transactions on Graphics (SIGGRAPH Asia), 2014
Project (code) | Paper | Data | Slides | Supplement | BibTex |
 |
AverageExplorer: Interactive Exploration and Alignment of Visual Data Collections Jun-Yan Zhu, Yong Jae Lee and Alexei A. Efros In ACM Transactions on Graphics (SIGGRAPH), 2014
See article in The New Yorker |
 |
MILCut: A Sweeping Line Multiple Instance Learning Paradigm for Interactive Image Segmentation Jiajun Wu*, Yibiao Zhao*, Jun-Yan Zhu, Siwei Luo and Zhuowen Tu In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014
|
 |
Reverse Image Segmentation: A High-Level Solution to a Low-Level Task Jiajun Wu, Jun-Yan Zhu, and Zhuowen Tu In British Machine Vision Conference (BMVC), 2014
|
 |
Unsupervised Object Class Discovery via Saliency-Guided Multiple Class Learning Jun-Yan Zhu, Jiajun Wu, Yan Xu, Eric Chang and Zhuowen Tu In IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2015 (an expanded journal version of our CVPR 2012 paper)
Project | Paper | Supplement | Poster | BibTex |
 |
Multiple Clustered Instance Learning for Histopathology Cancer Image Classification, Segmentation and Clustering Yan Xu*, Jun-Yan Zhu*, Eric I-Chao Chang and Zhuowen Tu In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2012 (See an expanded journal version at Medical Image Analysis (MIA), 2014
|
 |
Motion-Aware Gradient Domain Video Composition Tao Chen, Jun-Yan Zhu, Ariel Shamir and Shi-Min Hu In IEEE Transactions on Image Processing (TIP), 2013
|
Preprints
 |
CyCADA: Cycle-Consistent Adversarial Domain Adaptation Judy Hoffman, Eric Tzeng, Taesung Park, Jun-Yan Zhu, Phillip Isola, Alexei A. Efros, and Trevor Darrell In arXiv:1711.03213, 2017
|
Software
pix2pixHD: 2048x1024 image synthesis with conditional GANs.
BicycleGAN: multimodal image-to-image translation.
Interactive Deep Colorization: a user interface for real-time user-guided colorization.
Light Field Video: light field video applications (e.g. video refocusing, changing aperture and view).
CycleGAN: Torch implementation for learning an image-to-image translation without input-output pairs.
pix2pix: Torch implementation for learning a mapping from input images to output images.
pytorch CycleGAN & pix2pix: PyTorch implementation for both unpaired and paired image-to-image translation.
iGAN: a deep learning software that easily generates images with a few brushstrokes.
RealismCNN: code for predicting and improving visual realism in composite images.
MCILBoost: a boosting-based Multiple Instance Learning (MIL) software.
MirrorMirror: an expression training App that helps users mimic their own expressions.
SelectGoodFace: a program for selecting attractive/serious portraits from a personal photo collection.
FaceDemo: a simple 3D face alignment and warping demo.
Talks
Learning to Synthesize and Manipulate Natural Photos
MIT CSAIL, HKUST CSE Departmental Seminar, ICCV tutorial on GANs, O'Reilly AI, AI with the best, Y Conf, DEVIEW, ODSC West (2017)
Stanford, Facebook, CUHK, SNU (2017)
SIGGRAPH, NVIDIA Innovation Theater, Global AI Hackathon (2017)
Visual Manipulation and Synthesis on the Natural Image Manifold
Facebook, MSR, Berkeley BAIR, Tsinghua, Fudan Univ, ICML workshop "Visualization for Deep Learning" (2016)
Mirror Mirror: Crowdsourcing Better Portraits
SIGGRAPH Asia (2014)
What Makes Big Visual Data Hard?
SIGGRAPH Asia invited course "Data-Driven Visual Computing" (2014)
AverageExplorer: Interactive Exploration and Alignment of Visual Data Collections
SIGGRAPH (2014)
Discovering Objects and Harvesting Visual Concepts via Weakly Supervised Learning
Berkeley Visual Computing Lab (2014)
Awards
Facebook Fellowship (2015)
Outstanding Undergraduate Thesis in Tsinghua University (2012)
Excellent Undergraduate Student in Tsinghua University (2012)
National Scholarship, by Ministry of Education of China (2009 and 2010)
Singapore Technologies Engineering China Scholarship (2010, 2011, and 2012)
MISC
Here is my cat Aquarius.
