Hao Zhang

Postdoctoral Scholar
EECS, UC Berkeley

Email: sjtu.haozhang AT


I am currently a postdoctoral researcher at RISE Lab, UC Berkeley, working with Prof. Ion Stoica.

I am recently working on building end-to-end composable and automated systems for large-scale distributed DL. The most recent projects is Alpa that automates model-parallel training on large-scale distributed GPU/TPU clusters.

I study large-scale distributed ML in the joint context of ML and systems, concerning both performance and usability. My work spans across parallel ML programmability, representations of parallelisms, performance optimizations, system architectures, auto-parallelization techniques, and AutoML, with applications in computer vision, natural language processing, and healthcare.

I completed my Ph.D. at the School of Computer Science, Carnegie Mellon University. My advisor was Prof. Eric Xing. My Ph.D. thesis is Machine Learning Parallelism Could Be Adaptive, Composable and Automated. Several of works including Poseidon, Cavs, GeePS, and Alpa are parts of the CASL project and the Ray project, and now being commercialized at multiple start-ups including Petuum and AnyScale.



[ / / / / / ]

Lianmin Zheng*, Zhuohan Li*, Hao Zhang*, Yonghao Zhuang, Zhifeng Chen, Yanping Huang, Yida Wang, Yuanzhong Xu, Danyang Zhuo, Joseph E. Gonzalez, Ion Stoica
(* indicates equal contributions)
OSDI 2022
arXiv project presentation
Aurick Qiao, Sang Keun Choe, Suhas Jayaram Subramanya, Willie Neiswanger, Qirong Ho, Hao Zhang, Gregory R. Ganger, Eric P. Xing
OSDI 2021(Jay Lepreau Best Paper Award!)
arXiv code slides presentation
Zhuohan Li, Siyuan Zhuang, Shiyuan Guo, Danyang Zhuo, Hao Zhang, Dawn Song, Ion Stoica
ICML 2021
arXiv code
Hao Zhang, Peng Wu, Zhijie Deng, Christy Li, Aurick Qiao, Qirong Ho, Zeya Wang, Eric P. Xing
Preprint 2020
Hao Zhang, Christy Li, Zhijie Deng, Xiaodan Liang, Lawrence Carin, Eric P. Xing
NeurIPS 2020
Zhijie Deng, Xiao Yang, Hao Zhang, Yinpeng Dong, Jun Zhu
Preprint 2020
arXiv code
Hao Zhang*, Haowen Xu*, Zhiting Hu, Xiaodan Liang, Qirong Ho, Eric P. Xing
(* indicates equal contributions)
ICLR 2019
arXiv code
Wei Dai, Yi Zhou, Nanqing Dong, Hao Zhang, Eric P. Xing
ICLR 2019
Hao Zhang*, Shizhen Xu*, Graham Neubig, Wei Dai, Qirong Ho, Guangwen Yang, Eric P. Xing
(* indicates equal contributions)
ATC 2018 (Oral), AISys@SOSP 2017, MLSys@NeurIPS 2017
arXiv slides code medium post
Hao Zhang*, Zhijie Deng*, Xiaodan Liang, Luona Yang, Shizhen Xu, Jun Zhu, Eric P. Xing (* indicates equal contributions)
NeurIPS 2017(Nvidia Pioneer Research Award!)
arXiv code
Hao Zhang, Zeyu Zheng, Shizhen Xu, Wei Dai, Qirong Ho, Xiaodan Liang, Zhiting Hu, Jinliang Wei, Pengtao Xie, Eric P. Xing
ATC 2017 (Oral)
arXiv slides project zhihu post (in Chinese)
Henggang Cui, Hao Zhang, Gregory R. Ganger, Phillip B. Gibbons, Eric P. Xing
EuroSys 2016
slides code
Zhicheng Yan, Hao Zhang, Robinson Piramuthu, Vignesh Jagadeesh, Dennis DeCoste, Wei Di, Yizhou Yu
ICCV 2015
arXiv project