Whale: Efficient Giant Model Training over Heterogeneous GPUs

Xianyan Jia, Le Jiang, Ang Wang, Wencong Xiao, Ziji Shi, Jie Zhang 0078, Xinyuan Li, Langshi Chen, Yong Li, Zhen Zheng, Xiaoyong Liu, Wei Lin 0016. Whale: Efficient Giant Model Training over Heterogeneous GPUs. In Jiri Schindler, Noa Zilberman, editors, 2022 USENIX Annual Technical Conference, USENIX ATC 2022, Carlsbad, CA, USA, July 11-13, 2022. pages 673-688, USENIX Association, 2022. [doi]

Abstract

Abstract is missing.