Whale: Efficient Giant Model Training over Heterogeneous GPUs

Xianyan Jia, Le Jiang, Ang Wang, Wencong Xiao, Ziji Shi, Jie Zhang 0078, Xinyuan Li, Langshi Chen, Yong Li, Zhen Zheng, Xiaoyong Liu, Wei Lin 0016. Whale: Efficient Giant Model Training over Heterogeneous GPUs. In Jiri Schindler, Noa Zilberman, editors, 2022 USENIX Annual Technical Conference, USENIX ATC 2022, Carlsbad, CA, USA, July 11-13, 2022. pages 673-688, USENIX Association, 2022. [doi]

Authors

Xianyan Jia

This author has not been identified. Look up 'Xianyan Jia' in Google

Le Jiang

This author has not been identified. Look up 'Le Jiang' in Google

Ang Wang

This author has not been identified. Look up 'Ang Wang' in Google

Wencong Xiao

This author has not been identified. Look up 'Wencong Xiao' in Google

Ziji Shi

This author has not been identified. Look up 'Ziji Shi' in Google

Jie Zhang 0078

This author has not been identified. Look up 'Jie Zhang 0078' in Google

Xinyuan Li

This author has not been identified. Look up 'Xinyuan Li' in Google

Langshi Chen

This author has not been identified. Look up 'Langshi Chen' in Google

Yong Li

This author has not been identified. Look up 'Yong Li' in Google

Zhen Zheng

This author has not been identified. Look up 'Zhen Zheng' in Google

Xiaoyong Liu

This author has not been identified. Look up 'Xiaoyong Liu' in Google

Wei Lin 0016

This author has not been identified. Look up 'Wei Lin 0016' in Google