Whale: Efficient Giant Model Training over Heterogeneous GPUs

Xianyan Jia, Le Jiang, Ang Wang, Wencong Xiao, Ziji Shi, Jie Zhang 0078, Xinyuan Li, Langshi Chen, Yong Li, Zhen Zheng, Xiaoyong Liu, Wei Lin 0016. Whale: Efficient Giant Model Training over Heterogeneous GPUs. In Jiri Schindler, Noa Zilberman, editors, 2022 USENIX Annual Technical Conference, USENIX ATC 2022, Carlsbad, CA, USA, July 11-13, 2022. pages 673-688, USENIX Association, 2022. [doi]

@inproceedings{JiaJWXS0LCLZL022,
  title = {Whale: Efficient Giant Model Training over Heterogeneous GPUs},
  author = {Xianyan Jia and Le Jiang and Ang Wang and Wencong Xiao and Ziji Shi and Jie Zhang 0078 and Xinyuan Li and Langshi Chen and Yong Li and Zhen Zheng and Xiaoyong Liu and Wei Lin 0016},
  year = {2022},
  url = {https://www.usenix.org/conference/atc22/presentation/jia-xianyan},
  researchr = {https://researchr.org/publication/JiaJWXS0LCLZL022},
  cites = {0},
  citedby = {0},
  pages = {673-688},
  booktitle = {2022 USENIX Annual Technical Conference, USENIX ATC 2022, Carlsbad, CA, USA, July 11-13, 2022},
  editor = {Jiri Schindler and Noa Zilberman},
  publisher = {USENIX Association},
  isbn = {978-1-939133-29-8},
}