Hanayo: Harnessing Wave-like Pipeline Parallelism for Enhanced Large Model Training Efficiency

Ziming Liu, Shenggan Cheng, Haotian Zhou, Yang You 0001. Hanayo: Harnessing Wave-like Pipeline Parallelism for Enhanced Large Model Training Efficiency. In Dorian Arnold, Rosa M. Badia, Kathryn M. Mohror, editors, Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2023, Denver, CO, USA, November 12-17, 2023. ACM, 2023. [doi]

Authors

Ziming Liu

This author has not been identified. Look up 'Ziming Liu' in Google

Shenggan Cheng

This author has not been identified. Look up 'Shenggan Cheng' in Google

Haotian Zhou

This author has not been identified. Look up 'Haotian Zhou' in Google

Yang You 0001

This author has not been identified. Look up 'Yang You 0001' in Google