Hanayo: Harnessing Wave-like Pipeline Parallelism for Enhanced Large Model Training Efficiency

Ziming Liu, Shenggan Cheng, Haotian Zhou, Yang You 0001. Hanayo: Harnessing Wave-like Pipeline Parallelism for Enhanced Large Model Training Efficiency. In Dorian Arnold, Rosa M. Badia, Kathryn M. Mohror, editors, Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2023, Denver, CO, USA, November 12-17, 2023. ACM, 2023. [doi]

Abstract

Abstract is missing.