Hanayo: Harnessing Wave-like Pipeline Parallelism for Enhanced Large Model Training Efficiency

Ziming Liu, Shenggan Cheng, Haotian Zhou, Yang You 0001. Hanayo: Harnessing Wave-like Pipeline Parallelism for Enhanced Large Model Training Efficiency. In Dorian Arnold, Rosa M. Badia, Kathryn M. Mohror, editors, Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2023, Denver, CO, USA, November 12-17, 2023. ACM, 2023. [doi]

@inproceedings{LiuCZ023,
  title = {Hanayo: Harnessing Wave-like Pipeline Parallelism for Enhanced Large Model Training Efficiency},
  author = {Ziming Liu and Shenggan Cheng and Haotian Zhou and Yang You 0001},
  year = {2023},
  doi = {10.1145/3581784.3607073},
  url = {https://doi.org/10.1145/3581784.3607073},
  researchr = {https://researchr.org/publication/LiuCZ023},
  cites = {0},
  citedby = {0},
  booktitle = {Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2023, Denver, CO, USA, November 12-17, 2023},
  editor = {Dorian Arnold and Rosa M. Badia and Kathryn M. Mohror},
  publisher = {ACM},
}