SAPipe: Staleness-Aware Pipeline for Data Parallel DNN Training

Yangrui Chen, Cong Xie, Meng Ma, Juncheng Gu, Yanghua Peng, Haibin Lin, Chuan Wu 0001, Yibo Zhu. SAPipe: Staleness-Aware Pipeline for Data Parallel DNN Training. In Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, A. Oh, editors, Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 - December 9, 2022. 2022. [doi]

Authors

Yangrui Chen

This author has not been identified. Look up 'Yangrui Chen' in Google

Cong Xie

This author has not been identified. Look up 'Cong Xie' in Google

Meng Ma

This author has not been identified. Look up 'Meng Ma' in Google

Juncheng Gu

This author has not been identified. Look up 'Juncheng Gu' in Google

Yanghua Peng

This author has not been identified. Look up 'Yanghua Peng' in Google

Haibin Lin

This author has not been identified. Look up 'Haibin Lin' in Google

Chuan Wu 0001

This author has not been identified. Look up 'Chuan Wu 0001' in Google

Yibo Zhu

This author has not been identified. Look up 'Yibo Zhu' in Google