Measuring the Impact of Gradient Accumulation on Cloud-based Distributed Training

Zimeng Huang, Bo Jiang 0003, Tian Guo 0001, Yunzhuo Liu. Measuring the Impact of Gradient Accumulation on Cloud-based Distributed Training. In Yogesh Simmhan, Ilkay Altintas, Ana Lucia Varbanescu, Pavan Balaji, Abhinandan S. Prasad, Lorenzo Carnevale, editors, 23rd IEEE/ACM International Symposium on Cluster, Cloud and Internet Computing, CCGrid 2023, Bangalore, India, May 1-4, 2023. pages 344-354, IEEE, 2023. [doi]

Abstract

Abstract is missing.