Overlap Communication with Dependent Computation via Decomposition in Large Deep Learning Models

Shibo Wang, Jinliang Wei, Amit Sabne, Andy Davis, Berkin Ilbeyi, Blake Hechtman, Dehao Chen, Karthik Srinivasa Murthy, Marcello Maggioni, Qiao Zhang, Sameer Kumar, Tongfei Guo, Yuanzhong Xu, Zongwei Zhou. Overlap Communication with Dependent Computation via Decomposition in Large Deep Learning Models. In Tor M. Aamodt, Natalie D. Enright Jerger, Michael M. Swift, editors, Proceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 1, ASPLOS 2023, Vancouver, BC, Canada, March 25-29, 2023. pages 93-106, ACM, 2023. [doi]

Authors

Shibo Wang

This author has not been identified. Look up 'Shibo Wang' in Google

Jinliang Wei

This author has not been identified. Look up 'Jinliang Wei' in Google

Amit Sabne

This author has not been identified. Look up 'Amit Sabne' in Google

Andy Davis

This author has not been identified. Look up 'Andy Davis' in Google

Berkin Ilbeyi

This author has not been identified. Look up 'Berkin Ilbeyi' in Google

Blake Hechtman

This author has not been identified. Look up 'Blake Hechtman' in Google

Dehao Chen

This author has not been identified. Look up 'Dehao Chen' in Google

Karthik Srinivasa Murthy

This author has not been identified. Look up 'Karthik Srinivasa Murthy' in Google

Marcello Maggioni

This author has not been identified. Look up 'Marcello Maggioni' in Google

Qiao Zhang

This author has not been identified. Look up 'Qiao Zhang' in Google

Sameer Kumar

This author has not been identified. Look up 'Sameer Kumar' in Google

Tongfei Guo

This author has not been identified. Look up 'Tongfei Guo' in Google

Yuanzhong Xu

This author has not been identified. Look up 'Yuanzhong Xu' in Google

Zongwei Zhou

This author has not been identified. Look up 'Zongwei Zhou' in Google