Overlap Communication with Dependent Computation via Decomposition in Large Deep Learning Models

Shibo Wang, Jinliang Wei, Amit Sabne, Andy Davis, Berkin Ilbeyi, Blake Hechtman, Dehao Chen, Karthik Srinivasa Murthy, Marcello Maggioni, Qiao Zhang, Sameer Kumar, Tongfei Guo, Yuanzhong Xu, Zongwei Zhou. Overlap Communication with Dependent Computation via Decomposition in Large Deep Learning Models. In Tor M. Aamodt, Natalie D. Enright Jerger, Michael M. Swift, editors, Proceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 1, ASPLOS 2023, Vancouver, BC, Canada, March 25-29, 2023. pages 93-106, ACM, 2023. [doi]

@inproceedings{WangWSDIHCMMZKG23,
  title = {Overlap Communication with Dependent Computation via Decomposition in Large Deep Learning Models},
  author = {Shibo Wang and Jinliang Wei and Amit Sabne and Andy Davis and Berkin Ilbeyi and Blake Hechtman and Dehao Chen and Karthik Srinivasa Murthy and Marcello Maggioni and Qiao Zhang and Sameer Kumar and Tongfei Guo and Yuanzhong Xu and Zongwei Zhou},
  year = {2023},
  doi = {10.1145/3567955.3567959},
  url = {https://doi.org/10.1145/3567955.3567959},
  researchr = {https://researchr.org/publication/WangWSDIHCMMZKG23},
  cites = {0},
  citedby = {0},
  pages = {93-106},
  booktitle = {Proceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 1, ASPLOS 2023, Vancouver, BC, Canada, March 25-29, 2023},
  editor = {Tor M. Aamodt and Natalie D. Enright Jerger and Michael M. Swift},
  publisher = {ACM},
  isbn = {978-1-4503-9915-9},
}