LightSeq2: Accelerated Training for Transformer-Based Models on GPUs

Xiaohui Wang, Yang Wei, Ying Xiong, Guyue Huang, Xian Qian, Yufei Ding, Mingxuan Wang, Lei Li 0005. LightSeq2: Accelerated Training for Transformer-Based Models on GPUs. In SC22: International Conference for High Performance Computing, Networking, Storage and Analysis, Dallas, TX, USA, November 13-18, 2022. pages 1-14, IEEE, 2022. [doi]

@inproceedings{WangWXHQDWL22,
  title = {LightSeq2: Accelerated Training for Transformer-Based Models on GPUs},
  author = {Xiaohui Wang and Yang Wei and Ying Xiong and Guyue Huang and Xian Qian and Yufei Ding and Mingxuan Wang and Lei Li 0005},
  year = {2022},
  doi = {10.1109/SC41404.2022.00043},
  url = {https://doi.org/10.1109/SC41404.2022.00043},
  researchr = {https://researchr.org/publication/WangWXHQDWL22},
  cites = {0},
  citedby = {0},
  pages = {1-14},
  booktitle = {SC22: International Conference for High Performance Computing, Networking, Storage and Analysis, Dallas, TX, USA, November 13-18, 2022},
  publisher = {IEEE},
  isbn = {978-1-6654-5444-5},
}