Parm: Efficient Training of Large Sparsely-Activated Models with Dedicated Schedules

Xinglin Pan, Wenxiang Lin, Shaohuai Shi, Xiaowen Chu 0001, Weinong Sun, Bo Li 0001. Parm: Efficient Training of Large Sparsely-Activated Models with Dedicated Schedules. In IEEE INFOCOM 2024 - IEEE Conference on Computer Communications, Vancouver, BC, Canada, May 20-23, 2024. pages 1880-1889, IEEE, 2024. [doi]

@inproceedings{PanLS0S024,
  title = {Parm: Efficient Training of Large Sparsely-Activated Models with Dedicated Schedules},
  author = {Xinglin Pan and Wenxiang Lin and Shaohuai Shi and Xiaowen Chu 0001 and Weinong Sun and Bo Li 0001},
  year = {2024},
  doi = {10.1109/INFOCOM52122.2024.10621327},
  url = {https://doi.org/10.1109/INFOCOM52122.2024.10621327},
  researchr = {https://researchr.org/publication/PanLS0S024},
  cites = {0},
  citedby = {0},
  pages = {1880-1889},
  booktitle = {IEEE INFOCOM 2024 - IEEE Conference on Computer Communications, Vancouver, BC, Canada, May 20-23, 2024},
  publisher = {IEEE},
  isbn = {979-8-3503-8350-8},
}