Parm: Efficient Training of Large Sparsely-Activated Models with Dedicated Schedules

Xinglin Pan, Wenxiang Lin, Shaohuai Shi, Xiaowen Chu 0001, Weinong Sun, Bo Li 0001. Parm: Efficient Training of Large Sparsely-Activated Models with Dedicated Schedules. In IEEE INFOCOM 2024 - IEEE Conference on Computer Communications, Vancouver, BC, Canada, May 20-23, 2024. pages 1880-1889, IEEE, 2024. [doi]

Abstract

Abstract is missing.