Parm: Efficient Training of Large Sparsely-Activated Models with Dedicated Schedules

Xinglin Pan, Wenxiang Lin, Shaohuai Shi, Xiaowen Chu 0001, Weinong Sun, Bo Li 0001. Parm: Efficient Training of Large Sparsely-Activated Models with Dedicated Schedules. In IEEE INFOCOM 2024 - IEEE Conference on Computer Communications, Vancouver, BC, Canada, May 20-23, 2024. pages 1880-1889, IEEE, 2024. [doi]

Authors

Xinglin Pan

This author has not been identified. Look up 'Xinglin Pan' in Google

Wenxiang Lin

This author has not been identified. Look up 'Wenxiang Lin' in Google

Shaohuai Shi

This author has not been identified. Look up 'Shaohuai Shi' in Google

Xiaowen Chu 0001

This author has not been identified. Look up 'Xiaowen Chu 0001' in Google

Weinong Sun

This author has not been identified. Look up 'Weinong Sun' in Google

Bo Li 0001

This author has not been identified. Look up 'Bo Li 0001' in Google