ResMoE: Space-efficient Compression of Mixture of Experts LLMs via Residual Restoration

Mengting Ai, Tianxin Wei, Yifan Chen 0004, Zhichen Zeng 0001, Ritchie Zhao, Girish Varatkar, Bita Darvish Rouhani, Xianfeng Tang, Hanghang Tong, Jingrui He. ResMoE: Space-efficient Compression of Mixture of Experts LLMs via Residual Restoration. In Yizhou Sun, Flavio Chierichetti, Hady W. Lauw, Claudia Perlich, Wee Hyong Tok, Andrew Tomkins, editors, Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining, V.1, KDD 2025, Toronto, ON, Canada, August 3-7, 2025. pages 1-12, ACM, 2025. [doi]

Abstract

Abstract is missing.