A Provably Effective Method for Pruning Experts in Fine-tuned Sparse Mixture-of-Experts

Mohammed Nowaz Rabbani Chowdhury, Meng Wang 0003, Kaoutar El Maghraoui, Naigang Wang, Pin-Yu Chen, Christopher D. Carothers. A Provably Effective Method for Pruning Experts in Fine-tuned Sparse Mixture-of-Experts. In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024. OpenReview.net, 2024. [doi]

Abstract

Abstract is missing.