A Provably Effective Method for Pruning Experts in Fine-tuned Sparse Mixture-of-Experts

Mohammed Nowaz Rabbani Chowdhury, Meng Wang 0003, Kaoutar El Maghraoui, Naigang Wang, Pin-Yu Chen, Christopher D. Carothers. A Provably Effective Method for Pruning Experts in Fine-tuned Sparse Mixture-of-Experts. In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024. OpenReview.net, 2024. [doi]

@inproceedings{Chowdhury0MWCC24,
  title = {A Provably Effective Method for Pruning Experts in Fine-tuned Sparse Mixture-of-Experts},
  author = {Mohammed Nowaz Rabbani Chowdhury and Meng Wang 0003 and Kaoutar El Maghraoui and Naigang Wang and Pin-Yu Chen and Christopher D. Carothers},
  year = {2024},
  url = {https://openreview.net/forum?id=1oU4FKpVx5},
  researchr = {https://researchr.org/publication/Chowdhury0MWCC24},
  cites = {0},
  citedby = {0},
  booktitle = {Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024},
  publisher = {OpenReview.net},
}