Efficient Large Scale Language Modeling with Mixtures of Experts

Mikel Artetxe, Shruti Bhosale, Naman Goyal, Todor Mihaylov, Myle Ott, Sam Shleifer, Xi Victoria Lin, Jingfei Du, Srinivasan Iyer, Ramakanth Pasunuru, Giridharan Anantharaman, Xian Li, Shuohui Chen, Halil Akin, Mandeep Baines, Louis Martin, Xing Zhou, Punit Singh Koura, Brian O'Horo, Jeffrey Wang, Luke Zettlemoyer, Mona T. Diab, Zornitsa Kozareva, Veselin Stoyanov. Efficient Large Scale Language Modeling with Mixtures of Experts. In Yoav Goldberg, Zornitsa Kozareva, Yue Zhang, editors, Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, EMNLP 2022, Abu Dhabi, United Arab Emirates, December 7-11. pages 11699-11732, Association for Computational Linguistics, 2022. [doi]

Authors

Mikel Artetxe

This author has not been identified. Look up 'Mikel Artetxe' in Google

Shruti Bhosale

This author has not been identified. Look up 'Shruti Bhosale' in Google

Naman Goyal

This author has not been identified. Look up 'Naman Goyal' in Google

Todor Mihaylov

This author has not been identified. Look up 'Todor Mihaylov' in Google

Myle Ott

This author has not been identified. Look up 'Myle Ott' in Google

Sam Shleifer

This author has not been identified. Look up 'Sam Shleifer' in Google

Xi Victoria Lin

This author has not been identified. Look up 'Xi Victoria Lin' in Google

Jingfei Du

This author has not been identified. Look up 'Jingfei Du' in Google

Srinivasan Iyer

This author has not been identified. Look up 'Srinivasan Iyer' in Google

Ramakanth Pasunuru

This author has not been identified. Look up 'Ramakanth Pasunuru' in Google

Giridharan Anantharaman

This author has not been identified. Look up 'Giridharan Anantharaman' in Google

Xian Li

This author has not been identified. Look up 'Xian Li' in Google

Shuohui Chen

This author has not been identified. Look up 'Shuohui Chen' in Google

Halil Akin

This author has not been identified. Look up 'Halil Akin' in Google

Mandeep Baines

This author has not been identified. Look up 'Mandeep Baines' in Google

Louis Martin

This author has not been identified. Look up 'Louis Martin' in Google

Xing Zhou

This author has not been identified. Look up 'Xing Zhou' in Google

Punit Singh Koura

This author has not been identified. Look up 'Punit Singh Koura' in Google

Brian O'Horo

This author has not been identified. Look up 'Brian O'Horo' in Google

Jeffrey Wang

This author has not been identified. Look up 'Jeffrey Wang' in Google

Luke Zettlemoyer

This author has not been identified. Look up 'Luke Zettlemoyer' in Google

Mona T. Diab

This author has not been identified. Look up 'Mona T. Diab' in Google

Zornitsa Kozareva

This author has not been identified. Look up 'Zornitsa Kozareva' in Google

Veselin Stoyanov

This author has not been identified. Look up 'Veselin Stoyanov' in Google