Do Transformer Modifications Transfer Across Implementations and Applications?

Sharan Narang, Hyung Won Chung, Yi Tay, Liam Fedus, Thibault Févry, Michael Matena, Karishma Malkan, Noah Fiedel, Noam Shazeer, Zhenzhong Lan, Yanqi Zhou, Wei Li 0133, Nan Ding, Jake Marcus, Adam Roberts, Colin Raffel. Do Transformer Modifications Transfer Across Implementations and Applications?. In Marie-Francine Moens, Xuanjing Huang, Lucia Specia, Scott Wen-tau Yih, editors, Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, EMNLP 2021, Virtual Event / Punta Cana, Dominican Republic, 7-11 November, 2021. pages 5758-5773, Association for Computational Linguistics, 2021. [doi]

Authors

Sharan Narang

This author has not been identified. Look up 'Sharan Narang' in Google

Hyung Won Chung

This author has not been identified. Look up 'Hyung Won Chung' in Google

Yi Tay

This author has not been identified. Look up 'Yi Tay' in Google

Liam Fedus

This author has not been identified. Look up 'Liam Fedus' in Google

Thibault Févry

This author has not been identified. Look up 'Thibault Févry' in Google

Michael Matena

This author has not been identified. Look up 'Michael Matena' in Google

Karishma Malkan

This author has not been identified. Look up 'Karishma Malkan' in Google

Noah Fiedel

This author has not been identified. Look up 'Noah Fiedel' in Google

Noam Shazeer

This author has not been identified. Look up 'Noam Shazeer' in Google

Zhenzhong Lan

This author has not been identified. Look up 'Zhenzhong Lan' in Google

Yanqi Zhou

This author has not been identified. Look up 'Yanqi Zhou' in Google

Wei Li 0133

This author has not been identified. Look up 'Wei Li 0133' in Google

Nan Ding

This author has not been identified. Look up 'Nan Ding' in Google

Jake Marcus

This author has not been identified. Look up 'Jake Marcus' in Google

Adam Roberts

This author has not been identified. Look up 'Adam Roberts' in Google

Colin Raffel

This author has not been identified. Look up 'Colin Raffel' in Google