Emergent Modularity in Pre-trained Transformers

Zhengyan Zhang, Zhiyuan Zeng, Yankai Lin, Chaojun Xiao, Xiaozhi Wang, Xu Han 0007, Zhiyuan Liu 0001, Ruobing Xie, Maosong Sun, Jie Zhou 0016. Emergent Modularity in Pre-trained Transformers. In Anna Rogers, Jordan L. Boyd-Graber, Naoaki Okazaki, editors, Findings of the Association for Computational Linguistics: ACL 2023, Toronto, Canada, July 9-14, 2023. pages 4066-4083, Association for Computational Linguistics, 2023. [doi]

Abstract

Abstract is missing.