How Much Do Modifications to Transformer Language Models Affect Their Ability to Learn Linguistic Knowledge?

Simeng Sun, Brian Dillon, Mohit Iyyer. How Much Do Modifications to Transformer Language Models Affect Their Ability to Learn Linguistic Knowledge?. In Shabnam Tafreshi, João Sedoc, Anna Rogers, Aleksandr Drozd, Anna Rumshisky, Arjun R. Akula, editors, Proceedings of the Third Workshop on Insights from Negative Results in NLP, Insights@ACL 2022, Dublin, Ireland, May 26, 2022. pages 46-53, Association for Computational Linguistics, 2022. [doi]

Authors

Simeng Sun

This author has not been identified. Look up 'Simeng Sun' in Google

Brian Dillon

This author has not been identified. Look up 'Brian Dillon' in Google

Mohit Iyyer

This author has not been identified. Look up 'Mohit Iyyer' in Google