Systematicity Emerges in Transformers when Abstract Grammatical Roles Guide Attention

Ayush K. Chakravarthy, Jacob Labe Russin, Randall C. O'Reilly. Systematicity Emerges in Transformers when Abstract Grammatical Roles Guide Attention. In Daphne Ippolito, Liunian Harold Li, Maria Leonor Pacheco, Danqi Chen, Nianwen Xue, editors, Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Student Research Workshop, NAACL-HLT 2022, Hybrid Event / Seattle, WA, USA, July 10-15, 2022. pages 1-8, Association for Computational Linguistics, 2022. [doi]

Authors

Ayush K. Chakravarthy

This author has not been identified. Look up 'Ayush K. Chakravarthy' in Google

Jacob Labe Russin

This author has not been identified. Look up 'Jacob Labe Russin' in Google

Randall C. O'Reilly

This author has not been identified. Look up 'Randall C. O'Reilly' in Google