Outlier Dimensions that Disrupt Transformers are Driven by Frequency

Giovanni Puccetti, Anna Rogers, Aleksandr Drozd, Felice dell'Orletta. Outlier Dimensions that Disrupt Transformers are Driven by Frequency. In Yoav Goldberg, Zornitsa Kozareva, Yue Zhang, editors, Findings of the Association for Computational Linguistics: EMNLP 2022, Abu Dhabi, United Arab Emirates, December 7-11, 2022. pages 1286-1304, Association for Computational Linguistics, 2022. [doi]

Abstract

Abstract is missing.