The Shape of Learning: Anisotropy and Intrinsic Dimensions in Transformer-Based Models

Anton Razzhigaev, Matvey Mikhalchuk, Elizaveta Goncharova, Ivan V. Oseledets, Denis Dimitrov, Andrey Kuznetsov. The Shape of Learning: Anisotropy and Intrinsic Dimensions in Transformer-Based Models. In Yvette Graham, Matthew Purver, editors, Findings of the Association for Computational Linguistics: EACL 2024, St. Julian's, Malta, March 17-22, 2024. pages 868-874, Association for Computational Linguistics, 2024. [doi]

Abstract

Abstract is missing.