Transformers learn through gradual rank increase

Emmanuel Abbe, Samy Bengio, Enric Boix-Adserà, Etai Littwin, Joshua M. Susskind. Transformers learn through gradual rank increase. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, Sergey Levine, editors, Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023. 2023. [doi]

Authors

Emmanuel Abbe

This author has not been identified. Look up 'Emmanuel Abbe' in Google

Samy Bengio

This author has not been identified. Look up 'Samy Bengio' in Google

Enric Boix-Adserà

This author has not been identified. Look up 'Enric Boix-Adserà' in Google

Etai Littwin

This author has not been identified. Look up 'Etai Littwin' in Google

Joshua M. Susskind

This author has not been identified. Look up 'Joshua M. Susskind' in Google