Benjamin L. Edelman, Surbhi Goel, Sham M. Kakade, Cyril Zhang. Inductive Biases and Variable Creation in Self-Attention Mechanisms. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvári, Gang Niu 0001, Sivan Sabato, editors, International Conference on Machine Learning, ICML 2022, 17-23 July 2022, Baltimore, Maryland, USA. Volume 162 of Proceedings of Machine Learning Research, pages 5793-5831, PMLR, 2022. [doi]
Abstract is missing.