Krzysztof Choromanski, Shanda Li, Valerii Likhosherstov, Kumar Avinava Dubey, Shengjie Luo, Di He, Yiming Yang, Tamás Sarlós, Thomas Weingarten, Adrian Weller. Learning a Fourier Transform for Linear Relative Positional Encodings in Transformers. In Sanjoy Dasgupta, Stephan Mandt, Yingzhen Li, editors, International Conference on Artificial Intelligence and Statistics, 2-4 May 2024, Palau de Congressos, Valencia, Spain. Volume 238 of Proceedings of Machine Learning Research, pages 2278-2286, PMLR, 2024. [doi]
Abstract is missing.