Absolute Position Embedding Learns Sinusoid-like Waves for Attention Based on Relative Position

Yuji Yamamoto, Takuya Matsuzaki. Absolute Position Embedding Learns Sinusoid-like Waves for Attention Based on Relative Position. In Houda Bouamor, Juan Pino 0001, Kalika Bali, editors, Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023, Singapore, December 6-10, 2023. pages 15-28, Association for Computational Linguistics, 2023. [doi]

Abstract

Abstract is missing.