Length Extrapolation of Transformers: A Survey from the Perspective of Positional Encoding

Liang Zhao, Xiachong Feng, Xiaocheng Feng, Weihong Zhong, Dongliang Xu, Qing Yang 0033, Hongtao Liu, Bing Qin 0001, Ting Liu 0001. Length Extrapolation of Transformers: A Survey from the Perspective of Positional Encoding. In Yaser Al-Onaizan, Mohit Bansal, Yun-Nung Chen, editors, Findings of the Association for Computational Linguistics: EMNLP 2024, Miami, Florida, USA, November 12-16, 2024. pages 9959-9977, Association for Computational Linguistics, 2024. [doi]

Abstract

Abstract is missing.