Length Extrapolation of Transformers: A Survey from the Perspective of Positional Encoding

Liang Zhao, Xiachong Feng, Xiaocheng Feng, Weihong Zhong, Dongliang Xu, Qing Yang 0033, Hongtao Liu, Bing Qin 0001, Ting Liu 0001. Length Extrapolation of Transformers: A Survey from the Perspective of Positional Encoding. In Yaser Al-Onaizan, Mohit Bansal, Yun-Nung Chen, editors, Findings of the Association for Computational Linguistics: EMNLP 2024, Miami, Florida, USA, November 12-16, 2024. pages 9959-9977, Association for Computational Linguistics, 2024. [doi]

@inproceedings{ZhaoFFZXYL0024,
  title = {Length Extrapolation of Transformers: A Survey from the Perspective of Positional Encoding},
  author = {Liang Zhao and Xiachong Feng and Xiaocheng Feng and Weihong Zhong and Dongliang Xu and Qing Yang 0033 and Hongtao Liu and Bing Qin 0001 and Ting Liu 0001},
  year = {2024},
  url = {https://aclanthology.org/2024.findings-emnlp.582},
  researchr = {https://researchr.org/publication/ZhaoFFZXYL0024},
  cites = {0},
  citedby = {0},
  pages = {9959-9977},
  booktitle = {Findings of the Association for Computational Linguistics: EMNLP 2024, Miami, Florida, USA, November 12-16, 2024},
  editor = {Yaser Al-Onaizan and Mohit Bansal and Yun-Nung Chen},
  publisher = {Association for Computational Linguistics},
  isbn = {979-8-89176-168-1},
}