Characterizing and addressing the issue of oversmoothing in neural autoregressive sequence modeling

Ilia Kulikov, Maksim Eremeev, KyungHyun Cho. Characterizing and addressing the issue of oversmoothing in neural autoregressive sequence modeling. In Yulan He 0001, Heng Ji, Yang Liu 0005, Sujian Li, Chia-Hui Chang, Soujanya Poria, Chenghua Lin, Wray L. Buntine, Maria Liakata, Hanqi Yan, Zonghan Yan, Sebastian Ruder, Xiaojun Wan, Miguel Arana-Catania, Zhongyu Wei, Hen-Hsen Huang, Jheng-Long Wu, Min-Yuh Day, Pengfei Liu, Ruifeng Xu, editors, Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing, AACL/IJCNLP 2022 - Volume 1: Long Papers, Online Only, November 20-23, 2022. pages 1115-1124, Association for Computational Linguistics, 2022. [doi]

Abstract

Abstract is missing.