Repulsive Attention: Rethinking Multi-head Attention as Bayesian Inference

Bang An, Jie Lyu 0004, Zhenyi Wang, Chunyuan Li, Changwei Hu, Fei Tan, Ruiyi Zhang, Yifan Hu, Changyou Chen. Repulsive Attention: Rethinking Multi-head Attention as Bayesian Inference. In Bonnie Webber, Trevor Cohn, Yulan He, Yang Liu, editors, Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 16-20, 2020. pages 236-255, Association for Computational Linguistics, 2020. [doi]

Abstract

Abstract is missing.