Predicting Attention Sparsity in Transformers

Marcos V. Treviso, António Góis, Patrick Fernandes, Erick Rocha Fonseca, André F. T. Martins. Predicting Attention Sparsity in Transformers. In Andreas Vlachos 0001, Priyanka Agrawal, André F. T. Martins, Gerasimos Lampouras, Chunchuan Lyu, editors, Proceedings of the Sixth Workshop on Structured Prediction for NLP, SPNLP@ACL 2022, Dublin, Ireland, May 27, 2022. pages 67-81, Association for Computational Linguistics, 2022. [doi]

Abstract

Abstract is missing.