SparseBERT: Rethinking the Importance Analysis in Self-attention

Han Shi, Jiahui Gao, Xiaozhe Ren, Hang Xu, Xiaodan Liang, Zhenguo Li, James Tin-Yau Kwok. SparseBERT: Rethinking the Importance Analysis in Self-attention. In Marina Meila, Tong Zhang 0001, editors, Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event. Volume 139 of Proceedings of Machine Learning Research, pages 9547-9557, PMLR, 2021. [doi]

Authors

Han Shi

This author has not been identified. Look up 'Han Shi' in Google

Jiahui Gao

This author has not been identified. Look up 'Jiahui Gao' in Google

Xiaozhe Ren

This author has not been identified. Look up 'Xiaozhe Ren' in Google

Hang Xu

This author has not been identified. Look up 'Hang Xu' in Google

Xiaodan Liang

This author has not been identified. Look up 'Xiaodan Liang' in Google

Zhenguo Li

This author has not been identified. Look up 'Zhenguo Li' in Google

James Tin-Yau Kwok

This author has not been identified. Look up 'James Tin-Yau Kwok' in Google