Jump Self-attention: Capturing High-order Statistics in Transformers

Haoyi Zhou, Siyang Xiao, Shanghang Zhang, Jieqi Peng, Shuai Zhang 0026, Jianxin Li 0002. Jump Self-attention: Capturing High-order Statistics in Transformers. In Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, A. Oh, editors, Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 - December 9, 2022. 2022. [doi]

Authors

Haoyi Zhou

This author has not been identified. Look up 'Haoyi Zhou' in Google

Siyang Xiao

This author has not been identified. Look up 'Siyang Xiao' in Google

Shanghang Zhang

This author has not been identified. Look up 'Shanghang Zhang' in Google

Jieqi Peng

This author has not been identified. Look up 'Jieqi Peng' in Google

Shuai Zhang 0026

This author has not been identified. Look up 'Shuai Zhang 0026' in Google

Jianxin Li 0002

This author has not been identified. Look up 'Jianxin Li 0002' in Google