Low-Rank Bottleneck in Multi-head Attention Models

Srinadh Bhojanapalli, Chulhee Yun, Ankit Singh Rawat, Sashank J. Reddi, Sanjiv Kumar. Low-Rank Bottleneck in Multi-head Attention Models. In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event. Volume 119 of Proceedings of Machine Learning Research, pages 864-873, PMLR, 2020. [doi]

Authors

Srinadh Bhojanapalli

This author has not been identified. Look up 'Srinadh Bhojanapalli' in Google

Chulhee Yun

This author has not been identified. Look up 'Chulhee Yun' in Google

Ankit Singh Rawat

This author has not been identified. Look up 'Ankit Singh Rawat' in Google

Sashank J. Reddi

This author has not been identified. Look up 'Sashank J. Reddi' in Google

Sanjiv Kumar

This author has not been identified. Look up 'Sanjiv Kumar' in Google