MetaLA: Unified Optimal Linear Approximation to Softmax Attention Map

Yuhong Chou, Man Yao, Kexin Wang, Yuqi Pan, Rui-Jie Zhu 0003, Jibin Wu, Yiran Zhong, Yu Qiao, Bo Xu 0002, Guoqi Li. MetaLA: Unified Optimal Linear Approximation to Softmax Attention Map. In Amir Globersons, Lester Mackey, Danielle Belgrave, Angela Fan, Ulrich Paquet, Jakub M. Tomczak, Cheng Zhang 0005, editors, Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, December 10 - 15, 2024. 2024. [doi]

Abstract

Abstract is missing.