Fast Attention Over Long Sequences With Dynamic Sparse Flash Attention

Matteo Pagliardini, Daniele Paliotta, Martin Jaggi, François Fleuret. Fast Attention Over Long Sequences With Dynamic Sparse Flash Attention. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, Sergey Levine, editors, Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023. 2023. [doi]

@inproceedings{PagliardiniPJF23,
  title = {Fast Attention Over Long Sequences With Dynamic Sparse Flash Attention},
  author = {Matteo Pagliardini and Daniele Paliotta and Martin Jaggi and François Fleuret},
  year = {2023},
  url = {http://papers.nips.cc/paper_files/paper/2023/hash/bc222e8153a49c1b30a1b8ba96b35117-Abstract-Conference.html},
  researchr = {https://researchr.org/publication/PagliardiniPJF23},
  cites = {0},
  citedby = {0},
  booktitle = {Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023},
  editor = {Alice Oh and Tristan Naumann and Amir Globerson and Kate Saenko and Moritz Hardt and Sergey Levine},
}