A length adaptive algorithm-hardware co-design of transformer on FPGA through sparse attention and dynamic pipelining

Hongwu Peng, Shaoyi Huang, Shiyang Chen, Bingbing Li, Tong Geng, Ang Li, Weiwen Jiang, Wujie Wen, Jinbo Bi, Hang Liu, Caiwen Ding. A length adaptive algorithm-hardware co-design of transformer on FPGA through sparse attention and dynamic pipelining. In Rob Oshana, editor, DAC '22: 59th ACM/IEEE Design Automation Conference, San Francisco, California, USA, July 10 - 14, 2022. pages 1135-1140, ACM, 2022. [doi]

Abstract

Abstract is missing.