InstAttention: In-Storage Attention Offloading for Cost-Effective Long-Context LLM Inference

Xiurui Pan, Endian Li, Qiao Li, Shengwen Liang, Yizhou Shan, Ke Zhou, Yingwei Luo, Xiaolin Wang, Jie Zhang. InstAttention: In-Storage Attention Offloading for Cost-Effective Long-Context LLM Inference. In IEEE International Symposium on High Performance Computer Architecture, HPCA 2025, Las Vegas, NV, USA, March 1-5, 2025. pages 1510-1525, IEEE, 2025. [doi]

Abstract

Abstract is missing.