External Links

Cite Key

Statistics

  • References: 0
  • Cited by: 0
  • Reviews: 0
  • Bibliographies: 0

PDF

Researchr

Researchr is a web site for finding, collecting, sharing, and reviewing scientific publications, for researchers by researchers.

Sign up for an account to create a profile with publication list, tag and review your related work, and share bibliographies with your co-authors.

PyramidInfer: Pyramid KV Cache Compression for High-throughput LLM Inference

Dongjie Yang, Xiaodong Han, Yan Gao, Yao Hu, Shilin Zhang, Hai Zhao 0001. PyramidInfer: Pyramid KV Cache Compression for High-throughput LLM Inference. In Lun-Wei Ku, Andre Martins, Vivek Srikumar, editors, Findings of the Association for Computational Linguistics, ACL 2024, Bangkok, Thailand and virtual meeting, August 11-16, 2024. pages 3258-3270, Association for Computational Linguistics, 2024. [doi]

Abstract

Abstract is missing.

runs on