SpecInfer: Accelerating Large Language Model Serving with Tree-based Speculative Inference and Verification - researchr publication authors

researchr

You are not signed in
Sign in
Sign up

Xupeng Miao, Gabriele Oliaro, Zhihao Zhang, Xinhao Cheng, Zeyu Wang, Zhengxin Zhang, Rae Ying Yee Wong, Alan Zhu, Lijie Yang, Xiaoxiang Shi, Chunan Shi, Zhuoming Chen, Daiyaan Arfeen, Reyna Abhyankar, Zhihao Jia. SpecInfer: Accelerating Large Language Model Serving with Tree-based Speculative Inference and Verification. In Rajiv Gupta 0001, Nael B. Abu-Ghazaleh, Madan Musuvathi, Dan Tsafrir, editors, Proceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 3, ASPLOS 2024, La Jolla, CA, USA, 27 April 2024- 1 May 2024. pages 932-949, ACM, 2024. [doi]

This author has not been identified. Look up 'Xupeng Miao' in GoogleThis author has not been identified. Look up 'Gabriele Oliaro' in GoogleThis author has not been identified. Look up 'Zhihao Zhang' in GoogleThis author has not been identified. Look up 'Xinhao Cheng' in GoogleThis author has not been identified. Look up 'Zeyu Wang' in GoogleThis author has not been identified. Look up 'Zhengxin Zhang' in GoogleThis author has not been identified. Look up 'Rae Ying Yee Wong' in GoogleThis author has not been identified. Look up 'Alan Zhu' in GoogleThis author has not been identified. Look up 'Lijie Yang' in GoogleThis author has not been identified. Look up 'Xiaoxiang Shi' in GoogleThis author has not been identified. Look up 'Chunan Shi' in GoogleThis author has not been identified. Look up 'Zhuoming Chen' in GoogleThis author has not been identified. Look up 'Daiyaan Arfeen' in GoogleThis author has not been identified. Look up 'Reyna Abhyankar' in GoogleThis author has not been identified. Look up 'Zhihao Jia' in Google

runs on WebDSL