PLaD: Preference-based Large Language Model Distillation with Pseudo-Preference Pairs

Rongzhi Zhang, Jiaming Shen, Tianqi Liu 0002, Haorui Wang, Zhen Qin 0001, Feng Han, Jialu Liu, Simon Baumgartner, Michael Bendersky, Chao Zhang. PLaD: Preference-based Large Language Model Distillation with Pseudo-Preference Pairs. In Lun-Wei Ku, Andre Martins, Vivek Srikumar, editors, Findings of the Association for Computational Linguistics, ACL 2024, Bangkok, Thailand and virtual meeting, August 11-16, 2024. pages 15623-15636, Association for Computational Linguistics, 2024. [doi]

Abstract

Abstract is missing.