Mitigating Outlier Activations in Low-Precision Fine-Tuning of Language Models

Alireza Ghaffari, Justin Yu, Mahsa Ghazvini Nejad, Masoud Asgharian, Boxing Chen, Vahid Partovi Nia. Mitigating Outlier Activations in Low-Precision Fine-Tuning of Language Models. In Modesto Castrillón Santana, Maria De Marsico, Ana L. N. Fred, editors, Proceedings of the 13th International Conference on Pattern Recognition Applications and Methods, ICPRAM 2024, Rome, Italy, February 24-26, 2024. pages 478-484, SCITEPRESS, 2024. [doi]

Abstract

Abstract is missing.