MS-BERT: A Multi-layer Self-distillation Approach for BERT Compression Based on Earth Mover's Distance

Jiahui Huang, Bin Cao 0004, Jiaxing Wang, Jing Fan. MS-BERT: A Multi-layer Self-distillation Approach for BERT Compression Based on Earth Mover's Distance. In Honghao Gao, Xinheng Wang, editors, Collaborative Computing: Networking, Applications and Worksharing - 17th EAI International Conference, CollaborateCom 2021, Virtual Event, October 16-18, 2021, Proceedings, Part II. Volume 407 of Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, pages 316-334, Springer, 2021. [doi]