Rethinking Soft Labels for Knowledge Distillation: A Bias-Variance Tradeoff Perspective

Helong Zhou, Liangchen Song, Jiajie Chen, Ye Zhou, Guoli Wang, Junsong Yuan, Qian Zhang. Rethinking Soft Labels for Knowledge Distillation: A Bias-Variance Tradeoff Perspective. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021. [doi]

@inproceedings{ZhouSCZWYZ21,
  title = {Rethinking Soft Labels for Knowledge Distillation: A Bias-Variance Tradeoff Perspective},
  author = {Helong Zhou and Liangchen Song and Jiajie Chen and Ye Zhou and Guoli Wang and Junsong Yuan and Qian Zhang},
  year = {2021},
  url = {https://openreview.net/forum?id=gIHd-5X324},
  researchr = {https://researchr.org/publication/ZhouSCZWYZ21},
  cites = {0},
  citedby = {0},
  booktitle = {9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021},
  publisher = {OpenReview.net},
}