Why Is Public Pretraining Necessary for Private Model Training?

Arun Ganesh, Mahdi Haghifam, Milad Nasr, Sewoong Oh, Thomas Steinke 0002, Om Thakkar 0001, Abhradeep Guha Thakurta, Lun Wang. Why Is Public Pretraining Necessary for Private Model Training?. In Andreas Krause 0001, Emma Brunskill, KyungHyun Cho, Barbara Engelhardt, Sivan Sabato, Jonathan Scarlett, editors, International Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA. Volume 202 of Proceedings of Machine Learning Research, pages 10611-10627, PMLR, 2023. [doi]

@inproceedings{GaneshHNO00TW23,
  title = {Why Is Public Pretraining Necessary for Private Model Training?},
  author = {Arun Ganesh and Mahdi Haghifam and Milad Nasr and Sewoong Oh and Thomas Steinke 0002 and Om Thakkar 0001 and Abhradeep Guha Thakurta and Lun Wang},
  year = {2023},
  url = {https://proceedings.mlr.press/v202/ganesh23a.html},
  researchr = {https://researchr.org/publication/GaneshHNO00TW23},
  cites = {0},
  citedby = {0},
  pages = {10611-10627},
  booktitle = {International Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA},
  editor = {Andreas Krause 0001 and Emma Brunskill and KyungHyun Cho and Barbara Engelhardt and Sivan Sabato and Jonathan Scarlett},
  volume = {202},
  series = {Proceedings of Machine Learning Research},
  publisher = {PMLR},
}