GradInit: Learning to Initialize Neural Networks for Stable and Efficient Training

Chen Zhu, Renkun Ni, Zheng Xu 0002, Kezhi Kong, W. Ronny Huang, Tom Goldstein. GradInit: Learning to Initialize Neural Networks for Stable and Efficient Training. In Marc'Aurelio Ranzato, Alina Beygelzimer, Yann N. Dauphin, Percy Liang, Jennifer Wortman Vaughan, editors, Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, December 6-14, 2021, virtual. pages 16410-16422, 2021. [doi]

Authors

Chen Zhu

This author has not been identified. Look up 'Chen Zhu' in Google

Renkun Ni

This author has not been identified. Look up 'Renkun Ni' in Google

Zheng Xu 0002

This author has not been identified. Look up 'Zheng Xu 0002' in Google

Kezhi Kong

This author has not been identified. Look up 'Kezhi Kong' in Google

W. Ronny Huang

This author has not been identified. Look up 'W. Ronny Huang' in Google

Tom Goldstein

This author has not been identified. Look up 'Tom Goldstein' in Google