Delving into Variance Transmission and Normalization: Shift of Average Gradient Makes the Network Collapse

Yuxiang Liu, JiDong Ge, Chuanyi Li, Jie Gui. Delving into Variance Transmission and Normalization: Shift of Average Gradient Makes the Network Collapse. In Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Third Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational Advances in Artificial Intelligence, EAAI 2021, Virtual Event, February 2-9, 2021. pages 2216-2224, AAAI Press, 2021. [doi]

Authors

Yuxiang Liu

This author has not been identified. Look up 'Yuxiang Liu' in Google

JiDong Ge

This author has not been identified. Look up 'JiDong Ge' in Google

Chuanyi Li

This author has not been identified. Look up 'Chuanyi Li' in Google

Jie Gui

This author has not been identified. Look up 'Jie Gui' in Google