Resonance in Weight Space: Covariate Shift Can Drive Divergence of SGD with Momentum

Kirby Banman, Liam Peet-Pare, Nidhi Hegde 0001, Alona Fyshe, Martha White. Resonance in Weight Space: Covariate Shift Can Drive Divergence of SGD with Momentum. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net, 2022. [doi]

Abstract

Abstract is missing.