AdamP: Slowing Down the Slowdown for Momentum Optimizers on Scale-invariant Weights

Byeongho Heo, Sanghyuk Chun, Seong Joon Oh, Dongyoon Han, Sangdoo Yun, Gyuwan Kim, Youngjung Uh, Jung-Woo Ha 0001. AdamP: Slowing Down the Slowdown for Momentum Optimizers on Scale-invariant Weights. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021. [doi]

Authors

Byeongho Heo

This author has not been identified. Look up 'Byeongho Heo' in Google

Sanghyuk Chun

This author has not been identified. Look up 'Sanghyuk Chun' in Google

Seong Joon Oh

This author has not been identified. Look up 'Seong Joon Oh' in Google

Dongyoon Han

This author has not been identified. Look up 'Dongyoon Han' in Google

Sangdoo Yun

This author has not been identified. Look up 'Sangdoo Yun' in Google

Gyuwan Kim

This author has not been identified. Look up 'Gyuwan Kim' in Google

Youngjung Uh

This author has not been identified. Look up 'Youngjung Uh' in Google

Jung-Woo Ha 0001

This author has not been identified. Look up 'Jung-Woo Ha 0001' in Google