AdamP: Slowing Down the Slowdown for Momentum Optimizers on Scale-invariant Weights

Byeongho Heo, Sanghyuk Chun, Seong Joon Oh, Dongyoon Han, Sangdoo Yun, Gyuwan Kim, Youngjung Uh, Jung-Woo Ha 0001. AdamP: Slowing Down the Slowdown for Momentum Optimizers on Scale-invariant Weights. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021. [doi]

@inproceedings{HeoCOHYKUH21,
  title = {AdamP: Slowing Down the Slowdown for Momentum Optimizers on Scale-invariant Weights},
  author = {Byeongho Heo and Sanghyuk Chun and Seong Joon Oh and Dongyoon Han and Sangdoo Yun and Gyuwan Kim and Youngjung Uh and Jung-Woo Ha 0001},
  year = {2021},
  url = {https://openreview.net/forum?id=Iz3zU3M316D},
  researchr = {https://researchr.org/publication/HeoCOHYKUH21},
  cites = {0},
  citedby = {0},
  booktitle = {9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021},
  publisher = {OpenReview.net},
}