Frederik Kunstner, Jacques Chen, Jonathan Wilder Lavington, Mark Schmidt 0001. Noise Is Not the Main Factor Behind the Gap Between Sgd and Adam on Transformers, But Sign Descent Might Be. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net, 2023. [doi]
Abstract is missing.