Recommended citation: Kunstner, Frederik, et al. "Noise Is Not the Main Factor Behind the Gap Between Sgd and Adam on Transformers, But Sign Descent Might Be" https://www.proquest.com/openview/da3b1b61b3fd2f0ca38a3418172867d9/1?pq-origsite=gscholar&cbl=18750
