Scaling Laws vs Model Architectures: How does Inductive Bias Influence Scaling?

Yi Tay, Mostafa Dehghani 0001, Samira Abnar, Hyung Won Chung, William Fedus, Jinfeng Rao, Sharan Narang, Vinh Q. Tran 0002, Dani Yogatama, Donald Metzler. Scaling Laws vs Model Architectures: How does Inductive Bias Influence Scaling?. In Houda Bouamor, Juan Pino 0001, Kalika Bali, editors, Findings of the Association for Computational Linguistics: EMNLP 2023, Singapore, December 6-10, 2023. pages 12342-12364, Association for Computational Linguistics, 2023. [doi]

Abstract

Abstract is missing.