Scale Efficiently: Insights from Pretraining and Finetuning Transformers

Yi Tay, Mostafa Dehghani 0001, Jinfeng Rao, William Fedus, Samira Abnar, Hyung Won Chung, Sharan Narang, Dani Yogatama, Ashish Vaswani, Donald Metzler. Scale Efficiently: Insights from Pretraining and Finetuning Transformers. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net, 2022. [doi]

@inproceedings{Tay0RFACNYVM22,
  title = {Scale Efficiently: Insights from Pretraining and Finetuning Transformers},
  author = {Yi Tay and Mostafa Dehghani 0001 and Jinfeng Rao and William Fedus and Samira Abnar and Hyung Won Chung and Sharan Narang and Dani Yogatama and Ashish Vaswani and Donald Metzler},
  year = {2022},
  url = {https://openreview.net/forum?id=f2OYVDyfIB},
  researchr = {https://researchr.org/publication/Tay0RFACNYVM22},
  cites = {0},
  citedby = {0},
  booktitle = {The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022},
  publisher = {OpenReview.net},
}