How does the task complexity of masked pretraining objectives affect downstream performance?

Atsuki Yamaguchi, Hiroaki Ozaki, Terufumi Morishita, Gaku Morio, Yasuhiro Sogawa. How does the task complexity of masked pretraining objectives affect downstream performance?. In Anna Rogers, Jordan L. Boyd-Graber, Naoaki Okazaki, editors, Findings of the Association for Computational Linguistics: ACL 2023, Toronto, Canada, July 9-14, 2023. pages 10527-10537, Association for Computational Linguistics, 2023. [doi]

Abstract

Abstract is missing.