Difference-Masking: Choosing What to Mask in Continued Pretraining

Alex Wilf, Syeda Nahida Akter, Leena Mathur, Paul Pu Liang, Sheryl Mathew, Mengrou Shou, Eric Nyberg, Louis-Philippe Morency. Difference-Masking: Choosing What to Mask in Continued Pretraining. In Houda Bouamor, Juan Pino 0001, Kalika Bali, editors, Findings of the Association for Computational Linguistics: EMNLP 2023, Singapore, December 6-10, 2023. pages 13222-13234, Association for Computational Linguistics, 2023. [doi]

Authors

Alex Wilf

This author has not been identified. Look up 'Alex Wilf' in Google

Syeda Nahida Akter

This author has not been identified. Look up 'Syeda Nahida Akter' in Google

Leena Mathur

This author has not been identified. Look up 'Leena Mathur' in Google

Paul Pu Liang

This author has not been identified. Look up 'Paul Pu Liang' in Google

Sheryl Mathew

This author has not been identified. Look up 'Sheryl Mathew' in Google

Mengrou Shou

This author has not been identified. Look up 'Mengrou Shou' in Google

Eric Nyberg

This author has not been identified. Look up 'Eric Nyberg' in Google

Louis-Philippe Morency

This author has not been identified. Look up 'Louis-Philippe Morency' in Google