COCO-LM: Correcting and Contrasting Text Sequences for Language Model Pretraining

Yu Meng 0001, Chenyan Xiong, Payal Bajaj, Saurabh Tiwary, Paul Bennett 0001, Jiawei Han 0001, Xia Song. COCO-LM: Correcting and Contrasting Text Sequences for Language Model Pretraining. In Marc'Aurelio Ranzato, Alina Beygelzimer, Yann N. Dauphin, Percy Liang, Jennifer Wortman Vaughan, editors, Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, December 6-14, 2021, virtual. pages 23102-23114, 2021. [doi]

Authors

Yu Meng 0001

This author has not been identified. Look up 'Yu Meng 0001' in Google

Chenyan Xiong

This author has not been identified. Look up 'Chenyan Xiong' in Google

Payal Bajaj

This author has not been identified. Look up 'Payal Bajaj' in Google

Saurabh Tiwary

This author has not been identified. Look up 'Saurabh Tiwary' in Google

Paul Bennett 0001

This author has not been identified. Look up 'Paul Bennett 0001' in Google

Jiawei Han 0001

This author has not been identified. Look up 'Jiawei Han 0001' in Google

Xia Song

This author has not been identified. Look up 'Xia Song' in Google