MosaicBERT: A Bidirectional Encoder Optimized for Fast Pretraining

Jacob Portes, Alexander Trott, Sam Havens, Daniel King, Abhinav Venigalla, Moin Nadeem, Nikhil Sardana, Daya Khudia, Jonathan Frankle. MosaicBERT: A Bidirectional Encoder Optimized for Fast Pretraining. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, Sergey Levine, editors, Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023. 2023. [doi]

Authors

Jacob Portes

This author has not been identified. Look up 'Jacob Portes' in Google

Alexander Trott

This author has not been identified. Look up 'Alexander Trott' in Google

Sam Havens

This author has not been identified. Look up 'Sam Havens' in Google

Daniel King

This author has not been identified. Look up 'Daniel King' in Google

Abhinav Venigalla

This author has not been identified. Look up 'Abhinav Venigalla' in Google

Moin Nadeem

This author has not been identified. Look up 'Moin Nadeem' in Google

Nikhil Sardana

This author has not been identified. Look up 'Nikhil Sardana' in Google

Daya Khudia

This author has not been identified. Look up 'Daya Khudia' in Google

Jonathan Frankle

This author has not been identified. Look up 'Jonathan Frankle' in Google