What Happens To BERT Embeddings During Fine-tuning?

Amil Merchant, Elahe Rahimtoroghi, Ellie Pavlick, Ian Tenney. What Happens To BERT Embeddings During Fine-tuning?. In Afra Alishahi, Yonatan Belinkov, Grzegorz Chrupala, Dieuwke Hupkes, Yuval Pinter, Hassan Sajjad, editors, Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP, BlackboxNLP@EMNLP 2020, Online, November 2020. pages 33-44, Association for Computational Linguistics, 2020. [doi]