What is best for spoken language understanding: small but task-dependant embeddings or huge but out-of-domain embeddings?

Sahar Ghannay, Antoine Neuraz, Sophie Rosset. What is best for spoken language understanding: small but task-dependant embeddings or huge but out-of-domain embeddings?. In 2020 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2020, Barcelona, Spain, May 4-8, 2020. pages 8114-8118, IEEE, 2020. [doi]

@inproceedings{GhannayNR20,
  title = {What is best for spoken language understanding: small but task-dependant embeddings or huge but out-of-domain embeddings?},
  author = {Sahar Ghannay and Antoine Neuraz and Sophie Rosset},
  year = {2020},
  doi = {10.1109/ICASSP40776.2020.9053278},
  url = {https://doi.org/10.1109/ICASSP40776.2020.9053278},
  researchr = {https://researchr.org/publication/GhannayNR20},
  cites = {0},
  citedby = {0},
  pages = {8114-8118},
  booktitle = {2020 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2020, Barcelona, Spain, May 4-8, 2020},
  publisher = {IEEE},
  isbn = {978-1-5090-6631-5},
}