Improving Language Models by Retrieving from Trillions of Tokens

Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann, Trevor Cai, Eliza Rutherford, Katie Millican, George van den Driessche 0002, Jean-Baptiste Lespiau, Bogdan Damoc, Aidan Clark, Diego de Las Casas, Aurelia Guy, Jacob Menick, Roman Ring, Tom Hennigan, Saffron Huang, Loren Maggiore, Chris Jones, Albin Cassirer, Andy Brock, Michela Paganini, Geoffrey Irving, Oriol Vinyals, Simon Osindero, Karen Simonyan, Jack W. Rae, Erich Elsen, Laurent Sifre. Improving Language Models by Retrieving from Trillions of Tokens. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvári, Gang Niu 0001, Sivan Sabato, editors, International Conference on Machine Learning, ICML 2022, 17-23 July 2022, Baltimore, Maryland, USA. Volume 162 of Proceedings of Machine Learning Research, pages 2206-2240, PMLR, 2022. [doi]

Authors

Sebastian Borgeaud

This author has not been identified. Look up 'Sebastian Borgeaud' in Google

Arthur Mensch

This author has not been identified. Look up 'Arthur Mensch' in Google

Jordan Hoffmann

This author has not been identified. Look up 'Jordan Hoffmann' in Google

Trevor Cai

This author has not been identified. Look up 'Trevor Cai' in Google

Eliza Rutherford

This author has not been identified. Look up 'Eliza Rutherford' in Google

Katie Millican

This author has not been identified. Look up 'Katie Millican' in Google

George van den Driessche 0002

This author has not been identified. Look up 'George van den Driessche 0002' in Google

Jean-Baptiste Lespiau

This author has not been identified. Look up 'Jean-Baptiste Lespiau' in Google

Bogdan Damoc

This author has not been identified. Look up 'Bogdan Damoc' in Google

Aidan Clark

This author has not been identified. Look up 'Aidan Clark' in Google

Diego de Las Casas

This author has not been identified. Look up 'Diego de Las Casas' in Google

Aurelia Guy

This author has not been identified. Look up 'Aurelia Guy' in Google

Jacob Menick

This author has not been identified. Look up 'Jacob Menick' in Google

Roman Ring

This author has not been identified. Look up 'Roman Ring' in Google

Tom Hennigan

This author has not been identified. Look up 'Tom Hennigan' in Google

Saffron Huang

This author has not been identified. Look up 'Saffron Huang' in Google

Loren Maggiore

This author has not been identified. Look up 'Loren Maggiore' in Google

Chris Jones

This author has not been identified. Look up 'Chris Jones' in Google

Albin Cassirer

This author has not been identified. Look up 'Albin Cassirer' in Google

Andy Brock

This author has not been identified. Look up 'Andy Brock' in Google

Michela Paganini

This author has not been identified. Look up 'Michela Paganini' in Google

Geoffrey Irving

This author has not been identified. Look up 'Geoffrey Irving' in Google

Oriol Vinyals

This author has not been identified. Look up 'Oriol Vinyals' in Google

Simon Osindero

This author has not been identified. Look up 'Simon Osindero' in Google

Karen Simonyan

This author has not been identified. Look up 'Karen Simonyan' in Google

Jack W. Rae

This author has not been identified. Look up 'Jack W. Rae' in Google

Erich Elsen

This author has not been identified. Look up 'Erich Elsen' in Google

Laurent Sifre

This author has not been identified. Look up 'Laurent Sifre' in Google