Is attention required for ICL? Exploring the Relationship Between Model Architecture and In-Context Learning Ability

Ivan Lee, Nan Jiang, Taylor Berg-Kirkpatrick. Is attention required for ICL? Exploring the Relationship Between Model Architecture and In-Context Learning Ability. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net, 2024. [doi]

Authors

Ivan Lee

This author has not been identified. Look up 'Ivan Lee' in Google

Nan Jiang

This author has not been identified. Look up 'Nan Jiang' in Google

Taylor Berg-Kirkpatrick

This author has not been identified. Look up 'Taylor Berg-Kirkpatrick' in Google