The Information Pathways Hypothesis: Transformers are Dynamic Self-Ensembles

Md. Shamim Hussain, Mohammed J. Zaki, Dharmashankar Subramanian. The Information Pathways Hypothesis: Transformers are Dynamic Self-Ensembles. In Ambuj Singh, Yizhou Sun, Leman Akoglu, Dimitrios Gunopulos, Xifeng Yan, Ravi Kumar 0001, Fatma Ozcan, Jieping Ye, editors, Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD 2023, Long Beach, CA, USA, August 6-10, 2023. pages 810-821, ACM, 2023. [doi]

@inproceedings{HussainZS23,
  title = {The Information Pathways Hypothesis: Transformers are Dynamic Self-Ensembles},
  author = {Md. Shamim Hussain and Mohammed J. Zaki and Dharmashankar Subramanian},
  year = {2023},
  doi = {10.1145/3580305.3599520},
  url = {https://doi.org/10.1145/3580305.3599520},
  researchr = {https://researchr.org/publication/HussainZS23},
  cites = {0},
  citedby = {0},
  pages = {810-821},
  booktitle = {Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD 2023, Long Beach, CA, USA, August 6-10, 2023},
  editor = {Ambuj Singh and Yizhou Sun and Leman Akoglu and Dimitrios Gunopulos and Xifeng Yan and Ravi Kumar 0001 and Fatma Ozcan and Jieping Ye},
  publisher = {ACM},
}