On Orthogonality Constraints for Transformers

Aston Zhang, Alvin Chan, Yi Tay, Jie Fu, Shuohang Wang, Shuai Zhang 0007, Huajie Shao, Shuochao Yao, Roy Ka-Wei Lee. On Orthogonality Constraints for Transformers. In Chengqing Zong, Fei Xia, Wenjie Li 0002, Roberto Navigli, editors, Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, ACL/IJCNLP 2021, (Volume 2: Short Papers), Virtual Event, August 1-6, 2021. pages 375-382, Association for Computational Linguistics, 2021. [doi]

@inproceedings{ZhangCTFW0SYL20,
  title = {On Orthogonality Constraints for Transformers},
  author = {Aston Zhang and Alvin Chan and Yi Tay and Jie Fu and Shuohang Wang and Shuai Zhang 0007 and Huajie Shao and Shuochao Yao and Roy Ka-Wei Lee},
  year = {2021},
  url = {https://aclanthology.org/2021.acl-short.48},
  researchr = {https://researchr.org/publication/ZhangCTFW0SYL20},
  cites = {0},
  citedby = {0},
  pages = {375-382},
  booktitle = {Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, ACL/IJCNLP 2021, (Volume 2: Short Papers), Virtual Event, August 1-6, 2021},
  editor = {Chengqing Zong and Fei Xia and Wenjie Li 0002 and Roberto Navigli},
  publisher = {Association for Computational Linguistics},
  isbn = {978-1-954085-53-4},
}