k-Rater Reliability: The Correct Unit of Reliability for Aggregated Human Annotations

Ka Wong, Praveen K. Paritosh. k-Rater Reliability: The Correct Unit of Reliability for Aggregated Human Annotations. In Smaranda Muresan, Preslav Nakov, Aline Villavicencio, editors, Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), ACL 2022, Dublin, Ireland, May 22-27, 2022. pages 378-384, Association for Computational Linguistics, 2022. [doi]

@inproceedings{WongP22,
  title = {k-Rater Reliability: The Correct Unit of Reliability for Aggregated Human Annotations},
  author = {Ka Wong and Praveen K. Paritosh},
  year = {2022},
  url = {https://aclanthology.org/2022.acl-short.42},
  researchr = {https://researchr.org/publication/WongP22},
  cites = {0},
  citedby = {0},
  pages = {378-384},
  booktitle = {Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), ACL 2022, Dublin, Ireland, May 22-27, 2022},
  editor = {Smaranda Muresan and Preslav Nakov and Aline Villavicencio},
  publisher = {Association for Computational Linguistics},
  isbn = {978-1-955917-22-3},
}