Table of Contents

• Frontmatter [doi]
• Findings of the 2023 Conference on Machine Translation (WMT23): LLMs Are Here but Not Quite There YetTom Kocmi, Eleftherios Avramidis, Rachel Bawden, Ondrej Bojar, Anton Dvorkovich, Christian Federmann, Mark Fishel, Markus Freitag, Thamme Gowda, Roman Grundkiewicz, Barry Haddow, Philipp Koehn, Benjamin Marie, Christof Monz, Makoto Morishita, Kenton Murray, Makoto Nagata, Toshiaki Nakazawa, Martin Popel, Maja Popovic, Mariya Shmatova. 1-42 [doi]
• Findings of the WMT 2023 Biomedical Translation Shared Task: Evaluation of ChatGPT 3.5 as a Comparison SystemMariana L. Neves, Antonio Jimeno-Yepes, Aurélie Névéol, Rachel Bawden, Giorgio Maria Di Nunzio, Roland Roller, Philippe Thomas 0002, Federica Vezzani, Maika Vicente Navarro, Lana Yeganova, Dina Wiemann, Cristian Grozea. 43-54 [doi]
• Findings of the WMT 2023 Shared Task on Discourse-Level Literary Translation: A Fresh Orb in the Cosmos of LLMsLongyue Wang, Zhaopeng Tu, Yan Gu, Siyou Liu, Dian Yu, Qingsong Ma, Chenyang Lyu, Liting Zhou, Chao-Hong Liu, YuFeng Ma, Weiyu Chen, Yvette Graham, Bonnie Webber, Philipp Koehn, Andy Way, Yulin Yuan, Shuming Shi 0001. 55-67 [doi]
• Findings of the Second WMT Shared Task on Sign Language Translation (WMT-SLT23)Mathias Müller 0002, Malihe Alikhani, Eleftherios Avramidis, Richard Bowden, Annelies Braffort, Necati Cihan Camgöz, Sarah Ebling, Cristina España-Bonet, Anne Göhring, Roman Grundkiewicz, Mert Inan, Zifan Jiang, Oscar Koller, Amit Moryossef, Annette Rios, Dimitar Shterionov, Sandra Sidler-Miserez, Katja Tissi, Davy Van Landuyt. 68-94 [doi]
• Findings of the WMT 2023 Shared Task on Parallel Data CurationSteve Sloto, Brian Thompson 0001, Huda Khayrallah, Tobias Domhan, Thamme Gowda, Philipp Koehn. 95-102 [doi]
• Samsung R&D Institute Philippines at WMT 2023Jan Christian Blaise Cruz. 103-109 [doi]
• NAIST-NICT WMT'23 General MT Task SubmissionHiroyuki Deguchi, Kenji Imamura, Yuto Nishida, Yusuke Sakai 0010, Justin Vasselli, Taro Watanabe. 110-118 [doi]
• CUNI at WMT23 General Translation Task: MT and a Genetic AlgorithmJosef Jon, Martin Popel, Ondrej Bojar. 119-127 [doi]
• SKIM at WMT 2023 General Translation TaskKeito Kudo, Takumi Ito, Makoto Morishita, Jun Suzuki. 128-136 [doi]
• KYB General Machine Translation Systems for WMT23Ben Li, Yoko Matsuzaki, Shivam Kalkar. 137-142 [doi]
• Yishu: Yishu at WMT2023 Translation TaskLuo Min, Yixin Tan, Qiulin Chen. 143-149 [doi]
• PROMT Systems for WMT23 Shared General Translation TaskAlexander Molchanov, Vladislav Kovalenko. 150-154 [doi]
• AIST AIRC Submissions to the WMT23 Shared TaskMatiss Rikters, Makoto Miwa. 155-161 [doi]
• MUNI-NLP Submission for Czech-Ukrainian Translation Task at WMT23Pavel Rychlý, Yuliia Teslia. 162-165 [doi]
• Exploring Prompt Engineering with GPT Language Models for Document-Level Machine Translation: Insights and FindingsYangjian Wu, Gang Hu. 166-169 [doi]
• Treating General MT Shared Task as a Multi-Domain Adaptation Problem: HW-TSC's Submission to the WMT23 General MT Shared TaskZhanglin Wu, Daimeng Wei, Zongyao Li, Zhengzhe Yu, Shaojun Li, Xiaoyu Chen, Hengchao Shang, Jiaxin Guo, Yuhao Xie, Lizhi Lei, Hao Yang, Yanfei Jiang. 170-174 [doi]
• UvA-MT's Participation in the WMT 2023 General Translation Shared TaskDi Wu, Shaomu Tan, David Stap, Ali Araabi, Christof Monz. 175-180 [doi]
• Achieving State-of-the-Art Multilingual Translation Model with Minimal Data and ParametersHui Zeng. 181-186 [doi]
• IOL Research Machine Translation Systems for WMT23 General Machine Translation Shared TaskWenbo Zhang. 187-191 [doi]
• GTCOM and DLUT's Neural Machine Translation Systems for WMT23Hao Zong. 192-197 [doi]
• RoCS-MT: Robustness Challenge Set for Machine TranslationRachel Bawden, Benoît Sagot. 198-216 [doi]
• Multifaceted Challenge Set for Evaluating Machine Translation PerformanceXiaoyu Chen, Daimeng Wei, Zhanglin Wu, Ting Zhu, Hengchao Shang, Zongyao Li, Jiaxin Guo, Ning Xie, Lizhi Lei, Hao Yang, Yanfei Jiang. 217-223 [doi]
• Linguistically Motivated Evaluation of the 2023 State-of-the-art Machine Translation: Can ChatGPT Outperform NMT?Shushen Manakhimova, Eleftherios Avramidis, Vivien Macketanz, Ekaterina Lapshinova-Koltunski, Sergei Bagdasarov, Sebastian Möller 0001. 224-245 [doi]
• IIIT HYD's Submission for WMT23 Test-suite TaskAnanya Mukherjee, Manish Shrivastava 0001. 246-251 [doi]
• Test Suites Task: Evaluation of Gender Fairness in MT with MuST-SHE and INESBeatrice Savoldi, Marco Gaido, Matteo Negri, Luisa Bentivogli. 252-262 [doi]
• Biomedical Parallel Sentence Retrieval Using Large Language ModelsSheema Firdous, Sadaf Abdul-Rauf. 263-270 [doi]
• The Path to Continuous Domain Adaptation Improvements by HW-TSC for the WMT23 Biomedical Translation Shared TaskZhanglin Wu, Daimeng Wei, Zongyao Li, Zhengzhe Yu, Shaojun Li, Xiaoyu Chen, Hengchao Shang, Jiaxin Guo, Yuhao Xie, Lizhi Lei, Hao Yang, Yanfei Jiang. 271-274 [doi]
• Investigating Techniques for a Deeper Understanding of Neural Machine Translation (NMT) Systems through Data Filtering and Fine-tuning StrategiesLichao Zhu, Maria Zimina, Maud Bénard, Behnoosh Namdar, Nicolas Ballier, Guillaume Wisniewski, Jean-Baptiste Yunès. 275-281 [doi]
• MAX-ISI System at WMT23 Discourse-Level Literary Translation TaskLi An, Linghao Jin, Xuezhe Ma. 282-286 [doi]
• The MAKE-NMTVIZ System Description for the WMT23 Literary TaskFabien Lopez, Gabriela González Sáez, Damien Hansen, Mariam Nakhlé, Behnoosh Namdarzadeh, Nicolas Ballier, Marco Dinarelli, Emmanuelle Esperança-Rodier, Sui He, Sadaf Mohseni, Caroline Rossi, Didier Schwab, Jun Yang, Jean-Baptiste Yunès, Lichao Zhu. 287-295 [doi]
• DUTNLP System for the WMT2023 Discourse-Level Literary TranslationAnqi Zhao, Kaiyu Huang, Hao Yu, Degen Huang. 296-301 [doi]
• HW-TSC's Submissions to the WMT23 Discourse-Level Literary Translation Shared TaskYuhao Xie, Zongyao Li, Zhanglin Wu, Daimeng Wei, Xiaoyu Chen, Zhiqiang Rao, Shaojun Li, Hengchao Shang, Jiaxin Guo, Lizhi Lei, Hao Yang, Yanfei Jiang. 302-306 [doi]
• TJUNLP: System Description for the WMT23 Literary Task in Chinese to English Translation DirectionShaoLin Zhu, Deyi Xiong. 307-311 [doi]
• Machine Translation for Nko: Tools, Corpora, and Baseline ResultsMoussa Doumbouya, Baba Mamadi Diané, Solo Farabado Cissé, Djibrila Diané, Abdoulaye Sow, Séré Moussa Doumbouya, Daouda Bangoura, Fodé Moriba Bayo, Ibrahima Sory 2. Condé, Kalo Mory Diané, Chris Piech, Christopher D. Manning. 312-343 [doi]
• TTIC's Submission to WMT-SLT 23Marcelo Sandoval-Castañeda, Yanhong Li, Bowen Shi, Diane Brentari, Karen Livescu, Gregory Shakhnarovich. 344-350 [doi]
• KnowComp Submission for WMT23 Sign Language Translation TaskBaixuan Xu, Haochen Shi, Tianshi Zheng, Qing Zong, Weiqi Wang, Zhaowei Wang 0003, Yangqiu Song. 351-358 [doi]
• A Fast Method to Filter Noisy Parallel Data WMT2023 Shared Task on Parallel Data CurationNguyen-Hoang Minh-Cong, Nguyen Van Vinh, Nguyen Le Minh. 359-365 [doi]
• A Sentence Alignment Approach to Document Alignment and Multi-faceted Filtering for Curating Parallel Sentence Pairs from Web-crawled DataSteinthor Steingrimsson. 366-374 [doi]
• Document-Level Language Models for Machine TranslationFrithjof Petrick, Christian Herold, Pavel Petrushkov, Shahram Khadivi, Hermann Ney. 375-391 [doi]
• ChatGPT MT: Competitive for High- (but Not Low-) Resource LanguagesNathaniel R. Robinson, Perez Ogayo, David R. Mortensen, Graham Neubig. 392-418 [doi]
• Large Language Models Effectively Leverage Document-level Context for Literary Translation, but Critical Errors PersistMarzena Karpinska, Mohit Iyyer. 419-451 [doi]
• Identifying Context-Dependent Translations for Evaluation Set ProductionRachel Wicks, Matt Post. 452-467 [doi]
• Machine Translation with Large Language Models: Prompting, Few-shot Learning, and Fine-tuning with QLoRAXuan Zhang 0008, Navid Rajabi, Kevin Duh, Philipp Koehn. 468-481 [doi]
• Towards Effective Disambiguation for Machine Translation with Large Language ModelsVivek Iyer, Pinzhen Chen, Alexandra Birch. 482-495 [doi]
• A Closer Look at Transformer Attention for Multilingual TranslationJingyi Zhang 0002, Gerard de Melo, Hongfei Xu, Kehai Chen. 496-506 [doi]
• Bridging the Gap between Position-Based and Content-Based Self-Attention for Neural Machine TranslationFelix Schmidt, Mattia Di Gangi. 507-521 [doi]
• Visual Prediction Improves Zero-Shot Cross-Modal Machine TranslationTosho Hirasawa, Emanuele Bugliarello, Desmond Elliott, Mamoru Komachi. 522-535 [doi]
• The Gender-GAP Pipeline: A Gender-Aware Polyglot Pipeline for Gender Characterisation in 55 LanguagesBenjamin Muller, Belen Alastruey, Prangthip Hansanti, Elahe Kalbassi, Christophe Ropers, Eric Smith, Adina Williams, Luke Zettlemoyer, Pierre Andrews, Marta R. Costa-Jussà. 536-550 [doi]
• Towards Better Evaluation for Formality-Controlled English-Japanese Machine TranslationEdison Marrese-Taylor, Pin Chen Wang, Yutaka Matsuo. 551-560 [doi]
• There's No Data like Better Data: Using QE Metrics for MT Data FilteringJan-Thorsten Peter, David Vilar, Daniel Deutsch, Mara Finkelstein, Juraj Juraska, Markus Freitag. 561-577 [doi]
• Results of WMT23 Metrics Shared Task: Metrics Might Be Guilty but References Are Not InnocentMarkus Freitag, Nitika Mathur, Chi-kiu Lo, Eleftherios Avramidis, Ricardo Rei, Brian Thompson, Tom Kocmi, Frédéric Blain, Daniel Deutsch, Craig Stewart, Chrysoula Zerva, Sheila Castilho, Alon Lavie, George F. Foster. 578-628 [doi]
• Findings of the WMT 2023 Shared Task on Quality EstimationFrédéric Blain, Chrysoula Zerva, Ricardo Ribeiro, Nuno Miguel Guerreiro, Diptesh Kanojia, José G. C. de Souza, Beatriz Silva, Tânia Vaz, Yan Jingxuan, Fatemeh Azadi, Constantin Orasan, André Martins. 629-653 [doi]
• Findings of the Word-Level AutoCompletion Shared Task in WMT 2023Lemao Liu, Francisco Casacuberta, George Foster, Guoping Huang, Philipp Koehn, Geza Kovacs, Shuming Shi 0001, Taro Watanabe, Chengqing Zong. 654-662 [doi]
• Findings of the WMT 2023 Shared Task on Machine Translation with TerminologiesKirill Semenov, Vilém Zouhar, Tom Kocmi, Dongdong Zhang 0001, Wangchunshu Zhou, Yuchen Eleanor Jiang. 663-671 [doi]
• Findings of the WMT 2023 Shared Task on Automatic Post-EditingPushpak Bhattacharyya, Rajen Chatterjee, Markus Freitag, Diptesh Kanojia, Matteo Negri, Marco Turchi. 672-681 [doi]
• Findings of the WMT 2023 Shared Task on Low-Resource Indic Language TranslationSantanu Pal, Partha Pakray, Sahinur Rahman Laskar, Lenin Laitonjam, Vanlalmuansangi Khenglawt, Sunita Warjri, Pankaj Kundan Dadure, Sandeep Kumar Dash. 682-694 [doi]
• ACES: Translation Accuracy Challenge Sets at WMT 2023Chantal Amrhein, Nikita Moghe, Liane Guillou. 695-712 [doi]
• Challenging the State-of-the-art Machine Translation Metrics from a Linguistic PerspectiveEleftherios Avramidis, Shushen Manakhimova, Vivien Macketanz, Sebastian Möller 0001. 713-729 [doi]
• Tokengram_F, a Fast and Accurate Token-based chrF++ DerivativeSören Dreano, Derek Molloy, Noel Murphy. 730-737 [doi]
• Embed_Llama: Using LLM Embeddings for the Metrics Shared TaskSören Dreano, Derek Molloy, Noel Murphy. 738-745 [doi]
• eBLEU: Unexpectedly Good Machine Translation Evaluation Using Simple Word EmbeddingsMuhammad N. ElNokrashy, Tom Kocmi. 746-750 [doi]
• Cometoid: Distilling Strong Reference-based Machine Translation Metrics into Even Stronger Quality Estimation MetricsThamme Gowda, Tom Kocmi, Marcin Junczys-Dowmunt. 751-755 [doi]
• MetricX-23: The Google Submission to the WMT 2023 Metrics Shared TaskJuraj Juraska, Mara Finkelstein, Daniel Deutsch, Aditya Siddhant, Mehdi Mirzazadeh, Markus Freitag. 756-767 [doi]
• GEMBA-MQM: Detecting Translation Quality Error Spans with GPT-4Tom Kocmi, Christian Federmann. 768-775 [doi]
• Metric Score Landscape Challenge (MSLC23): Understanding Metrics' Performance on a Wider Landscape of Translation QualityChi-kiu Lo, Samuel Larkin, Rebecca Knowles. 776-799 [doi]
• MEE4 and XLsim : IIIT HYD's Submissions' for WMT23 Metrics Shared TaskAnanya Mukherjee, Manish Shrivastava 0001. 800-805 [doi]
• Quality Estimation Using Minimum Bayes RiskSubhajit Naskar, Daniel Deutsch, Markus Freitag. 806-811 [doi]
• Evaluating Metrics for Document-context Evaluation in Machine TranslationVikas Raunak, Tom Kocmi, Matt Post. 812-814 [doi]
• Semantically-Informed Regressive Encoder ScoreVasiliy Viskov, George Kokush, Daniil Larionov, Steffen Eger, Alexander Panchenko. 815-821 [doi]
• Empowering a Metric with LLM-assisted Named Entity Annotation: HW-TSC's Submission to the WMT23 Metrics Shared TaskZhanglin Wu, Yilun Liu, Min Zhang 0005, Xiaofeng Zhao, Junhao Zhu, Ming Zhu, Xiaosong Qiao, Jingfei Zhang, Miaomiao Ma, Yanqing Zhao, Song Peng, Shimin Tao, Hao Yang, Yanfei Jiang. 822-828 [doi]
• Unify Word-level and Span-level Tasks: NJUNLP's Participation for the WMT2023 Quality Estimation Shared TaskXiang Geng, Zhejian Lai, Yu Zhang, Shimin Tao, Hao Yang, Jiajun Chen, Shujian Huang. 829-834 [doi]
• HW-TSC 2023 Submission for the Quality Estimation Shared TaskYuang Li, Chang Su 0001, Ming Zhu, Mengyao Piao, Xinglin Lyu, Min Zhang, Hao Yang. 835-840 [doi]
• Scaling up CometKiwi: Unbabel-IST 2023 Submission for the Quality Estimation Shared TaskRicardo Rei, Nuno M. Guerreiro, José Pombal, Daan van Stigt, Marcos V. Treviso, Luísa Coheur, José G. C. de Souza, André Martins. 841-848 [doi]
• SurreyAI 2023 Submission for the Quality Estimation Shared TaskArchchana Sindhujan, Diptesh Kanojia, Constantin Orasan, Tharindu Ranasinghe. 849-855 [doi]
• MMT's Submission for the WMT 2023 Quality Estimation Shared TaskYulong Wu, Viktor Schlegel, Daniel Beck, Riza Batista-Navarro. 856-862 [doi]
• IOL Research's Submission for WMT 2023 Quality Estimation Shared TaskZeyu Yan. 863-871 [doi]
• SJTU-MTLAB's Submission to the WMT23 Word-Level Auto Completion TaskXingyu Chen, Rui Wang. 872-876 [doi]
• PRHLT's Submission to WLAC 2023Ángel Navarro, Miguel Domingo, Francisco Casacuberta. 877-881 [doi]
• KnowComp Submission for WMT23 Word-Level AutoCompletion TaskYi Wu, Haochen Shi, Weiqi Wang, Yangqiu Song. 882-889 [doi]
• Terminology-Aware Translation with Constrained Decoding and Large Language Model PromptingNikolay Bogoychev, Pinzhen Chen. 890-896 [doi]
• Lingua Custodia's Participation at the WMT 2023 Terminology Shared TaskJingshu Liu, Mariam Nakhlé, Gaëtan Caillout, Raheel Qader. 897-901 [doi]
• Domain Terminology Integration into Machine Translation: Leveraging Large Language ModelsYasmin Moslem, Gianfranco Romani, Mahdi Molaei, John D. Kelleher, Rejwanul Haque, Andy Way. 902-911 [doi]
• OPUS-CAT Terminology Systems for the WMT23 Terminology Shared TaskTommi Nieminen. 912-918 [doi]
• VARCO-MT: NCSOFT's WMT'23 Terminology Shared Task SubmissionGeon-Woo Park, Junghwa Lee, Meiying Ren, Allison Shindell, Yeonsoo Lee. 919-925 [doi]
• HW-TSC's Participation in the WMT 2023 Automatic Post Editing Shared TaskJiawei Yu, Min Zhang, Yanqing Zhao, Xiaofeng Zhao, Yuang Li, Chang Su 0001, Yinglu Li, Miaomiao Ma, Shimin Tao, Hao Yang 0006. 926-930 [doi]
• Neural Machine Translation for English - Manipuri and English - AssameseGoutam Agrawal, Rituraj Das, Anupam Biswas, Dalton Meitei Thounaojam. 931-934 [doi]
• GUIT-NLP's Submission to Shared Task: Low Resource Indic Language TranslationMazida A. Ahmed, Kuwali Talukdar, Parvez Aziz Boruah, Shikhar Kumar Sarma, Kishore Kashyap. 935-940 [doi]
• NICT-AI4B's Submission to the Indic MT Shared Task in WMT 2023Raj Dabre, Jay P. Gala, Pranjal A. Chitale. 941-949 [doi]
• Machine Translation Advancements for Low-Resource Indian Languages in WMT23: CFILT-IITB's Effort for Bridging the GapPranav Gaikwad, Meet Doshi, Sourabh Dattatray Deoghare, Pushpak Bhattacharyya. 950-953 [doi]
• Low-Resource Machine Translation Systems for Indic LanguagesIvana Kvapilíková, Ondrej Bojar. 954-958 [doi]
• MUNI-NLP Systems for Low-resource Indic Machine TranslationEdoardo Signoroni, Pavel Rychlý. 959-966 [doi]
• NITS-CNLP Low-Resource Neural Machine Translation Systems of English-Manipuri Language PairKshetrimayum Boynao Singh, Avichandra Singh Ningthoujam, Loitongbam Sanayai Meetei, Sivaji Bandyopadhyay, Thoudam Doren Singh. 967-971 [doi]
• IACS-LRILT: Machine Translation for Low-Resource Indic LanguagesDhairya Suman, Atanu Mandal, Santanu Pal, Sudip Kumar Naskar. 972-977 [doi]
• IOL Research Machine Translation Systems for WMT23 Low-Resource Indic Language Translation Shared TaskWenbo Zhang. 978-982 [doi]
• Trained MT Metrics Learn to Cope with Machine-translated ReferencesJannis Vamvas, Tobias Domhan, Sony Trenous, Rico Sennrich, Eva Hasler. 983-995 [doi]
• Training and Meta-Evaluating Machine Translation Evaluation Metrics at the Paragraph LevelDaniel Deutsch, Juraj Juraska, Mara Finkelstein, Markus Freitag. 996-1013 [doi]
• Automating Behavioral Testing in Machine TranslationJavier Ferrando, Matthias Sperber, Hendra Setiawan, Dominic Telaar, Sasa Hasan. 1014-1030 [doi]
• One Wide Feedforward Is All You NeedTelmo Pires, António Vilarinho Lopes, Yannick Assogba, Hendra Setiawan. 1031-1044 [doi]
• A Benchmark for Evaluating Machine Translation Metrics on Dialects without Standard OrthographyNoëmi Aepli, Chantal Amrhein, Florian Schottmann, Rico Sennrich. 1045-1065 [doi]
• The Devil Is in the Errors: Leveraging Large Language Models for Fine-grained Machine Translation EvaluationPatrick Fernandes, Daniel Deutsch, Mara Finkelstein, Parker Riley, André Martins, Graham Neubig, Ankush Garg, Jonathan H. Clark, Markus Freitag, Orhan Firat. 1066-1083 [doi]