Human Evaluation of Machine Translation of Specialized Lexical Combinations: Testing Google Translate and DeepL
DOI:
https://doi.org/10.26512/belasinfieis.v9.n4.2020.26755Keywords:
Machine translation. Legal texts. Specialized lexical combinations. Translation evaluation.Abstract
This study evaluates the translations of specialized lexical combinations (L’HOMME; BERTRAND, 2000) produced by two different machine translation systems, i.e., Google Translate and DeepL. The selected lexical combinations have a verb+noun structure and were extracted from a legal genre known as “arbitration clauses”. A methodology for selecting the specialized lexical combinations and evaluating the adequacy and fluency of the translations is presented. The results are in line with recent research in the field of neural machine translation (NMT) suggesting that the outputs of this type of machine translation system are more “fluent” than “adequate” (KOEHN; KNOWLES, 2017). As for terminological accuracy, different terms in the source language received the same translation in the target language within the same sentence, which contradicts the idea that machine translation generates consistent results and is, thus, more appropriate for specialized translation. Nevertheless, despite the shortcomings discussed in the study, the translations of the lexical combinations were considered quite satisfactory.
Downloads
References
ARNOLD, Doug. Why translation is difficult for computers. In: SOMERS, Harold (Ed.). Computers and translation: a translator’s guide. Amsterdam: John Benjamins, 2003. p. 119-142. doi: https://doi.org/10.1075/btl.35.11arn
CASTILHO, Sheila; O’BRIEN, Sharon. Acceptability of machine-translated content: A multi-language evaluation by translators and end-users. Linguistica Antverpiensia ”“ New Series: Themes in Translation Studies, v. 16, p. 120-136, 2017. Disponível em: https://lans-tts.uantwerpen.be/index.php/LANS-TTS/article/view/430. Acesso em: 26 dez. 2019.
CASTILHO, Sheila; DOHERTY, Stephen; GASPARI, Federico; MOORKENS, Joss. Approaches to Human and Machine Translation Quality Assessment. In: MOORKENS, Joss; CASTILHO, Sheila; GASPARI, Federico; DOHERTY, Stephen (Ed.). Translation quality assessment: From principles to practice. Cham: Springer, 2018. p. 9-38. doi: https://doi.org/10.1007/978-3-319-91241-7_2
CASTRO, Marcílio M. Dicionário de Direito, Economia e Contabilidade ”“ Português-Inglês/Inglês-Português. Rio de Janeiro: Forense, 2013.
CONSTANT, Mathieu et al. Multiword expression processing: A survey. Computational Linguistics, v. 43, n. 4, p. 1-92, 2017. Disponível em: https://www.mitpressjournals.org/doi/full/10.1162/COLI_a_00302. Acesso em: 29 dez. 2019. doi: https://doi.org/10.1162/COLI_a_00302
CRYSTAL, David. The language revolution. Cambridge/Malden: Polity Press, 2004.
PÉREZ-HERNÁNDEZ, Chantal; ORTIZ, Antonio Moreno; FABER, Pamela. Lexicografia Computacional y Lexicografia de Corpus. Revista Española de Lingüística Aplicada: Panorama de la investigación em lingúistica informática, v. 1, p. 175-214, 1999. Disponível em: https://www.researchgate.net/publication/28106208_Lexicografia_computacional_y_lexicografia_de_corpus. Acesso em: 26 dez. 2019.
FIEDERER, Rebecca; O’BRIEN, Sharon. Quality and machine translation: A realistic objective? The Journal of Specialized Translation, v. 11, p. 52-74, 2009. Disponível em: https://www.jostrans.org/issue11/art_fiederer_obrien.pdf. Acesso em: 26 dez 2019.
FROTA, Maria Paula. Erros e lapsos de tradução: um tema para o ensino. Cadernos de Tradução, Florianópolis, v. 1, n. 17, p. 141-156, 2006. Disponível em: https://periodicos.ufsc.br/index.php/traducao/article/view/6859. Acesso em: 26 dez 2019.
HASSAN, Hany et al. Achieving human parity on automatic Chinese to English news translation. Relatório da Microsoft. 2018. Disponível em: https://www.microsoft.com/en-us/research/publication/achieving-human-parity-on-automatic-chinese-to-english-news-translation. Acesso em: 10 dez 2019.
HEID, Ulrich. On the way words work together ”“ Topics in lexical combinatorics. In: MARTIN, Willy et al. (Org.). Proceedings of the Euralex ‘94. Amsterdam: EURALEX, 2014. p. 226-257.
KOEHN, Philipp; KNOWLES, Rebecca. Six challenges for neural machine translation. In: Proceedings of the First Workshop on Neural Machine Translation. Vancouver (Canadá), 2017. p. 28-39. doi: https://doi.org/10.18653/v1/W17-3204
L’HOMME, Marie-Claude; BERTRAND, Claudine. Specialized lexical combinations: Should they be described as collocations or in terms of selectional restrictions. In: EVERT, Stefan et al. Proceedings of the Ninth Euralex International Congress. Stuttgart: EURALEX, 2000. p. 497-506.
MARTÃN, Beatriz. Translations quality assessment of google translate and microsoft bing translation. 2016. 45f. TCC (Graduação em Estudos Ingleses) ”“ Faculdade de Filosofia e Letras, Departamento de Filologia Inglesa, Universidad de Valladolid, Valladolid, 2017. Disponível em: https://uvadoc.uva.es/bitstream/handle/10324/22596/TFG_F_2017_7.pdf;jsessionid=153C84743864859CEF93398BDB9D818A?sequence=1. Acesso em: 26 dez 2019.
MUNDAY, Jeremy. Introducing translation studies: Theories and applications. 3. ed. Londres: Routledge. 2012. doi: https://doi.org/10.4324/9780203121252
MUZII, Luigi. Quality assessment and economic sustainability of translation. Rivista internazionale di técnica della traduzione, v. 9, p. 15-38, 2006. Disponível em: https://www.openstarts.units.it/bitstream/10077/2891/1/ritt9_05muzii.pdf. Acesso em: 27 dez. 2019.
ORENHA, Adriane. Unidades fraseológicas especializadas: colocações e colocações estendidas em contratos sociais e estatutos sociais traduzidos no modo juramentado e não-juramentado. 2009. 290f. Tese (Doutorado em Estudos Linguísticos) ”“ Instituto de Biociências, Letras e Ciências Exatas, Universidade Estadual Paulista, São José do Rio Preto, 2009. Disponível em: https://repositorio.unesp.br/bitstream/handle/11449/103524/orenha_a_dr_sjrp.pdf?sequence=1&isAllowed=y. Acesso em: 27 dez. 2019.
PIMENTEL, Janine. Specialized verbs and specialized uses of verbs in a comparable corpus of judgments produced in Canada, Portugal and Brazil. In: DROUIN, Patrick et al. (Org.). Multiple perspectives on terminological variation. Amsterdam/Filadélfia: John Benjamins, 2017. p. 109-130. doi: https://doi.org/10.1075/tlrp.18.05pim
RAMISCH, Carlos. Multi-word terminology extraction for domain-specific documents. Advisor: Christian Boitet. 2009. 77f. Mémoire (Master de Recherche en Infromatique ”“ Option Intelligence Artificielle et Web) ”“ Institut National Polytechnique de Grenoble, Grenoble, 2009. Disponível em: https://pdfs.semanticscholar.org/79b6/a43105df36fe6e507adc8fa67dfa4d9f9d19.pdf?_ga=2.54638955.1171411335.1577447828-749771379.1577447828. Acesso em: 27 dez. 2019.
SAG, Ivan A.; BALDWIN, Timothy; BOND, Francis; COPESTAKE, Ann; FLICKINGER, Dan. Multiword expressions: A pain in the neck for NLP. In: INTERNATIONAL CONFERENCE ON INTELLIGENT TEXT PROCESSING AND COMPUTATIONAL LINGUISTICS, 3., 2002, Cidade do México. Proceedings… Cidade do México: [s.e.], 2002. p. 1-15. doi: https://doi.org/10.1007/3-540-45715-1_1
SYCZ-OPOŃ, Joanna. Machine translation ”“ Can it assist in professional translation of contracts?, International Journal of Legal Discourse, v. 20, p. 81-200, 2014. Disponível em: https://pressto.amu.edu.pl/index.php/cl/article/view/6524. Acesso em: 27 dez. 2019.
TAGNIN, Stella. Os corpora: Instrumentos de auto-ajuda para o tradutor. Cadernos de Tradução, Florianópolis, v.9, n. 9, p. 191-213, 2002. Disponível em: https://periodicos.ufsc.br/index.php/traducao/article/view/5986/5690. Acesso em: 27 dez. 2019.
TEIXEIRA, Michel. O jogo da avaliação: um estudo prático sobre tradução automática. 2018. 134f. Dissertação (Mestrado em Estudos da Linguagem) ”“ Programa de Pós-Graduação em Estudos da Linguagem, Pontifícia Universidade Católica do Rio de Janeiro, Rio de Janeiro, 2018. Disponível em: https://www.maxwell.vrac.puc-rio.br/41711/41711.PDF. Acesso em: 27 dez. 2019.
VILLAVICENCIO, Aline; RAMISCH, Carlos. Chutando o balde ou batendo as botas? Processamento de linguagem natural e expressões multipalavra na linguagem cotidiana e científica. In: PERNA, Cristina; DELGADO, Heloisa; FINATTO, Maria José (Org.) Linguagens especializadas em corpora modos de dizer e interfaces de pesquisa. Porto Alegre: EDIPUCRS, 2010. p. 29-49.
WAQUIL, Marina. Tradução de textos especializados: unidades fraseológicas especializadas e técnicas tradutórias. 2013. 206f. Dissertação (Mestrado em Letras) ”“ Programa de Pós-Graduação em Letras, Universidade Federal do Rio Grande do Sul, Porto Alegre, 2013. Disponível em: https://www.lume.ufrgs.br/bitstream/handle/10183/72737/000883531.pdf?sequence=1&isAllowed=y. Acesso em: 27 dez. 2019.
WU, Yonghui et al. Google’s neural machine translation system: Bridging the gap between human and machine translation. ArXiv, p. 1-23, 2016. Disponível em: https://arxiv.org/pdf/1609.08144.pdf. Acesso em: 10 dez. 2019.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2020 CC BY
This work is licensed under a Creative Commons Attribution 4.0 International License.
Given the public access to this journal, the texts are free to use but requires the recognition of the original authorship and initial publication in this journal to be properly stated.
 The journal allows the use of works published for non-commercial purposes, including the right to submit the work to publicly accessible databases. Published contributions are the sole and exclusive responsibility of the author(s).Â
