Use this URL to cite or link to this record in EThOS:
Title: The mat sat on the cat : investigating structure in the evaluation of order in machine translation
Author: McCaffery, Martin
ISNI:       0000 0004 6498 9647
Awarding Body: University of St Andrews
Current Institution: University of St Andrews
Date of Award: 2017
Availability of Full Text:
Access from EThOS:
Access from Institution:
We present a multifaceted investigation into the relevance of word order in machine translation. We introduce two tools, DTED and DERP, each using dependency structure to detect differences between the structures of machine-produced translations and human-produced references. DTED applies the principle of Tree Edit Distance to calculate edit operations required to convert one structure into another. Four variants of DTED have been produced, differing in the importance they place on words which match between the two sentences. DERP represents a more detailed procedure, making use of the dependency relations between words when evaluating the disparities between paths connecting matching nodes. In order to empirically evaluate DTED and DERP, and as a standalone contribution, we have produced WOJ-DB, a database of human judgments. Containing scores relating to translation adequacy and more specifically to word order quality, this is intended to support investigations into a wide range of translation phenomena. We report an internal evaluation of the information in WOJ-DB, then use it to evaluate variants of DTED and DERP, both to determine their relative merit and their strength relative to third-party baselines. We present our conclusions about the importance of structure to the tools and their relevance to word order specifically, then propose further related avenues of research suggested or enabled by our work.
Supervisor: Nederhof, Mark-Jan Sponsor: Engineering and Physical Sciences Research Council (EPSRC)
Qualification Name: Thesis (Ph.D.) Qualification Level: Doctoral
EThOS ID:  DOI: Not available
Keywords: Machine translation ; Natural language processing ; Translation evaluation ; Dependency structure ; Evaluation dataset ; Tree edit distance ; P308.M3 ; Machine translating ; Natural language processing (Computer science) ; Translators (Computer programs)