Evaluation of MOLTO Technologies

We present the results of an evaluation of the translation quality in the MOLTO use cases, namely, the tourist phrasebook, ACE-in-GF and AceWiki, patents and mathematics. The results for cultural heritage are still pending. All evaluations were made by native or near-native level speakers of each language, 34 evaluators in all. The MOLTO translations were compared to Google, Bing and Systran, and the evaluators were to choose the best, either to accept as it is or for post-editing. Both automatic evaluation metrics and the percentage of the evaluators' preferred translations suggest that MOLTO method fares significantly better for the task; however, there is variation between languages. The time and effort needed to fix the languages that get the poorest results is a factor which we continue to evaluate.