5 months ago
November 25, 2020
New Research Flips the Script on CAT Tools — Literally
A November 2020 paper by leading machine translation researchers may impact the way future translation productivity (a.k.a. CAT) tools are designed.
The paper’s authors are Samuel Läubli, CTO of Swiss language technology company TextShuttle; Rico Sennrich, SNSF Professor at the University of Zurich; and contributors from Lilt: CEO Spence Green; Principal Research Scientist Patrick Simianer; Director of Research Joern Wuebker; and Senior Research Scientist Geza Kovacs.
Although qualitative research has suggested that segmented text may disrupt the flow of a translator’s work, the authors describe the concept of text presentation as “overlooked” in research, and set out to understand how segmentation and orientation could influence a translator’s speed and efficiency.
Prior to the experiment, researchers surveyed eight professional translators on their experiences working with CAT tools. The researchers were particularly interested in the translators’ thoughts on how documents are split up in CAT tools.
“A number of interviewees tell us that it is hard for them to translate without knowing what the source and target texts look like,” the authors wrote. Researchers offered an alternative setup to the display found on most CAT tools: “a screen showing two entire pages: the left one containing the source text, the right one being empty. In that sense, our drawing resembles the ‘print layout’ available in Microsoft Word or Google Docs, but with two parallel documents.”
Only one interviewee reacted negatively, suggesting that the proposed display was reminiscent of the days before CAT tools, when translators had no choice but to jump back and forth between documents.
Several translators mentioned that segments sometimes help them focus on their work, and that long segments can be unhelpful. Overall, however, feedback was positive, and interviewees praised the system’s ease of use and predicted that it would improve translation quality.
Test Drive
The researchers then measured the speed and accuracy of 20 professional German-to-English translators working with different text presentations on three tasks: text reproduction, error identification, and revision.
For text reproduction, researchers timed translators as they typed the source text into a target text space, and calculated the translators’ accuracy based on the number of mistyped errors per text.
According to the post-experiment survey, the majority of translators (65%) preferred top-bottom orientation overall, versus the left-right orientation most (85%) use in their daily work
To quantify error identification, researchers measured the translators’ speed and accuracy in finding errors in suggestions from the translation memory or machine translation (MT) system.
For revision, researchers focused on the translators’ accuracy in fixing errors that had been inserted into sentences.
It turns out that the best kind of text presentation depends on the task at hand. According to the post-experiment survey, the majority of translators (65%) preferred top-bottom orientation overall, versus the left-right orientation most (85%) use in their daily work.
Slator 2020 Medtech Translation and Localization Report
Top-bottom orientation of sentences was associated with faster text reproduction, while left-right orientation led to faster revision “for lexical cohesion,” using the appropriate connections between words. Compared to unsegmented text, sentence-by-sentence presentation helped translators copy text and find errors within sentences more quickly, but did not enable faster revision.
“Letting translators switch between a segmented and continuous view of the document they are translating may enable them to focus on local context and consider global context when needed,” the authors concluded, adding that this view was supported by feedback from the interviewees.
Looking ahead, the researchers recommended testing under more realistic work conditions by integrating experimental user interfaces (UI) into richer prototypes or, ideally, into full-fledged CAT tools. They noted that adding other factors, such as real-time suggestions from translation memory and/or from MT, could also be helpful.
“Future work will have to investigate whether the strong focus on single sentences in MT system outputs — and/or in the UI layout of CAT tools — has a priming effect on professional translators,” the researchers said.
They also pointed out that, as MT quality continues to improve, it could reduce the volume of writing necessary to come up with translations that are publication-ready and, in turn, increase the need for UIs optimized for revision.
Your information will never be shared with third parties. No Spam.