Swiss Science Foundation Grants USD 0.5m to Take Neural MT Beyond the Sentence

Switzerland’s National Science Foundation is one of the Swiss government’s main vehicles to support innovation. In 2016, the foundation supported over 3,200 new projects, funding them to the tune of nearly CHF 0.93bn (USD 0.95bn).

The projects run across all categories with CHF 264m allocated to the social sciences, CHF 337m allocated to math, physics, engineering etc., and CHF 334m allocated to biology and medicine.

Getting funded is a highly competitive process and a successful application requires jumping through a number of admin hoops as well as convincing the foundation of the value of the research.

One of the projects that successfully navigated the funding maze was “Rich Context in Neural Machine Translation” (CoNTra) by Martin Volk and his collaborators Annette Rios and Rico Sennrich. The project is set to receive CHF 0.46m over the course of the next three years.

Volk is a professor at the Institute of Computational Linguistics of the University of Zurich. It is not the first time Volk received funding from the Foundation. In 2010, 2013, and 2014, he obtained nearly CHF 1.6m for three machine translation-related projects.

Beyond the Sentence

Slator met with Volk and Samuel Läubli, a Ph.D. student at Volk’s Institute, in Zurich to discuss the project and how the recent boom in artificial intelligence (AI) is affecting many areas in natural language processing, including machine translation.

In addition to their collaboration at the University of Zurich, Volk and Läubli are partners in a spin-off company called TextShuttle, which develops custom machine translation (MT) engines and advises enterprises on how best to leverage MT, including neural MT. 

Volk told Slator that the inherent ambiguity of natural language is the core challenge for all things machine translation. That is why one of the CoNTra project’s main goals is to figure out how to infuse machine translation with much more contextual information beyond the sentence level.

Volk explains: “When Google Translate has to translate a text as someone is inputting it, the system has to split up the text by the sentence and send each sentence to different processing units.

Due to the speed at which the translation has to be served up, those units have no way of communicating with each other and, hence, are unable to process context that goes beyond the sentence level.”

“For us, speed is not the key. We want to drill down on how information from sentences preceding and following the one being translated can be used to improve the translation,” Volk says.

While neural machine translation’s by now well known improved fluency is partially achieved by its ability to include more context within the same sentence, NMT does not go beyond the sentence level yet either, according to Volk.

The abstract for CoNTra’s project submission explains the challenge in somewhat more technical language: “It has been rather cumbersome to include document-level information into SMT systems. Neural MT provides a more elegant and straightforward way to include lexical preferences, which have been derived from neighboring sentences. This will help to improve lexical choice which is still one of the major problems in machine translation.”

What makes Volk’s project different from many other NMT research projects is its language focus. “Everyone does English,” Volk says, so they decided to focus on Switzerland’s national languages – German, French, and Italian – first.

Pings on LinkedIn

How does the current AI boom impact computer linguistics? Volk says that they had been doing NLP for two decades in relative quiet but the current AI boom definitely brought a lot of new researchers into the field. “There is a lot more competition now,” he observes. On the flip side, Volk says his graduates’ job prospects have improved as well.

Ph.D. student and TextShuttle partner Läubli concurs. “You definitely notice it when you log on to LinkedIn. There are a lot more requests along the lines of ‘hey, great opportunity with company X’.”

Gradually Approaching Human Quality

Volk’s take on future of machine translation? “I think that as computing power inevitably increases and neural learning mechanisms improve, machine translation quality will gradually approach the quality of a professional human translator over the coming two decades. There will be a point where in commercial translation (not literature) there will no longer be a need for a professional human translator.”

“Ten years ago I would have probably been more cautious in my assessment,” he continues. “But looking at what these systems can do and how large the corpus of existing translations has become, I don’t see a reason why these learning mechanisms should not enable computers to achieve human translation quality at some point.”

Meanwhile, Volk’s CoNTra project has been ongoing for over six months and is slated to end in 2019. No doubt, much progress will have been made by then.

Image: University of Zurich and Federal Institute of Technology (ETH) at night