European researchers are getting more prolific in their work on machine translation. The June 2017 issue of The Prague Bulletin of Mathematical Linguistics (PBML) featured 29 research papers, 20 of which are on machine translation (MT).

The peer-reviewed journal is published by the Institute of Formal and Applied Linguistics (ÚFAL) of the Computer School of Charles University (formerly the University of Prague), the oldest university in the Czech Republic. It has published research on linguistics since 1964, covering topics such as machine translation, corpus linguistics, morphology, syntax, and semantics, among others.

The current issue was released on the heels of the annual conference of the European Association for Machine Translation (EAMT) held in Prague from May 9 to 21 and hosted by ÚFAL, a well-known MT research group.

Research papers on machine translation (MT) jumped from six in 2016 to 20 this year, marking a 70% increase. Studies on neural machine translation have also appeared in the journal for the first time. Of the 20 papers, 12 are papers on neural MT.

In the current issue, a total of 85 researchers, mostly from European universities and research institutions, have contributed to the collection, which is the largest to date since the journal started publishing the electronic version in January 2007.

“It’s a solid conference and the quality of the work is always very good,” said Dr. John Tinsley, CEO & Co-Founder, Iconic Translation Machines Ltd., and a co-author of a research paper on neural MT presented at the conference.

MT Tour D’Horizon

The published works were an interesting mix of approaches to advancing MT, such as addressing the problems of modelling and training neural networks through improved algorithms, augmenting recurrent neural network (RNN) encoders with additional layers of convolutional neural network (CNN) encoders to capture a wider context, and extending Natural Language Processing (NLP) applications to benefit MT systems.

Some studies have delved into new approaches for specific language pairs, including papers on vocabulary reduction for neural MT from Turkish to English, rule-based MT for the Italian–Sardinian language pair, and comparing language related issues for NMT and PBMT between German and English.

One prolific contributor to the journal is Dublin City University’s ADAPT Centre for Digital Content Technology, which published six papers authored by 18 of its researchers, led by its Deputy Director, Professor Andy Way.

The center published four papers on neural MT, and two on the quality of MT of user-generated content such as social media posts, and the use of Machine Translation Quality Estimation (MTQE) for improving professional translation workflows (a joint undertaking with researchers from the University of Wolverhampton in the UK).

Senior researchers from the German Center for Artificial Intelligence (DFKI), including Georg Rehm, Network Manager of META-NET, an EU-funded Network of Excellence consisting of 60 research centers dedicated to building the technological foundations of a multilingual European information society, have also contributed four papers to the journal.

Many other papers were results of collaborations between universities and top research institutions or commercial translation companies.

One such collaboration is the paper on how MT may be improved by learning from human post-edits, a solution that remains unexplored in the neural MT framework, according to the research authors from Fondazione Bruno Kessler (FBK) and the University of Trento in Italy. The paper noted that continuous learning from a stream of incoming post-edits could be a promising solution for cutting costs in retraining neural MT models from scratch.

Meanwhile, top executives and researchers of language service provider (LSP) KantanMT, led by its CEO Tony O’Dowd, have teamed up with Anita Ramm of the University of Stuttgart in Germany for a paper which presented an open source tool for a commercial Statistical Machine Translation (SMT) platform.

Cautious not to Promise too Much

Despite the buzz around on NMT, however, one paper, in particular, poured a little bit of cold water on the excitement.

The paper, entitled “Is Neural Machine Translation the New State of the Art?”, studied three use-cases of NMT and compared it against statistical machine translation (SMT) systems where the evaluation was also performed by humans – NMT for E-Commerce Product Listing, NMT for Patent Domain, and NMT for Massive Open Online Courses (MOOC).

Research authors Sheila Castilho, Joss Moorkens, Federico Gaspari, Iacer Calixto from the ADAPT Centre at Dublin City University, and Dr. John Tinsley from Iconic Translations, noted that “although promising results are being reported when comparing NMT with other MT paradigms using automatic metrics, when human evaluation is added to the comparison, the results are not yet so clear-cut.”

 “There are also limitations for the neural models that cannot be overlooked and still need to be addressed”

“NMT no doubt represents a step forward for the MT field. However, there are also limitations for the neural models that cannot be overlooked and still need to be addressed,” the paper further stated. “In our view, at this stage, researchers and industry need to be cautious not to promise too much, and allow for more research to address the limitations of NMT and more extensive human evaluations to be performed, addressing as many text types, domains and language pairs as possible.”

Tinsley told Slator that the paper was based on a collaboration carried out between his company, Iconic Translation Machines, and the ADAPT Centre at Dublin City University in Ireland.

“In it, we carried out a case study with one of our clients to compare an existing production Chinese-English MT engine with a neural MT engine,” he said, adding that they’ve since expanded the case study to include Korean and Japanese.

“The findings still stand,” he noted. “There are some cases where neural MT can be effective in the short term, depending on the nature of the project (content type, language, training data), but there are also cases where existing approaches are more than fit for purpose for now.”