The quote probably no one was expecting from SlatorCon Remote March 2023 came during the Intento panel on GPT moderated by Slator Co-founder Andrew Smart.
“It’s so yesterday,” said Intento Co-Founder and CEO Konstantin Savenkov. But of course, context is everything; more specifically, GPT-3 is “so yesterday” relative to its latest successor, GPT-4.
In another nod to the past, panelist Jon Ritzdorf, Senior Manager of Global Content Solutions at Procore, pointed out the similarities of the current discourse around GPT to that surrounding neural machine translation (MT) back in 2016, complete with concerns about privacy, speed, accuracy, and reliability.
“We’re literally having the exact same discussions,” Ritzdorf said, adding that he believes these issues will be resolved quickly, as they were with MT: “In fact, because there are even more eyes on this than there were on neural MT, they’re probably going to solve them a lot faster.”
Indeed, in more ways than one, speed is likely to be a major factor in the practical adoption of GPT in workflows, pointed out VMware’s Senior Technical Localization Program Manager, Martin Xiao.
Xiao’s internal assessments show that Google Translate is stronger than ChatGPT, thanks in part to the fact that VMware already trains Google Translate with its own data. But inference speed also counts, Xiao explained. Google’s API takes just milliseconds to respond, while ChatGPT takes several seconds.
“That’s not acceptable for a production environment because there are thousands and millions of strings that need to get the machine translation in one second or several seconds,” Xiao added.
Time saved is a major factor in deciding how to modify workflows. Procore, which Ritzdorf described as Salesforce for the traditionally pen-and-paper construction industry, has been experimenting with using GPT to audit source content.
Once GPT identifies and rewrites source segments that could be problematic for the target market, the text can be run through MT or sent to human translators. Savings in time — and money — multiply very quickly, once users take the time to train GPT.
“Could you imagine, suddenly, three million words across the board are scanned and changed and grammatically correct and done correctly in context,” Ritzdorf said. “What used to take three months now just takes minutes, probably.”r
Free Form Craziness
For all the praise it has earned, OpenAI acknowledges that GPT-4’s MT currently maxes out at 85.5% accuracy.
“Where MT is probably still very good at structured stuff, GPT might be a little better at the free-form craziness that we all really use in real life,” — Jon Ritzdorf, Senior Manager of Global Content Solutions, Procore
That is “not even close to [a] human level of translation,” Xiao said. But with a better algorithm to improve MT quality to a score of over 95% accuracy, he added, the impact on the language industry could be significant.
Xiao said that he has used a Transformer to train a GPT model to automatically post-edit MT, and has tried to fit ChatGPT with VMware’s style guide and terminology — both with good results.
Slator Machine Translation Expert-in-the-Loop Report
And beyond MT, panelists emphasized, there seems to be much potential. Based on feedback from the Procore support team on GPT-generated examples of Spanish chat logs, Ritzdorf believes GPT might be a good candidate for free-form content — chatting, talking, and texting.
“Where MT is probably still very good at structured stuff, GPT might be a little better at the free-form craziness that we all really use in real life,” he said.
Savenkov agreed, adding, “It actually creates pretty good localized copy using local Idioms and local context.”
Users, then, can choose to tap into one of two immediate potential benefits of GPT: low-cost MT or highly automated source quality improvement and post-editing. Either way, the language industry benefits from a huge increase in the amount of content created. While someone needs to translate it, “less impactful content” is unlikely to be handled exclusively by humans.
In the face of a well-worn narrative that threatens to scare language professionals away from GPT and similar models, Ritzdorf, a part-time adjunct professor of localization, said that the current challenge is training the next generation to embrace language technology.
“For too many years we’ve had this hammer, and so all we see is nails. Now we finally have a whole toolkit available,” Ritzdorf explained. “How do we train them to take language technology and apply it to thousands of different problems?”
For those who missed SlatorCon Remote March 2023 in real-time, recordings will be available in due course via our Pro and Enterprise plans.
Your information will not be shared with third parties. No Spam.