This article is about how Crowdin sees the future of translation and the role of LLM in it.
I recently came across an article by Arnaud Rinquin of Slite entitled “How we skipped conventional translation tools“. This article caught my attention because Crowdin is a translation tool. For now, let’s just use one fact from the article: it has been proven that LLM can produce 95% of ready-to-publish translations in real-life UI localization projects. Conventional NMT did about 30%.
We’ve seen similar results in our synthetic experiments. I’ve referenced Slite to make what follows more credible.
How did such a huge leap in efficiency happen with a technology that was deliberately not designed for the translation industry? Well, one way is by letting humans give context when instructing the machine to do translations.
Google Translate knows language perfectly. It knows far more words in English than I do (or any human), but it still struggles to translate “Name” from English to Polish. This is because translation depends on context. It’s either Nazwa (a city name) or Imię (a person’s name). There’s no way for you to provide that context to Google Translate, so the machine makes the assumption. Where you have assumptions, you should expect mistakes.
UI localization projects, in particular, are full of short texts that are impossible to translate without context.
You may have seen a prediction on social media that in 5 years’ time, every person will have a personal AI assistant. This assistant will know a lot about you, who you are, and what you do and thus be very efficient in helping you with your daily tasks.
At Crowdin, we believe that every multilingual company will have multiple AI assistants, one of which will be their own AI translator. A machine that will be fine-tuned for that specific company, knowing their preferences, and what kind of translations they expect for each target audience.
- I’m hoping/assuming that in 5 years, LLM will not have advanced to the point where personalized content is generated for a website visitor or mobile app user in real time.
- I assume that an enterprise would still require a human to review AI translations for mission-critical content, even if they know there will be little or no changes needed.
Now, with this information, the team at Crowdin was thinking, what should the modern translation tool look like today? How can we make general-purpose LLM more useful for translation tasks?
If in some localization projects, AI can produce 95% publishable translations, how do we replicate that success for many Crowdin customers?
How can we let our enterprise customers start training their tailored AI translator today so it improves over time?
This second part of this post is more of a guide on how to use AI in Crowdin. I have to admit that the technology described below is quite new, intentionally built with compromises to get it released quickly. The technology has been proven to work as described, though. The purpose of this guide is to show how the transformation of the translation workflows on a buyer’s side and introduce the concepts that will be used in future localization projects.
Context is a King Again
Just as Slite’s experiment and many of our internal experiments have shown, the more context humans provide to the machine, the better quality translations can be expected.
Crowdin gives our customers plenty of ways to provide that context.
– Project level
– File level
– Key/string level
– Translator session level
A project-level context is what’s often called a “prompt”. A localization manager starts with a pre-configured prompt and can modify and extend the prompt with all relevant information about this project. Before executing the prompt, Crowdin will populate it with the necessary context, such as translation memory matches, glossary terms, and even translations to other languages that were approved by linguists, etc.
The best prompt should include information about your domain, the company, the product, consumer, the more, the better.
Then, a file context. This is most important when translating content like Zendesk articles or a Word document with your app’s release notes. If you have more than one file in your Crowdin project, there needs to be something you can tell the machine about the content of those files and what kind of translations are expected.
Key/string level context is crucial for UI localization projects. Texts in the UI copy are short, even a human would struggle to translate them without sufficient context (textual, screenshots, or in-context translation tool). It’s a good idea to provide string-level context even in content-heavy projects such as website translation projects. Texts on the main page of the website that appears between images would need some additional context for the machine to translate them best.
The session level is for linguists, not project managers. Linguists working with Crowdin editor have a chat with the LLM model of your choice and can ask for assistance whenever needed. The more context linguists provide, the better help they get.
There’s one more thing we recommend you do. Crowdin AI Assistant has an option to “send a whole file” when requesting translations. Even if that’s a translation for a single segment of a big file, this option might affect your budget, but very often, a whole file contains a lot of context, and the LLM can produce consistent translations with translations you already have in that file.
One more thing, which is not a feature but rather a recommendation. If you, as a Crowdin project manager, are bilingual, we recommend that you experiment with prompt and context many times until you see translations are “good enough”, at least for a few files out of a big project. Having that, you are ready to pre-translate other target languages and invite proofreaders for your target locales.
Providing context in the way the machine expects is the actual workflow change. Previously, when buying translations from the agency, a human on the other end would organically get a lot of context about your project. If you were talking to a game localization company, you never had to explicitly say, “I am localizing a mobile game about zombies for teenagers”.
UI localization projects have got even more changes to the workflow. Now a content designer or developer has to provide context for every title of the button they create. I mean, this should have been done even when humans were doing the translations, but in the AI era, developers would have to spend more time describing the use of each key.
If you succeed with the tools described above and provide enough context, I cannot immediately guarantee 95% good translations, but the percentage would definitely be much higher than the 30% expected from NMT.
Company AI Translator
Even if your localization project achieves 95% valid translations after providing sufficient context. What about the remaining 5%?
In the recent months, LLM vendors have started to offer a fine-tuning feature. Remember when ChatGPT asks for a “thumbs up” or “thumbs down”? That is exactly what fine-tuning is.
In the case of translations on Crowdin, an approved translation from LLM is considered a thumbs-up, if a human made edits, it’s a thumbs-down, plus the edited translation is sent back to the LLM model.
Crowdin’s AI assistant allows you to fine-tune a stock LLM model by exposing your existing TMs and glossaries. This fine-tuning is a great way to show the machine the kind of translations you expect and the terminology you use.
Best of all, fine-tuning can be done incrementally. Any time you have a significant amount of new entries in your TM or glossary, you can do the fine-tuning or the previously fine-tuned LLM model, and we expect it to happen in real-time sometime soon (with no need to fine-tune once a week manually).
We believe this is a way to create a customized AI translator for every company. The more edits humans make, the better translations LLM is expected to deliver in the future.
Data Security and LLM Models
When we presented the above solution to our early adopters, especially enterprise customers, the first concern that popped up was security.
Quick answer: Crowdin allows you to bring your own API key from your LLM provider. Most LLM providers like OpenAI or Microsoft Azure would have a privacy framework for enterprise customers. The agreement that would regulate how the customer’s data is treated when exposed to the LLM.
We deliberately decided to ask our enterprise customers to bring their own API keys. Except for addressing the privacy issue, Crowdin encourages our customers to fully own the LLM models trained while using Crowdin. This way, we do not create a vendor lock for a client. And, of course, the price, Crowdin would not charge clients extra for using LLM in Crowdin.
For small localization projects and users who are on our free subscription plan, and for companies that just want to experiment with AI on Crowdin, we offer a $5 credit with the possibility to top up the balance to enjoy the managed LLM models, which are fine-tuned to be a generally better translator than the stock models.
Final Thoughts and Other Concerns
This article is even more optimistic than I expected. What I have learned over the years is that reality often breaks our successful lab experiments. In fact, the current reality shows that the APIs of most LLM vendors are just not reliable yet for large translation projects. Crowdin integration is a new, frequently updated technology that is not even well documented. The overall workflow is quite different, and rebuilding the process on the client side is tedious.
But I’m a big believer that all the rough edges will be polished at a tremendous speed. I would encourage all of our customers to start experimenting and looking for ways to take advantage of the new technology as soon as possible.
All the above technologies are immediately available on Crowdin. Crowdin offers a 24×7 tech support team that is happy to consult you with your AI adoption.
- When using LLM, provide enough context
- Start fine-tuning the LLM model for your business early
- Keep your data secure