Implementing Language AI in the Enterprise with Translated’s John Tinsley

SlatorPod #198 - Translated AI Solutions with John Tinsley

John Tinsley, the VP of AI Solutions at language AI agency Translated, joins SlatorPod to talk about the challenges, advancements, and future directions of AI in the language industry. 

John shares his journey from founding machine translation (MT) pioneer ICONIC, selling it during the height of the pandemic to RWS, and now his current role focusing on connecting technology and capabilities with customer needs at Translated.

He touches on the challenges of managing the noise around AI and the excitement and potential of generative AI, particularly in the context of language. He discusses the impact of large language models on translation and the challenges of multilingual content generation.

John mentions the importance of having the right data for AI and highlights a new product initiative called Human-in-the-Loop. This initiative focuses on automating the process of improving MT by constantly fine-tuning it based on user feedback and human data.

Subscribe on YoutubeApple PodcastsSpotifyGoogle Podcasts, and elsewhere

He also explores the dynamic landscape of innovation in the AI field, discussing the sources of innovation, the role of big tech companies, and the challenges of keeping up with the rapidly evolving research landscape.

Looking ahead, John underscores the importance of ensuring enterprise readiness in MT, considering factors beyond just good output, such as fitting into existing workflows, cost-effectiveness, and scalability.

Transcript

Florian: Today on the pod, we welcome John Tinsley, so John is now the VP of AI Solutions at language AI agency Translated. Hi, John, and thanks for joining.

John: Hey, Florian, as we say, long time listener, first time caller to SlatorPod.

Florian: Just give us a bit of a background about your entrepreneurial career and trajectory and some of the milestones, from Iconic to RWS to Translated, et cetera.

John: Yeah, and apologies if I get the years wrong, because the last four years have just blended into this kind of blob of not being able to remember, was that 21, 22? Yeah, so long story short, I co-founded a company called Iconic Translation Machines back in 2012 as a spin out from Dublin City University, and essentially we built machine translation, custom machine translation engines for enterprise. We raised a few rounds of funding, we chugged along, kept the lights on, and then, as you said, 2016, 2017, we had this big paradigm shift with neural MT, and that really caused the company to take off. So we were one of the first commercial providers of neural MT, along with Translated, ironically, so that was a good time. So we continued to grow the business from there and then ultimately we sold it to RWS in the middle of 2020, actually, at the height of the pandemic. It was an unusual time, for sure, and then things kind of evolved from there. Not long after, RWS bought SDL, who also had a big machine translation team. So I basically spent a year and a half there working on the integration of Iconic machine translation and what was formerly SDL’s machine translation, which has since been rebranded as Language Weaver, and then I left once that integration was complete. I took some time off, traveled around, did some kind of work locally, some investing, some advising of businesses in completely different spaces to kind of get my head clear, and then ended up coming back, and I started with Translated, then back in April of last year. So I’ve been here nine months at this stage. It’s been a fun journey and kind of, not what I’m doing today, not tremendously different than what I was doing before, but obviously in a different landscape, and so that’s kind of what’s kept it exciting.

Florian: When I saw that you came back to the industry, I’m like you can truly not leave. You can check out, but you can never leave because you’d been away for a couple of years. I follow you on Twitter, and I saw some of the other stuff you were doing. I’m like, yeah, that seems like he’s slowly detaching, but then, boom, you’re back at Translated. So what motivated your return to kind of machine translation, language AI, and to Translated?

John: As you say, I never really went away. I was there in the background scheming, but very, very legitimately it was FOMO. If nothing had changed, if it was still neural MT, I would have been like, I’ve done that, box checked. I’ll do something else. I’ll do broadband. I’ll do coffee. We’ll see what it is, but it was really a case of the generative AI. That’s first check on saying AI today. The ball got rolling on that, and it was getting faster, and things were evolving, and I wasn’t at the coalface of it for the first time, and I really felt I was missing out. And there is an element of the longer you’re away, the more distant you are in the memory. So I was kind of thinking, I’ll keep my eyes and ears open just in case there’s anything, or maybe I’ll do something myself. Let’s play it by ear, but I think Marco had similar ideas, and he reached out to me before I took some time off and just kind of planted a little seed in my head. And I’ve known Marco and the Translated team, Alessandro, Claudia, for a long, long time, and I know that the technology that they had here is proprietary. It’s really cutting edge. It was the closest thing I thought in the market to what I did before. Like I said, we were both kind of moved into the neural space at the same time, and I thought this would be a fun place. So actually, as a bit of trivia, the Matecat product, so Translated’s CAT Tool, originally came out of a European Commission funded project back in 2010 to 2012. And I was one of the independent reviewers for the European Commission who reviewed Matecat. So I basically kind of, let’s say, was advising on how this original prototype of the product was getting built. That’s where I met the guys, and I knew them, I liked them, used to meet them at events, and, yeah, one thing led to another, met with Marco a few times, talked about the role, talked about the vision, and I bought it, and so here I am.

SlatorPod – News, Analysis, Guests

The weekly language industry podcast. On Youtube, Apple Podcasts, Spotify, Google Podcasts, and all other major platforms.

SlatorPod – News, Analysis, Guests

Florian: As VP of AI solutions at Translated, what are some of the key problems you’re working on? Are you working mostly kind of in-house or like client-facing, or just describe the role and just outline some of the key things that keep you busy at the moment?

John: Yeah, it’s a really enjoyable role. It faces both directions. I see it like it’s a connective role, so I’m connecting our technology and capabilities that we have in-house with our customers because we have products and we have services and the off-the-shelf stuff. But then customers come with interesting challenges and we don’t necessarily have something today, but we could build it or we could stitch a few things together, so solutionizing. So doing a lot of that, and then building kind of bridges internally between teams, because you have the sales, marketing who are at the coalface with customers, and you have product and engineering internally and kind of passing the information that we’re hearing from customers, that we’re seeing in the market, making sure that makes it into the products, making sure that makes it into the marketing materials, into the hands of our salespeople so they can have those conversations with customers as well. It’s super dynamic. It was even the same when I did machine translation back in the day. You think this is going to be the use case, and it is for a while, but then something else comes up and the technology evolves, and obviously the technology landscape today is ever evolving. And so keeping on top of that, bringing that to the table as well, or managing expectations, or having conversations with customers about what is ready and what’s not ready and where we are with a piece of technology, I think is a really important part of the role as well, so that’s what fills my time.

Florian: You mentioned FOMO. So since you came back and we had this LLM, I don’t know, revolution or whatever, I guess we can call it revolution for once. It’s not actually exaggerated. Since we have this LLM, GPT-4 level technology now, what do you think, in your view, were like the top two, three, four things that have changed for machine translation, for kind of the narrow use case of machine translation?

John: Not to give a facetious answer as the first answer, but the biggest change has been the level of noise. Just like we had this exact conversation about neural MT, what was it six, seven years ago, and then trying to manage the hype and manage the expectations. Whatever the noise was back then, around that paradigm shift, it’s orders of magnitude louder today. So actually trying to cut through that and communicate effectively to a colleague, to a prospective customer, to an existing customer, what is real, what is near term, what is still down the line is the biggest impact that it’s had, to be honest. In terms of actually what we’re delivering day-to-day, in terms of solutions, technology, it hasn’t changed that much yet in terms of what we’re actually selling, let’s say. In terms of what we’re doing internally, it’s changed massively because now we have basically like a SWOT team. So we have our engineering team, our product team that works on the machine translation product, but we kind of have an R&D, SWOT team that’s working on, okay, what is our product going to look like in six months and 12 months? What elements are going to be enterprise-ready versus not enterprise-ready? And I think that’s been kind of the biggest change day-to-day. But what it has done from that landscape of research and development, it’s removed the guardrails. Like when you’re doing research on MT, neural MT, transformers, whatever the case may be, there’s a few things that you can try in the context of translation. Now it’s like the sky is the limit. Yeah and I think you’re aware of some of the conversations that are going on out there. Why create the content and then have a separate step that translates it? Why not just create it in the languages that we want it in? So I think that’s going to be a key direction in the future. A key question that this industry, an existential question that this industry is going to have to ask is the difference between content creation and localization, and where do they meet and do they become one thing? Let’s say, to summarize, there’s noise that we deal with in communication. There is practical stuff that we do today from a development perspective and then there’s the forward-looking stuff, like what is this going to mean if and when it reaches a certain point? We, this industry, are kind of in a privileged position. I don’t think people realize it as much. We are at the forefront of all of this stuff, all this generative AI, LLMs. The biggest challenge that they have are language. The second L in LLM is language. When we talk about transformers in neural machine translation, the T in GPT stands for transformer. All these problems started, all these technologies started with the problem of language, and we’re in a super exciting time about working out how we can use it for ourselves, for customers, and it’s really exciting, and I would hate not to be involved in it.

Florian: You mentioned enterprise-ready and also just the crazy hype. So do you think there’s this kind of misperception that a lot of the stuff that you’re seeing on Twitter or in the media or on LinkedIn and that people have these lip-sync things and all of that, do you feel there’s maybe a lot of consumer-grade stuff that people assume is just going to be ready for massive enterprise adoption and assume that this is ready now? And is this enterprise making things enterprise-ready, is that going to be a long, long journey and full of complexity? Or do you think there’s a misperception by all this kind of short, clippy hype stuff that’s out there now?

John: Yeah, but I mean, that’s how you capture the attention, that’s how you capture the imagination. It’s with the snappy stuff like, wow, look at that, it translated those few sentences really nicely. Or look at that, it turned me from speaking English into speaking Swahili, and it got my lips kind of right, that’s really neat. Do that at scale, how much does that cost to do at scale? Can it even be done at scale? Or are these models so big and so slow that it’s not possible? They’re the questions that obviously don’t get asked at the start, and so when things are on the hype cycle and they’re creeping up to that apex, they’re not really the questions that we care about. It’s the wow factor. It’s the stuff that builds the hype, and then we have to become a little bit curmudgeonly and start to say, yeah, but now let’s think about the real world application. How is this actually going to work in practice? Even if you think about, so forget about things like speed and cost, you just look at like interfaces. So, yeah, GPT-4 can translate, but it’s not built like a translation interface. You don’t have the source box where you type your content and you get your translation back. You have to write a prompt to tell the thing to do the translation for you. And how you write that prompt has a huge impact on the quality of the result that it produces because it’s generative and so even just things like that. So that’s why prompt engineering has been the hot topic in more recent times because without having to do anything fancy around models and training and things like that, you can have a huge influence on the output just by the way you ask the question. So that will max out at some point and then be like, okay, everyone’s refined the prompts, but maybe refining the prompt means you have to write a lot, and that costs a lot and makes it a lot slower, and then you get into optimizing. So I think we’re still at a peak of the cycle, but we’re getting to a point now where people are starting to ask questions. And I think if you were to do a survey and ask big corporations, you’ve kicked the tires on AI, you’ve done little integrations, nice apps, who is actually using it at enterprise scale and for what tasks? And I think if you ask that question where the task is machine translation, having replaced a dedicated enterprise machine translation solution, I don’t think you’ll find many people that have done it for anything except a few sentences here and there, or something super that doesn’t require low latency, or it’s not going to cost an arm and a leg, and I think that’s where we are. And there’s still a lot of explaining around that that yes, it can translate and it can do it quite well in some cases, but actually, you, enterprise customer, think about your scale and it falls apart quickly. And that’s not to say that it won’t get there, and then we’re into that question that everybody always asks, which is when with these things, when will it get there? When will it become cost effective? But we’re still in that stage where everything’s so dynamic. How many versions of GPT have there been or exist? And they all have different cost models and now we’re starting to see this wave of open-source LLMs that are tuned for language-specific, but for some languages. So we’re still in that phase where, and it happened with neural MT as well, right, where there were so many different open-source tools from Facebook, from Google, Amazon had one, there was like open NMT from Harvard. And it took a while before the dust settled and kind of people started to migrate towards one or the other, and I think we’ll see something similar here with translation. But there’s a very strong chance that enterprise, good enterprise neural machine translation as it is today, will still be around for some time yet. But as I like to say, I wrote it on a LinkedIn post not that long ago, if you’re using neural machine translation, you’re using pretty sophisticated AI already. It’s not generative AI, but it’s still pretty sophisticated, and it’s based on very similar technology to the GPTs and the other LLMs that you’re seeing today.

SlatorCon Silicon Valley 2024 | $ 1,340

SlatorCon Silicon Valley 2024 | $ 1,340

A rich online conference which brings together our research and network of industry leaders.

Buy Tickets

Register Now

Florian: Do you see any traction for this prompt and then multiple language output anywhere so far, or is it just more kind of a playful idea?

John: Again, not at any sort of scale, but the big challenge currently with multilingual and LLMs, it’s only multilingual for a relatively, it’s only good multilingual for a relatively small set of languages, and then the quality very, very quickly drops off a cliff because one way I’ve been describing it is that the big models that are out there are arguably multilingual by accident, by default. They weren’t trying to be multilingual. In some cases, non-English data was actually filtered out as noise, but because there was so much web data in there, it got in there anyway, and it managed to actually be able to translate pretty well on the basis of that. But you can nearly guess which languages it was good for, the super well resourced ones. But then, as I said, quality really drops off a cliff when you get into the long-tail languages or the lesser resource languages, and that’s still a problem even for, how to say, regular machine translation or for current approaches to machine translation, low-resource languages are always a challenge, and I think that challenge is only amplified with generative AI because they’re so much more data hungry. So you need more data, but the languages are still low-resource, and so you have a bit of a Catch-22 there. So I see huge future potential for multilingual content generation, but it could end up only being for a subset of languages, and we end up having different approaches for different languages. And when you get there, it becomes a data problem, which is why data for AI is such a hot topic as well.

Florian: Are you involved very much in the kind of human, we call it expert-in-the-loop type of part of Translated, the Matecat, the UI, or where you basically still need a human linguist? Or are you fully on just the automated MT side of things?

John: More on the automated side. I’m obviously intimately familiar with what happens on the other side and suffice to say, machine translation is a native part of that workflow. Like every segment that goes through Translated’s human translation workflow gets translated by ModernMT and that is provided as a suggestion to the linguists, and those suggestions get fed back and continue to improve the MT. But some of the innovations we’re working on, the R&D are around pure automation, and that’s kind of where my focus is on some of those product initiatives. One of which is the idea, we’re calling it Human-in-the-Loop. It’s the idea of you have a machine translation product and so one of the challenges with machine translation is if you don’t have a workflow where it’s being post-edited, ergo, you’re improving the output and the machine is learning for that, if you have like large scale raw MT, how does it get better? And the answer today is you have to have some process in place that identifies the issues, creates more data, you retrain the engines and so on. So we’re kind of thinking we do that for customers where on a periodic basis we do a big quality control, big evaluation, generate human data and feed that back to the MT so that it can learn, but what if we could automate that process? So that’s something that we’re looking at basically using a technology that’s been around for a long time, but is just starting to emerge as good enough to be useful, which is quality estimation. And so basically saying, if we have users who are translating super large volumes of machine translated content, is there a way that we can use quality estimation to pick off the segments that it’s not doing well on and in the background, push them through our human translation workflows to revise those segments exclusively for the purpose of fine-tuning the MT. So basically, instead of doing periodic retraining, you have like a constant daily micro fine-tuning, so that over the course of time, a certain percentage of everything that you machine translated has been captured in the background and revised for the purpose of improving the MT. That’s something that we’ve started to roll out and it’s pretty neat because you don’t have to do anything. The more you use the MT, the better it gets, basically, and the user can define how much they invest in making it better.

Florian: And you feel that component is something that you need to own in-house?

John: Yeah, so I mean, we have our own quality estimation. So quality estimation, like I said, has been around for a long time. WMT, shared task, they compare MT engines, they also compare, they evaluate the automatic evaluation. But in the same way as you have general purpose machine translation, and then you have machine translation that’s customized to make it better for an individual company’s content type, quality estimation should follow the same kind of idea because what constitutes good for one user doesn’t necessarily constitute good for another because it will depend on the use case. So we have what we call adaptive quality estimation, which is basically over time, as you revise things and you do human assessments, the quality estimation can learn what was good for one customer and what was good for another customer and adapt based on that. So over time, the quality estimation adapts as well, and it starts to get higher precision at finding those lower quality segments, and it’s kind of a virtuous cycle from there. And so because of that, yeah, we need that QE to be built into the workflow so that this can be fully automated and happen in the background without any impact on the day to day.

Florian: You need a lot of volume for that, right? You need like Airbnb sized volumes. I’m mentioning Airbnb because it’s one of the kind of flagship clients of Translated, it’s been disclosed everywhere. So, yeah, I mean, just for these types of, running these types of massive evaluations and basically funneling it with human feedback, so you need a lot of volume for that.

John: If you’re translating a few thousand words every week, MT is probably not a super critical component in your workflow and reviewing some of that to make it better is not going to have a huge impact. But if you’re translating millions of words on a daily basis or a weekly basis and you’re picking off 3, 5% of that and revising it, then it’s going to have a measurable impact in the short term.

Florian: How important is like text translation? So these giant volumes versus kind of that new world of multimodal at Translated, right. I mean, even two years ago, very few people were talking about speech-to-speech and all these cool things that we’re seeing now, it was much more a text based industry. So where do you see that going?

John: Specifically for us in Translated, it’s a relatively new part of the business. As a proportion of overall revenue, it’s still small enough, it’s still in, let’s say, single digit percentages, but it’s growing quite fast, which I think is reflective of A, the changes in the market to the types of content that are being created, thus driving the demand for this type of service, and then the other thing that’s aiding the service is the technology that has also improved. And I think again with Translated, having the products underpinned by the fact that we have the human workflow built for text. If you build a product that works for audiovisual, you can kind of have the same idea. So instead of just machine translation post-editing, you have transcription or speech recognition, plus some post-editing that leads into machine translation, plus revision, that potentially leads into speech generation plus some revision. So it’s kind of the same idea of some automation, plus a human-in-the-loop. And that’s why I describe like these types of human-in-the-loop processes for machine translation, they apply to all types of AI, whether translation tasks or not. They will work to a certain level, and then when you want to make them better, you need some sort of feedback loop, whether it’s multilingual or not. But as I said, the language is a tough challenge for these tasks. So you get a lot of bang for your buck when you have the human-in-the-loop involved. So for us, it fits really nicely with what we do. We have the technology, we have the people. Like I said, it’s a pretty fast growing arm of the business.

LocJobs.com I Recruit Talent. Find Jobs

LocJobs is the new language industry talent hub, where candidates connect to new opportunities and employers find the most qualified professionals in the translation and localization industry.

LocJobs.com I Recruit Talent. Find Jobs

Florian: So Matedub, is that tech or is it a service? Or you have the tech and then you can also do the service. Matedub, it’s a new product I saw.

John: Yeah, so we have two. One is Matesub, which is for subtitling, so you stop at the text part and then Matedub is dubbing, where we have the extra step, where we’re doing the voice as well. So if you think about that, that’s like the studio that sits on top and everything connects to that. So when you get to the human part, that connects to our TMS and pushes that content into the human workflows and then back into the dubbing platform because you need specific tools for that task, like the timing and the segments and stuff, like aligning stuff to the video. So the Matedub platform gives you the ability to do the stuff that is needed to be done for dubbing and then has the connectors to our transcription, to our machine translation, to the voice generation technology. But if our client came and said, we want to use our own voice generation technology, we can plug that into Matedub, but we still have the benefit of other parts of our technology in the workflow and our people in the workflow as well, so that’s the idea. It’s basically like a studio that sits on top and facilitates all the things that we do and also the things that you need to do specifically for a task like dubbing.

Florian: This is such an arguably even more disrupted field over the past just twelve months, right? There’s so much that’s going on there with all this kind of lip-sync dubbing that we’re seeing out there. We spoke about it before that it might not be enterprise-ready yet, but companies like Translated will make it enterprise-ready because they understand what the enterprise needs and then this is going to hit the market, and who knows, maybe I’m going to lip-sync dub this podcast at some point in the future. It’s a little niche content.

John: It is. Yeah. It’s funny. We have our whole team in Rome this week doing our kind of kickoff for the year, and it’s been a topic and one of the things we’ve been talking about this lip syncing a lot and we’ve been talking about the creepiness factor of it. When we’re talking about dubbing, we do that kind of, I think it was called the UN style dubbing, where I’ll start to speak in one language and then you hear the voiceover that comes in and you know it’s not John speaking the language and you listen to it. A lot of the European news channels do that when they have interviews with people speaking different languages. Nice idea to have the lip-syncing happen, but how does the individual feel about that? Because I did one this morning for the purposes of a demo internally, and I just look weird. Your mouth is weird. It does this stuttering stuff. I don’t know how tolerant people are going to be of that. I think that’s still an open question.

Florian: In the last podcast, I spoke to Esther about it because we had this speech by Milei, the Argentinian President, right? And it wasn’t just that it was non-permissioned. I mean, I’m sure nobody asked him, but it was his voice. I mean they took his voice profile and then made him speak English and it was lip-synced as well. I don’t know. Yeah, I also wonder how people will feel about it. Milei retweeted, I think, the German version of it, so maybe he, or probably more likely his social media team doesn’t care. But I don’t know, if you make this available to millions of people and have millions of people lip-synced without their permission, seems something that might not be cool.

John: I would ask, what is the value add actually, over a voiceover or something like that? Do you really need to see the words look like they’re coming out of the guy’s mouth? And if you know what that person looks like, you will be able to tell that it’s not natural. As good as it is, as I said, it’s that creepiness factor. When you’re looking at something, there’s just something not right there. I think that’s what that looks like. And arguably, the cons are more, like you say, it’s like doing it without permission, deep fakes. I think probably there’s more risks to it than the value that it’s actually adding, which is making something look a little bit more normal. I don’t know. I’m not convinced.

Florian: We had MARZ on the podcast. They’re very specifically for the lip-sync problem for Hollywood-grade, so arguably it’s not creepy, and I would see a use case, for example, an initial one in Netflix show for kids, right? So my kids watch dubbed shows from the US or Australia, and I mean, you see it so clearly that they don’t speak German, and the lips, I mean, I can literally see what they’re saying in English underneath, right? So that would be a massive improvement over the current experience and it’s permissioned, so I guess these actors wouldn’t mind.

John: It’s funny, I was telling someone the story yesterday when we used to get ads on TV in Ireland, so I guess whoever had the advertising budget for the company in the UK, they just said, yeah, we’ll just run that ad in Ireland as well, and they had probably, like a super cheap budget for dubbing it into an Irish accent. So they’re still basically saying the same words, but it’s just in a different accent, but you could still even see that the mouth movements weren’t right because of the differences in pronunciation. And we always thought there was something jarring about it as a kid and never quite knew what it was, so I don’t know. A kid’s going to grow up today with just loads of jarring stuff. Unless it’s been super Hollywood-level produced, they’re just going to be seeing weird stuff and going to be a little bit confused about it.

Florian: Do you work on any kind of live speech-to-speech? Is that something on your radar at all?

John: Not really. We have done it, but we don’t do it as a kind of an ongoing thing. It’s super specialized, and from a pure technology perspective, it’s pretty hard to do, and some folks do it quite well. So it’s not an area where we’re particularly targeting. I think we’ve a lot on our plate with the other things, so it’s not an area that we’re super focused on.

Florian: I’m asking also because it kind of brings up one of the issues when you have these big tech companies, in the case of speech-to-speech, it’s almost, I feel like a pet project of Meta, right? Because they want to have us interact in the metaverse and have language not be a barrier. So when you have these trillion-dollar companies come in with their billions, in these areas wouldn’t you maybe just wait another year or two and have them launch some type of next level framework you want to build on? It must be a really difficult decision, where you’re going to go and start developing proprietarily and where you’re just going to wait.

John: 100%. It’s kind of what I said, we need to pick our battles. Meta can do this, Google can do this, but not everybody can. So we can do some things, but like you said, you have to pick your battles. And that’s not to say, to continue the analogy, that you lay down your sword and just Meta will take care of that and we’ll work with it when they do it. You can still take the areas that are of particular interest to you or value to you and focus on them and then, yeah, build on something else. Because I don’t think there aren’t that many companies in the world that, are even the ones that are building top class technology, are doing all of the basic research themselves. That’s how technology comes. Universities used to be the source of all of the innovation. Now a lot of it is happening in the Big Tech companies and everyone can avail from that and I think that’s absolutely fine. We’re doing fundamental research at Translated, but not necessarily all of it. And if we see something useful that comes out, if Mistral released something or there’s another version of LLaMA, we’ll take it, we’ll play with it, we’ll use it if it works, if we can, or if we’ll reimplement something similar. So I think yeah, it’s a multipronged approach to it. Personally that’s how we always worked in the past. You’re a small team, you have limited resources, so yeah, you pick your battles and so live kind of speech-to-speech is not one that we’re targeting. If the technology comes out and everyone starts doing it, we might jump on that too. But in the meantime, we’ll focus on, like I said, the MT has always been there and there’s always scope, and then this AV stuff in terms of the dubbing and subtitling is growing. So we’ll keep our focus a little bit more there.

Florian: How do you keep up, especially with all these new models being launched and what’s open source, what’s not? Other than, of course, reading Slator, but what are some of the sources that you go to?

John: Fundamentally, it’s hard. We have a visitor here who asked me, hey, have you heard of this thing? And I said, I haven’t, and then they asked somebody else, and they had just heard of it yesterday, and I think it’s called Mamba. It’s this new approach that’s different from transformers. It’s this new kind of new way of building models that are smaller. And it’s just like, okay, that’s a new thing, and then you have to, like with all news these days, you have to find the source. So this Mamba thing was on some website, and it got picked up by press. You follow the, what’s the source? What’s the paper? Who did it? It was from Princeton. Okay. Reputable source, but it wasn’t peer reviewed. Okay. Like I said about the noise earlier on, there’s just so much out there, it’s quite hard to just filter through it. You give yourself a pat on the back and absolutely. We rely on media or some intermediary to maybe do some of that work and point us towards the best stuff. And then we obviously just have folks internally who are more day to day. And it would be like your typical, even in the MT space or the computational linguistics space, some conferences are more valuable than others. So if you have papers that get accepted at a certain conference or you know who the people are or you know where they work, that adds credibility to it. And so you tend to focus your reading or your time that you have to get up to speed or to catch up on some of those things, so kind of referral kind of thing. There’s so much out there, you need some sort of filter, and so you kind of filter it on the basis of that. But like I said, you get captured by the headline, you determine it’s something that’s interesting to you, you read some summary of it, and then you go to the source, and then that brings you back to that whole point of enterprise-ready. Did they just do it on some sort of little test set for one language, like super narrow, or was it actually broad and legitimate and fair? But like I said, it’s hard. There’s a lot out there.

Florian: They started writing these really catchy headlines for these research papers, like four or five years ago. It was super geeky and very precise, and now it’s like shocking, and all these terms, and like okay.

John: I coined this back when we were talking about neural MT. I said it’s like the intersection of science and marketing. When Google had that paper, the human parity thing and stuff like that, so I’m like, oh, it’s like researchers who are maybe thinking if we give this a snappier title, like our marketing team does, or like TechCrunch did, then maybe we’ll get more eyeballs on it. And I think it’s like the tail wagging the dog, it’s leading to it, which is funny.

Florian: And where do you see a lot of the innovation right now? I mean, Big Tech, open source, or still very much kind of academic? I mean, we’re seeing these collaborative efforts, like when you look at some of these papers that are coming out. And has there been a shift, especially in the kind of narrower MT domain since the LLM came out, since the LLMs, like the GPT-4 LLMs came out?

John: Even in the last few years, maybe to pre-pandemic onwards, there’s been a bit more shift in where the origin of a lot of this research coming from bigger tech companies as opposed to traditionally from certain sets of universities. And even with that, the sets of universities where the research was originating from would depend on the technology because machine translation came from a very specific set of universities, like there was Dublin City University, there was like Aachen in Germany, there was Groningen in the Netherlands. For particular reasons of how research in Europe evolved, your Harvards and Yales weren’t doing machine translation, they were focused on other things. So that’s evolved that this is coming a little bit more from Big Tech, but then you have curveballs that come like Mistral when it got released, there was no pomp and circumstance about it. It’s like a small French company released this in a super techie way, just pushed it out there, didn’t make a song and a dance about it, and turns out to be pretty good. Just when you think you know the answers, the questions get changed. It’s kind of like, where is it going to come from next? And nothing has settled down. We’re in the middle of a tornado and everything’s gone around, and like I said before, wouldn’t have it any other way.

Florian: What’s on your plate for the next 12 months, like any exciting things you’re working on? Any innovations for 2024?

John: One thing I’m going to be doing a lot more this year that I haven’t done in a number of years is getting back on the road. I only started to do it towards the tail end of last year and missed the value in it because I think when we couldn’t do it, we convinced ourselves that, hey, look, we can do everything digitally. We don’t need to talk to people in person anymore. And once you started doing that again, the value in that is huge. So, I mean, independent of any sort of technology developments, I’m looking forward to that, like sitting down in front of people and talking to them about their challenges. The one to many experience that you get at conferences where you can have, like I did a trip to San Francisco just before the end of the year, and I maybe had 20 meetings, and I just came home. I was sitting on the plane on the way home going, that was awesome. I’ve evolved from Monday to Friday in how I saw something and in how I was thinking about something and how I described things. So I’m looking forward to doing that and I fully suspect that that will inform a lot of some of the directions that we end up taking. But specifically on some of the innovations and the tech for this year, this product that I mentioned, Human-in-the-Loop, is a big release for us. You’re going to start to see a lot about it across our channels. I’m doing a big plug, my webinar next Thursday. This process that it described of basically improving MT in the background without having to do anything, so that’s a big thing for us. And this is just the first version of this that we’re releasing, and we have a roadmap and a product team working on it. And so that’s going to be something that we’re going to be bringing out a lot. And I would say watch this space in terms of, I suspect there will be something significant from us this year around large language models and something multilingual. It’s in the works, like everything that’s in the works, there’s things to work out still. I will continue to bang the drum about enterprise readiness because it means more than just good output. It’s good output delivered in a way that fits into companies existing workflows and is cost-effective and scales because content is exploding still. It continues to explode. It’s in the tornado, and things can’t be getting more expensive because it won’t work. So that’s definitely something that we’re going to be refining, and when the time is right, you’ll hear all about it. So watch this space.