Get Smart: Review Less
25 Mar 2024
6 mins read
Have you ever been on holiday and seen one of those signs? You know the ones, right? The ones where someone has “had a go” at translation and not quite got it right. Like the one in the Copenhagen airline ticket office:
We take your bags and send them in all directions.
Or perhaps the one from the car rental firm in Tokyo:
When passenger of foot heave in sight, tootle the horn. Trumpet him melodiously at first, but if he still obstacles your passage then tootle him with vigor.
One of my favourites though is this sign in Wales:
The text in Welsh below the arrow reads “I am not in the office at the moment. Send any work for translation.”
You might wonder how these signs ended up being printed. Well, reviewing translations just adds cost; you’re paying for another pair of eyes to look at every piece of text that has been translated. Reviewing translations also adds time to the project so when you’re hoping to stem the tide of all those Welsh-speaking drivers of heavy goods vehicles, timeliness could be key.
Evaluating translation quality
There are several ways to evaluate the quality of a translation. Let’s look at some of these below.
Human evaluation
The most obvious solution for evaluating the translation quality is to have a “Human in the Loop” (HITL). As a native speaker of the target language and someone who understands the source language, a reviewer can look at the proposed translation and evaluate it based on criteria such as:
- Accuracy: Does the translation accurately convey the meaning of the source text, including all its nuances and implications? This includes factual accuracy, as well as cultural and linguistic accuracy.
- Consistency: The translation should be consistent in terms of its terminology, style, and tone. This is especially important for longer or more complex texts.
- Fluency: The translation should read smoothly and naturally in the target language. It should be free from awkward phrasing and grammatical errors.
- Style: The translation should match the style of the source text. This includes the level of formality, the register of language, and the overall tone.
- Cultural appropriateness: The translation should be culturally appropriate for the target audience. This means avoiding any offensive or insensitive language or imagery.
Of course, as above, this can add significant time and cost to the project but is the most reliable method for checking translation quality.
Automated quality tools
There are a several automated quality tools available today. Some of these are integrated functionality in existing CAT tools, some are standalone tools, and some are add-ons for other tools e.g., plugins for Trados Studio. Generically, these tools take a bilingual translation document and run through each segment looking for potential errors. They will check, for example:
- Empty or forgotten translations.
- Significant mismatches in source and target translation length.
- Forbidden characters in the translation.
- Terminology – checking that the approved terms are used, and no forbidden terms are used.
- Inconsistencies, punctuation, spelling.
- Numbers, dates, times, measurements, and currencies.
Some tools are more in-depth and feature-rich than others, but you get the idea. Note, however, that there’s no real “intelligence” to these tools; they are just running algorithms on the source and target text and identifying those segments where one of the rules has been broken. These automated tools are good at catching obvious errors, but they don’t do the same job as a human reviewer.
Round-trip translation
If you don’t speak the target language, a simple way to evaluate the quality of a translation is to translate the text back to the source language and compare with the original. Using a machine translation engine to do this will prevent some obvious errors. Using the above:
- Original Text: No entry for heavy goods vehicles. Residential site only.
- Translated: Nid wyf yn y swyddfa ar hyn o bryd. Anfonwch unrhyw waith I’w gyfieithu.
- Translated back: I am not in the office at the moment. Send any work for translation.
One would hope that this translation is then questioned and the time and expense of creating the sign is saved. Whilst round-trip translation can catch obvious errors like the one above, and may also provide some levels of amusement, it is generally not considered a reliable way of evaluating the quality of translation.
Harnessing Artificial Intelligence
From everything that has been covered above, it’s clear that our ideal solution is to have a human reviewer checking the quality of the translations. It’s also clear that this will add time and cost to the project. What we need is for our reviewer to be more productive and spend less time checking translations that already look good. Fortunately for us, the advent of Large Language Models (LLMs) has provided a new and interesting possibility to help us improve productivity and reduce costs.
One of the interesting things you can do with LLMs is to give them a persona. Consider the following prompt for example:
From now on, act as a product marketer. Pay close attention to style and tone of your response. Provide outputs that a product marketer would regarding the input. Write a short marketing announcement for a new AI-powered translation review function in my translation management software.
You might get:
Introducing Smart Review: Revolutionize your translation workflow!
Are you tired of wasting valuable time and resources reviewing translations manually? Say goodbye to costly errors and hello to efficiency with our cutting-edge AI-powered translation review function, exclusively available in Trados Enterprise.
🌐 Seamlessly integrated. Our AI-driven translation review seamlessly integrates into your existing workflow, making it easier than ever to ensure linguistic excellence in your translations.
🤖 AI precision. Harness the power of artificial intelligence to review translations with unmatched precision. Our AI algorithms catch nuanced errors that may go unnoticed by human eyes, ensuring your translations are of the highest quality.
🚀 Boost productivity. Smart Review significantly speeds up the review process, allowing your team to focus on what matters most: delivering translations that resonate with your global audience.
💼 Cost-effective solution. Save time and money by reducing the need for extensive manual reviews. With Smart Review, you can achieve top-notch translations without breaking the bank.
🔒 Security and accuracy. Rest easy knowing that your confidential translation data is handled with utmost security and accuracy. Our advanced AI technology always maintains the integrity of your content.
🌍 Go global with confidence. Expand your reach and conquer new markets confidently, knowing that your translations are backed by AI-powered precision.
Don't miss out on the future of translation management. Elevate your translations to a whole new level with Smart Review. Get started today and experience the difference!
Sounds amazing right? Count me in! Where do I sign? Even more amazing is that what’s written above is pretty much exactly what we’re working on! There are, however, a couple of things that you should know about the above text:
- I did some very light post-editing on the content but it’s around 95% generated by an LLM.
- This was my second go at generating the marketing content; the first LLM I tried took the product marketing role to heart and disappeared for a three-hour lunch on expenses. Just kidding my product marketing friends 😆.
Using this persona approach, we can get AI to act as a linguistic reviewer. We can also provide it with much more data to help evaluate the segment accurately. This includes:
- Terminology that might be present in the source text that we want to make sure has been translated properly in the target text.
- Sample translations from the same document or from the translation memory that have similar source text and can be used to provide the desired style and tone.
- Instructions as to what we want to achieve – we’d like to get a score out of 100 and a reason.
We can ask it to check the translations for accuracy, consistency, fluency, style, and cultural appropriateness. All good I guess but how is this saving time and money? Well, what if we now ask our human reviewer to ignore segments with any score above a certain threshold? Our human reviewer can now be laser-focused on exactly the translations where there might be issues. Rather than reviewing 100% of the content, perhaps they’re only reviewing 80%. The time and effort that is saved means faster turnaround and lower costs which is, after all, what we all want from our translation projects.
Delivering Smart Review
Smart Review is another interactive feature delivered through the Trados Copilot interface. It is available to reviewers in both the desktop and browser-based editors. Keep your eyes and ears open for more news through our community and our social media channels.
With our initial implementation, which is in technical preview right now, you’ll need a subscription to Azure OpenAI to configure Smart Review. You will also need to contact us to enable this feature on your account.
David Pooley
All from David PooleyRelated Articles
