How Accurate Is Machine Translation, Really?

How Accurate Is Machine Translation? - How Accurate Is Translation?

Machine translation has improved enormously over the last decade, and it is now good enough to be useful for a huge range of everyday tasks. It is also good enough to be dangerous, because it usually sounds fluent even when it is wrong. A smooth English sentence feels trustworthy, but fluency and accuracy are different things. A translation can read beautifully and still say the opposite of the original.

This guide explains how accurate machine translation really is, where it tends to fail, how you can test a result without knowing the other language and when to bring in a human. It applies to any translation tool, including our own image translator.

Opened book
Photo by Aaron Burden on Unsplash

Two separate steps, two kinds of errors

When you translate a picture, two things happen. First, software reads the text in the image. Second, software translates the text. Mistakes can happen at either stage, and it helps to know which one you are dealing with.

StageTypical errorHow to spot it
Reading the imageWrong letters, missing words, jumbled orderThe recognized text does not match the picture
Translating the wordsWrong meaning, odd phrasing, lost toneThe recognized text is right, but the English sounds strange or contradicts context

Always check the recognized text first. If it is wrong, the translation will be wrong, no matter how good the translator is. Our guide on why OCR misreads text covers the first stage in detail.

How good is it, really?

There is no single accuracy number, because it depends on three things: the language pair, the type of text and the quality of the input.

  • Language pair. Widely used pairs with lots of training material, such as Spanish and English, generally do better than rarer ones.
  • Type of text. Plain, factual sentences translate better than jokes, poetry, marketing slogans and slang.
  • Input quality. Clean, complete sentences work better than fragments, abbreviations and noisy text from a blurry photo.

As a broad rule, machine translation is good at conveying the gist of straightforward text and weaker at tone, nuance, humor and precise legal or technical meaning. We cannot give a precise percentage, and we are skeptical of anyone who does without describing their test.

Where machine translation fails

Idioms and figures of speech

Expressions rarely translate word for word. A phrase that means “that is not my problem” in one language may come out as a literal image of animals or food in another.

Words with several meanings

A single word can have different meanings depending on context. On a sign, one word might mean “closed” or “out of order”, and without context the tool has to guess.

Negation and small words

A missing “not” turns a warning into an invitation. Small words are easy to lose, especially in fragments from a picture.

Numbers, dates and units

Decimal commas, day and month order, and different units can all cause confusion. A date of 03/04 can mean two different days, depending on the country.

Gender and formality

Many languages mark gender and politeness, while English does not. Information can be lost or guessed incorrectly when translating in either direction.

Names

Names of people, places and brands are sometimes translated when they should not be, or left alone when they should be changed.

Specialist vocabulary

Legal, medical and technical terms have precise meanings. A general tool may choose a common meaning, not the specialist one.

Tone

Polite requests, firm warnings and casual jokes may all sound the same after translation.

How to test a translation without knowing the language

You do not have to speak the original language to run useful checks.

  1. Read for sense. Does each sentence make sense in context? A sentence that suddenly turns strange is a warning sign.
  2. Check the facts. Numbers, dates, times, names and amounts should match the picture exactly.
  3. Look for consistency. The same term should translate the same way each time it appears.
  4. Try back translation. Translate the English result back into the original language with a second tool and compare it with the original. If the meaning has drifted, something went wrong. This is a rough test, since errors can go in both directions, but it often exposes serious problems.
  5. Use a second tool. If two different tools agree, you can be a little more confident. If they disagree, investigate.
  6. Search the original phrase. Searching a tricky phrase in its original language, with a word such as “meaning”, often brings up explanations and examples.
  7. Ask a person. A native speaker, even a friend, can check a key sentence in seconds.

Red flags that suggest a problem

  • A sentence that does not make logical sense.
  • Repeated odd words or phrases.
  • Numbers that differ from the picture.
  • A warning that sounds too casual, or an instruction that sounds too extreme.
  • Names or brands turned into everyday words.
  • A very short result from a long block of text, which may mean lines were dropped.

How much risk does the content carry?

Type of contentRisk if wrongRecommended approach
Casual chat, social postsLowMachine translation is usually fine
Menus, signs, directionsLow to mediumFine for the gist, ask staff to confirm key points
Product labels and instructionsMediumDouble check numbers and warnings
Allergens and medicineHighConfirm with a pharmacist, doctor or manufacturer
Contracts, leases, official lettersHighUse a qualified translator for anything you will sign or be held to
Medical, legal, immigration documentsVery highUse a certified professional

When a human translator is worth it

Professional translators understand context, culture and specialist terms, and they take responsibility for their work. Some situations call for one: documents you sign, documents you submit to authorities, medical information, legal disputes, published content and anything where a small mistake could be costly. Many official bodies require a certified translation. A machine translation can still help you prepare, since you can understand the outline and prepare your questions before you meet the translator.

How to get better machine translations

  • Start with clean input. A sharp photo and the correct source language make a big difference. See our photo settings guide.
  • Translate complete sentences. Fragments lose context.
  • Avoid slang when you can. If you are the writer, use plain language.
  • Add context. If a word is ambiguous, translate it inside a longer phrase.
  • Review and edit. Treat the output as a draft.

A worked example

Imagine a notice taped to an apartment door in Spanish. After translation you read: “The water will be cut off on Tuesday from 9 to 14.” That seems clear. You then check the original text in the left box and see the word for Tuesday spelled with an accent you did not expect. You compare it with the picture and confirm the day. You note that the time uses a 24 hour clock, so 14 means 2 in the afternoon. You also notice the English does not say which building the notice applies to, so you look at the picture again and see an address line that was cropped out. Three quick checks turned a plausible translation into a reliable plan.

What accuracy claims to be skeptical about

Be wary of claims such as “99 percent accurate” or “perfect translation”. Accuracy depends on the language, the text and the way it is measured. A tool that scores well on formal news articles may do poorly on handwritten notes or slang. Reputable tools say what they are good at, and where they struggle.

A back translation example

Suppose a label says, in Spanish, that a product “no debe usarse en niños menores de tres años”, which means it must not be used by children under three. A good translation says exactly that. A faulty one might drop the word for “not”, and tell you that the product is suitable for children under three. If you translate that faulty English back into Spanish with a second tool, you would see an affirmative sentence that does not match the original. The mismatch is your alarm. Back translation is not perfect, but it is a fast, free way to catch a serious reversal of meaning, which is among the most dangerous kinds of error.

Machine translation and tone

Tone matters in real life. A polite request in one language may sound blunt in another, and a firm legal notice may sound friendly. When you are translating a message you will reply to, remember that the other person wrote it with a particular tone. If a message seems harsh or oddly casual after translation, it may simply be a side effect of the process. Before you react, consider asking the sender what they meant, in simple words.

Frequently asked questions

Is machine translation accurate enough to trust?

For everyday gist, usually yes. For important decisions, treat it as a draft and verify it.

Why does a translation sound fluent but wrong?

Modern systems are very good at fluent sentences, but they can still choose a wrong meaning. Fluency does not guarantee accuracy.

How can I check a translation if I do not know the language?

Check numbers and names, use a second tool, try a back translation and ask a person for key sentences.

Are some languages better than others?

Yes. Languages with a lot of available text generally translate better than less common ones.

Should I use machine translation for legal documents?

Use it to understand the outline only. Rely on a qualified translator for anything legal.

Next steps

Try the translator on a picture, and then apply the checks above to the result. For more on the safety side, read whether image translation is safe and private, and read our disclaimer for the limits of the service.