A futuristic hologram depiction of technology superimposed over a person typing on a keyboard

Lionbridge Now Shares GPT-4 Machine Translation Results via Its Tracker

Lionbridge has added GPT-4 to its Machine Translation Tracker, providing insight into the performance of the most promising GPT Large Language Model currently available for professional translation

Last updated: October 24, 2023 10:24PM

It’s Essential To Include Large Language Models — And Especially GPT-4 — In Machine Translation Comparative Evaluations

GPT-4, the most advanced and useful Large Language Model (LLM) within the GPT family at the time of this writing, is more suited to perform Machine Translation tasks than the earlier GPT models.

How well does this generative AI (GenAI) / LLM model currently perform Machine Translation (MT)?
How does it compare to the major Neural MT engines?

You’ll get the answers to these questions and be able to monitor the LLM’s progress over time via the Lionbridge Machine Translation Tracker, the industry’s longest-standing assessment tool that analyzes MT engine performance.

Due to the mounting shift from the Neural MT paradigm to the LLM MT paradigm, Lionbridge has added GPT-4 to its MT Tracker and now measures GPT-4 Machine Translation performance results for select domains and language pairs. Additionally, the Tracker measures the results of two other LLMs and the five major Neural MT (NMT) engines.

The Lionbridge Machine Translation Tracker evaluates the following AI engines:
Large Language Models	Major Neural MT Engines
GPT-4	Google NMT
GPT-3.5	Bing NMT
GPT-Davinci	Amazon
	DeepL
	Yandex

How Does Translation Performance Differ Between Large Language Models and Neural MT Engines?

At present, Neural MT engines generally perform better than LLMs, but only by a little. While LLMs cannot yet replace Neural MT engines, in part due to current cost considerations and issues with performance and scalability, we may apply the technology in specific ways to enhance the translation workflow and reduce costs.

In May 2023, we reported that the GPT-4 model outperformed a neural MT engine in one instance. We consider this a harbinger of what’s to come; its utility will only grow as it matures.

The more we test GenAI/LLM models, the more we learn about their idiosyncrasies, limitations, and opportunities. You can get the latest GPT-4 Machine Translation data from our original research via our MT Tracker and accompanying commentaries.

Read our October 2023 expert’s commentary to learn:

Noteworthy peculiarities associated with GPT-4.
Ways to mitigate unfavorable issues when using the technology.
How GPT-4’s current translation performance compares with other engines.

READ THE COMMENTARY

“While we may have to live with more translation output uncertainty as we shift to the LLM MT paradigm, it may be possible to use some mechanisms and best practices to make that variability somewhat controllable.”

— Rafa Moral, Lionbridge Vice President, Innovation

How Will the Lionbridge MT Tracker Help You?

The Tracker assessment tool provides essential data on engine performance, enabling global companies to stay abreast of exciting GenAI/LLM technology advancements.

However, developing an effective MT solution that will meet your company’s needs can be tricky. That’s because the most advantageous, best-of-breed approach leverages multiple engines based on the engine’s ability to handle the unique aspects of the content. Adding LLMs into the mix of options makes the already challenging undertaking even more complicated.

Get in touch

Want to learn how Lionbridge can tailor a solution leveraging LLM / generative AI opportunities and applications to produce measurable business value? Reach out to us today.