Printing problems?
If you are having trouble printing information from SimulTrans´ website, please visit our printing advice page.

Evaluating the Quality of Localization Deliverables, Part II

In May, the SimulTrans Localization Series featured two speakers, Manfred Hall, Senior QA Engineer for Kodak's Software and Integration Group, and Addison Phillips, Manager of Operations at SimulTrans.

Phillips, a former software development engineer, demonstrated a series of translation tools SimulTrans uses to assist in the quality assurance process. He also explained the history of machine translation and how current translation memory tools evolved to help fill the gaps left by the early technology.

Hall explained the specific QA processes he employs at Kodak. His company's primary goal is to achieve the same level of quality in the localized version of the software as in the original version. He also shared Kodak's "keys to quality" and the company's. step-by step quality assurance process.

The evening's discussion was a continuation of the April session. Guest speakers that evening were Phillips and Emmanuel Uren, Manger of International Software Quality Assurance at Siebel Systems, Inc. Together, they addressed the more theoretical aspects of assembling a quality-focused localization program.

Addison Phillips

Tools used at SimulTrans pertain to both quality and process. There are three basic categories:

• Tools for people interested in localizing software
• Tools designed for Help and content localizers, and related software
• Various other tools useful to people in the localization business

Machine Translation and Lexical Analysis

Machine translation, the forerunner of translation tools, emerged some twenty years ago. Early machine translation efforts strived to use artificial intelligence and lexical analysis to save time and money, and provide more consistent translation. Companies with very large documentation sets made major commitments to using the technology.

However, companies had to commit to "living in the machine translation world." Technical writers were forced to write in an awkward style of English easily understood by the machine. Companies also had to build large dictionaries and follow certain processes. Following the initial effort, editors still had to make the final text appear to be written by a human.

Translation Memory Tools

Translation memory evolved to fill the gap left by straight machine translation. Among current translation memory tools are TRADOS, Transit, TM/2, and XL8. The TRADOS suite of tools takes in translated text that has been turned into RTF. The information is then stored and kept readily available for the next round of translations.

Unfortunately, the tools are linked to certain text formats. For example, TM/2 prefers SGML and HTLM, while TRADOS works best with a specific flavor of RTF.

New Approaches

Modern tagging systems include XML, which consistently tags all files with items regarding terminology. Because tagging systems build up structures that are bigger than a translation memory file, programs are currently being developed to address storage issues.

Corel Catalyst 2.5

Corel's Catalyst product is designed around a specific philosophy: internationalize your source code. You can then perform"object code translation." If you take the compiled English program and localize its elements, it should run as efficiently as the English version.

The Corel product is solid only on the Win32 platform; it does not work on Macintosh, UNIX, or Win16 platforms. In general, SimulTrans has accomplished higher levels of leveraging, more intelligent leveraging, and a higher level of quality assurance because of the product. It's a solid contributor to simultaneous release efforts and can be essential to the QA process.

TRADOS and TermLink

TRADOS aids recycling and terminology consistency. However, it provides consistency mostly at the segment (or software string) level and falls short regarding how strings are assembled. You may need an additional tool such as as TermLink to work with the technologist.

Use of such tools requires you to pull terminology out of products ahead of time, have a translator view it, then return it to the terminology database ahead of translating the next block. Pluses include savings in terms of the number of words translated and amount of time (20% to 40%) to translate. Consistency is also guaranteed.

Tools for Help/HTML

Notable QA tools include HelpQA or HTMLQA, produced by Translation Craft. HTMLQA is very similar to HelpQA in terms of navigation.

Other Interesting Tools

One notable string capture product is CaptureEZE. While it allows you to capture with the menus pulled down, it routinely requires a timing device. Another good tool is HyperSnap. And, if you're doing a great deal of graphic work, ThumbsPlus allows you to catalogue numerous graphics, complete them, and convert them to a necessary format.

Manfred Hall

As an American company, Kodak is developing software primarily in English. Our QA goal is to achieve the same level of quality in the localized version of the software as in the non-localized version, both functionaly and linguistically.

The Challenge

We deal primarily with high-end copier/printer products. The control software for such printers displays the options the printer is capable of.

This complexity results in localization problems regarding layout, especially for a language such as German which expands greatly. As a result, the entire product needs to be thoroughly QA'd.

What Is Quality?

Linguistic quality applies to license statements, terminology consistency, grammar, and documentation. Functional quality refers to installation, whether the product features country-specific defaults (such as paper size), etc.

There are gray areas. The software may be online Help and feature functional aspects, such as hotlinks, but also linguistic aspects such as time/date formats. The user interface is another gray area where the layout can be considered a functional aspect while terminology and consistency can be considered a linguistic aspect.

Keys to Quality

Non-localized software needs to be deemed translatable, localizable, and fully funtioning. Non-localized software that doesn't work properly cannot be localized.

The localization vendor should fully understand the product's function. A consistent glossary of terms should be made available to the vendor, along with a clear definition of deliverables. Other key points: priorities, schedule, and cost as they relate to quality.

Constraints

Routinely, if something doesn't work in the English version of software, it won't work in the localized version either. Also, hardcoded strings and hardcoded defaults can hamper localization efforts.

An existing release of a localized product can also pose a problem. It's often very difficult to introduce a new code release of the English product. Resources are often finished and the developers are nowhere around. If you would require a new formal release of the English software, it could be extremely costly.

Product changes just prior to shipping can be problematic. Regardless of how minor, each change must be incorporated into the localization process. As a result, localization runs late.

Out of my experience at Kodak come two general scenarios regarding localization.

Scenario 1

We translate the entire install of the product. Basically, we hand off an installation disk and we run that installation disk. The source file and documentation are the first items given to the localization vendor. The vendor then provides us with a translatability assessment which allows us to make changes early on.

At some point we receive the installable product. The localization QAs within Kodak test the product. If problems are detected, the product is returned to the localization vendor. Once corrections are made, we send out a formal approval to the localization vendor, software development, and media reproduction. Media reproduction then starts using the product.

Within this scenario are our keys to success. We have developed essential documentation on how to build the product, i.e., which strings to translate, which strings not to translate. The translatability assessment is also key because it assesses early on whether or not code changes are needed. Maintaining technical contact throughout the process is essential.

Localization QA within Kodak is only functional QA. Linguistic QA is part of the vendor's QA. He or she is in charge of delivering linguistically correct product. Admittedly, a functional issue proves far more critical than a linguistical issue. If the system crashes, that's not acceptable. If a word is spelled wrong, it can be fixed in a later release.

The Kodak localization QA process includes people who know the weak points of our product and where to discover problems early on. An experienced QA person in our business manages to create the first crash of the software within minutes.

Scenario 2

Scenario 2 concerns strings only and is usually done for UNIX software. We expect the translated strings back in the same file format. Development either creates that string or asks if they can abbreviate it. It often comes down to lack of space.

Our QA then receives the install of the product from Kodak development. We routinely send either the product or the screenshots to the localization vendor or our in-country reviewers for linguistic review. With a strings-only focus, the translator never sees the string contact (which can result in some loss of quality).

We are trying to move more toward Scenario 2. It's proven to be more efficient, even with Windows products.

Development documentationa detailed description of contactsis key to the process. This can be a problem because few developers enjoy documenting.

The linguistic quality of translated strings is essential. A consistent glossary becomes critical because so much material has already been translated. Communication regarding context issues throughout the cycle is also critical.

Finally, there needs to be a thorough linguistic review of the software or screenshots, either by the localization vendor or in-country reviewer.

Testing

A test that fails to find any problems is not a successful test. The reason is simple: There is no software without bugs. If you find more problems than average, it simply means that the testing procedure was a good one. If you find fewer problems, you should think about your test factors.

QA Approach

During our ramp-up process we identify potential weaknesses of non-localized products in early releases and beta releases. We check basic items such as the user interface influence on the protocol layer. We then establish a test plan with the English software using the orthogonal array approach where we test all features but not all combinations. We also focus on requirements for a target country such as default paper size.

We then perform internally a "translatability test." We start testing a so-called lead language, usually German. Because we encounter problems inherent to all languages, we try to eliminate those in the first language. Because it is relatively early on in the process, we can still address coding related problems.

We're currently translating German, French, Dutch, Swedish, Italian, Norwegian, Spanish, and Finnish. No Japanese, Chinese, or any other multibyte character sets. Nearly 40% of all European machines are sold in Germany.

Localization QA

Localization QA is really a one-by-one test according to a test plan. The regression is built in because we do a full test, usually with the same test plan for every single release. We do not rely on a simple regression test. Generally, we start with a fully integrated test, which means we print from the application to the printer. In the instance of failure, we try to identify the responsible subsystem in terms of where to hand it off: the localization vendor or to our development team. We try to determine the exact cause of the problem. And we establish a test case. We do that because we have a Notes database for reporting problems, so we have to establish a test case to re-create the problem. This is part of our general QA process.

Problem Reports

We use the rating system of 4 (critical) to 0 (enhancement). Zero is very uncommon in linguistics or in translation, but this is our general bug reporting system. For a product to pass, it can have no critical or severe bugs. We have a so-called quality bug matrix, which is a complex formula which allows us to combine those severities. We maintain one Lotus Notes database per product for all languages, including English.

Localization QA Profile

The Localization QA group consists of one engineer-manager and two full-time technicians who are also senior testers. One to two students/technicians are also on call. The QA group is supported by the European product specialist for a corresponding product, which sits in the same group. This structure provides us with contacts to software development, test plans, etc. They often know every bug in the product.

Localization QA Schedule

We routinely test for each driver, and test one to five days per language, per cycle. The vendor has three days to respond to a problem report or to fix it. We are currently driving three to five cycles per language.

Typical Localization QA problems

Layout problems such as stretching, diacriticals, functionality, software overruns, and some product-specific items are typical.

What Localization QA Finds

Linguistic issues are commonplace. For example, "finishing" as it is used in the term "finishing set-up," is a noun, not a verb. During the German translation process, it often comes out a verb. That's why we start up with German as a lead language: If we find that word improperly translated, we usually question the entire linguistic quality of the entire product.

Basic Functional Issues

Functional issues are often caused by a buffer not being long enough for the translated string. Different defaults for different countries, such as paper size and time/date format, pose another common problem. User interface issues range from clipped/malformatted text to buffer overruns.

Help issues form another important category. Although testers do not speak the language, they are able to catch common problems in the on-line Help package, such as formatting, inconsistencies with the UI, and functional issues with hotlinks

Protocol can be a little more difficult. Whenever you translate parts of the protocol, you must be careful to translate the right elements. And, again, you have to know where to look. You cannot blame a translation vendor because he or she must look at the protocol. We try to avoid it from a development standpoint and help the translation vendor find that kind of problem.

Baseline

In summary, you need a correct, high-quality, translatable original before you go into translation. Otherwise, it won't work. Approximately 10% of the problems we find, especially during ramp-up and translatability, are problems with the English software. Also, testing must be done by highly qualified personnel. A technical person needs to verify the function, and a linguist to verify the language.