1. MT Scoping

Essential: expectation management! Ideally, the requirements of process managers and important stakeholders are considered from the start. Have lots of conversations with project managers, IT experts, translators, and main user groups. If everyone is aware of machine translation use cases and resources, you can take specific steps to select MT systems. We at blc support our customers in the scoping process: We conduct stakeholder interviews and provide consulting for all areas of machine translation before, while, and after system integration.

2. MT Pre-Processing

After scoping, it is vital to test the suitability of available resources for machine translation training. After extracting relevant data from various file formats, the data needs to be cleaned. In addition to the linguistic and formal correctness of bilingual translation units, we at blc also check terminology resources. If necessary, we will use term extraction to include more resources. For data-sensitive use cases, we apply anonymization procedures. After ensuring the quality of the data, we aggregate domain-specific content and prepare it for MT training. We pay particular attention to the creation of representative test sets. These test sets are then applied to evaluate the quality of the trained engines.

3. MT Training

Using customer data, we train your MT engines in selected MT systems. MT system offerings differ in terms of available language directions, training resources, configuration options, and pre-trained base engines, which are then specialized with customer data. Note that training can take from less than one up to several days, depending on data volume. With these newly trained MT engines, we then translate our previously created test sets. These translations are the base of the following MT evaluation.

4. MT Evaluation

The evaluation of MT quality is one of the most important steps, in an MT selection process but also to assess the quality of your productive MT engines. Automatic evaluation compares machine-generated translation with the human translation of the same set of sentences. This results in a score that expresses the similarity between machine and human translation. The best-known score, which is mostly used at the moment is BLEU (Bilingual Evaluation Understudy).

BLEU and comparable metrics are not able to assess the accuracy of a machine translation in detail. Therefore, human evaluation is still the most precise measure of assessing the MT. Human evaluation can be either holistic, analytical, or both. Holistic evaluation measures the readability and accuracy of machine translation using scales. It provides a fast, qualified mean value across all sentences in a test set. Analytical evaluation provides more detailed information. Based on a defined error metric, the evaluators annotate error types in machine translation. The summary of these results provides valuable information for optimizing MT engines.

Another important factor in MT evaluation is the analysis of post-editing effort. As part of the MT evaluation, the so-called edit distance provides information about how much effort was needed by a post-editor to finalize each machine-translated sentence. The smaller the edit distance for a sentence, the fewer changes had to be made, and the better the MT output. Together with automatic and human evaluation, the edit distance provides a comprehensive picture of the quality and suitability of MT engines at a given time. 

5. MT System Selection

You may be wondering how to choose the right MT system provider. The results of the MT evaluation serve as the basis for selecting an MT system provider that is most suitable for you. On the other hand, requirements and financial framework also play an important role in the decision process. In this phase, an objective comparison and presentation of the most important system features and evaluation results is vital. The cost/benefit ratio in a sound business case as well as interviews with MT system providers and reference customers complete the picture. Once all professional and technical questions have been answered, you can select your MT system based on solid information.

6. MT Integration & Post-Editing

After you have decided on a system, the implementation of your MT use cases must be tackled. The central aim here is to integrate machine translation into your company’s existing system landscape: translation and content management systems are connected to the MT system, MT self-service portals are set up for company-wide MT use, and workflows are configured for a smooth post-editing process. To ensure that the MT system remains up-to-date, we support you with MT evaluation and re-training processes. Here the automatic quality estimation of MT plays an important role: Customer-specific estimation models are trained based on neural methods, resulting in a quality risk estimation score. This score helps post-editors, to keep the effort of fine-tuning the MT to a minimum.

We help you to build up your MT process, and we are also happy to provide support in your productive MT process, should you need it.