Evaluating language models on low-resource pairs

Abstract

This work is devoted to the evaluation of the Facebook M2M100_418M and Alirezamsh Small100 models for low-resource language pairs. For this study, parallel corpora were selected for the following language pairs: Japanese-Ukrainian, Korean-Ukrainian, Turkish-Ukrainian, Vietnamese-Ukrainian, and Chinese-Ukrainian. The models were assessed based on their performance in translating these language pairs. Evaluation metrics included BLEU and ChrF scores, which measure the quality of the translations. Additionally, differences between the target and translated sentences were analyzed. The study aims to highlight the strengths and weaknesses of each model when working with low-resource languages. A comparative analysis of the results provides insights into the effectiveness of these models. The findings can be useful for future improvements in machine translation for underrepresented language pairs.