Обробка природної мови та комп’ютерна лінгвістика

Abstract

This report is devoted to the topic of Natural Language Processing (NLP) and Computational Linguistics (CL). It provides an overview of these fields, including their history, key concepts, and applications. The report also discusses the challenges facing these fields, such as the ambiguity of human language and the need for large amounts of annotated data to train models. One particular problem addressed in this report is the difficulty of handling rare or unseen words in NLP models, and it presents various ways to resolve this problem, including the use of sub word units, character-level models, and transfer learning techniques. The report concludes with a comparison of these techniques and their effectiveness in addressing the challenge of rare words. The report highlights the importance of NLP and CL in processing natural language text and presents solutions to some of the challenges facing these fields.