Порівняльний аналіз сучасних методів ефективного за параметрами донавчання LLM

Abstract

This work presents a comparative analysis of modern Parameter-Efficient Fine-Tuning (PEFT) methods for Large Language Models (LLMs). Existing approaches are systematized into four categories: adapter-based methods, low-rank adaptation (LoRA) and its variants, prompt-based tuning, and hybrid approaches. For each category, the mathematical foundations, computational complexity, and practical trade-offs between parameter count, training efficiency and downstream task performance are analyzed. A comparative analysis on the GLUE benchmark is presented. Key open challenges including rank collapse, static compute allocation, and limited multilingual evaluation are identified. Recommendations for selecting PEFT methods depending on computational resources and task characteristics are formulated.