Порівняння класичного та стохастичного градієнтного спуску в задачах оптимізації

Abstract

The relevance of this study lies in the need to improve the efficiency of training intelligent systems in Big Data environments. The object of research is the iterative processes of minimising the loss function in machine learning tasks. The paper provides a comparative analysis of Batch Gradient Descent and Stochastic Gradient Descent (SGD) algorithms, focusing on convergence speed, stability, and computational complexity. It is shown that while the classical approach provides high precision for small datasets, the stochastic method is economically feasible for large-scale systems due to its O (1) iteration complexity and its ability to escape local minima under noise. The results of the study can be used to optimise the infrastructure of modern neural network training