Адаптация активационных функций в глубоких нейронных сетях

Abstract

In this paper the modification of PReLU used in Deep Neural Networks (DNN) is proposed. This modification called Adaptive Parametric Rectifier Linear Unit (AdPReLU) tunes both synaptic weights of neuron and parameters of activation function that permits to accelerate the learning process of DNN in whole and to reduce the amount of network`s hidden layers. The proposed AdPReLU is generalization of activation functions usually used in known DNN.