A competitive learning scheme for deep neural network pattern classifier training

To reduce the computational complexity of training a deep neural network architecture using large data sets of 3D scenes, a competitive learning scheme was devised. The proposed algorithm pits a neural network learning algorithm, in this case the standard Adam optimiser, against an evolutionary algorithm that is used to select the most difficult training examples. The overall scheme is similar to a predator-prey system, where the predator (the neural network) strives to optimise its ability to capture (identify) the prey (the training patterns), and the evolutionary procedure selects the prey that so far evaded capture. As a consequence of the evolutionary process, the neural network is presented only a fraction of the training examples, and the computational complexity of the learning procedure is reduced. Experimental evidence showed that the proposed scheme allows reducing the deep neural network training time on different model sets, sometimes significantly, without affecting the recognition accuracy. The proposed predator-prey scheme is fairly independent of the ANN type and training algorithm employed, and has the potential to be beneficial to a wide range of deep learning applications, where practical implementations are often hindered by the time complexity of the training process.