Parallel design of sparse deep belief network with multi-objective optimization

Yangyang Li, Shuangkang Fang, Xiaoyu Bai, Licheng Jiao, Naresh Marturi

Research output: Contribution to journalArticlepeer-review

3 Citations (Scopus)
197 Downloads (Pure)

Abstract

Deep belief network (DBN) is an import deep learning model and restricted Boltzmann machine (RBM) is one of its basic models. The traditional DBN and RBM have numerous redundant features. Hence an improved strategy is required to perform sparse operations on them. Previously, we have proposed our own sparse DBN (SDBN): using a multi-objective optimization (MOP) algorithm to learn sparse features, which solves the contradiction between the reconstruction error and network sparsity of RBM. Due to the optimization algorithm and millions of parameters of the network itself, the training process is difficult. Therefore, in this paper, we propose an efficient parallel strategy to speed up the training of SDBN networks. Self-adaptive Quantum Multi-objectives Evolutionary algorithm based on Decomposition (SA-QMOEA/D) that we have proposed as the multi-objective optimization algorithm has the hidden parallelism of populations. Based on this, we not only parallelize the DBN network but also realize the parallelism of the multi-objective optimization algorithm. In order to further verify the advantages of our approach, we apply it to the problem of facial expression recognition (FER). The obtained experimental results demonstrate that our parallel algorithm achieves a significant speedup performance and a higher accuracy rate over previous CPU implementations and other conventional methods.
Original languageEnglish
Pages (from-to)24-42
Number of pages19
JournalInformation Sciences
Volume533
Early online date17 May 2020
DOIs
Publication statusPublished - Sep 2020

Keywords

  • Restricted Boltzmann machine
  • Deep belief network
  • Multi-objective optimization
  • Parallel acceleration
  • Facial expression recognition
  • GPU

ASJC Scopus subject areas

  • Software
  • Information Systems and Management
  • Artificial Intelligence
  • Theoretical Computer Science
  • Control and Systems Engineering
  • Computer Science Applications

Fingerprint

Dive into the research topics of 'Parallel design of sparse deep belief network with multi-objective optimization'. Together they form a unique fingerprint.

Cite this