Adaptive weighting of handcrafted feature losses for facial expression recognition

Weicheng Xie, Linlin Shen, Jinming Duan

Research output: Contribution to journalArticlepeer-review


Due to the importance of facial expressions in human-machine interaction, a number of handcrafted features and deep neural networks have been developed for facial expression recognition. While a few studies have shown the similarity between the handcrafted features and the features learned by deep network, a new feature loss is proposed to use feature bias constraint of handcrafted and deep features to guide the deep feature learning during the early training of network. The feature maps learned with and without the proposed feature loss for a toy network suggest that our approach can fully explore the complementarity between handcrafted features and deep features. Based on the feature loss, a general framework for embedding the traditional feature information into deep network training was developed and tested using the FER2013, CK+, Oulu-CASIA, and MMI datasets. Moreover, adaptive loss weighting strategies are proposed to balance the influence of different losses for different expression databases. The experimental results show that the proposed feature loss with adaptive weighting achieves much better accuracy than the original handcrafted feature and the network trained without using our feature loss. Meanwhile, the feature loss with adaptive weighting can provide complementary information to compensate for the deficiency of a single feature.
Original languageEnglish
JournalIEEE Transactions on Cybernetics
Early online date2 Aug 2019
Publication statusE-pub ahead of print - 2 Aug 2019


  • Deep feature loss
  • expression recognition
  • handcrafted feature
  • loss adaptive weighting


Dive into the research topics of 'Adaptive weighting of handcrafted feature losses for facial expression recognition'. Together they form a unique fingerprint.

Cite this