Dealing with a missing sensor in a multilabel and multimodal automatic affective states recognition system

Jesus Joel Rivas, Felipe Orihuela-Espina, Luis Enrique Sucar, Nadia Bianchi-Berthouze

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

Data from multiple sensors can boost the automatic recognition of multiple affective states in a multilabel and multimodal recognition system. At any time, the streaming from any of the contributing sensors can be missing. This work proposes a method for dealing with a missing sensor in a multilabel and multimodal automatic affective states recognition system. The proposed method, called Hot Deck using Conditional Probability Tables (HD-CPT), is incorporated into a multimodal affective state recognition system for compensating the loss of a sensor using the recorded historical information of the sensor and its interaction with the other available sensors. In this work, we consider a multilabel classifier, named Circular Classifier Chain, for the automatic recognition of four states: tiredness, anxiety, pain, and engagement; combined with a multimodal classifier based on three sensors: fingers pressure, hand movements, and facial expressions; which was adapted for coping with the problem of a missing sensor in a virtual rehabilitation platform for post-stroke patients. A dataset of five post-stroke patients who attended ten longitudinal rehabilitation sessions was used for the evaluation. The inclusion of HD-CPT compensated for the loss of one sensor with results above those obtained with only the remaining sensors available. HD-CPT prevents the system from collapsing when a sensor fails, providing continuity of operation with results that attenuate the loss of the sensor. The proposed method HD-CPT can provide robustness for the naturalistic everyday use of an affective states recognition system.

Original languageEnglish
Title of host publication2021 9th International Conference on Affective Computing and Intelligent Interaction (ACII)
PublisherInstitute of Electrical and Electronics Engineers (IEEE)
Pages1-8
Number of pages8
ISBN (Electronic)9781665400190
ISBN (Print)9781665400206 (PoD)
DOIs
Publication statusPublished - 15 Nov 2021
Event9th International Conference on Affective Computing and Intelligent Interaction, ACII 2021 - Nara, Japan
Duration: 28 Sep 20211 Oct 2021

Publication series

NameInternational Conference on Affective Computing and Intelligent Interaction and workshops
PublisherIEEE
ISSN (Print)2156-8103
ISSN (Electronic)2156-8111

Conference

Conference9th International Conference on Affective Computing and Intelligent Interaction, ACII 2021
Country/TerritoryJapan
CityNara
Period28/09/211/10/21

Bibliographical note

Funding Information:
This project was supported by a grant from the project FORDECyT 296737 “Consorcio en Inteligencia Artificial”. We would also like to acknowledge support for the Scholarship No. 434867 from the Mexican Research Council CONACYT.

Publisher Copyright:
© 2021 IEEE.

Keywords

  • automatic affective states recognition
  • classifier chains
  • facial expressions
  • fingers pressure
  • hand movements
  • missing sensor
  • multilabel classification
  • multimodal classification
  • virtual rehabilitation

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computer Vision and Pattern Recognition
  • Human-Computer Interaction

Fingerprint

Dive into the research topics of 'Dealing with a missing sensor in a multilabel and multimodal automatic affective states recognition system'. Together they form a unique fingerprint.

Cite this