A Hybrid DCNN-SVM Model for Classifying Neonatal Sleep and Wake States Based on Facial Expressions in Video

Muhammad Awais; Xi Long; Bin Yin; Saadullah Farooq Abbasi; Saeed Akbarzadeh; Chunmei Lu; Xinhua Wang; Laishuan Wang; Jiong Zhang; Jeroen Dudink; Wei Chen

doi:10.1109/JBHI.2021.3073632

A Hybrid DCNN-SVM Model for Classifying Neonatal Sleep and Wake States Based on Facial Expressions in Video

Muhammad Awais, Xi Long^*, Bin Yin, Saadullah Farooq Abbasi, Saeed Akbarzadeh, Chunmei Lu^*, Xinhua Wang, Laishuan Wang, Jiong Zhang, Jeroen Dudink, Wei Chen^*

^*Corresponding author for this work

Electronic, Electrical and Systems Engineering

Research output: Contribution to journal › Article › peer-review

Abstract

Sleep is a natural phenomenon controlled by the central nervous system. The sleep-wake pattern, which functions as an essential indicator of neurophysiological organization in the neonatal period, has profound meaning in the prediction of cognitive diseases and brain maturity. In recent years, unobtrusive sleep monitoring and automatic sleep staging have been intensively studied for adults, but much less for neonates. This work aims to investigate a novel video-based unobtrusive method for neonatal sleep-wake classification by analyzing the behavioral changes in the neonatal facial region. A hybrid model is proposed to monitor the sleep-wake patterns of human neonates. The model combines two algorithms: deep convolutional neural network (DCNN) and support vector machine (SVM), where DCNN works as a trainable feature extractor and SVM as a classifier. Data was collected from nineteen Chinese neonates at the Children's Hospital of Fudan University, Shanghai, China. The classification results are compared with the gold standard of video-electroencephalography scored by pediatric neurologists. Validations indicate that the proposed hybrid DCNN-SVM model achieved reliable performances in classifying neonatal sleep and wake states in RGB video frames (with the face region detected), with an accuracy of 93.8 ± 2.2% and an F1-score 0.93 ± 0.3.

Original language	English
Article number	9405399
Pages (from-to)	1441-1449
Number of pages	9
Journal	IEEE Journal of Biomedical and Health Informatics
Volume	25
Issue number	5
Early online date	15 Apr 2021
DOIs	https://doi.org/10.1109/JBHI.2021.3073632
Publication status	Published - May 2021

Bibliographical note

Funding:
This work was supported in part by the Shanghai Municipal Science and Technology Major Project under Grant 2017SHZDZX01, and in part by the National Key R&D Program of China under Grant 2017YFE0112000.

Keywords

Neonatal sleep monitoring
video and image analysis
facial expression
deep convolutional neural network
support vector machine

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

Access to Document

10.1109/JBHI.2021.3073632

Cite this

@article{dbba0b9cc362473db7adf1c0b03a1b43,

title = "A Hybrid DCNN-SVM Model for Classifying Neonatal Sleep and Wake States Based on Facial Expressions in Video",

abstract = "Sleep is a natural phenomenon controlled by the central nervous system. The sleep-wake pattern, which functions as an essential indicator of neurophysiological organization in the neonatal period, has profound meaning in the prediction of cognitive diseases and brain maturity. In recent years, unobtrusive sleep monitoring and automatic sleep staging have been intensively studied for adults, but much less for neonates. This work aims to investigate a novel video-based unobtrusive method for neonatal sleep-wake classification by analyzing the behavioral changes in the neonatal facial region. A hybrid model is proposed to monitor the sleep-wake patterns of human neonates. The model combines two algorithms: deep convolutional neural network (DCNN) and support vector machine (SVM), where DCNN works as a trainable feature extractor and SVM as a classifier. Data was collected from nineteen Chinese neonates at the Children's Hospital of Fudan University, Shanghai, China. The classification results are compared with the gold standard of video-electroencephalography scored by pediatric neurologists. Validations indicate that the proposed hybrid DCNN-SVM model achieved reliable performances in classifying neonatal sleep and wake states in RGB video frames (with the face region detected), with an accuracy of 93.8 ± 2.2% and an F1-score 0.93 ± 0.3.",

keywords = "Neonatal sleep monitoring, video and image analysis, facial expression, deep convolutional neural network, support vector machine",

author = "Muhammad Awais and Xi Long and Bin Yin and Abbasi, {Saadullah Farooq} and Saeed Akbarzadeh and Chunmei Lu and Xinhua Wang and Laishuan Wang and Jiong Zhang and Jeroen Dudink and Wei Chen",

note = "Funding: This work was supported in part by the Shanghai Municipal Science and Technology Major Project under Grant 2017SHZDZX01, and in part by the National Key R&D Program of China under Grant 2017YFE0112000.",

year = "2021",

month = may,

doi = "10.1109/JBHI.2021.3073632",

language = "English",

volume = "25",

pages = "1441--1449",

journal = "IEEE Journal of Biomedical and Health Informatics",

number = "5",

}

TY - JOUR

T1 - A Hybrid DCNN-SVM Model for Classifying Neonatal Sleep and Wake States Based on Facial Expressions in Video

AU - Awais, Muhammad

AU - Long, Xi

AU - Yin, Bin

AU - Abbasi, Saadullah Farooq

AU - Akbarzadeh, Saeed

AU - Lu, Chunmei

AU - Wang, Xinhua

AU - Wang, Laishuan

AU - Zhang, Jiong

AU - Dudink, Jeroen

AU - Chen, Wei

N1 - Funding: This work was supported in part by the Shanghai Municipal Science and Technology Major Project under Grant 2017SHZDZX01, and in part by the National Key R&D Program of China under Grant 2017YFE0112000.

PY - 2021/5

Y1 - 2021/5

N2 - Sleep is a natural phenomenon controlled by the central nervous system. The sleep-wake pattern, which functions as an essential indicator of neurophysiological organization in the neonatal period, has profound meaning in the prediction of cognitive diseases and brain maturity. In recent years, unobtrusive sleep monitoring and automatic sleep staging have been intensively studied for adults, but much less for neonates. This work aims to investigate a novel video-based unobtrusive method for neonatal sleep-wake classification by analyzing the behavioral changes in the neonatal facial region. A hybrid model is proposed to monitor the sleep-wake patterns of human neonates. The model combines two algorithms: deep convolutional neural network (DCNN) and support vector machine (SVM), where DCNN works as a trainable feature extractor and SVM as a classifier. Data was collected from nineteen Chinese neonates at the Children's Hospital of Fudan University, Shanghai, China. The classification results are compared with the gold standard of video-electroencephalography scored by pediatric neurologists. Validations indicate that the proposed hybrid DCNN-SVM model achieved reliable performances in classifying neonatal sleep and wake states in RGB video frames (with the face region detected), with an accuracy of 93.8 ± 2.2% and an F1-score 0.93 ± 0.3.

AB - Sleep is a natural phenomenon controlled by the central nervous system. The sleep-wake pattern, which functions as an essential indicator of neurophysiological organization in the neonatal period, has profound meaning in the prediction of cognitive diseases and brain maturity. In recent years, unobtrusive sleep monitoring and automatic sleep staging have been intensively studied for adults, but much less for neonates. This work aims to investigate a novel video-based unobtrusive method for neonatal sleep-wake classification by analyzing the behavioral changes in the neonatal facial region. A hybrid model is proposed to monitor the sleep-wake patterns of human neonates. The model combines two algorithms: deep convolutional neural network (DCNN) and support vector machine (SVM), where DCNN works as a trainable feature extractor and SVM as a classifier. Data was collected from nineteen Chinese neonates at the Children's Hospital of Fudan University, Shanghai, China. The classification results are compared with the gold standard of video-electroencephalography scored by pediatric neurologists. Validations indicate that the proposed hybrid DCNN-SVM model achieved reliable performances in classifying neonatal sleep and wake states in RGB video frames (with the face region detected), with an accuracy of 93.8 ± 2.2% and an F1-score 0.93 ± 0.3.

KW - Neonatal sleep monitoring

KW - video and image analysis

KW - facial expression

KW - deep convolutional neural network

KW - support vector machine

U2 - 10.1109/JBHI.2021.3073632

DO - 10.1109/JBHI.2021.3073632

M3 - Article

VL - 25

SP - 1441

EP - 1449

JO - IEEE Journal of Biomedical and Health Informatics

JF - IEEE Journal of Biomedical and Health Informatics

IS - 5

M1 - 9405399

ER -

A Hybrid DCNN-SVM Model for Classifying Neonatal Sleep and Wake States Based on Facial Expressions in Video

Abstract

Bibliographical note

Keywords

UN SDGs

Access to Document

Fingerprint

Cite this