Compensating for object variability in DNN–HMM object-centered human activity recognition

Research output: Chapter in Book/Report/Conference proceedingConference contribution


This paper describes a deep neural network –hidden Markov model (DNN-HMM) human activity recognition system based on instrumented objects and studies compensation strategies to deal with object variability. The sensors, comprising an accelerometer, gyroscope, magnetometer and force-sensitive resistors (FSRs), are packaged in a coaster attached to the base of an object, here a mug. Results are presented for recognition of actions involved in manipulating a mug. Evaluations are performed using over 24 hours of data recordings containing sequences of actions, labelled without time-stamp information. We demonstrate the importance of data alignments. While the DNN-HMM system achieved error rate below 0.1% for matched train-test conditions, this increased up to 26.5% for highly mismatched conditions. The error rate averaged over all conditions was 1.4% when using multi-condition training and decreased to 0.8% by employing feature augmentation. The use of FSR feature compensation, specific to weight variability, resulted in 0.24% error rate.


Original languageEnglish
Title of host publication2019 27th European Signal Processing Conference (EUSIPCO)
Publication statusPublished - 18 Nov 2019
Event27th European Signal Processing Conference - A Coruña, Spain
Duration: 2 Sep 20196 Sep 2019

Publication series

NameSignal Processing Conference (EUSIPCO), European
ISSN (Print)2219-5491
ISSN (Electronic)2076-1465


Conference27th European Signal Processing Conference
Abbreviated titleEUSIPCO 2019
CityA Coruña


  • Action recognition, Compensation, DNN-HMM, Deep neural networks, Feature augmentation, Hidden Markov models, Instrumented objects, Sensors