Object-centred recognition of human activity

This paper describes an approach to real-time human activity recognition using hidden Markov models (HMMs) and sensorised objects, and its application to rehabilitation of stroke patients with apraxia or action disorganisation syndrome
(AADS). Results are presented for the task of making a cup of tea. Unlike speech or other sequential decoding problems where HMMs have previously been successfully applied, human actions can occur simultaneously or at least in overlapping time. The solution proposed in this paper is based on a parallel,
asynchronous set of detectors, each responsible for the detection of one of the component sub-goals of the tea-making task. The inputs to these detectors are formed from the outputs of sensors attached to the objects involved in that sub-goal, plus hand coordinate data. The sensors, comprising an accelerometer and
three force-sensitive resistors, are packaged in a coaster which can be easily attached to the base of a mug or jug. In tests on complete tea-making trials, error rates range from less than 5% for sub-goals where all of the objects involved are sensorised, to up to 30% for detectors that rely on hand-coordinate data alone. The complete set of detectors runs in real-time. It is concluded that
a set of parallel HMM-based sub-goal detectors combined with fully sensorised objects, is a viable, accurate and easily deployable approach to real-time object-centred human activity recognition.