Human operator cognitive availability aware Mixed-Initiative control

Giannis Petousakis; Manolis Chiou; Grigoris Nikolaou; Rustam Stolkin

doi:10.1109/ICHMS49158.2020.9209582

Human operator cognitive availability aware Mixed-Initiative control

Giannis Petousakis, Manolis Chiou, Grigoris Nikolaou, Rustam Stolkin

Metallurgy and Materials

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

This paper presents a Cognitive Availability Aware Mixed-Initiative Controller for remotely operated mobile robots. The controller enables dynamic switching between different levels of autonomy (LOA), initiated by either the AI and/or the human operator. The controller leverages a state-of-The-Art computer vision method and an off-The-shelf web camera to infer the cognitive availability of the operator and inform the AI-initiated LOA switching. This constitutes a qualitative advancement over previous Mixed-Initiative (MI) controllers. The controller is evaluated in a disaster response experiment, in which human operators have to conduct an exploration task with a remote robot. MI systems are shown to effectively assist the operators, as demonstrated by quantitative and qualitative results in performance and workload. Additionally, some insights into the experimental difficulties of evaluating complex MI controllers are presented.

Original language	English
Title of host publication	Proceedings of the 2020 IEEE International Conference on Human-Machine Systems, ICHMS 2020
Editors	Giancarlo Fortino, Fei-Yue Wang, Andreas Nurnberger, David Kaber, Rino Falcone, David Mendonca, Zhiwen Yu, Antonio Guerrieri
Publisher	Institute of Electrical and Electronics Engineers (IEEE)
ISBN (Electronic)	9781728158716
DOIs	https://doi.org/10.1109/ICHMS49158.2020.9209582
Publication status	Published - Sept 2020
Event	1st IEEE International Conference on Human-Machine Systems, ICHMS 2020 - Virtual, Rome, Italy Duration: 7 Sept 2020 → 9 Sept 2020

Publication series

Name	Proceedings of the 2020 IEEE International Conference on Human-Machine Systems, ICHMS 2020

Conference

Conference	1st IEEE International Conference on Human-Machine Systems, ICHMS 2020
Country/Territory	Italy
City	Virtual, Rome
Period	7/09/20 → 9/09/20

Bibliographical note

Funding Information:
ACKNOWLEDGMENT This work was supported by the following grants of UKRI-EPSRC: EP/P017487/1 (Remote Sensing in Extreme Environments); EP/R02572X/1 (National Centre for Nuclear Robotics); EP/P01366X/1 (Robotics for Nuclear Environments). Stolkin was also sponsored by a Royal Society Industry Fellowship.

Publisher Copyright:
© 2020 IEEE.

Copyright:
Copyright 2020 Elsevier B.V., All rights reserved.

ASJC Scopus subject areas

Human-Computer Interaction

Access to Document

10.1109/ICHMS49158.2020.9209582

Cite this

Petousakis, G., Chiou, M., Nikolaou, G., & Stolkin, R. (2020). Human operator cognitive availability aware Mixed-Initiative control. In G. Fortino, F-Y. Wang, A. Nurnberger, D. Kaber, R. Falcone, D. Mendonca, Z. Yu, & A. Guerrieri (Eds.), Proceedings of the 2020 IEEE International Conference on Human-Machine Systems, ICHMS 2020 [9209582] (Proceedings of the 2020 IEEE International Conference on Human-Machine Systems, ICHMS 2020). Institute of Electrical and Electronics Engineers (IEEE). https://doi.org/10.1109/ICHMS49158.2020.9209582

Petousakis, Giannis ; Chiou, Manolis ; Nikolaou, Grigoris et al. / Human operator cognitive availability aware Mixed-Initiative control. Proceedings of the 2020 IEEE International Conference on Human-Machine Systems, ICHMS 2020. editor / Giancarlo Fortino ; Fei-Yue Wang ; Andreas Nurnberger ; David Kaber ; Rino Falcone ; David Mendonca ; Zhiwen Yu ; Antonio Guerrieri. Institute of Electrical and Electronics Engineers (IEEE), 2020. (Proceedings of the 2020 IEEE International Conference on Human-Machine Systems, ICHMS 2020).

@inproceedings{b9de2327427f415d8fe9dbadf39be8fb,

title = "Human operator cognitive availability aware Mixed-Initiative control",

abstract = "This paper presents a Cognitive Availability Aware Mixed-Initiative Controller for remotely operated mobile robots. The controller enables dynamic switching between different levels of autonomy (LOA), initiated by either the AI and/or the human operator. The controller leverages a state-of-The-Art computer vision method and an off-The-shelf web camera to infer the cognitive availability of the operator and inform the AI-initiated LOA switching. This constitutes a qualitative advancement over previous Mixed-Initiative (MI) controllers. The controller is evaluated in a disaster response experiment, in which human operators have to conduct an exploration task with a remote robot. MI systems are shown to effectively assist the operators, as demonstrated by quantitative and qualitative results in performance and workload. Additionally, some insights into the experimental difficulties of evaluating complex MI controllers are presented.",

author = "Giannis Petousakis and Manolis Chiou and Grigoris Nikolaou and Rustam Stolkin",

note = "Funding Information: ACKNOWLEDGMENT This work was supported by the following grants of UKRI-EPSRC: EP/P017487/1 (Remote Sensing in Extreme Environments); EP/R02572X/1 (National Centre for Nuclear Robotics); EP/P01366X/1 (Robotics for Nuclear Environments). Stolkin was also sponsored by a Royal Society Industry Fellowship. Publisher Copyright: {\textcopyright} 2020 IEEE. Copyright: Copyright 2020 Elsevier B.V., All rights reserved.; 1st IEEE International Conference on Human-Machine Systems, ICHMS 2020 ; Conference date: 07-09-2020 Through 09-09-2020",

year = "2020",

month = sep,

doi = "10.1109/ICHMS49158.2020.9209582",

language = "English",

series = "Proceedings of the 2020 IEEE International Conference on Human-Machine Systems, ICHMS 2020",

publisher = "Institute of Electrical and Electronics Engineers (IEEE)",

editor = "Giancarlo Fortino and Fei-Yue Wang and Andreas Nurnberger and David Kaber and Rino Falcone and David Mendonca and Zhiwen Yu and Antonio Guerrieri",

booktitle = "Proceedings of the 2020 IEEE International Conference on Human-Machine Systems, ICHMS 2020",

}

Petousakis, G, Chiou, M, Nikolaou, G & Stolkin, R 2020, Human operator cognitive availability aware Mixed-Initiative control. in G Fortino, F-Y Wang, A Nurnberger, D Kaber, R Falcone, D Mendonca, Z Yu & A Guerrieri (eds), Proceedings of the 2020 IEEE International Conference on Human-Machine Systems, ICHMS 2020., 9209582, Proceedings of the 2020 IEEE International Conference on Human-Machine Systems, ICHMS 2020, Institute of Electrical and Electronics Engineers (IEEE), 1st IEEE International Conference on Human-Machine Systems, ICHMS 2020, Virtual, Rome, Italy, 7/09/20. https://doi.org/10.1109/ICHMS49158.2020.9209582

Human operator cognitive availability aware Mixed-Initiative control. / Petousakis, Giannis; Chiou, Manolis; Nikolaou, Grigoris et al.

Proceedings of the 2020 IEEE International Conference on Human-Machine Systems, ICHMS 2020. ed. / Giancarlo Fortino; Fei-Yue Wang; Andreas Nurnberger; David Kaber; Rino Falcone; David Mendonca; Zhiwen Yu; Antonio Guerrieri. Institute of Electrical and Electronics Engineers (IEEE), 2020. 9209582 (Proceedings of the 2020 IEEE International Conference on Human-Machine Systems, ICHMS 2020).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Human operator cognitive availability aware Mixed-Initiative control

AU - Petousakis, Giannis

AU - Chiou, Manolis

AU - Nikolaou, Grigoris

AU - Stolkin, Rustam

N1 - Funding Information: ACKNOWLEDGMENT This work was supported by the following grants of UKRI-EPSRC: EP/P017487/1 (Remote Sensing in Extreme Environments); EP/R02572X/1 (National Centre for Nuclear Robotics); EP/P01366X/1 (Robotics for Nuclear Environments). Stolkin was also sponsored by a Royal Society Industry Fellowship. Publisher Copyright: © 2020 IEEE. Copyright: Copyright 2020 Elsevier B.V., All rights reserved.

PY - 2020/9

Y1 - 2020/9

N2 - This paper presents a Cognitive Availability Aware Mixed-Initiative Controller for remotely operated mobile robots. The controller enables dynamic switching between different levels of autonomy (LOA), initiated by either the AI and/or the human operator. The controller leverages a state-of-The-Art computer vision method and an off-The-shelf web camera to infer the cognitive availability of the operator and inform the AI-initiated LOA switching. This constitutes a qualitative advancement over previous Mixed-Initiative (MI) controllers. The controller is evaluated in a disaster response experiment, in which human operators have to conduct an exploration task with a remote robot. MI systems are shown to effectively assist the operators, as demonstrated by quantitative and qualitative results in performance and workload. Additionally, some insights into the experimental difficulties of evaluating complex MI controllers are presented.

AB - This paper presents a Cognitive Availability Aware Mixed-Initiative Controller for remotely operated mobile robots. The controller enables dynamic switching between different levels of autonomy (LOA), initiated by either the AI and/or the human operator. The controller leverages a state-of-The-Art computer vision method and an off-The-shelf web camera to infer the cognitive availability of the operator and inform the AI-initiated LOA switching. This constitutes a qualitative advancement over previous Mixed-Initiative (MI) controllers. The controller is evaluated in a disaster response experiment, in which human operators have to conduct an exploration task with a remote robot. MI systems are shown to effectively assist the operators, as demonstrated by quantitative and qualitative results in performance and workload. Additionally, some insights into the experimental difficulties of evaluating complex MI controllers are presented.

UR - http://www.scopus.com/inward/record.url?scp=85093976594&partnerID=8YFLogxK

U2 - 10.1109/ICHMS49158.2020.9209582

DO - 10.1109/ICHMS49158.2020.9209582

M3 - Conference contribution

AN - SCOPUS:85093976594

T3 - Proceedings of the 2020 IEEE International Conference on Human-Machine Systems, ICHMS 2020

BT - Proceedings of the 2020 IEEE International Conference on Human-Machine Systems, ICHMS 2020

A2 - Fortino, Giancarlo

A2 - Wang, Fei-Yue

A2 - Nurnberger, Andreas

A2 - Kaber, David

A2 - Falcone, Rino

A2 - Mendonca, David

A2 - Yu, Zhiwen

A2 - Guerrieri, Antonio

PB - Institute of Electrical and Electronics Engineers (IEEE)

T2 - 1st IEEE International Conference on Human-Machine Systems, ICHMS 2020

Y2 - 7 September 2020 through 9 September 2020

ER -

Petousakis G, Chiou M, Nikolaou G, Stolkin R. Human operator cognitive availability aware Mixed-Initiative control. In Fortino G, Wang F-Y, Nurnberger A, Kaber D, Falcone R, Mendonca D, Yu Z, Guerrieri A, editors, Proceedings of the 2020 IEEE International Conference on Human-Machine Systems, ICHMS 2020. Institute of Electrical and Electronics Engineers (IEEE). 2020. 9209582. (Proceedings of the 2020 IEEE International Conference on Human-Machine Systems, ICHMS 2020). doi: 10.1109/ICHMS49158.2020.9209582

Human operator cognitive availability aware Mixed-Initiative control

Abstract

Publication series

Conference

Bibliographical note

ASJC Scopus subject areas

Access to Document

Fingerprint

Cite this