The effect of known decision support reliability on outcome quality and visual information foraging in joint decision making

Sandra Dorothee Starke; Chris Baber

doi:10.1016/j.apergo.2020.103102

The effect of known decision support reliability on outcome quality and visual information foraging in joint decision making

Sandra Dorothee Starke^*, Chris Baber

^*Corresponding author for this work

Research output: Contribution to journal › Article › peer-review

4 Citations (Scopus)

Abstract

Decision support systems (DSSs) are being woven into human workflows from aviation to medicine. In this study, we examine decision quality and visual information foraging for DSSs with different known reliability levels. Thirty-six participants completed a financial fraud detection task, first unsupported and then supported by a DSS which highlighted important information sources. Participants were randomly allocated to four cohorts, being informed that the system's reliability was 100%, 90%, 80% or undisclosed. Results showed that only a DSS known to be 100% reliable resulted in participants systematically following its suggestions, increasing the percentage of correct classifications to a median of 100% while halving both, decision time and number of visually attended information sources. In all other conditions, the DSS had no effect on most visual sampling metrics, while decision quality of the human-DSS team was below the reliability level of the DSS. Knowledge of an even slightly unreliable system hence had a profound impact on joint decision making, with participants trusting their significantly worse performance more than the DSSs suggestions.

Original language	English
Article number	103102
Journal	Applied Ergonomics
Volume	86
DOIs	https://doi.org/10.1016/j.apergo.2020.103102
Publication status	Published - Jul 2020

Bibliographical note

Funding Information:
We would like to thank all of our participants for their time, Joanne Claire Kitchen for her help with mobilising participants and two anonymous reviewers for their constructive feedback on the original manuscript draft. This work was funded through the European Union FP7 project SPEEDD (grant number 619435 ).

Funding Information:
We would like to thank all of our participants for their time, Joanne Claire Kitchen for her help with mobilising participants and two anonymous reviewers for their constructive feedback on the original manuscript draft. This work was funded through the European Union FP7 project SPEEDD (grant number 619435).

Publisher Copyright:
© 2020 Elsevier Ltd

ASJC Scopus subject areas

Human Factors and Ergonomics
Physical Therapy, Sports Therapy and Rehabilitation
Safety, Risk, Reliability and Quality
Engineering (miscellaneous)

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

Access to Document

10.1016/j.apergo.2020.103102

Cite this

@article{abe3c11a2f3a457b850c13cff89f692b,

title = "The effect of known decision support reliability on outcome quality and visual information foraging in joint decision making",

abstract = "Decision support systems (DSSs) are being woven into human workflows from aviation to medicine. In this study, we examine decision quality and visual information foraging for DSSs with different known reliability levels. Thirty-six participants completed a financial fraud detection task, first unsupported and then supported by a DSS which highlighted important information sources. Participants were randomly allocated to four cohorts, being informed that the system's reliability was 100%, 90%, 80% or undisclosed. Results showed that only a DSS known to be 100% reliable resulted in participants systematically following its suggestions, increasing the percentage of correct classifications to a median of 100% while halving both, decision time and number of visually attended information sources. In all other conditions, the DSS had no effect on most visual sampling metrics, while decision quality of the human-DSS team was below the reliability level of the DSS. Knowledge of an even slightly unreliable system hence had a profound impact on joint decision making, with participants trusting their significantly worse performance more than the DSSs suggestions.",

author = "Starke, {Sandra Dorothee} and Chris Baber",

note = "Funding Information: We would like to thank all of our participants for their time, Joanne Claire Kitchen for her help with mobilising participants and two anonymous reviewers for their constructive feedback on the original manuscript draft. This work was funded through the European Union FP7 project SPEEDD (grant number 619435 ). Funding Information: We would like to thank all of our participants for their time, Joanne Claire Kitchen for her help with mobilising participants and two anonymous reviewers for their constructive feedback on the original manuscript draft. This work was funded through the European Union FP7 project SPEEDD (grant number 619435). Publisher Copyright: {\textcopyright} 2020 Elsevier Ltd",

year = "2020",

month = jul,

doi = "10.1016/j.apergo.2020.103102",

language = "English",

volume = "86",

journal = "Applied Ergonomics",

issn = "0003-6870",

publisher = "Elsevier",

}

TY - JOUR

T1 - The effect of known decision support reliability on outcome quality and visual information foraging in joint decision making

AU - Starke, Sandra Dorothee

AU - Baber, Chris

N1 - Funding Information: We would like to thank all of our participants for their time, Joanne Claire Kitchen for her help with mobilising participants and two anonymous reviewers for their constructive feedback on the original manuscript draft. This work was funded through the European Union FP7 project SPEEDD (grant number 619435 ). Funding Information: We would like to thank all of our participants for their time, Joanne Claire Kitchen for her help with mobilising participants and two anonymous reviewers for their constructive feedback on the original manuscript draft. This work was funded through the European Union FP7 project SPEEDD (grant number 619435). Publisher Copyright: © 2020 Elsevier Ltd

PY - 2020/7

Y1 - 2020/7

N2 - Decision support systems (DSSs) are being woven into human workflows from aviation to medicine. In this study, we examine decision quality and visual information foraging for DSSs with different known reliability levels. Thirty-six participants completed a financial fraud detection task, first unsupported and then supported by a DSS which highlighted important information sources. Participants were randomly allocated to four cohorts, being informed that the system's reliability was 100%, 90%, 80% or undisclosed. Results showed that only a DSS known to be 100% reliable resulted in participants systematically following its suggestions, increasing the percentage of correct classifications to a median of 100% while halving both, decision time and number of visually attended information sources. In all other conditions, the DSS had no effect on most visual sampling metrics, while decision quality of the human-DSS team was below the reliability level of the DSS. Knowledge of an even slightly unreliable system hence had a profound impact on joint decision making, with participants trusting their significantly worse performance more than the DSSs suggestions.

AB - Decision support systems (DSSs) are being woven into human workflows from aviation to medicine. In this study, we examine decision quality and visual information foraging for DSSs with different known reliability levels. Thirty-six participants completed a financial fraud detection task, first unsupported and then supported by a DSS which highlighted important information sources. Participants were randomly allocated to four cohorts, being informed that the system's reliability was 100%, 90%, 80% or undisclosed. Results showed that only a DSS known to be 100% reliable resulted in participants systematically following its suggestions, increasing the percentage of correct classifications to a median of 100% while halving both, decision time and number of visually attended information sources. In all other conditions, the DSS had no effect on most visual sampling metrics, while decision quality of the human-DSS team was below the reliability level of the DSS. Knowledge of an even slightly unreliable system hence had a profound impact on joint decision making, with participants trusting their significantly worse performance more than the DSSs suggestions.

UR - http://www.scopus.com/inward/record.url?scp=85082719754&partnerID=8YFLogxK

U2 - 10.1016/j.apergo.2020.103102

DO - 10.1016/j.apergo.2020.103102

M3 - Article

C2 - 32342892

AN - SCOPUS:85082719754

SN - 0003-6870

VL - 86

JO - Applied Ergonomics

JF - Applied Ergonomics

M1 - 103102

ER -

The effect of known decision support reliability on outcome quality and visual information foraging in joint decision making

Abstract

Bibliographical note

ASJC Scopus subject areas

UN SDGs

Access to Document

Fingerprint

Cite this