HARK No More:: On the Preregistration of CHI Experiments

Andy Cockburn; Carl Gutwin; Alan Dix

doi:10.1145/3173574.3173715

HARK No More: On the Preregistration of CHI Experiments

Andy Cockburn, Carl Gutwin, Alan Dix

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

29 Citations (Scopus)

Abstract

Experimental preregistration is required for publication in many scientific disciplines and venues. When experimental intentions are preregistered, reviewers and readers can be confident that experimental evidence in support of reported hypotheses is not the result of HARKing, which stands for Hypothesising After the Results are Known. We review the motivation and outcomes of experimental preregistration across a variety of disciplines, as well as previous work commenting on the role of evaluation in HCI research. We then discuss how experimental preregistration could be adapted to the distinctive
characteristics of Human-Computer Interaction empirical research, to the betterment of the discipline.

Original language	English
Title of host publication	Proceedings of ACM CHI 2018 Conference on Human Factors in Computing Systems
Publisher	Association for Computing Machinery
Number of pages	12
ISBN (Print)	978-1-4503-5620-6
DOIs	https://doi.org/10.1145/3173574.3173715
Publication status	Published - 21 Apr 2018
Event	ACM CHI 2018 Conference on Human Factors in Computing Systems (CHI 2018) - Montreal, Canada Duration: 21 Apr 2018 → 26 Apr 2018 https://chi2018.acm.org/

Conference

Conference	ACM CHI 2018 Conference on Human Factors in Computing Systems (CHI 2018)
Country/Territory	Canada
City	Montreal
Period	21/04/18 → 26/04/18
Internet address	https://chi2018.acm.org/

Bibliographical note

CHI 2018 - Best paper award.

Access to Document

10.1145/3173574.3173715Licence: None: All rights reserved

Cite this

@inproceedings{52fb344157d54ea79ff7450d43fffcfb,

title = "HARK No More:: On the Preregistration of CHI Experiments",

abstract = "Experimental preregistration is required for publication in many scientific disciplines and venues. When experimental intentions are preregistered, reviewers and readers can be confident that experimental evidence in support of reported hypotheses is not the result of HARKing, which stands for Hypothesising After the Results are Known. We review the motivation and outcomes of experimental preregistration across a variety of disciplines, as well as previous work commenting on the role of evaluation in HCI research. We then discuss how experimental preregistration could be adapted to the distinctivecharacteristics of Human-Computer Interaction empirical research, to the betterment of the discipline.",

author = "Andy Cockburn and Carl Gutwin and Alan Dix",

note = "CHI 2018 - Best paper award.; ACM CHI 2018 Conference on Human Factors in Computing Systems (CHI 2018) ; Conference date: 21-04-2018 Through 26-04-2018",

year = "2018",

month = apr,

day = "21",

doi = "10.1145/3173574.3173715",

language = "English",

isbn = "978-1-4503-5620-6 ",

booktitle = "Proceedings of ACM CHI 2018 Conference on Human Factors in Computing Systems",

publisher = "Association for Computing Machinery ",

url = "https://chi2018.acm.org/",

}

TY - GEN

T1 - HARK No More:

T2 - ACM CHI 2018 Conference on Human Factors in Computing Systems (CHI 2018)

AU - Cockburn, Andy

AU - Gutwin, Carl

AU - Dix, Alan

N1 - CHI 2018 - Best paper award.

PY - 2018/4/21

Y1 - 2018/4/21

N2 - Experimental preregistration is required for publication in many scientific disciplines and venues. When experimental intentions are preregistered, reviewers and readers can be confident that experimental evidence in support of reported hypotheses is not the result of HARKing, which stands for Hypothesising After the Results are Known. We review the motivation and outcomes of experimental preregistration across a variety of disciplines, as well as previous work commenting on the role of evaluation in HCI research. We then discuss how experimental preregistration could be adapted to the distinctivecharacteristics of Human-Computer Interaction empirical research, to the betterment of the discipline.

AB - Experimental preregistration is required for publication in many scientific disciplines and venues. When experimental intentions are preregistered, reviewers and readers can be confident that experimental evidence in support of reported hypotheses is not the result of HARKing, which stands for Hypothesising After the Results are Known. We review the motivation and outcomes of experimental preregistration across a variety of disciplines, as well as previous work commenting on the role of evaluation in HCI research. We then discuss how experimental preregistration could be adapted to the distinctivecharacteristics of Human-Computer Interaction empirical research, to the betterment of the discipline.

U2 - 10.1145/3173574.3173715

DO - 10.1145/3173574.3173715

M3 - Conference contribution

SN - 978-1-4503-5620-6

BT - Proceedings of ACM CHI 2018 Conference on Human Factors in Computing Systems

PB - Association for Computing Machinery

Y2 - 21 April 2018 through 26 April 2018