Touchscreen typing as optimal supervisory control

Jussi P.P. Jokinen, Aditya Acharya, Mohammad Uzair, Xinhui Jiang, Antti Oulasvirta

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Citations (Scopus)
35 Downloads (Pure)

Abstract

Traditionally, touchscreen typing has been studied in terms of motor performance. However, recent research has exposed a decisive role of visual attention being shared between the keyboard and the text area. Strategies for this are known to adapt to the task, design, and user. In this paper, we propose a unifying account of touchscreen typing, regarding it as optimal supervisory control. Under this theory, rules for controlling visuo-motor resources are learned via exploration in pursuit of maximal typing performance. The paper outlines the control problem and explains how visual and motor limitations afect it. We then present a model, implemented via reinforcement learning, that simulates co-ordination of eye and fnger movements. Comparison with human data affrms that the model creates realistic fnger-and eye-movement patterns and shows human-like adaptation. We demonstrate the model's utility for interface development in evaluating touchscreen keyboard designs.

Original languageEnglish
Title of host publicationCHI '21
Subtitle of host publicationProceedings of the 2021 CHI Conference on Human Factors in Computing Systems
PublisherAssociation for Computing Machinery
ISBN (Electronic)9781450380966
DOIs
Publication statusPublished - 6 May 2021
EventCHI '21: CHI Conference on Human Factors in Computing Systems: Making Waves, Combining Strengths - Virtual, Virtual, Online, Japan
Duration: 8 May 202113 May 2021
https://chi2021.acm.org/

Publication series

NameCHI: Conference on Human Factors in Computing Systems

Conference

ConferenceCHI '21: CHI Conference on Human Factors in Computing Systems
Abbreviated titleCHI 2021
Country/TerritoryJapan
CityVirtual, Online
Period8/05/2113/05/21
Internet address

Bibliographical note

Acknowledgments
This research has been supported by the Academy of Finland projects BAD and Human Automata, and Finnish Center for AI (FCAI). Anqi Yang helped with the UI development. The project has a website at
https://userinterfaces.aalto.f/touchscreen-typing/

Keywords

  • Computational modelling
  • Rational adaptation
  • Touchscreen typing

ASJC Scopus subject areas

  • Human-Computer Interaction
  • Computer Graphics and Computer-Aided Design
  • Software

Fingerprint

Dive into the research topics of 'Touchscreen typing as optimal supervisory control'. Together they form a unique fingerprint.

Cite this