Towards sound accessibility in virtual reality

Dhruv Jain; Sasa Junuzovic; Eyal Ofek; Mike Sinclair; John R. Porter; Chris Yoon; Swetha Machanavajhala; Meredith Ringel Morris

doi:10.1145/3462244.3479946

Towards sound accessibility in virtual reality

Dhruv Jain, Sasa Junuzovic, Eyal Ofek, Mike Sinclair, John R. Porter, Chris Yoon, Swetha Machanavajhala, Meredith Ringel Morris

Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

Virtual reality (VR) leverages sight, hearing, and touch senses to convey virtual experiences. For d/Deaf and hard of hearing (DHH) people, however, information conveyed through sound may not be accessible. While prior work has explored making every day sounds accessible to DHH users, the context of VR is, as yet, unexplored. In this paper, we provide a first comprehensive investigation of sound accessibility in VR. Our primary contributions include a design space for developing visual and haptic substitutes of VR sounds to support DHH users and prototypes illustrating several points within the design space. We also characterize sound accessibility in commonly used VR apps and discuss findings from early evaluations of our prototypes with 11 DHH users and 4 VR developers.

Original language	English
Title of host publication	ICMI '21
Subtitle of host publication	Proceedings of the 2021 International Conference on Multimodal Interaction
Publisher	Association for Computing Machinery (ACM)
Pages	80-91
Number of pages	12
ISBN (Electronic)	9781450384810
DOIs	https://doi.org/10.1145/3462244.3479946
Publication status	Published - 18 Oct 2021
Event	ICMI '21: International Conference on Multimodal Interaction - Montreal, Canada Duration: 18 Oct 2021 → 22 Oct 2021 https://icmi.acm.org/2021/

Conference

Conference	ICMI '21
Abbreviated title	ICMI 2021
Country/Territory	Canada
City	Montreal
Period	18/10/21 → 22/10/21
Internet address	https://icmi.acm.org/2021/

Access to Document

10.1145/3462244.3479946Licence: None: All rights reserved

Cite this

@inproceedings{16cab84c564b4af28866db6a5771c8fa,

title = "Towards sound accessibility in virtual reality",

abstract = "Virtual reality (VR) leverages sight, hearing, and touch senses to convey virtual experiences. For d/Deaf and hard of hearing (DHH) people, however, information conveyed through sound may not be accessible. While prior work has explored making every day sounds accessible to DHH users, the context of VR is, as yet, unexplored. In this paper, we provide a first comprehensive investigation of sound accessibility in VR. Our primary contributions include a design space for developing visual and haptic substitutes of VR sounds to support DHH users and prototypes illustrating several points within the design space. We also characterize sound accessibility in commonly used VR apps and discuss findings from early evaluations of our prototypes with 11 DHH users and 4 VR developers.",

author = "Dhruv Jain and Sasa Junuzovic and Eyal Ofek and Mike Sinclair and {R. Porter}, John and Chris Yoon and Swetha Machanavajhala and {Ringel Morris}, Meredith",

year = "2021",

month = oct,

day = "18",

doi = "10.1145/3462244.3479946",

language = "English",

pages = "80--91",

booktitle = "ICMI '21",

publisher = "Association for Computing Machinery (ACM)",

address = "United States",

note = "ICMI '21 : International Conference on Multimodal Interaction, ICMI 2021 ; Conference date: 18-10-2021 Through 22-10-2021",

url = "https://icmi.acm.org/2021/",

}

Jain, D, Junuzovic, S, Ofek, E, Sinclair, M, R. Porter, J, Yoon, C, Machanavajhala, S & Ringel Morris, M 2021, Towards sound accessibility in virtual reality. in ICMI '21: Proceedings of the 2021 International Conference on Multimodal Interaction. Association for Computing Machinery (ACM), pp. 80-91, ICMI '21, Montreal, Quebec, Canada, 18/10/21. https://doi.org/10.1145/3462244.3479946

TY - GEN

T1 - Towards sound accessibility in virtual reality

AU - Jain, Dhruv

AU - Junuzovic, Sasa

AU - Ofek, Eyal

AU - Sinclair, Mike

AU - R. Porter, John

AU - Yoon, Chris

AU - Machanavajhala, Swetha

AU - Ringel Morris, Meredith

PY - 2021/10/18

Y1 - 2021/10/18

N2 - Virtual reality (VR) leverages sight, hearing, and touch senses to convey virtual experiences. For d/Deaf and hard of hearing (DHH) people, however, information conveyed through sound may not be accessible. While prior work has explored making every day sounds accessible to DHH users, the context of VR is, as yet, unexplored. In this paper, we provide a first comprehensive investigation of sound accessibility in VR. Our primary contributions include a design space for developing visual and haptic substitutes of VR sounds to support DHH users and prototypes illustrating several points within the design space. We also characterize sound accessibility in commonly used VR apps and discuss findings from early evaluations of our prototypes with 11 DHH users and 4 VR developers.

AB - Virtual reality (VR) leverages sight, hearing, and touch senses to convey virtual experiences. For d/Deaf and hard of hearing (DHH) people, however, information conveyed through sound may not be accessible. While prior work has explored making every day sounds accessible to DHH users, the context of VR is, as yet, unexplored. In this paper, we provide a first comprehensive investigation of sound accessibility in VR. Our primary contributions include a design space for developing visual and haptic substitutes of VR sounds to support DHH users and prototypes illustrating several points within the design space. We also characterize sound accessibility in commonly used VR apps and discuss findings from early evaluations of our prototypes with 11 DHH users and 4 VR developers.

U2 - 10.1145/3462244.3479946

DO - 10.1145/3462244.3479946

M3 - Conference contribution

SP - 80

EP - 91

BT - ICMI '21

PB - Association for Computing Machinery (ACM)

T2 - ICMI '21

Y2 - 18 October 2021 through 22 October 2021

ER -