Overview of the 2017 Spoken CALL Shared Task

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Standard

Overview of the 2017 Spoken CALL Shared Task. / Baur, Claudia; Chua, Cathy; Gerlach, Johanna; Rayner, Manny; Russell, Martin; Strik, Helmer; Wei, Xizi.

Proceedings of the 7th ISCA Workshop on Speech and Language Technology in Education. ed. / O Engwall; J Lopes; I Leite. Stockholm, Sweden, 2017.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Harvard

Baur, C, Chua, C, Gerlach, J, Rayner, M, Russell, M, Strik, H & Wei, X 2017, Overview of the 2017 Spoken CALL Shared Task. in O Engwall, J Lopes & I Leite (eds), Proceedings of the 7th ISCA Workshop on Speech and Language Technology in Education. Stockholm, Sweden. <http://www.slate2017.org/proceedings.pdf>

APA

Baur, C., Chua, C., Gerlach, J., Rayner, M., Russell, M., Strik, H., & Wei, X. (2017). Overview of the 2017 Spoken CALL Shared Task. In O. Engwall, J. Lopes, & I. Leite (Eds.), Proceedings of the 7th ISCA Workshop on Speech and Language Technology in Education http://www.slate2017.org/proceedings.pdf

Vancouver

Baur C, Chua C, Gerlach J, Rayner M, Russell M, Strik H et al. Overview of the 2017 Spoken CALL Shared Task. In Engwall O, Lopes J, Leite I, editors, Proceedings of the 7th ISCA Workshop on Speech and Language Technology in Education. Stockholm, Sweden. 2017

Author

Baur, Claudia ; Chua, Cathy ; Gerlach, Johanna ; Rayner, Manny ; Russell, Martin ; Strik, Helmer ; Wei, Xizi. / Overview of the 2017 Spoken CALL Shared Task. Proceedings of the 7th ISCA Workshop on Speech and Language Technology in Education. editor / O Engwall ; J Lopes ; I Leite. Stockholm, Sweden, 2017.

Bibtex

@inproceedings{7b8da0907f814f828fb4c5be328d4e89,
title = "Overview of the 2017 Spoken CALL Shared Task",
abstract = "We present an overview of the shared task for spoken CALL. Groups competed on a prompt-response task using English language data collected, through an online CALL game, from Swiss German teens in their second and third years of learningEnglish. Each item consists of a written German prompt and an audio file containing a spoken response. The task is to accept linguistically correct responses and reject linguistically incorrect ones, with “linguistically correct” being defined by a gold standard derived from human annotations; scoring wasperformed using a metric defined as the ratio of the relative rejection rates on incorrect and correct responses. The task received twenty entries from nine different groups. We present the task itself, the results, a tentative analysis of what makes items challenging, a comparison between different metrics, andsuggestions for a continuation.",
keywords = "CALL, Shared Task, Automatic speech recognition, Metrics",
author = "Claudia Baur and Cathy Chua and Johanna Gerlach and Manny Rayner and Martin Russell and Helmer Strik and Xizi Wei",
year = "2017",
month = aug,
day = "25",
language = "English",
editor = "O Engwall and J Lopes and I Leite",
booktitle = "Proceedings of the 7th ISCA Workshop on Speech and Language Technology in Education",

}

RIS

TY - GEN

T1 - Overview of the 2017 Spoken CALL Shared Task

AU - Baur, Claudia

AU - Chua, Cathy

AU - Gerlach, Johanna

AU - Rayner, Manny

AU - Russell, Martin

AU - Strik, Helmer

AU - Wei, Xizi

PY - 2017/8/25

Y1 - 2017/8/25

N2 - We present an overview of the shared task for spoken CALL. Groups competed on a prompt-response task using English language data collected, through an online CALL game, from Swiss German teens in their second and third years of learningEnglish. Each item consists of a written German prompt and an audio file containing a spoken response. The task is to accept linguistically correct responses and reject linguistically incorrect ones, with “linguistically correct” being defined by a gold standard derived from human annotations; scoring wasperformed using a metric defined as the ratio of the relative rejection rates on incorrect and correct responses. The task received twenty entries from nine different groups. We present the task itself, the results, a tentative analysis of what makes items challenging, a comparison between different metrics, andsuggestions for a continuation.

AB - We present an overview of the shared task for spoken CALL. Groups competed on a prompt-response task using English language data collected, through an online CALL game, from Swiss German teens in their second and third years of learningEnglish. Each item consists of a written German prompt and an audio file containing a spoken response. The task is to accept linguistically correct responses and reject linguistically incorrect ones, with “linguistically correct” being defined by a gold standard derived from human annotations; scoring wasperformed using a metric defined as the ratio of the relative rejection rates on incorrect and correct responses. The task received twenty entries from nine different groups. We present the task itself, the results, a tentative analysis of what makes items challenging, a comparison between different metrics, andsuggestions for a continuation.

KW - CALL

KW - Shared Task

KW - Automatic speech recognition

KW - Metrics

M3 - Conference contribution

BT - Proceedings of the 7th ISCA Workshop on Speech and Language Technology in Education

A2 - Engwall, O

A2 - Lopes, J

A2 - Leite, I

CY - Stockholm, Sweden

ER -