Harnessing Uncertainty - Multi-label Dysfluency Classification with Uncertain Labels

Melanie Jouaiti; Kerstin Dautenhahn

doi:10.1007/978-3-031-20980-2_26

Harnessing Uncertainty - Multi-label Dysfluency Classification with Uncertain Labels

Melanie Jouaiti^*, Kerstin Dautenhahn

^*Corresponding author for this work

Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

Manually labelled datasets inherently contain errors or uncertain/imprecise labelling as sometimes experts cannot agree or are not sure. This issue is even more prominent in multi-label datasets as some labels may be missing. However, discarding samples with high uncertainty may lead to the loss of valuable data. In this paper, we study two datasets where the uncertainty is explicit in the expert annotations. We give an overview of the different approaches available to deal with uncertainty and evaluate them on two dysfluency datasets. Our results show that adopting methods that embrace uncertainty leads to better results than using only labels with high certainty and performs better than current state of the art results.

Original language	English
Title of host publication	Speech and Computer
Subtitle of host publication	24th International Conference, SPECOM 2022, Gurugram, India, November 14–16, 2022, Proceedings
Editors	S.R. Mahadeva Prasanna, Alexey Karpov, K. Samudravijaya, Shyam S. Agrawal
Publisher	Springer
Pages	302-311
Number of pages	10
ISBN (Electronic)	9783031209802
ISBN (Print)	9783031209796
DOIs	https://doi.org/10.1007/978-3-031-20980-2_26
Publication status	Published - 10 Nov 2022
Event	24th International Conference on Speech and Computer, SPECOM 2022 - Gurugram, India Duration: 14 Nov 2022 → 16 Nov 2022

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	13721 LNAI
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	24th International Conference on Speech and Computer, SPECOM 2022
Country/Territory	India
City	Gurugram
Period	14/11/22 → 16/11/22

Bibliographical note

Funding Information:
Acknowledgments. This research was undertaken, in part, thanks to funding from the Canada 150 Research Chairs Program.

Publisher Copyright:
© 2022, Springer Nature Switzerland AG.

Keywords

Dysfluency classification
Transfer learning
Uncertainty

ASJC Scopus subject areas

Theoretical Computer Science
General Computer Science

Access to Document

10.1007/978-3-031-20980-2_26Licence: None: All rights reserved

Cite this

Jouaiti, M., & Dautenhahn, K. (2022). Harnessing Uncertainty - Multi-label Dysfluency Classification with Uncertain Labels. In S. R. M. Prasanna, A. Karpov, K. Samudravijaya, & S. S. Agrawal (Eds.), Speech and Computer: 24th International Conference, SPECOM 2022, Gurugram, India, November 14–16, 2022, Proceedings (pp. 302-311). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 13721 LNAI). Springer. https://doi.org/10.1007/978-3-031-20980-2_26

Jouaiti, Melanie ; Dautenhahn, Kerstin. / Harnessing Uncertainty - Multi-label Dysfluency Classification with Uncertain Labels. Speech and Computer: 24th International Conference, SPECOM 2022, Gurugram, India, November 14–16, 2022, Proceedings. editor / S.R. Mahadeva Prasanna ; Alexey Karpov ; K. Samudravijaya ; Shyam S. Agrawal. Springer, 2022. pp. 302-311 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{df2aec6170fd4ea4aa542186dd084779,

title = "Harnessing Uncertainty - Multi-label Dysfluency Classification with Uncertain Labels",

abstract = "Manually labelled datasets inherently contain errors or uncertain/imprecise labelling as sometimes experts cannot agree or are not sure. This issue is even more prominent in multi-label datasets as some labels may be missing. However, discarding samples with high uncertainty may lead to the loss of valuable data. In this paper, we study two datasets where the uncertainty is explicit in the expert annotations. We give an overview of the different approaches available to deal with uncertainty and evaluate them on two dysfluency datasets. Our results show that adopting methods that embrace uncertainty leads to better results than using only labels with high certainty and performs better than current state of the art results.",

keywords = "Dysfluency classification, Transfer learning, Uncertainty",

author = "Melanie Jouaiti and Kerstin Dautenhahn",

note = "Funding Information: Acknowledgments. This research was undertaken, in part, thanks to funding from the Canada 150 Research Chairs Program. Publisher Copyright: {\textcopyright} 2022, Springer Nature Switzerland AG.; 24th International Conference on Speech and Computer, SPECOM 2022 ; Conference date: 14-11-2022 Through 16-11-2022",

year = "2022",

month = nov,

day = "10",

doi = "10.1007/978-3-031-20980-2_26",

language = "English",

isbn = "9783031209796",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer",

pages = "302--311",

editor = "Prasanna, {S.R. Mahadeva} and Alexey Karpov and K. Samudravijaya and Agrawal, {Shyam S.}",

booktitle = "Speech and Computer",

}

Jouaiti, M & Dautenhahn, K 2022, Harnessing Uncertainty - Multi-label Dysfluency Classification with Uncertain Labels. in SRM Prasanna, A Karpov, K Samudravijaya & SS Agrawal (eds), Speech and Computer: 24th International Conference, SPECOM 2022, Gurugram, India, November 14–16, 2022, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 13721 LNAI, Springer, pp. 302-311, 24th International Conference on Speech and Computer, SPECOM 2022, Gurugram, India, 14/11/22. https://doi.org/10.1007/978-3-031-20980-2_26

Harnessing Uncertainty - Multi-label Dysfluency Classification with Uncertain Labels. / Jouaiti, Melanie; Dautenhahn, Kerstin.
Speech and Computer: 24th International Conference, SPECOM 2022, Gurugram, India, November 14–16, 2022, Proceedings. ed. / S.R. Mahadeva Prasanna; Alexey Karpov; K. Samudravijaya; Shyam S. Agrawal. Springer, 2022. p. 302-311 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 13721 LNAI).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Harnessing Uncertainty - Multi-label Dysfluency Classification with Uncertain Labels

AU - Jouaiti, Melanie

AU - Dautenhahn, Kerstin

N1 - Funding Information: Acknowledgments. This research was undertaken, in part, thanks to funding from the Canada 150 Research Chairs Program. Publisher Copyright: © 2022, Springer Nature Switzerland AG.

PY - 2022/11/10

Y1 - 2022/11/10

N2 - Manually labelled datasets inherently contain errors or uncertain/imprecise labelling as sometimes experts cannot agree or are not sure. This issue is even more prominent in multi-label datasets as some labels may be missing. However, discarding samples with high uncertainty may lead to the loss of valuable data. In this paper, we study two datasets where the uncertainty is explicit in the expert annotations. We give an overview of the different approaches available to deal with uncertainty and evaluate them on two dysfluency datasets. Our results show that adopting methods that embrace uncertainty leads to better results than using only labels with high certainty and performs better than current state of the art results.

AB - Manually labelled datasets inherently contain errors or uncertain/imprecise labelling as sometimes experts cannot agree or are not sure. This issue is even more prominent in multi-label datasets as some labels may be missing. However, discarding samples with high uncertainty may lead to the loss of valuable data. In this paper, we study two datasets where the uncertainty is explicit in the expert annotations. We give an overview of the different approaches available to deal with uncertainty and evaluate them on two dysfluency datasets. Our results show that adopting methods that embrace uncertainty leads to better results than using only labels with high certainty and performs better than current state of the art results.

KW - Dysfluency classification

KW - Transfer learning

KW - Uncertainty

UR - http://www.scopus.com/inward/record.url?scp=85142740961&partnerID=8YFLogxK

U2 - 10.1007/978-3-031-20980-2_26

DO - 10.1007/978-3-031-20980-2_26

M3 - Conference contribution

AN - SCOPUS:85142740961

SN - 9783031209796

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 302

EP - 311

BT - Speech and Computer

A2 - Prasanna, S.R. Mahadeva

A2 - Karpov, Alexey

A2 - Samudravijaya, K.

A2 - Agrawal, Shyam S.

PB - Springer

T2 - 24th International Conference on Speech and Computer, SPECOM 2022

Y2 - 14 November 2022 through 16 November 2022

ER -

Jouaiti M, Dautenhahn K. Harnessing Uncertainty - Multi-label Dysfluency Classification with Uncertain Labels. In Prasanna SRM, Karpov A, Samudravijaya K, Agrawal SS, editors, Speech and Computer: 24th International Conference, SPECOM 2022, Gurugram, India, November 14–16, 2022, Proceedings. Springer. 2022. p. 302-311. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-031-20980-2_26