Multi-label Dysfluency Classification

Melanie Jouaiti; Kerstin Dautenhahn

doi:10.1007/978-3-031-20980-2_25

Multi-label Dysfluency Classification

Melanie Jouaiti^*, Kerstin Dautenhahn

^*Corresponding author for this work

Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

Stuttering is a neuro-developmental disorder represented in 1% of the population. Dysfluency classification is still an open research question, with concerns of which feature representation or which classifier to use. Another issue, which has been neglected so far, is how to deal with audio samples that contain more than one type of dysfluency. Research has mostly preferred considering only single-labels problems, in part due to the lack of substantial multi-labels datasets. However, the FluencyBank and SEP-28K datasets are now available and contain multi-label data, which should pave the way for more research taking this aspect into account. In this paper, we give an overview of different ways to handle multi-label classification and compare them, while fine-tuning the ResNet50 network to perform multi-label dysfluency classification. We show that, fine-tuning the ResNet50, independently of the label representation, performs better than current state of the art results.

Original language	English
Title of host publication	Speech and Computer
Subtitle of host publication	24th International Conference, SPECOM 2022, Gurugram, India, November 14–16, 2022, Proceedings
Editors	S.R. Mahadeva Prasanna, Alexey Karpov, K. Samudravijaya, Shyam S. Agrawal
Publisher	Springer
Pages	290-301
Number of pages	12
ISBN (Electronic)	9783031209802
ISBN (Print)	9783031209796
DOIs	https://doi.org/10.1007/978-3-031-20980-2_25
Publication status	Published - 10 Nov 2022
Event	24th International Conference on Speech and Computer, SPECOM 2022 - Gurugram, India Duration: 14 Nov 2022 → 16 Nov 2022

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	13721 LNAI
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	24th International Conference on Speech and Computer, SPECOM 2022
Country/Territory	India
City	Gurugram
Period	14/11/22 → 16/11/22

Bibliographical note

Funding Information:
Acknowledgments. This research was undertaken, in part, thanks to funding from the Canada 150 Research Chairs Program.

Publisher Copyright:
© 2022, Springer Nature Switzerland AG.

Keywords

Dysfluency classification
Multi-label classification
Transfer learning

ASJC Scopus subject areas

Theoretical Computer Science
General Computer Science

Access to Document

10.1007/978-3-031-20980-2_25Licence: None: All rights reserved

Cite this

Jouaiti, M., & Dautenhahn, K. (2022). Multi-label Dysfluency Classification. In S. R. M. Prasanna, A. Karpov, K. Samudravijaya, & S. S. Agrawal (Eds.), Speech and Computer: 24th International Conference, SPECOM 2022, Gurugram, India, November 14–16, 2022, Proceedings (pp. 290-301). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 13721 LNAI). Springer. https://doi.org/10.1007/978-3-031-20980-2_25

Jouaiti, Melanie ; Dautenhahn, Kerstin. / Multi-label Dysfluency Classification. Speech and Computer: 24th International Conference, SPECOM 2022, Gurugram, India, November 14–16, 2022, Proceedings. editor / S.R. Mahadeva Prasanna ; Alexey Karpov ; K. Samudravijaya ; Shyam S. Agrawal. Springer, 2022. pp. 290-301 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{e555b441a7f0410dab65539f3846877f,

title = "Multi-label Dysfluency Classification",

abstract = "Stuttering is a neuro-developmental disorder represented in 1% of the population. Dysfluency classification is still an open research question, with concerns of which feature representation or which classifier to use. Another issue, which has been neglected so far, is how to deal with audio samples that contain more than one type of dysfluency. Research has mostly preferred considering only single-labels problems, in part due to the lack of substantial multi-labels datasets. However, the FluencyBank and SEP-28K datasets are now available and contain multi-label data, which should pave the way for more research taking this aspect into account. In this paper, we give an overview of different ways to handle multi-label classification and compare them, while fine-tuning the ResNet50 network to perform multi-label dysfluency classification. We show that, fine-tuning the ResNet50, independently of the label representation, performs better than current state of the art results.",

keywords = "Dysfluency classification, Multi-label classification, Transfer learning",

author = "Melanie Jouaiti and Kerstin Dautenhahn",

note = "Funding Information: Acknowledgments. This research was undertaken, in part, thanks to funding from the Canada 150 Research Chairs Program. Publisher Copyright: {\textcopyright} 2022, Springer Nature Switzerland AG.; 24th International Conference on Speech and Computer, SPECOM 2022 ; Conference date: 14-11-2022 Through 16-11-2022",

year = "2022",

month = nov,

day = "10",

doi = "10.1007/978-3-031-20980-2_25",

language = "English",

isbn = "9783031209796",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer",

pages = "290--301",

editor = "Prasanna, {S.R. Mahadeva} and Alexey Karpov and K. Samudravijaya and Agrawal, {Shyam S.}",

booktitle = "Speech and Computer",

}

Jouaiti, M & Dautenhahn, K 2022, Multi-label Dysfluency Classification. in SRM Prasanna, A Karpov, K Samudravijaya & SS Agrawal (eds), Speech and Computer: 24th International Conference, SPECOM 2022, Gurugram, India, November 14–16, 2022, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 13721 LNAI, Springer, pp. 290-301, 24th International Conference on Speech and Computer, SPECOM 2022, Gurugram, India, 14/11/22. https://doi.org/10.1007/978-3-031-20980-2_25

Multi-label Dysfluency Classification. / Jouaiti, Melanie; Dautenhahn, Kerstin.
Speech and Computer: 24th International Conference, SPECOM 2022, Gurugram, India, November 14–16, 2022, Proceedings. ed. / S.R. Mahadeva Prasanna; Alexey Karpov; K. Samudravijaya; Shyam S. Agrawal. Springer, 2022. p. 290-301 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 13721 LNAI).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Multi-label Dysfluency Classification

AU - Jouaiti, Melanie

AU - Dautenhahn, Kerstin

N1 - Funding Information: Acknowledgments. This research was undertaken, in part, thanks to funding from the Canada 150 Research Chairs Program. Publisher Copyright: © 2022, Springer Nature Switzerland AG.

PY - 2022/11/10

Y1 - 2022/11/10

N2 - Stuttering is a neuro-developmental disorder represented in 1% of the population. Dysfluency classification is still an open research question, with concerns of which feature representation or which classifier to use. Another issue, which has been neglected so far, is how to deal with audio samples that contain more than one type of dysfluency. Research has mostly preferred considering only single-labels problems, in part due to the lack of substantial multi-labels datasets. However, the FluencyBank and SEP-28K datasets are now available and contain multi-label data, which should pave the way for more research taking this aspect into account. In this paper, we give an overview of different ways to handle multi-label classification and compare them, while fine-tuning the ResNet50 network to perform multi-label dysfluency classification. We show that, fine-tuning the ResNet50, independently of the label representation, performs better than current state of the art results.

AB - Stuttering is a neuro-developmental disorder represented in 1% of the population. Dysfluency classification is still an open research question, with concerns of which feature representation or which classifier to use. Another issue, which has been neglected so far, is how to deal with audio samples that contain more than one type of dysfluency. Research has mostly preferred considering only single-labels problems, in part due to the lack of substantial multi-labels datasets. However, the FluencyBank and SEP-28K datasets are now available and contain multi-label data, which should pave the way for more research taking this aspect into account. In this paper, we give an overview of different ways to handle multi-label classification and compare them, while fine-tuning the ResNet50 network to perform multi-label dysfluency classification. We show that, fine-tuning the ResNet50, independently of the label representation, performs better than current state of the art results.

KW - Dysfluency classification

KW - Multi-label classification

KW - Transfer learning

UR - http://www.scopus.com/inward/record.url?scp=85142668855&partnerID=8YFLogxK

U2 - 10.1007/978-3-031-20980-2_25

DO - 10.1007/978-3-031-20980-2_25

M3 - Conference contribution

AN - SCOPUS:85142668855

SN - 9783031209796

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 290

EP - 301

BT - Speech and Computer

A2 - Prasanna, S.R. Mahadeva

A2 - Karpov, Alexey

A2 - Samudravijaya, K.

A2 - Agrawal, Shyam S.

PB - Springer

T2 - 24th International Conference on Speech and Computer, SPECOM 2022

Y2 - 14 November 2022 through 16 November 2022

ER -

Jouaiti M, Dautenhahn K. Multi-label Dysfluency Classification. In Prasanna SRM, Karpov A, Samudravijaya K, Agrawal SS, editors, Speech and Computer: 24th International Conference, SPECOM 2022, Gurugram, India, November 14–16, 2022, Proceedings. Springer. 2022. p. 290-301. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-031-20980-2_25

Multi-label Dysfluency Classification

Abstract

Publication series

Conference

Bibliographical note

Keywords

ASJC Scopus subject areas

Access to Document

Fingerprint

Cite this