Matching Acoustic and Perceptual Measures of Phonation Assessment in Disordered Speech - A Case Study

Melanie Jouaiti; Pippa Kirby; Ravi Vaidyanathan

doi:10.21437/Interspeech.2023-1729

Matching Acoustic and Perceptual Measures of Phonation Assessment in Disordered Speech - A Case Study

Melanie Jouaiti, Pippa Kirby, Ravi Vaidyanathan

Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

Speech/voice disorders are common in People Living with Dementia (PLwD). Fluctuations in speech quality can serve as biomarkers of cognitive deterioration but there is a gap in automated assessment of speech collected in unstructured environs. Our organisation has deployed Alexa in the households of 14 PLwD to track self-reported mental and physical state as well as use of language.

n this work, we present a case study analysing highly variable speech over time, providing potential insights into cognitive changes. Alexa data gathered from the participant was manually annotated with speech assessment labels. Those labels are matched to openSMILE features by performing a feature importance analysis to isolate critical features that contribute to the perceptual ratings. We can assess phonation with a F1-score of 0.55, breathiness: 0.71, roughness: 0.60, asthenia: 0.65, strain: 0.74. This work is a first step towards automatic speech assessment to monitor cognitive impairment over time.

Original language	English
Title of host publication	INTERSPEECH 2023
Publisher	ISCA
Pages	4508-4512
Number of pages	5
DOIs	https://doi.org/10.21437/Interspeech.2023-1729
Publication status	Published - 24 Aug 2023
Event	24th International Speech Communication Association, Interspeech 2023 - Dublin, Ireland Duration: 20 Aug 2023 → 24 Aug 2023

Publication series

Name	Interspeech
ISSN (Electronic)	2958-1796

Conference

Conference	24th International Speech Communication Association, Interspeech 2023
Country/Territory	Ireland
City	Dublin
Period	20/08/23 → 24/08/23

Bibliographical note

Funding Information:
This work is supported by the UK Dementia Research Institute (UKDRI-7003) which receives its funding from UK DRI Ltd, funded by the UK Medical Research Council, Alzheimer’s Society and Alzheimer’s Research UK.

Publisher Copyright:
© 2023 International Speech Communication Association. All rights reserved.

Keywords

computational paralinguistics
human-computer interaction
speech recognition

ASJC Scopus subject areas

Language and Linguistics
Human-Computer Interaction
Signal Processing
Software
Modelling and Simulation

Access to Document

10.21437/Interspeech.2023-1729

Cite this

@inproceedings{d9e2ace659cc41bfb4692b4a21750166,

title = "Matching Acoustic and Perceptual Measures of Phonation Assessment in Disordered Speech - A Case Study",

abstract = "Speech/voice disorders are common in People Living with Dementia (PLwD). Fluctuations in speech quality can serve as biomarkers of cognitive deterioration but there is a gap in automated assessment of speech collected in unstructured environs. Our organisation has deployed Alexa in the households of 14 PLwD to track self-reported mental and physical state as well as use of language. n this work, we present a case study analysing highly variable speech over time, providing potential insights into cognitive changes. Alexa data gathered from the participant was manually annotated with speech assessment labels. Those labels are matched to openSMILE features by performing a feature importance analysis to isolate critical features that contribute to the perceptual ratings. We can assess phonation with a F1-score of 0.55, breathiness: 0.71, roughness: 0.60, asthenia: 0.65, strain: 0.74. This work is a first step towards automatic speech assessment to monitor cognitive impairment over time.",

keywords = "computational paralinguistics, human-computer interaction, speech recognition",

author = "Melanie Jouaiti and Pippa Kirby and Ravi Vaidyanathan",

note = "Funding Information: This work is supported by the UK Dementia Research Institute (UKDRI-7003) which receives its funding from UK DRI Ltd, funded by the UK Medical Research Council, Alzheimer{\textquoteright}s Society and Alzheimer{\textquoteright}s Research UK. Publisher Copyright: {\textcopyright} 2023 International Speech Communication Association. All rights reserved.; 24th International Speech Communication Association, Interspeech 2023 ; Conference date: 20-08-2023 Through 24-08-2023",

year = "2023",

month = aug,

day = "24",

doi = "10.21437/Interspeech.2023-1729",

language = "English",

series = "Interspeech",

publisher = "ISCA",

pages = "4508--4512",

booktitle = "INTERSPEECH 2023",

}

TY - GEN

T1 - Matching Acoustic and Perceptual Measures of Phonation Assessment in Disordered Speech - A Case Study

AU - Jouaiti, Melanie

AU - Kirby, Pippa

AU - Vaidyanathan, Ravi

N1 - Funding Information: This work is supported by the UK Dementia Research Institute (UKDRI-7003) which receives its funding from UK DRI Ltd, funded by the UK Medical Research Council, Alzheimer’s Society and Alzheimer’s Research UK. Publisher Copyright: © 2023 International Speech Communication Association. All rights reserved.

PY - 2023/8/24

Y1 - 2023/8/24

N2 - Speech/voice disorders are common in People Living with Dementia (PLwD). Fluctuations in speech quality can serve as biomarkers of cognitive deterioration but there is a gap in automated assessment of speech collected in unstructured environs. Our organisation has deployed Alexa in the households of 14 PLwD to track self-reported mental and physical state as well as use of language. n this work, we present a case study analysing highly variable speech over time, providing potential insights into cognitive changes. Alexa data gathered from the participant was manually annotated with speech assessment labels. Those labels are matched to openSMILE features by performing a feature importance analysis to isolate critical features that contribute to the perceptual ratings. We can assess phonation with a F1-score of 0.55, breathiness: 0.71, roughness: 0.60, asthenia: 0.65, strain: 0.74. This work is a first step towards automatic speech assessment to monitor cognitive impairment over time.

AB - Speech/voice disorders are common in People Living with Dementia (PLwD). Fluctuations in speech quality can serve as biomarkers of cognitive deterioration but there is a gap in automated assessment of speech collected in unstructured environs. Our organisation has deployed Alexa in the households of 14 PLwD to track self-reported mental and physical state as well as use of language. n this work, we present a case study analysing highly variable speech over time, providing potential insights into cognitive changes. Alexa data gathered from the participant was manually annotated with speech assessment labels. Those labels are matched to openSMILE features by performing a feature importance analysis to isolate critical features that contribute to the perceptual ratings. We can assess phonation with a F1-score of 0.55, breathiness: 0.71, roughness: 0.60, asthenia: 0.65, strain: 0.74. This work is a first step towards automatic speech assessment to monitor cognitive impairment over time.

KW - computational paralinguistics

KW - human-computer interaction

KW - speech recognition

UR - http://www.scopus.com/inward/record.url?scp=85171522546&partnerID=8YFLogxK

U2 - 10.21437/Interspeech.2023-1729

DO - 10.21437/Interspeech.2023-1729

M3 - Conference contribution

AN - SCOPUS:85171522546

T3 - Interspeech

SP - 4508

EP - 4512

BT - INTERSPEECH 2023

PB - ISCA

T2 - 24th International Speech Communication Association, Interspeech 2023

Y2 - 20 August 2023 through 24 August 2023

ER -

Matching Acoustic and Perceptual Measures of Phonation Assessment in Disordered Speech - A Case Study

Abstract

Publication series

Conference

Bibliographical note

Keywords

ASJC Scopus subject areas

Access to Document

Fingerprint

Cite this