“What is on your mind?” Automated Scoring of Mindreading in Childhood and Early Adolescence

Venelin Kovatchev; Phillip Smith; Mark Lee; Imogen Grumley Traynor; Irene Luque Aguilera; R.T. Devine

doi:10.18653/v1/2020.coling-main.547

“What is on your mind?” Automated Scoring of Mindreading in Childhood and Early Adolescence

Venelin Kovatchev
, Phillip Smith
, Mark Lee
, Imogen Grumley Traynor
, Irene Luque Aguilera
, R.T. Devine

Research output: Chapter in Book/Report/Conference proceeding › Chapter (peer-reviewed) › peer-review

41 Downloads (Pure)

Abstract

In this paper we present the first work on the automated scoring of mindreading ability in middle childhood and early adolescence. We create MIND-CA, a new corpus of 11,311 question-answer pairs in English from 1,066 children aged from 7 to 14. We perform machine learning experiments and carry out extensive quantitative and qualitative evaluation. We obtain promising results, demonstrating the applicability of state-of-the-art NLP solutions to a new domain and task.

Original language	English
Title of host publication	Proceedings of the 28th International Conference on Computational Linguistics
Editors	Donia Scott, Nuria Bel, Chengqing Zong
Place of Publication	Barcelona, Spain
Publisher	International Committee on Computational Linguistics
Pages	6217
Number of pages	6228
DOIs	https://doi.org/10.18653/v1/2020.coling-main.547
Publication status	Published - 2 Dec 2020
Event	International Conference on Computational Linguistics - Barcelona, Spain Duration: 8 Dec 2020 → 13 Dec 2020 Conference number: 28 https://www.aclweb.org/anthology/volumes/2020.coling-main/

Publication series

Name	Proceedings of the International Conference on Computational Linguistics

Conference

Conference	International Conference on Computational Linguistics
Abbreviated title	COLING
Country/Territory	Spain
City	Barcelona
Period	8/12/20 → 13/12/20
Internet address	https://www.aclweb.org/anthology/volumes/2020.coling-main/

Access to Document

10.18653/v1/2020.coling-main.547Licence: Creative Commons: Attribution (CC BY)

KovatchevV2023AutomatedFinal published version, 171 KBLicence: Creative Commons: Attribution (CC BY)

https://www.aclweb.org/anthology/2020.coling-main.547Licence: Creative Commons: Attribution (CC BY)

Machine learning and deep learning systems for automated measurement of ‘advanced’ theory of mind: reliability and validity in children and adolescents
Devine, R. T., Kovatchev, V., Grumley Traynor, I., Smith, P. & Lee, M., 28 Feb 2023, In: Psychological Assessment. 35, 2, p. 165-177 13 p.
Research output: Contribution to journal › Article › peer-review

Open Access
File
324 Downloads (Pure)
Can vectors read minds better than experts? Comparing data augmentation strategies for the automated scoring of children's mindreading ability
Kovatchev, V., Smith, P., Lee, M. & Devine, R., 6 Aug 2021, Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). Zong, C., Xia, F., Li, W. & Navigli, R. (eds.). Association for Computational Linguistics, ACL, Vol. 1. p. 1196-1206 11 p. (International Joint Conference on Natural Language Processing (IJCNLP)).
Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Open Access
File
138 Downloads (Pure)

Mindreading, Psychopathological and Social Adjustment in Middle Childhood
Devine, R. (Principal Investigator)
The Wellcome Trust
1/08/19 → 31/01/22
Project: Research

Cite this

Kovatchev, V., Smith, P., Lee, M., Grumley Traynor, I., Luque Aguilera, I., & Devine, R. T. (2020). “What is on your mind?” Automated Scoring of Mindreading in Childhood and Early Adolescence. In D. Scott, N. Bel, & C. Zong (Eds.), Proceedings of the 28th International Conference on Computational Linguistics (pp. 6217). (Proceedings of the International Conference on Computational Linguistics). International Committee on Computational Linguistics. https://doi.org/10.18653/v1/2020.coling-main.547

Kovatchev, Venelin ; Smith, Phillip ; Lee, Mark et al. / “What is on your mind?” Automated Scoring of Mindreading in Childhood and Early Adolescence. Proceedings of the 28th International Conference on Computational Linguistics. editor / Donia Scott ; Nuria Bel ; Chengqing Zong. Barcelona, Spain : International Committee on Computational Linguistics, 2020. pp. 6217 (Proceedings of the International Conference on Computational Linguistics).

@inbook{dfcbc00e66b4478a9091d98158017999,

title = "“What is on your mind?” Automated Scoring of Mindreading in Childhood and Early Adolescence",

abstract = "In this paper we present the first work on the automated scoring of mindreading ability in middle childhood and early adolescence. We create MIND-CA, a new corpus of 11,311 question-answer pairs in English from 1,066 children aged from 7 to 14. We perform machine learning experiments and carry out extensive quantitative and qualitative evaluation. We obtain promising results, demonstrating the applicability of state-of-the-art NLP solutions to a new domain and task.",

author = "Venelin Kovatchev and Phillip Smith and Mark Lee and \{Grumley Traynor\}, Imogen and \{Luque Aguilera\}, Irene and R.T. Devine",

year = "2020",

month = dec,

day = "2",

doi = "10.18653/v1/2020.coling-main.547",

language = "English",

series = "Proceedings of the International Conference on Computational Linguistics",

publisher = "International Committee on Computational Linguistics",

pages = "6217",

editor = "Donia Scott and Nuria Bel and Chengqing Zong",

booktitle = "Proceedings of the 28th International Conference on Computational Linguistics",

note = "International Conference on Computational Linguistics , COLING ; Conference date: 08-12-2020 Through 13-12-2020",

url = "https://www.aclweb.org/anthology/volumes/2020.coling-main/",

}

Kovatchev, V , Smith, P , Lee, M, Grumley Traynor, I, Luque Aguilera, I & Devine, RT 2020, “What is on your mind?” Automated Scoring of Mindreading in Childhood and Early Adolescence. in D Scott, N Bel & C Zong (eds), Proceedings of the 28th International Conference on Computational Linguistics. Proceedings of the International Conference on Computational Linguistics, International Committee on Computational Linguistics, Barcelona, Spain, pp. 6217, International Conference on Computational Linguistics , Barcelona, Spain, 8/12/20. https://doi.org/10.18653/v1/2020.coling-main.547

“What is on your mind?” Automated Scoring of Mindreading in Childhood and Early Adolescence. / Kovatchev, Venelin ; Smith, Phillip ; Lee, Mark et al.
Proceedings of the 28th International Conference on Computational Linguistics. ed. / Donia Scott; Nuria Bel; Chengqing Zong. Barcelona, Spain: International Committee on Computational Linguistics, 2020. p. 6217 (Proceedings of the International Conference on Computational Linguistics).

Research output: Chapter in Book/Report/Conference proceeding › Chapter (peer-reviewed) › peer-review

TY - CHAP

T1 - “What is on your mind?” Automated Scoring of Mindreading in Childhood and Early Adolescence

AU - Kovatchev, Venelin

AU - Smith, Phillip

AU - Lee, Mark

AU - Grumley Traynor, Imogen

AU - Luque Aguilera, Irene

AU - Devine, R.T.

N1 - Conference code: 28

PY - 2020/12/2

Y1 - 2020/12/2

N2 - In this paper we present the first work on the automated scoring of mindreading ability in middle childhood and early adolescence. We create MIND-CA, a new corpus of 11,311 question-answer pairs in English from 1,066 children aged from 7 to 14. We perform machine learning experiments and carry out extensive quantitative and qualitative evaluation. We obtain promising results, demonstrating the applicability of state-of-the-art NLP solutions to a new domain and task.

AB - In this paper we present the first work on the automated scoring of mindreading ability in middle childhood and early adolescence. We create MIND-CA, a new corpus of 11,311 question-answer pairs in English from 1,066 children aged from 7 to 14. We perform machine learning experiments and carry out extensive quantitative and qualitative evaluation. We obtain promising results, demonstrating the applicability of state-of-the-art NLP solutions to a new domain and task.

UR - http://dx.doi.org/10.18653/v1/2020.coling-main.547

U2 - 10.18653/v1/2020.coling-main.547

DO - 10.18653/v1/2020.coling-main.547

M3 - Chapter (peer-reviewed)

T3 - Proceedings of the International Conference on Computational Linguistics

SP - 6217

BT - Proceedings of the 28th International Conference on Computational Linguistics

A2 - Scott, Donia

A2 - Bel, Nuria

A2 - Zong, Chengqing

PB - International Committee on Computational Linguistics

CY - Barcelona, Spain

T2 - International Conference on Computational Linguistics

Y2 - 8 December 2020 through 13 December 2020

ER -

Kovatchev V , Smith P , Lee M, Grumley Traynor I, Luque Aguilera I, Devine RT. “What is on your mind?” Automated Scoring of Mindreading in Childhood and Early Adolescence. In Scott D, Bel N, Zong C, editors, Proceedings of the 28th International Conference on Computational Linguistics. Barcelona, Spain: International Committee on Computational Linguistics. 2020. p. 6217. (Proceedings of the International Conference on Computational Linguistics). doi: 10.18653/v1/2020.coling-main.547

“What is on your mind?” Automated Scoring of Mindreading in Childhood and Early Adolescence

Abstract

Publication series

Conference

Access to Document

Fingerprint

Research output

Machine learning and deep learning systems for automated measurement of ‘advanced’ theory of mind: reliability and validity in children and adolescents

Can vectors read minds better than experts? Comparing data augmentation strategies for the automated scoring of children's mindreading ability

Projects

Mindreading, Psychopathological and Social Adjustment in Middle Childhood

Cite this