Automating the Analysis of Speech Acts in Teams to Understand Distributed Sensemaking

We extend previous work on applying computational linguistics in understanding distributed sensemaking. In an experiment, teams of three respond to incidents in the C3Fire simulation under different levels of shared information, and radio communications were automatically transcribed (with an overall accuracy of around 80% for these recordings). The transcription was analyzed by computer to classify speech acts. We compared heuristics and regular expressions, supervised machine learning (using linear SVM), and unsupervised learning (using BERT). For this small corpus of utterances, SVM provides acceptable performance (around 79% accuracy) with minimal computational demand compared with the other approaches. In terms of team communication, differences in information conditions are identified (particularly in terms of speech acts relating to statements about “fire” and “rescue,” and statements about “reasoning and planning”). The study demonstrates the potential of automated analysis of team communications and indicates when teams might struggle with sensemaking.