Representational interactions during audiovisual speech entrainment: redundancy in left posterior superior temporal gyrus and synergy in left motor cortex

Hyojin Park, Robin A. A. Ince, Philippe G. Schyns, Gregor Thut, Joachim Gross

Research output: Contribution to journalArticlepeer-review

8 Citations (Scopus)
200 Downloads (Pure)

Abstract

Integration of multimodal sensory information is fundamental to many aspects of human behavior, but the neural mechanism underlying these processes remain mysterious. For example, during face-to-face communication we know that the brain integrates the dynamic auditory and visual inputs but we do not yet understand where and how such integration mechanisms support speech comprehension. Here we quantify representational interactions between dynamic audio and visual speech signals and show that different brain regions exhibit different types of representational interaction. With a novel information theoretic measure, we found that theta (3-7 Hz) oscillations in the posterior superior temporal gyrus/sulcus (pSTG/S) represent auditory and visual inputs redundantly (i.e. represent common features of the two) whereas the same oscillations in left motor and inferior temporal cortex represent the inputs synergistically (i.e. the instantaneous relationship between audio and visual inputs is also represented). Importantly, redundant coding in the left pSTG/S and synergistic coding in the left motor cortex predict behavior – i.e. speech comprehension performance. Our findings therefore demonstrate that processes classically described as integration can have different statistical properties and may reflect distinct mechanisms that occur in different brain regions to support audiovisual speech comprehension.
Original languageEnglish
Article numbere2006558
Number of pages26
JournalPLoS Biology
Volume16
Issue number8
DOIs
Publication statusPublished - 6 Aug 2018

Keywords

  • Speech
  • Entrainment
  • Audiovisual integration
  • Information theory
  • Partial Information Decomposition (PID)
  • Redundancy
  • Synergy

Fingerprint

Dive into the research topics of 'Representational interactions during audiovisual speech entrainment: redundancy in left posterior superior temporal gyrus and synergy in left motor cortex'. Together they form a unique fingerprint.

Cite this