UNIVERSITY OF CALIFORNIA Los Angeles

Neuroimaging Studies of the Role of Speech Motor Areas in Speech Perception

A dissertation submitted in partial satisfaction of the requirements for the degree Doctor of Philosophy in Neuroscience

by

Stephen Murray Wilson

2006

The dissertation of Stephen Murray Wilson is approved.

_________________________________ Mirella Dapretto

_________________________________ Patricia Keating

_________________________________ Roger Woods

_________________________________ Marco Iacoboni, Committee Chair

University of California, Los Angeles 2006

ii

TABLE OF CONTENTS

List of figures .................................................................................................................vi List of tables..................................................................................................................vii Acknowledgements ......................................................................................................viii Vita ................................................................................................................................xi Publications and presentations .......................................................................................xii Abstract of the dissertation ...........................................................................................xvi Chapter 1 A review of the role of speech motor areas in speech perception....................1 1.1 Introduction ........................................................................................................1 1.2 Functional neuroanatomy of frontal speech motor areas ......................................6 1.3 Phonological and phonetic tasks........................................................................15 1.4 Effortful extraction of phonetic cues .................................................................27 1.5 Passive speech perception .................................................................................29 1.6 Functional roles of frontal regions in speech perception ....................................44 1.7 Summary and conclusions.................................................................................51 Chapter 2 Listening to speech activates motor areas involved in speech production..........................................................................................................54 2.1 Abstract ............................................................................................................54 2.2 Introduction ......................................................................................................54 2.3 Materials and Methods ......................................................................................55 2.3.1 Subjects and experimental design .............................................................56 2.3.2 Image acquisition .....................................................................................58 2.3.3 Image analysis..........................................................................................59 2.4 Results ..............................................................................................................63 2.5 Discussion.........................................................................................................69

iii

Chapter 3 Neural responses to non-native phonemes varying in producibility: Evidence for the sensorimotor nature of speech perception.................................70 3.1 Abstract ............................................................................................................70 3.2 Introduction ......................................................................................................71 3.3 Materials and methods ......................................................................................73 3.3.1 Stimuli .....................................................................................................73 3.3.2 Scanning procedure ..................................................................................78 3.3.3 Image analysis..........................................................................................79 3.4 Results ..............................................................................................................82 3.4.1 Group analyses .........................................................................................82 3.4.2 Region of interest (ROI) analyses .............................................................86 3.4.3 Functional connectivity analyses ..............................................................88 3.5 Discussion.........................................................................................................90 Chapter 4 Beyond superior temporal cortex: Intersubject correlations in speech comprehension ..............................................................................................................97 4.1 Abstract ............................................................................................................97 4.2 Introduction ......................................................................................................98 4.3 Materials and methods ....................................................................................101 4.3.1 Participants.............................................................................................101 4.3.2 Experimental design ...............................................................................101 4.3.3 Image acquisition ...................................................................................103 4.3.4 Image processing....................................................................................103 4.3.5 Standard analysis....................................................................................104 4.3.6 Intersubject correlational analysis ...........................................................105 4.4 Results ............................................................................................................108 4.5 Discussion.......................................................................................................117 4.5.1 Default mode network ............................................................................119

iv

4.5.2 Involvement of the bilateral inferior frontal gyrus in speech comprehension .......................................................................................124 4.5.3 Premotor cortex......................................................................................127 4.5.4 Regions differentially implicated in audiovisual speech perception...............................................................................................128 4.5.5 Superior temporal cortex ........................................................................130 4.5.6 Conclusion .............................................................................................131 Chapter 5 Conclusion ................................................................................................132 References...................................................................................................................136

v

LIST OF FIGURES

Figure 1.1 Activation peaks from studies of speech production and orolaryngeal motor control, phonological processing, and speech comprehension under phonetically challenging conditions ...........................................................................7 Figure 1.2 Activation peaks from studies of syllable and pseudoword perception, and comprehension of single words in isolation .......................................................31 Figure 1.3 Activation peaks from studies of sentence and narrative comprehension.........................................................................................................38 Figure 1.4 All activation peaks from Figures 1.1, 1.2 and 1.3........................................45 Figure 2.1 Areas activated by passive listening to meaningless monosyllables in three representative subjects.....................................................................................64 Figure 2.2 Characterization of the relationships between listening and motor areas ........................................................................................................................67 Figure 3.1 Speech-responsive regions and areas sensitive to the factors of nativeness and producibility.....................................................................................83 Figure 3.2 Region of interest (ROI) analyses.................................................................87 Figure 3.3 Functional connectivity analyses..................................................................89 Figure 4.1 Materials and methods ...............................................................................102 Figure 4.2 Auditory speech comprehension and audiovisual speech comprehension.......................................................................................................109 Figure 4.3 Audiovisual speech comprehension relative to auditory speech comprehension, and auditory speech comprehension relative to audiovisual speech comprehension ...........................................................................................115

vi

LIST OF TABLES

Table 1.1 Speech production and orolaryngeal motor studies ..........................................9 Table 1.2 Phonological processing studies ....................................................................16 Table 1.3 Studies of speech comprehension under phonetically challenging conditions ................................................................................................................28 Table 1.4 Syllable and pseudoword perception studies..................................................32 Table 1.5 Isolated word comprehension studies ............................................................36 Table 1.6 Sentence comprehension studies ...................................................................39 Table 1.7 Narrative comprehension studies...................................................................41 Table 2.1 Areas activated by listening to speech in 6 or more subjects, and PrCG/CS motor areas activated by producing speech or bimanual movement ..........65 Table 3.1 Phonemes used in the study...........................................................................76 Table 3.2 Areas activated in each contrast of interest ....................................................84 Table 4.1 Regions significantly correlated across subjects, or activated or deactivated relative to rest for auditory-only narratives ..........................................111 Table 4.2 Regions significantly correlated across subjects, or activated or deactivated relative to rest for audiovisual narratives .............................................113 Table 4.3 Regions which were significantly more correlated across audiovisual subjects than auditory-only subjects, or which were activated for audiovisual narratives relative to audio-only narratives.............................................................117 Table 4.4 Regions which were significantly more correlated across auditory-only subjects than audiovisual subjects, or which were activated for auditory-only narratives relative to audiovisual narratives............................................................118

vii

ACKNOWLEDGEMENTS

I am very grateful to the numerous people who have done so much for me during my graduate studies at the University of California, Los Angeles. I thank Marco Iacoboni, my committee chair and advisor, for always being generous with his time, advice and encouragement. Pat Keating gave me an invaluable phonetician’s perspective on my research, and Mirella Dapretto and Roger Woods provided key comments and criticisms which shaped the studies in my dissertation. My interest in functional MRI was first sparked by Mark Cohen and Steve Engel, who both taught me a great deal. Arnie Scheibel’s course on neuroanatomy was inspirational, and I learned a lot from Susan Bookheimer about the anatomy of language regions. Other wonderful teachers at UCLA who I would like to thank in particular are Francisco Bezanilla (neurophysiology), Susie Curtiss (neurolinguistics), Süreyya Er (Turkish), Nancy Ezer (Hebrew), Bruce Hayes (phonology) and Tim Stowell (syntax). I learned an enormous amount about language and the brain from my collaborators Liz Bates and Fred Dick at UC San Diego, and Nina Dronkers and David Wilkins at the Center for Aphasia and Related Disorders in Martinez, CA. I am grateful to Suzie Vader for her flawless administration of the neuroscience interdepartmental program, and to Leona Mattoni for making sure that I got paid. I thank my friends and colleagues Lisa Aziz-Zadeh, Tim Arbisi-Kelm, Jen Bramen, Meredith Braskie, Arne Ekstrom, Leo Fernandino, Amy Hubbard, Jonas Kaplan, Katie Karlsgodt, Lisa Koski, Ingo Meister, Eric Mooshagian, Istvan Molnar-Szakacs, Roy

viii

Mukamel, Kuniko Neilsen, Libby O’Hare, Mike Oldham and Lucina Uddin for all the fun times, camaraderie, and conversations about science and many other things. I am indebted to my parents Trish and Murray Wilson and my brother Michael for their constant love and support. I would like to acknowledge specific assistance and funding sources for the three empirical studies described in this dissertation. For the study reported in chapter 2, I thank my collaborators Ayşe Saygin, Marty Sereno and Marco Iacoboni. I am grateful to Katrin Amunts for providing cytoarchitectonic maps, and to the UCSD Center for functional MRI. This study was supported by NSF grants REC 0107077 and BCS 0224321. For the study in chapter 3, I thank my co-author Marco Iacoboni, Peter Ladefoged for recording the stimuli, Mirella Dapretto, Patricia Keating, Roger Woods, Lisa Aziz-Zadeh, Amy Hubbard, Jonas Kaplan, Istvan Molnar-Szakacs and John Barresi and Greg Hickok for helpful discussions, and Keith Worsley and Henry Tehrani for technical assistance. This study was supported by NSF grant REC0107077 and NIMH grant MH63680. For the study described in chapter 4, I thank my collaborators Istvan Molnar-Szakacs and Marco Iacoboni, and I am grateful to Susan Arick, Lisa Aziz-Zadeh, Susan Duncan, Susan Goldin-Meadow, Amy Hubbard, Jonas Kaplan and Roy Mukamel for their assistance in the design, implementation and analysis of the study. This project was supported by NSF grant REC0107077 and NIMH grant MH63680. For generous support of all three studies I also wish to thank the Brain Mapping Medical Research Organization, Brain Mapping Support Foundation, Pierson-Lovelace

ix

Foundation, The Ahmanson Foundation, William M. and Linda R. Dietel Philanthropic Fund at the Northern Piedmont Community Foundation, Tamkin Foundation, Jennifer Jones-Simon Foundation, Capital Group Companies Charitable Foundation, Robson Family and Northstar Fund. The projects described were supported by Grant Numbers RR12169, RR13642 and RR00865 from the National Center for Research Resources (NCRR), a component of the National Institutes of Health (NIH).

x

VITA

October 19, 1975

Born, Armidale, NSW, Australia

1997

B.A. (Honors), Linguistics University of Sydney, NSW, Australia

1997–99

Research Assistant University of Sydney, NSW, Australia, and Diwurruwurru-jaru Aboriginal Corp., Katherine, NT, Australia

2000–03

President’s Fellowship University of California, Los Angeles, CA

2000–06

Graduate Student Researcher Department of Linguistics University of California, Los Angeles, CA

2001–02

Teaching Assistant Department of Linguistics University of California, Los Angeles, CA

2002

M.A., Linguistics University of California, Los Angeles, CA

2002, 2003

Visiting Scholar Center for Research in Language University of California, San Diego, CA

2003–05

Teaching Assistant Neuroscience Interdepartmental Program University of California, Los Angeles, CA

2003–06

Graduate Student Researcher Neuroscience Interdepartmental Program University of California, Los Angeles, CA

2005–06

Dissertation Year Fellowship University of California, Los Angeles, CA

xi

PUBLICATIONS AND PRESENTATIONS Aziz-Zadeh, L., Iacoboni, M., Zaidel, E., Wilson, S., & Mazziotta, J. (2004). Left hemisphere motor facilitation in response to manual action sounds. European Journal of Neuroscience, 19, 2609–2612. Aziz-Zadeh, L., Wilson, S. M., & Iacoboni, M. (2005). A comparison of premotor areas activated by action observation and action phrases. Program No. 852.14. 2005 Abstract Viewer/Itinerary Planner. Washington, DC: Society for Neuroscience. Online. Aziz-Zadeh, L., Wilson, S. M., Rizzolatti, G., & Iacoboni, M. (2006). Congruent embodied representations for visually presented actions and linguistic phrases describing actions. Current Biology, 16, 1818–1823. Baldo, J. V., Wilson, S. M. & Dronkers, N. F. (2003). Two distinct brain regions crucial for problem solving revealed with voxel-based lesion symptom mapping. Program No. 195.8. 2003 Abstract Viewer and Itinerary Planner. Washington, DC: Society for Neuroscience. CD-ROM. Bates, E., Wilson, S. M., Saygin, A. P., Dick, F., Sereno, M. I., Knight, R. T., & Dronkers, N. F. (2003). Voxel-based lesion-symptom mapping. Nature Neuroscience, 6, 448–450. Borovsky, A., Saygin, A. P., Wilson, S. M., & Dronkers, N. F. (2004). Lesion mapping of conversational speech production deficits in aphasia. Program No. 595.2. 2004 Abstract Viewer and Itinerary Planner. Washington, DC: Society for Neuroscience. CD-ROM. Dick, F., Dronkers, N. F., Pizzamiglio, L., Saygin, A. P., Small, S. L., & Wilson, S. (2005). Language and the brain. In M. Tomasello, & D. I. Slobin (Eds.), Beyond nature-nurture: Essays in honor of Elizabeth Bates (pp. 237–260). Mahwah, NJ: Lawrence Erlbaum. Dick, F., Saygin, A. P., Wilson, S., Sereno, M. I., Pitzalis, S., Galati, G., Bentrovato, S., D’Amico, S., Pizzamiglio, L. & Bates, E. (2003). What is involved, and what is necessary for complex linguistic and non-linguistic auditory processing—a quantitative comparison. Cognitive Neuroscience Society Annual Meeting Program 2003. Journal of Cognitive Neuroscience, Suppl, 210. Iacoboni, M., Kaplan, J., & Wilson, S. (in press). A neural architecture for imitation and intentional relations. In C. Nehaniv, & K. Dautenhahn (Eds.), Imitation and Social Learning in Robots, Humans and Animals: Behavioral, Social and Communicative Dimensions. Cambridge, UK: Cambridge University Press.

xii

Iacoboni, M., & Wilson, S. (2006). Beyond a single area: motor control and language within a neural architecture for imitation encompassing Broca’s area. Cortex, 43, 503–506. Molnar-Szakacs, I., Wilson, S. M., & Iacoboni, M. (2005). I see what you’re saying: the neural correlates of gesture perception. Program No. 128.7. 2005 Abstract Viewer/Itinerary Planner. Washington, DC: Society for Neuroscience. Online. Molnar-Szakacs, I., Wilson, S. M., & Iacoboni, M. (2006). A Point of View: Subconscious Perspective Taking During Observation of Natural Gestures. Annual Meeting of the Organization for Human Brain Mapping, Florence, Italy: June, 2006. NeuroImage, 31(S1). Saygin, A. P., Dick, F., Wilson, S. M., Dronkers, N. F., & Bates, E. (2003). Neural resources for processing language and environmental sounds: Evidence from aphasia. Brain, 126, 928–945. Saygin, A. P., Dronkers, N., Wilson, S., Ludy, C. & Bates, E. (2003). Pantomime interpretation and reading comprehension in patients with aphasia. Cognitive Neuroscience Society Annual Meeting Program 2003. Journal of Cognitive Neuroscience, Suppl, 96. Saygin, A. P. & Wilson, S. (2002). Paradigm reanalysis and the representation of morphologically complex words in Turkish. In M. Andronis, E. Debenport, A. Pycha, & K. Yoshimura (Eds.), CLS 38-2: The panels (pp. 285–298). Chicago: Chicago Linguistic Society. Saygin, A. P., Wilson, S. M., & de Sa, V. R. (2004). Visual form facilitates audiovisual synchrony detection. Paper presented at the Annual Meeting of the Vision Sciences Society, April 30 – May 5, 2004, Sarasota, FL. Saygin, A. P., Wilson, S. M., Dick, F., Dronkers, N., Bates, E. (2003). Neural correlates of non-linguistic impairments in aphasia. Annual Meeting of the Organization for Human Brain Mapping, New York, NY: June 2003. NeuroImage, 19(2). CD-ROM. Saygin, A. P., Wilson, S. M., Dronkers, N. F., & Bates, E. (2004). Action comprehension in aphasia: Linguistic and non-linguistic deficits and their lesion correlates. Neuropsychologia, 42, 1788–1804. Saygin, A. P., Wilson, S. M., Hagler Jr., D. J., Bates, E., & Sereno, M. I. (2003). Brain areas involved in the processing of biological motion: Lesion-symptom mapping and fMRI. Annual Meeting of the Organization for Human Brain Mapping, New York, NY: June 2003. NeuroImage, 19(2). CD-ROM.

xiii

Saygin, A. P., Wilson, S. M., Hagler Jr., D. J., Bates, E., & Sereno, M. I. (2003). Visual and motor areas in the human brain are involved in the perception of point-light biological motion. Program No. 438.11. 2003 Abstract Viewer and Itinerary Planner. Washington, DC: Society for Neuroscience. CD-ROM. Saygin, A. P., Wilson, S. M., Hagler, D., Bates, E., & Sereno, M. I. (2004). Point-light biological motion perception activates human premotor cortex. Journal of Neuroscience, 24, 6181–6188. Wilson, S. (1997). Coverbs and complex predicates in Wagiman. Paper presented at the Second and Third International Workshops on Australian Aboriginal Languages. Melbourne, 1997, and Nijmegen, 1998. Wilson, S. (1998). Wakgala mahan matjjin Wagiman: Stories in the Wagiman language of Australia's Northern Territory. Katherine: Diwurruwurru-jaru Aboriginal Corporation. Wilson, S. (1999). Coverbs and complex predicates in Wagiman. Stanford: CSLI Publications. Wilson, S. (2002). A computer program to assist in learning a morphologically complex language. Paper presented at the UC Language Consortium Conference on Language Learning and Teaching, University of California, Irvine, March 9, 2002. Wilson, S. (2002). A neural network with delay lines for modeling the time course of word recognition. Cognitive Neuroscience Society Annual Meeting Program 2002. Journal of Cognitive Neuroscience, Suppl, 157. Wilson, S. M. (2003). A phonetic study of voiced, voiceless and alternating stops in Turkish. CRL Newsletter, 15(1), 1–13. Wilson, S. (2003). Lexically specific constructions in the acquisition of inflection in English. Journal of Child Language, 30, 75–115. Wilson, S. M., Bates, E., Saygin, A. P., Dick, F., Sereno, M., Knight, R. T. & Dronkers, N. F. (2003). Voxel-based lesion-symptom mapping. Cognitive Neuroscience Society Annual Meeting Program 2003. Journal of Cognitive Neuroscience, Suppl, 114. Wilson, S. M., Bates, E., Saygin, A. P., Dick, F., Sereno, M., Knight, R. & Dronkers, N. (2003). Voxel-based lesion-symptom mapping: An application to speech fluency and language comprehension. Annual Meeting of the Organization for Human Brain Mapping, New York, NY: June 2003. NeuroImage, 19(2). CD-ROM. Wilson, S., & Harvey, M. (1999). Wagiman online dictionary. Available online at: http://www1.aiatsis.gov.au/exhibitions/e_access/digital/a339234/home.html.

xiv

Wilson, S. M., & Iacoboni, M. (2004). Shared neural systems for imitation and language. Paper presented at the 11th Altenberg Workshop in Theoretical Biology: Analog Communication: Evolution, Brain Mechanisms, Dynamics, Simulation. Konrad Lorenz Institute for Evolution and Cognition Research, Altenberg, Austria, September 27, 2004. Wilson, S. M. & Iacoboni, M. (2005). Brain areas modulated by listening to non-native phonemes varying in producibility. Program No. 535.3. 2005 Abstract Viewer/Itinerary Planner. Washington, DC: Society for Neuroscience. Online. Wilson, S. M. & Iacoboni, M. (2006). Neural responses to non-native phonemes varying in producibility: evidence for the sensorimotor nature of speech perception. NeuroImage, 33, 316–325. Wilson, S. M., Molnar-Szakacs, I., & Iacoboni, M. (2006). Involvement of default state areas in audiovisual speech comprehension revealed by model-free intersubject correlational analysis. Annual Meeting of the Organization for Human Brain Mapping, Florence, Italy: June, 2006. NeuroImage, 31(S1). Wilson, S. & Saygin, A. P. (2003). Adverbs and functional heads in Turkish: Linear order and scope. In Carmichael, L., Huang, C.-H., & Samiian, V. (Eds.), Proceedings of the 2001 Western Conference in Linguistics (vol. 13). Fresno, CA: CSU Publications. Wilson, S. M., & Saygin, A. P. (2004). Grammaticality judgment in aphasia: deficits are not specific to syntactic structures, aphasic syndromes or lesion sites. Journal of Cognitive Neuroscience, 16, 238–252. Wilson, S. M., Saygin, A. P., Schleicher, E., Dick, F., & Bates, E. (2003). Grammaticality judgment under non-optimal processing conditions: Deficits induced in normal participants resemble those observed in aphasic patients. Brain and Language, 87, 67–68. Wilson, S. M., Saygin, A. P., Sereno, M. I., & Iacoboni, M. (2003). Listening to speech activates motor areas involved in speech production. Program No. 770.3. 2003 Abstract Viewer and Itinerary Planner. Washington, DC: Society for Neuroscience. CD-ROM. Wilson, S. M., Saygin, A. P., Sereno, M. I., & Iacoboni, M. (2004). Listening to speech activates motor areas involved in speech production. Nature Neuroscience, 7, 701– 702. Wilson, S. M., Saygin, A. P., Sereno, M. I., & Iacoboni, M. (2004). Two distinct foci for speech production in the anterior and posterior segments of primary motor cortex. Program No. 595.9. 2004 Abstract Viewer and Itinerary Planner. Washington, DC: Society for Neuroscience. CD-ROM.

xv

ABSTRACT OF THE DISSERTATION

Neuroimaging Studies of the Role of Speech Motor Areas in Speech Perception

by

Stephen Murray Wilson Doctor of Philosophy in Neuroscience University of California, Los Angeles, 2006 Professor Marco Iacoboni, Chair

The role of superior temporal cortex in speech perception is well established, but there is also much evidence suggestive of an ancillary role for frontal speech motor areas in the perceptual process. In this dissertation, three functional magnetic resonance imaging (fMRI) studies are presented which support a role for speech motor areas in speech perception. In the first study, subjects listened passively to monosyllables, and produced the same speech sounds. Listening to speech activated bilaterally a premotor cortical region largely overlapping a speech production motor area centered just posteriorly. These findings support the view that the motor system is recruited in mapping the

xvi

acoustic signal to a phonetic code. The next study examined neural responses to unfamiliar non-native phonemes varying in the extent to which they can be articulated. Both superior temporal (auditory) and precentral (motor) areas were activated by passive speech perception, and both distinguished non-native from native phonemes. Furthermore, speech-responsive motor regions and superior temporal sites were functionally connected. However, only in auditory areas did activity covary with the producibility of non-native phonemes. These data suggest that auditory areas are crucial for the transformation from acoustic signal to phonetic code, but the motor system also plays an active role, perhaps in generating candidate phonemic categorizations. In the final study, subjects were presented with auditory and audiovisual narratives, and modelfree intersubject correlational analyses were employed to reveal areas that were modulated in a consistent way across subjects during narrative comprehension. The intersubject correlational analyses revealed an extended network of areas not typically reported in previous studies of narrative speech comprehension, including extensive bilateral inferior frontal and premotor regions. These results support a role for frontal areas in speech perception and higher level linguistic processes. In sum, at least two ventral premotor regions appear to be important for speech perception: one located in Brodmann Area 6 which is argued to be involved in attention to phonetic form, and a region in dorsal Brodmann Area 44 which may code articulatory representations. Motor areas may be especially important for speech perception under perceptually challenging conditions such as comprehending speech in background noise.

xvii