NEUROREPORT

BRAIN IMAGING

Neural mechanisms underlying semantic and orthographic processing in Chinese ^English bilinguals Guosheng Ding, Conrad Perry,1 Danling Peng,CA Lin Ma,2 Dejun Li,2 Shiyong Xu, Qian Luo, Duo Xu and Jing Yang Department of Psychology, Beijing Normal University, Beijing,100875, PR China; 1The University of Hong Kong, Hong Kong; 2CPLA No. 301 Hospital, PR China CA

Corresponding Author: [email protected]

Received 20 May 2003; accepted 31 May 2003 DOI: 10.1097/01.wnr.0000087906.78892.8e

Brain activation underlying language processing in Chinese^English bilinguals was examined using fMRI in an orthographic search and a semantic classi¢cation task. In both tasks, brain areas activated by Chinese characters and English words were very similar to tasks examining Chinese reading using Chinese pinyin (an alphabetic Chinese script) and Chinese characters. However, the degree of later-

alization was di¡erent, with English words (second language) causing much more right hemisphere activation than Chinese characters (native language). These di¡erences support the hypothesis that second language usage causes more right hemisphere activation than native language usage. NeuroReport c 2003 Lippincott Williams & Wilkins. 14:1557^1562


Key words: Bilinguals; Neural mechanism; Orthographic processing; Right hemisphere; Semantic processing

INTRODUCTION Differences between how bilinguals use their native and second language have been a question of interest for a long time [1]. It has been suggested that second-language usage causes more right hemisphere processing than nativelanguage usage [2]. This suggestion is not without controversy, however, and has been criticized on the grounds that uncommon results tend to be reported and published [3]. A second issue concerning bilingualism is whether the same brain areas are used in both languages or whether there are areas specifically dedicated to each language. Recently, PET, fMRI, and ERPs have been used to examine this question. A review by Illes et al. [4] showed that the results were mixed. Some studies found differences while others did not. There are a number of potential reasons why some studies may have failed to find differences, and why differences found by different studies have not been the same. These include the specific type of task used, whether early or late bilinguals were examined, and what languages were compared [4]. The use of different tasks is extremely important, since different types of processing may lead to different asymmetries [2]. To further investigate bilingual processing with fMRI, we examined the performance of Chinese–English bilinguals on

c Lippincott Williams & Wilkins 0959- 4965


an orthographic search task, which emphasizes form-related aspects of the language, and a semantic categorization task, which emphasizes semantic processing. In the orthographic task, participants judged whether Chinese characters contained the radical , or whether the English words contained the letter ‘D’. In the semantic categorization task, subjects judged whether a given exemplar was an animal (e.g. dog, yes; box, no). Previous research has shown that these two types of task cause different amounts of processing to occur at brain areas thought to be used in orthographic and semantic processing [4]. However, perceptual orthographic tasks may activate semantics [5] and phonology [6] automatically. This means that direct subtraction of activation caused by native- and second-language usage may obscure areas responsible for such processing. Thus, we examined the activation left from the direct subtraction of activation caused by native- and second-language usage, and also examined activation caused by native- and second-language usage separately.

MATERIALS AND METHODS Participants: Six participants (mean age 22.67 years, range 21–24 years), three male and three female, participated in the experiment. The native language of the participants was Chinese and their second language was English. All were

Vol 14 No 12 26 August 2003

15 5 7

Copyright © Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.

NEUROREPORT

G. DING ETAL.

Participants were counterbalanced such that half performed the orthographic task first and the other half performed the semantic task first.

senior or junior students in the English Department of Beijing Normal University and all were fluent bilinguals. The average age at which they started learning English was 12.17 years (range 11–13), and they had an average of 10.5 years (range 9–11) English experience. All had passed the level 8 national English test for English major students in China, which means that they were proficient at reading, listening, speaking, and writing. All participants were right handed and had normal vision. Before the experiment, participants were given an informal interview to ensure they did not have a history of intelligence, reading, or oral language deficit.

Imaging: Brain images were collected with a 1.5 T GE Signa scanner located in the Beijing 301 Hospital in China. The stimuli were presented with a Panasonic projector (resolution 800  600, luminance 1500 lumen) that was linked to an IBM Notebook. Functional images (EPI) were acquired for 12 adjacent 8 mm slices which covered the whole cerebral cortex using the following acquisition parameters: TR (repetition time) ¼ 2400 ms; TE ¼ 65 ms; flip angle ¼ 901; FOV ¼ 240  240 mm; matrix size ¼ 64  64; NEX (number of excitations for each run) ¼ 100. Anatomical images (SPGR) were acquired using the following parameters: TE ¼ 40 ms; flip angle ¼ 301; FOV ¼ 240  240 mm; matrix size ¼ 256  256; 60 sagittal 2.5 mm slices.

Materials and procedure: Forty Chinese single-character words and 40 English words were used. Half were used in the orthographic task and half were used in the semantic task. All the Chinese and English words were concrete nouns. In the orthographic task, participants were asked to judge whether the Chinese words contained the component (radical) , or whether the English words contained the letter D. Half the Chinese stimuli contained the radical and half the English stimuli contained the letter D. In the semantic categorization task, participants were asked to judge whether the word displayed was a type of animal. Half the stimuli in each of the Chinese and English groups were animal names. If the response was yes, participants pinched a small gasbag with their right hand. If the response was no, no response was given. Each participant performed in two runs. In one of the runs, the orthographic search task was performed and in the other the semantic categorization task was performed. Each run lasted 4 min and consisted of eight blocks (Fig. 1). Of the eight blocks, four were word judgment blocks (experimental blocks) of 30 s each, and four were control blocks of 27 s each. Experimental and control blocks were always presented one after the other. Within the experimental blocks, two were composed of 10 Chinese characters and the other two were composed of 10 English words. Each word was presented for 1000 ms and followed by a 2000 ms blank interval during which the subject made their response. Each control block was composed of 9 asterisks, which were presented for 1000 ms, followed by a 2000 ms blank interval. Participants saw the stimuli passively without any response. The order of the experimental and the control blocks is displayed in Fig. 1. A 3 s instruction screen was shown before each experimental block. For the orthographic task, the instruction was ‘contains ?’ (in Chinese) before the Chinese blocks or ‘contains D?’ (in English) before the English blocks. For the semantic task, the instructions were ‘is it animal?’ (in the relevant language). Participants were also shown detailed instructions on paper immediately before the experiment and also familiarized with the procedure by the experimenter in a practice session.

fMRI data analysis: The fMRI data were analyzed by AFNI 2.2 (Analysis of Functional Neural Image [7]). For each participant, the first four volumes in each scan series that were collected before equilibrium magnetization was reached were discarded. All the functional images were then registered to the volume collected nearest in time to the high-resolution SPGR anatomical scan to correct potential small head motions. Next, two boxcar reference (input) functions for Chinese and English stimuli were built according to the time series of their presentation, convolved with a g-variate function to account for the slow hemodynamic response. A linear regression was then used to calculate the least squares fit of each voxel time series to the reference function by which the response (intensity of activation) to each stimulus category at each voxel was indicated. This procedure generated activation (intensity) maps for each condition of each participant. Individual activation maps were resampled and normalized to the Talairach and Tournoux stereotaxic atlas [8] with 2  2  2 spatial resolution, and smoothed using a Gaussian filter with FWHM ¼ 9 mm. After spatial normalization, individual activation maps were analyzed at a group level using a two-factor ANOVA in AFNI. The category of the stimuli (Chinese/English) was a fixed factor while the difference between participants was a random factor. Four contrasts were used for each task: Chinese vs baseline, English vs baseline, Chinese vs baseline minus English vs baseline, and English vs baseline minus Chinese vs baseline. The uncorrected threshold for the contrasts was set at p ¼ 0.01. To correct for multiple comparisons, only brain areas with an activation volume 4 220 mm3 were considered true activation. These were determined via Monte Carlo simulation [9].

Task clue English

Chinese

*

* 3s Fig. 1.

15 5 8

30s

27s

3s

30s

English

Chinese 27s

*

* 3s

30s

27s

3s

30s

27s

Arrangement of the blocks in each run.

Vol 14 No 12 26 August 2003

Copyright © Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.

NEUROREPORT

LANGUAGE PROCESSING IN BILINGUALS

ROI analysis: To examine lateralization differences, we calculated asymmetry indexes (AIs) in four regions of interest (ROIs) that were identified based on previous findings examining brain areas associated with reading: (1) the middle/inferior frontal gyri (BA 44,45,46,10,9) [10–12]; (2) the middle/inferior temporal and fusiform gyri (BA21,37) [10,13,14]; (3) the superior/inferior parietal lobules (BA 7/ 40) [15,16]; and (4) the occipital gyrus (BA 17/18) [10,14]. Although the posterior superior temporal gyrus (Wernickes’s area) was active in a number of neuroimaging studies examining reading [14,17,18], it was not activated in our study. We therefore did not use it as a ROI. The ROIs were demarcated on the high-resolution SPGR images slice-by-slice according to the boundary of Brodmann areas in the atlas of Talairach and Tournoux [8] and the Talairach Daemon database, provided by Lancaster and Fox of RIC-UTHSCSA, that is embedded in the AFNI software. For each ROI, regional activation volume size in each condition for each subject was computed by averaging the volume size in each ROI based on the individual activation maps. Laterality was evaluated by calculating an asymmetry index (AI) for each condition based on the regional activation volume size (AI ¼ sum(voxels(LR))/sum (voxels (L + R)) [10,11]. The value of AI ranges from 1 to + 1, with a negative value indicating right hemispheric dominance and a positive value indicating left hemispheric dominance.

RESULTS Behavioral data: The average reaction times and error rates from the orthographic task were 611 ms (93% accuracy) and 592 ms (96% accuracy) for the Chinese and English

stimuli, respectively. For the semantic task, reaction times and error rates were 504 ms (97% accuracy) and 642 ms (97% accuracy) for the Chinese and English stimuli, respectively. There was a significant interaction between task type and language (p o 0.001). Planned comparisons showed that no significant differences existed in the orthographic task, (p 4 0.1) but reaction times were faster in the semantic task in Chinese than in English (p o 0.001). These results suggest that the different types of task caused different types of processing to occur.

fMRI data: In the orthographic task, common brain areas activated by both Chinese and English stimuli were the left fusiform gyrus, the middle occipital gyrus, the posterior central gyrus, and the left inferior parietal lobule. The Chinese characters also activated left middle/inferior temporal gyri, bilateral rectus gyri, the anterior cingulate gyrus, the right fusiform gyrus, and the thalamus. Alternatively, the English words activated the right inferior parietal lobule, the left supramarginal gyrus and the precentral gyrus. A direct contrast showed that, compared to English words, Chinese words activated the left posterior middle temporal gyrus, while compared to Chinese words, English words activated bilateral parietal inferior lobes and the posterior central gyrus (Table 1, Fig. 2a). The assymetry indexes (AIs) in the four brain ROIs for Chinese were  (where  indicates no activation), 0.83, 1.0, and 1.0, respectively, and for English were , 1.0, 0.19, and 0.11 (Figs. 3a and 4a). These results suggest that brain activation induced by the Chinese orthographic search task was strongly left lateralized but brain activation induced by

Table 1. Brain activation in the orthographic search task (p o 0.01). Brain areas Chinese vs baseline Posterior middle/inferior temporal gyrus; fusiform gyrus Anterior cingulate gyrus Posterior rectus gyrus Posterior rectus gyrus Inferior parietal lobule Fusiform gyrus Middle occipital gyrus Middle temporal gyrus Thalamus English vs baseline Inferior parietal lobule Inferior parietal lobule Fusiform gyrus Inferior parietal lobule/supramarginal gyrus Middle occipital gyrus Precentral gyrus Precentral gyrus Middle occipital gyrus, cuneus Posterior central gyrus Chinese vs baseline minus English vs baseline Posterior middle temporal gyrus English vs baseline minus Chinese vs baseline Inferior parietal lobule/supramarginal gyrus Inferior parietal lobule Inferior parietal lobule Posterior central gyrus Posterior central gyrus

Brodmann’s area

Coordinates (x,y,z)

Volume (mm3)

21/37 (L) 24 (R) 25 (L,R) 25 (R) 40/2/5 (L) 37 (R) 17/18 (L) 21 (L)

48,53,12 25,2,35 1,25,11 9,28,15 40,34,41 33,44,11 25,83,7 45,60,2 16,16,9

2836 525 520 344 307 290 237 235 223

40 (L) 40 (R) (L) 7/40 (L) 19 (R) 3/4 (R) 3/4 (L) 17,18 (L) 40/5 (R)

49,27,44 59,21,41 15,81,1 59,35,20 26,77,1 29,26,65 23,23,43 27,87,1 47,45,45

867 485 367 346 345 329 327 278 227

21 (L)

53,9,10

1176

39,40 (L) 40 (L) 40 (R) 2 (R) 2 (L)

47,51,27 55,43,27 55,47,27 16,45,64 7,48,68

410 255 227 273 255

L, Left Hemisphere; R, Right Hemisphere.

Vol 14 No 12 26 August 2003

15 59

Copyright © Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.

NEUROREPORT

G. DING ETAL.

R

CHINESE

L

R

ENGLISH

L

T Value 28.4

4.04 Z=−12mm

Z=40mm

Z=0mm

Z=44mm

orthographic task

(a) R

L

CHINESE

Z=−12mm

R

Z=48mm

ENGLISH

Z=−12mm

Z=28mm

L

Z=44mm

semantic task

(b)

volume(mm3)

Fig. 2. Brain activation patterns induced by Chinese and English words in the orthographic search task (a) and the semantic classi¢cation task (b), compared to the ¢xation (uncorrected p o 0.01 and cluster size Z 220 mm3, see alsoTables 1 and 2). L, left hemisphere; R, right hemisphere.

4000 3500 3000 2500 2000 1500 1000 500 0

Left Hemisphere Right Hemisphere

Chinese

(a)

English

Orthographic Task

(b)

Chinese English Semantic task

Fig. 3. The sum of activation volume (mm3) induced by Chinese and English words in the orthographic search task (a) and the semantic classi¢cation task (b) as a function of hemisphere, computed based on the averaged activation map.The ¢gure shows that the processing of Chinese was more left lateralized than the processing of English in both the orthographic search and the semantic classi¢cation task.

the English task was not, particularly at parietal and occipital areas. In the semantic task, the left middle and posterior temporal lobe and the fusiform gyrus (BA21/37) were activated by both Chinese and English stimuli. The Chinese stimuli also activated the left superior frontal gyrus, the middle frontal gyrus (BA8/9), and the left inferior occipital gyrus (BA 19), while the English stimuli activated bilateral parietal lobules and the right inferior frontal gyrus. Direct comparison between Chinese and English showed that the Chinese characters induced stronger activation in the left middle temporal gyrus, and the left middle frontal gyrus, while English words induced more right hemisphere activation in right frontal areas and the right precentral gyrus (Table 2, Fig. 3b). Comparison between activation in the left and right hemispheres showed that the English semantic judgment task activated the right hemisphere more compared to Chinese semantic judgment task (Fig. 3b). The AI value for

15 6 0

Chinese were 1.0, 1.0, , 1.0 at the four ROIs , while those for English were 1.0, 1.0, 0.45,  (Fig. 4b). The results showed that the temporal area was activated the most with both Chinese and English stimuli. This suggests that the temporal lobe played an important role in the processing of semantic information. In addition, some regions in the right parietal area and frontal area were involved in the processing of English words. These results suggest that native-language processing in the task was strongly left lateralized, whereas the right hemisphere, particularly the right frontal and parietal areas, was used more in second-language processing.

DISCUSSION The extent to which strategies and brain areas differ when bilinguals use their native and second languages is a relatively controversial subject. In this study, we examined reading with Chinese–English bilinguals in a perceptual orthographic task, designed to emphasize form related differences, and a semantic categorization task, designed to emphasize semantic processing differences. The idea was to investigate differences and similarities between native and second language usage at different levels of processing. The results we found in the orthographic search task showed that English words caused more right hemisphere activation than Chinese characters. These findings support the hypothesis that second-language processing is more right hemisphere dominated than native-language processing. They also show that even low-level search tasks using a non-native language can activate the linguistic system, since areas activated by the task were similar to those found in other language tasks [16]. Differences between languages were also found in the semantic classification task. Again, more right hemisphere activation was found in English than Chinese. One potential interpretation of these results is that the differences in lateralization merely reflect differences in the

Vol 14 No 12 26 August 2003

Copyright © Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.

NEUROREPORT

LANGUAGE PROCESSING IN BILINGUALS

Table 2. Brain activation in the semantic classi¢cation task (p o 0.01). Brain areas Chinese vs baseline Posterior inferior temporal gyrus/fusiform gyrus Superior frontal gyrus/middle frontal gyrus Middle temporal gyrus Fusiform gyrus/inferior occipital gyrus English vs baseline Posterior inferior temporal gyrus/fusiform gyrus Inferior parietal lobule Middle inferior temporal gyrus Inferior frontal gyrus Superior parietal lobule/inferior parietal lobule Chinese vs baseline minus English vs baseline Middle temporal gyrus Middle frontal gyrus English vs baseline minus Chinese vs baseline Inferior frontal gyrus End of precentral gyrus

Brodmann’s area

Coordinates (x,y,z)

Volume (mm3)

37 (L) 8,9 (L) 21 (L) 19 (L)

46,68, 9 11,40,56 62,10,11 39,74,9

812 480 385 247

37 (L) 40 (R) 22 (L) 44 (R) 7/40 (L)

44,49,21 57,32,44 60,30,15 52,18,31 31,46,52

1454 758 628 456 364

21 (L) 8,9 (L)

59,6,9 35,15,57

331 302

44 (R) 4/6 (R)

55,25,31 56,3,30 59,17,31

315 458 363

L, Left Hemisphere; R, Right Hemisphere.

1

3 24

1 2 3 4 −0.5

0

0.5

1

(a) Orthographic Task

1 CHINESE ENGLISH 1 : middle/inferior frontal gyri 2 : middle/inferior temporal; fusiform gyri 3 : superior/inferior parietal lobules 4 : occipital gyri

2 3 4 −1 (b)

−0.5

0

0.5

1

Semantic Task

Fig. 4. Asymmetry indexes for the selected brain regions of interest (ROIs) in the orthographic search task (a) and the semantic classi¢cation task (b). Areas are (1) the middle/inferior frontal gyrus, (2) the middle/inferior temporal and fusiform gyrus, (3) the superior/inferior parietal lobule and (4) the occipital gyrus.

nature of the written form of the two languages, since English uses Roman letters whereas Chinese uses characters. These different written forms have significantly different visual properties, and thus might cause a difference in lateralization when being processed. Based on a comparison with the study of Chen et al. [19], however, this possibility appears unlikely. In their fMRI study, a relatively bilateral pattern of activation was found when subjects read Chinese characters and Chinese pinyin (an alphabetic script that is composed of Roman letters that can be used instead of Chinese characters), even though different brain areas were activated by the two different scripts. In our study, activation was much more left lateralized in Chinese compared to English. This suggests that it was not just the

different visual characteristics of the script that caused a different lateralization pattern, since Chen et al. [19] did not find this difference using the same language but two different types of script, but rather, it was due to processing related to more general language functions that reading shares with other language tasks (e.g. accessing the mental lexicon). The ROI analysis in our study provides further evidence about lateralization differences in different languages and tasks. In particular, in the orthographic search task the difference appeared only in the occipital and parietal areas, whilst in the semantic classification task the difference appeared in the frontal and parietal area (Figs. 2 and 4a,b). In other words, even though the right hemisphere was

Vol 14 No 12 26 August 2003

15 61

Copyright © Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.

NEUROREPORT activated more in English than Chinese in both tasks, the distribution in cerebral cortex activation differed across the tasks, which supports the hypothesis that the use of different tasks may be an important factor leading to the inconsistent pattern of results amongst bilingual studies [2]. This may be one reason why our results differ to those of Chee et al. [11], who did not find significant lateralization differences in Chinese–English bilinguals performing a stem completion task (i.e. trying to think of which letters are needed to complete a partially presented word). In this case, the stem completion task may involve a different set of processes than either our semantic categorization or orthographic search task. There was a specific difference between this study and that of Chen et al. [19] which is of particular importance. In our study, when English activation was subtracted from Chinese, the left middle frontal gyrus was activated (particularly BA 9). In the study of Chen et al. it was the opposite, where pinyin minus Chinese caused activation in left BA 9. The activation of this area in native-language processing has been found to be activated in other semantic classification tasks [20] as well as reading tasks requiring the generation of semantic alternatives [12]. Such a dissociation allows us to make a hypothesis about semantic processing in bilinguals, since the difference was not found in the orthographic search task. In particular, previous fMRI studies have suggested that left BA9 is responsible for maintaining item sets in working memory [21,22] and previous behavioral data has suggested people might be able to recall learnt relationships from memory [23] when performing semantic categorization. Thus the activation in our task may represent the retrieval of learnt relationships between instances and categories into working memory. In this case, since these relationships would have been learnt better in our participants native compared to their second language, they may have relied on them more when using their native language.

G. DING ETAL.

In conclusion, our results support the hypothesis that second-language processing is more right hemisphere dominated than native-language processing. This was true in both tasks we used, which emphasized quite different aspects of reading. Our results also support the possibility that people use learnt relationships between instances and categories more in their first compared to second language.

REFERENCES 1. Albert ML and Obler LK. The Bilingual Brain. Neuropsychological and Neurolinguistic Aspects of Bilingualism. New York: Springer-Verlag; 1978. 2. Proverbio AM, E`ok B and Zani A. J Cogn Neurosci 14, 994–1017 (2002). 3. Fabbro F. The Neurolinguistics of Bilingualism, An Introduction. Hove, East Sussex: Psychology Press; 1999. 4. Illes J, Francis WS, Desmond JE et al. Brain Lang 70, 347–363 (1999). 5. Price CJ, Wise RJS and Frackowiak RSJ. Cerebr Cortex 6, 62–70 (1996). 6. Ziegler JC, Van Orden GC and Jacobs AM. J Exp Psychol Hum Percept Perform 23, 845–860 (1997). 7. Cox RW. Comput Biomed Res 29, 162–173 (1996). 8. Talairach J and Tournoux P. Co-Planar Stereotaxic Atlas of the Human Brain. New York: Thieme; 1988. 9. Forman SD, Cohen JD, Fitzgerald M et al. Magn Reson Med 33, 636–647 (1995). 10. Tan LH, Spinks JA, Gao JH et al. Hum Brain Mapp 10, 16–27 (2000). 11. Chee MWL, Tan EWL and Thiel T. J Neurosci 19, 3050–3056 (1999). 12. Tan LH, Liu HL, Perfetti CA et al. Neuroimage 13, 826–846 (2001). 13. Fiez JA and Peterson SE. Proc Natl Acad Sci USA 95, 914–921 (1998). 14. Petersen SE, Fox PT, Posner MI et al. Nature 331, 585–589 (1988). 15. Rumsey JM, Horwitz B, Donohue C et al. Brain 120, 739–759 (1997). 16. Warburton E, Wise RJS, Price CJ et al. Brain 119, 159–180 (1996). 17. Price CJ, Wise RJS, Warburton E et al. Brain 119, 919–931 (1996). 18. Booth JR, Burman DD, Meyer JR et al. Neuroimage 16, 7–22 (2002). 19. Chen Y, Fu S, Iversen SD et al. J Cogn Neurosci 14, 1088–1098 (2002). 20. Hugdahl K, Lundervold A, Ersland L et al. Int J Neurosci 99, 49–58 (1999). 21. Cabeza R and Nyberg L. J Cogn Neurosci 12, 1–47 (2000). 22. Petrides M. J Neurosci 20, 7496–7503 (2000). 23. Fodor J and Lepore E. Cognition 58, 253–270 (1996).

Acknowledgements: This research was supported by the National Program of Basic Research of China (G199905400) and a grant from the Chinese Natural Science Foundation (30270462) and the National Panding Project (95-special- 09) awarded to the third author.The second author is support by a UDF grant from the University of Hong Kong.

15 62

Vol 14 No 12 26 August 2003

Copyright © Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.