Experiments on sentence comprehension

The initial motivation for experiments on sentence comprehension comes from a study published in Science in which the brain responses to deviant words in sentence contexts were investigated (Hagoort et. al, 2004). In their study, the ERP elicited by deviant words (e.g., ‘gravity’ in S2, below) in reading visually presented sentences was compared with that by words that are congruent with the sentence context (e.g., ‘Germany’ in S1).

S1:    ‘The fall of the Berlin Wall reunited Germany’        Congruent sentence

S2:    ‘The fall of the Berlin Wall reunited gravity’            Incongruent sentence

It was found that such a semantic violation elicits a more negative brain wave component, N400, that peaks at around 400 ms after stimulus onsets, than that elicited by congruent sentences.

Our attempt to conduct a Chinese version of Hagoort's has started to reveal novel findings. In our experiment, an incongruent sentence (S4) was composed by replacing a word in a congruent sentence (S3, ‘An increase was observed in the number of young drug abusers’) with a deviant word such that anomalies in meaning were introduced. An example of test materials used in our experiment is given below:

S3:    青少年 濫用 藥物 的 情況 有所 增加                    Congruent sentence

S4:    青少年 濫用 山火 的 情況 有所 增加                    Incongruent sentence

Sentences were presented visually to the subjects in a word-by-word manner. Participants were asked to read the sentences silently and to understand them. EEG segments time-locked to the onset of the target words were extracted and averaged through which ERPs to congruent and incongruent sentences were obtained for each subject.

Experiments on speech perception

A fundamental aspect of perception, that has long been our interest, is its categorical nature, i.e. the mapping of a continual physical world into discrete mental categories. In speech perception, though the tonal contours of a syllable can vary physically continuously between a low frequency tone (e.g., ‘罵’ /maa6/ in Cantonese) and a high frequency one (e.g., ‘媽’ /maa1/ in Cantonese), native speakers often perceive the continuum as a series of discrete categories. As a consequence, when asked to discriminate pairs of stimuli, the performance for pairs that cross a categorical boundary is much better than that for within category pairs.

We are interested in zooming in to look at brain activities, temporally and spatially, to address the issue of when the perception of linguistic categories emerges as well as the sources of the respective neural generators, for instance, whether and when brain activities exhibit a hemispheric dominance.

In our EEG experiment, 11 speech stimuli, all pronouncing the Mandarin syllable /yi/ were used, as described in Wang (1976). Stimulus #1 is clearly heard as the Mandarin tone two word 姨 /yi2/ while stimulus #11 is clearly heard as the tone one word 衣 /yi1/. All adjacent stimuli are separated from each other by a frequency difference of 3 Hz. Previous studies done by the PI (Wang, 1976) have shown that the category boundary in this tonal continuum is on stimulus #7

While the dominant neurolinguistic theory suggests a serial model of language processing, orthography to phonology and phonology to semantics, our brain waves data, however, show an early sensitivity, in a time window around 150 ms to 200 ms after stimulus onsets, in the brain responses to semantic manipulations (figure on the right). This suggests that the brain processes for language are parallel in nature, implying the involvements of multiple neural circuits that subserve different types of processing. Such a suggestion is consistent with the mosaic nature of language behaviour.

Grand average ERPs, of 21 subjects, to target words of congruent (blue curve) and incongruent (red curve) sentences at an electrode site on the left posterior region of the scalp.

During the experiment, subjects were asked to watch a movie of their selection; the movie was watched without sound but with subtitles. Simultaneously, they were presented with sequences of stimuli via earphones to which they were told not to pay any attention. Using an odd-ball paradigm, stimulus #8 (the “standard” with a high probability of occurrence in a sequence) was mixed with either a across-category deviant (stimulus #6) or a within-category deviant. The magnitude (in microvolt) and latency (in millisecond) of an ERP component called Mismatch Negativity (MMN) were extracted as the dependent variables of the study, as shown in the figure on the left.

Mismatch Negativities, obtained by subtracting ERPs to standard stimuli from ERPs to two types of deviant stimuli, recorded from 19 subjects.

References:

1. (1)P. Hagoort, L. Hald, M. Bastiaansen, and K. M. Petersson, "Integration of word meaning and world knowledge in language comprehension," Science, vol. 304, pp. 438-441, 2004
   

2. (2)W. S.-Y. Wang, "Language change," Annals of the New York Academy of Science, vol. 280, pp. 61-72, 1976.