One task that we solve daily is our ability to see an object, identify it, and name it. Our investigation focused on this last ability.
MPA 1996
Friday, May 3, 10:15
One task that we solve daily is our ability to see an object, identify it, and name it. Our investigation focused on this last ability.
So suppose you saw this object and were asked to name it.
Having already identified the object based on its visual features, we must now select the appropriate name corresponding to the concept.
As we saw in Zenzi Griffin's talk, language production models typically identify at least two stages of processing that must be involved: a semantic process which maps conceptual features to word representations and a phonological process which maps words onto their sounds.
There are a number of debates concerning different architectural features of the model. The debate that we focused on, is whether processing flows from top to bottom, in a two-stage modular fashion or a cascading fashion. The modular model states that semantic processing must be completed prior to the onset of processing at the lower phonological stage. The cascading model allows processing at the phonological level to begin based on partial processing at the semantic level.
Schreifers, Meyer, and Levelt examined this issue using a picture/word interference task. Our experiments used the same task, so I will explain it in some detail here. Speakers quickly named a picture, while ignoring an auditorily presented distractor word. Schreifers, et al. manipulated two factors, the time between the onset of the distractor word and the onset of the picture (which I will refer to as SOA), and the relationship between the distractor word and the picture. In the early SOA condition the distractor word was begun 150 ms before the onset of the picture, at the late SOA the distractor began 150 ms after the onset of the picture. Distractor words were either semantically related to the picture name (like turtle for a frog picture), phonologically related (frost), or unrelated (piano), which provided the picture naming baseline.
Here are the results of their experiment. At the early SOA semantic distractors interfered with picture naming. That is, hearing the word "turtle", slightly before seeing the frog, slowed picture naming. Phonological distractors, at this early SOA, did not affect picture naming. At the late SOA condition, a different pattern emerged; semantic distractors had no effect, while, phonological distractors facilitated picture naming.
Schriefers et al. interpreted these results as support for a modular model of language production. The early SOA results reflect an early semantic stage, at which there is no phonological processing. The late SOA results reflect a later phonological stage of processing, during which there is no semantic processing.
Dell and O'Seaghdha suggested that the partial phonological relationship between the distractor word and the picture name may not have been strong enough to result in measurable phonological effects at the early SOA. An implication of this suggestion is that if one could strengthen the phonological relationship between the distractor and the picture name, then you should be able to see evidence of overlapping semantic and phonological processing.
The three experiments presented here used homophone and non-homophone pictures. Homophone pictures offered us a chance to maximize the phonological relationship between the picture and distractor. Consider the homophone ball. It is a word with two distinct unrelated meanings; as in "toy-ball" and "formal dance-ball". Our representational assumption is that homophones are represented at the semantic level as two distinct representations that share a single representation at the phonological level. Thus the phonological relationship between these two separate semantic representations of the homophones is maximized. Our prediction was that if semantic and phonological processing cascade, then by using homophones we should be able to find evidence of overlapping semantic and phonological processing.
The next slide lays out the design and procedure of the first two experiments.
We only used the early SOA condition. There were two kinds of pictures, half were designed to elicit non-homophones like frog and half were designed to elicit homophones like ball. For the non-homophone pictures distractors were either phonologically, semantically, or unrelated to the picture. For the homophone pictures there were: unrelated distractors, appropriate distractors, which were semantically related to the depicted meaning of the homophone, like game for the toy-ball picture. The critical condition was the inappropriate distractors.
These were words semantically related to the non-depicted meaning of the homophone. So "dance" is semantically related to the formal dance meaning of ball, which is in turn phonologically related to the toy ball meaning. In other words, dance is a phonologically mediated semantic distractor for the toy ball picture.
Both the modular and cascading models predicted that for the non-homophone pictures we should replicate the Schriefers et al pattern; semantic distractors should cause interference, while the partially overlapping phonological distractors should have little or no effect.
For the homophone pictures, the two models make different predictions. Both models predict that appropriate distractors, like game, should result in interference.
For the inappropriate condition, with phonologically mediated semantic distractors, the modular model predicts no effect. In contrast the cascading model predicts that there should be evidence of overlapping semantic and phonological processing, because of the strong phonological relationship between the two different semantic representations. We were uncertain whether to expect semantic interference or phonological facilitation.
Here are the results of the first experiment.
Looking first at the non-homophone pictures, recall that we predicted semantic interference and no effect of phonological distractors. This is indeed what we found.
Turning the homophone pictures. Beginning with the inappropriate condition where the modular and cascading views had contrasting predictions. The modular view predicted that there should be no effect of inappropriate distractors, because this would require overlapping semantic and phonological processing. The cascading view did predict an effect, either interference or facilitation. The results were that inappropriate distractors facilitated picture naming. That is, speakers named the toy-ball picture faster if they heard "dance" slightly before seeing the picture of the ball. Thus, these results support the cascading model, in which semantic and phonological processing overlap.
Recall that both the modular and cascading models predicted that appropriate distractors should have resulted in interference. There was a small trend towards interference, however this difference was not significant.
One possible reason was that many of the homophone pictures were drawn specifically for the experiment. So it was possible these pictures were harder to recognize, which may have resulted in ceiling effect that obscured the semantic interference.
So we replicated the first experiment with one difference. This time we showed the speakers the pictures with their names before they participated in the picture/word interference task. We felt that the pre-exposure should reduce any picture recognition difficulty differences existing between the homophone and non-homophone pictures.
The pattern of results replicated those seen in experiment 1. Again with the non-homophones we replicated the Schriefers et al results.
With homophone pictures, inappropriate distractors facilitated picture naming.
And, once again, the predicted semantic interference of appropriate distractors showed a only a small non-significant trend towards interference. The only apparent effect of pre-exposing the speakers with the pictures and their names was an overall speeding of picture naming across all of the conditions.
So what should we take home from the results of Experiments 1 & 2?
The primary point is that we found evidence of overlapping semantic and phonological processing in the inappropriate condition. That is, hearing a word like "dance" shortly before seeing the toy ball picture, facilitated naming. This phonologically mediated semantic priming supports the cascading model over the modular model.
Second, we replicated the Schriefers et al results with non-homophone pictures, demonstrating that partial phonological overlap may not be enough to result in early phonological priming.
And finally, we are left with a curious reduction in semantic interference in the homophone appropriate distractor condition. And this doesn't seem to be related to picture recognizability.
So what's going on in this appropriate distractor condition?
Looking back at the semantic and appropriate distractors in the first two experiments we identified two kinds of semantic relations: semantic competitors (words from the same semantic category) and associates (related but not competitors). The homophone appropriate distractors were primarily associates, while the non-homophone semantic distractors were primarily competitors. Wheeldon and Monsell found that semantic competitors cause interference while associates typically do not.
So in the final experiment, we manipulated the kind of semantic relation in the semantic and appropriate conditions for the two picture types.
We predicted that for both the Non-homophone and homophone pictures, competitors should result in interference, while associates should not.
The results supported our predictions. Competitors produced semantic interference for both homophones and non-homophones, while associates had no affects.
From these results we concluded that the reduced semantic interference seen in the appropriate condition of the first two experiments was most likely due to a high proportion of associatively related distractors. When these were replace with competitor distractors, semantic interference was observed.
Overall, our results, like Zenzi's, support a model of language production in which semantic and phonological processing exist in a cascading relationship. Using homophone pictures allowed us to strengthen the phonological relationship between the picture and distractor.
The major result supporting this claim was that phonologically mediated semantic distractors facilitated picture naming, demonstrating that semantic and phonological processes must overlap.
For more details about this project you may want to take a look at the poster that we presented at the CUNY Sentence Processing conference.