Knowledge Bank

Although numerous studies have documented the existence of auditory-visual integration for both perfectly intelligible and compromised speech, research has yet to determine if it is these compromised situations that optimize this process. There is compelling evidence suggesting that listeners benefit from the addition of visual cues when auditory information is compromised in some way. However, studies in multimodal perception document a large degree of variability in the amount of benefit listeners receive from auditory-visual integration. The present study examined the role of the auditory signal in auditory-visual integration in order to explore how characteristics of the auditory signal might influence and change this process. The present study addressed several specific questions including: Does the amount of available information in the auditory signal affect auditory-visual integration ability? How does the integration process change as auditory information available in the speech signal is altered? Is the amount of information available in the auditory signal a contributing factor to the large degree of variability noted in the amount of benefit listeners receive from auditory-visual integration? The present study examined if the amount of information available in the auditory signal affected the auditory-visual integration process. Listeners were presented with degraded speech stimuli containing different amounts of information and their speech perception abilities in a number of conditions were measured. The stimuli were degraded using 2, 4, 6, and 8 bandpass filter channels. The temporal fine structure was then removed from the speech syllable stimuli and was replaced with narrowband noise, modulated by the temporal envelope retained in the filtered speech waveform. Performance was examined in four different conditions: auditory-only (A), visual-only (V), congruent auditory + visual (AV) and discrepant auditory + visual (AV). Results of this study demonstrated that degrading auditory stimuli, regardless to what degree, decreases overall speech perception ability in the A and AV conditions. Results also revealed a decrease in the amount of auditory-visual integration listeners achieved for discrepant stimuli when compared to the amount of McGurk-type integration they are able to achieve when the auditory signal is perfect. Although overall integration ability is decreased for degraded stimuli, removing information from the auditory signal does not inhibit auditory-visual integration ability. Surprisingly, the amount of auditory-visual integration did not change across the majority of the different levels of auditory signal degrading. Finally, there do appear to be difference in the type of stimulus materials used and the way that listeners integrate auditory and visual speech. The present study has provided some important insight to the process of auditory-visual integration. It is clear from the results of this study that removing information from the auditory signal does affect auditory-visual integration, but does not inhibit this ability. However, as the amount of signal degrading is altered only one level of degrading showed slightly significant differences in the amount of auditory-visual integration that was achieved at that level. These results suggest that the amount of information available in the auditory signal may play a slight role in the variability noted in the amount of benefit listeners receive from auditory-visual integration.