The Role of Information Redundancy in Audiovisual Speech Integration
MetadataShow full item record
Publisher:The Ohio State University
Series/Report no.:The Ohio State University. Department of Speech and Hearing Science Honors Theses; 2007
When most people think about communication, they think of its auditory aspect. Communication is made up of much more; speech perception is dependent on the integration of different senses, namely the auditory and visual systems. An everyday example of this is when someone tries to have a conversation at a noisy restaurant; a person may unconsciously pay attention to the speaker’s facial movements in order to gain some visual information in an imperfect auditory situation. In general, listeners are able to use visual cues in impoverished auditory situations (like a restaurant, or a hearing loss.) However, this process also occurs when the auditory signal provides sufficient information alone. In 1998, Grant and Seitz conducted a study that found that listeners differ greatly in their perceptions of auditory-visual speech. This study generated a lot of questions about how integration occurs, namely what promotes “optimal integration.” Research shows that many factors may be involved: it may be characteristics of the listener, the talker, or of the acoustic signal that influence the amount of integration. The present study looked at the characteristics of the acoustics, namely whether removal of fine spectral information from the speech signal would elicit more use of visual cues, and thus elicit greater audiovisual integration. The auditory stimuli were degraded by removing the spectral fine structure and replacing it with noise, but retaining the envelope structure. These stimuli were then output through 2-,4-,6-, and 8-channel bandpass filters. Ten listeners with normal hearing were tested under auditory-only, visual-only, and auditory-plus-visual presentations. Results showed substantial auditory-visual integration over all conditions. Also, significant cross-talker effects were found in the 2- and 4-channel auditory-only condition. However, the degree of integration produced by the talkers was not related to auditory intelligibility. The results of this study have implications for our understanding of the auditory-visual integration process.
This project was supported by an ASC Undergraduate Scholarship and an SBS Undergraduate Research Scholarship.
Items in Knowledge Bank are protected by copyright, with all rights reserved, unless otherwise indicated.