Visual and Auditory Factors Facilitating Multimodal Speech Perception

Please use this identifier to cite or link to this item:

Show full item record

Files Size Format View
Visual_and_Auditory_Factors_Facilitating.pdf 79.23Kb PDF View/Open

Title: Visual and Auditory Factors Facilitating Multimodal Speech Perception
Creators: Ver Hulst, Pamela
Advisor: Weisenberger, Janet
Issue Date: 2006-06
Abstract: Speech perception is often described as a unimodal process, when in reality it involves the integration of multiple sensory modalities, specifically, vision and hearing. Individuals use visual information to fill in missing pieces of auditory information when hearing has been compromised, such as with a hearing loss. However, individuals use visual cues even when auditory cues are perfect, and cannot ignore the integration that occurs between auditory and visual inputs when listening to speech. It is well known that individuals differ in their ability to integrate auditory and visual speech information, and likewise that some individuals produce clearer speech signals than others, either auditorily or visually. Clark (2005) found that some talkers in a study of the McGurk effect, produced much stronger ‘integration effects’ than did other talkers. One possible underlying mechanism of auditory + visual integration is the substantial redundancy found in the auditory speech signal. But how much redundancy is necessary for effective integration? And what auditory and visual characteristics make a good integration talker? The present study examined these questions by comparing the auditory intelligibility, visual intelligibility, and the degree of integration for speech sounds that were highly reduced in auditory redundancy, produced by 7 different talkers. Performance of participants under four conditions: 1) degraded auditory only, 2) visual only, 3) degraded auditory + visual, and 4) non-degraded auditory + visual, was examined. Results indicate across-talker differences in auditory and auditory + visual intelligibility. Degrading the auditory stimulus did not affect the overall amount of McGurk-type integration, but did influence the type of McGurk integration observed.
Series/Report no.: The Ohio State University. Department of Speech and Hearing Science Honors Theses; 2006
Keywords: Multimodal Speech Perception
McGurk effect
Sponsors: The present study was supported by an ASC Undergraduate Research Scholarship and by the SBS Undergraduate Research Scholarship.
Bookmark and Share
Attribution 2.5 Generic This item is licensed under a Creative Commons License:
Attribution 2.5 Generic