Making the rainbow connection: Audio-visual crossmodal correspondences between colors and musical instrument timbres

Loading...
Thumbnail Image

Date

2019-03

Journal Title

Journal ISSN

Volume Title

Publisher

Research Projects

Organizational Units

Journal Issue

Abstract

Music is an art form for the sense of hearing, but the words we use to describe musical sounds intriguingly recruit imagery from a much wider range of sensory experience: the ‘bright’ blare of a trumpet, the ‘sweet’ song of an oboe, or the ‘sharp’ exclamation of a piccolo. Are these associations simply the result of co-opting and re-defining existing words, or are they evidence of deeper perceptual connections among the senses? In a previous research project, I interviewed two dozen musicians and asked them to describe the timbres, or sound qualities, of musical instruments. I noticed that the musicians often volunteered color-related words to describe musical timbres. For example, interviewees described the bass clarinet as “a very deep, dark, navy-blue colored sound,” the saxophone as “orange,” the oboe as “yellow, with green undertones,” the flute as a “sky blue color,” and the piccolo as “fuchsia pink.” In addition to naming colors, participants often used words with visual, and specifically color-related, meanings: “light,” “bright,” “dark,” “warm,” and “cool” were among the top 25 most frequently used words. Informally, timbre is also referred to as “tone color,” and the language used by musicians in these interviews suggests that this alternate name may be more than mere analogy. The nature of these connections is scientifically interesting—how widespread are cross-sensory associations and what are their origins? Does metaphorical language predict how people match color and sound? The answers to these questions are not only important scientifically, but also artistically. Audio-visual correspondences are particularly important for the relatively new genre of music visualization, in which music is intentionally paired or co-created with color, shape, and movement (think Disney’s “Fantasia). Understanding the psychology of expectation for audio-visual correspondences allows theorists to consider and analyze how such art plays with expectations and provides artists with new resources. Inspired by my observations of color-related language in the interviews, I designed a series of empirical studies to investigate whether there are latent, commonly shared correspondences between musical instrument sounds and colors. Such potential timbre-color correspondences have not been explored by any published study to date. Based on the interview data and extant studies from psychology, ethology, and neuroscience that address how sounds may be mapped to visual features, I compiled a list of descriptive terms for sound that I predicted would tap into associations between timbre and color. In the first study, I hypothesized that sounds that are rated as smaller, brighter, higher, and happier are associated with lighter colors, while sounds that are rated as bigger, darker, lower, and sadder are associated with darker colors. Furthermore, I posited that instruments that were considered “warm” would be more likely to be matched with warm colors, while “cool” sounding instruments would be more likely to be matched with cool colors. First, participants listened to sounds of six different musical instruments and chose colors that they believed best matched the sound quality or character of the instrument. Then, they listened to the same six instruments and rated how well each of the adjectives (e.g. “bright,” “warm,” etc.) described those sounds. The 106 participants in the first study were visitors recruited from the Center for Science and Industry (COSI) in Columbus, and the study was completed using headphones and iPads with apps custom-built for the experiment. The results of the first study support the first hypothesis that smaller, brighter, higher, and happier sounds correspond with lighter colors, and bigger, darker, lower, and sadder sounds correspond with darker colors. The second hypothesis, that warm colors correspond with warm sounds and cool colors with cool sounds, is not supported. Intriguingly, negative correlations between ratings of words that seem to be opposites (e.g. “bright” and “dark”) were not as strong as would be anticipated if these pairs were true opposites, from which I make the recommendation that future studies in timbre should avoid bipolar scales that assume opposites, allowing for more complexity in responses. These results support and clarify the role of cross-sensory, metaphorical language in structuring such correspondences. Questions of cross-sensory mapping and metaphor are relevant not only to musical research on timbre and music visualization, but also to cognitive science, psychology, linguistics, and neuroscience, and so these results will contribute to multiple fields of inquiry.

Description

The Arts: 1st Place (The Ohio State University Edward F. Hayes Graduate Research Forum)

Keywords

crossmodal correspondences, timbre, qualia, audiovisual, crossmodal correspondences, timbre, qualia, audiovisual

Citation