The relationship between sound and vision

Sound and vision | School of Physics and Astronomy

the relationship between sound and vision

I've always been intrigued with the relationship between sound and image They can provide visual meaning to a song's narrative, or create a. provides insights into how the brain combines sound and vision. Among the implications of the study: It might not be as easy as many. J. Donald Harris, Ph.D., was Head, Sound Section, U.S. Naval On the other hand, the similarities between audition and vision are often striking. . a relationship between threshold levels of hearing and vision, at least in.

The researchers presented combinations of beeps and flashes in a one-hour period. In the second part of the study, subjects were asked to sit facing a black screen, behind which were five speakers. A projector mounted overhead was used to flash bursts of light onto the screen, at the same spots where the speakers were located.

the relationship between sound and vision

The researchers played brief bursts of sound and triggered flashes of light, in various combinations, and asked participants to identify where they originated. The participants used a computer mouse to point at the part of the screen where they thought the sounds and flashes occurred; each was presented with trials.

Brain Reconciles Sight and Sound in Different Ways

When the flash and sound occurred in different locations from each other, most participants struggled to correctly identify where the sound occurred. The effect was similar to what happens when people watch a ventriloquist with a puppet or a magician using misdirection to pull off an illusion, Odegaard said.

The relationship between color and music - Siana Altiise - TEDxPeachtree

Given that the two tasks both involved brief sights and sounds, the researchers figured that the people who were fooled less on one task would be fooled less on the other as well, but the researchers were surprised to find no correlation in how much subjects combined the two senses on the two tasks.

A week later, the participants were given both of the same tasks again.

the relationship between sound and vision

The results were the same—although they performed consistently on each individual task, their abilities to bind the two senses in the two different tests showed no correlation. A second group heard the right-to-left sound during both phases.

Sound and vision work hand in hand, UCLA psychologists report

And a third group heard sound moving in the opposite direction — from left to right — during both phases. Then, each participant experienced trials in all three conditions.

the relationship between sound and vision

As Shams expected, the participants were best able to identify the phase in which the dots moved horizontally when the sound moved in the same direction as the dots but remained stationary during the random phase. The researchers found that the sound that moved in the opposite direction neither enhanced nor worsened the participants' visual perception. Surprisingly, the sound that traveled leftward both when the dots moved leftward and when the dots moved randomly — that is, sound that provided no useful information for choosing between the two phases — also helped people correctly choose the phase with the horizontal motion.

Because the sound was identical in both phases, if the participants closed their eyes they, would have a chance of successfully performing the task based on sound alone; with their eyes open, however, the interaction between sound and vision led to a significant improvement in detection of visual motion.

Sound and vision work hand in hand, UCLA psychologists report | UCLA

Scientists have believed that each of the senses produces an estimate relevant for the task, and then these votes get combined subconsciously according to rules that take into account which sense is more reliable. This is how the senses interact in how we perceive the world. However, our findings show that the senses of hearing and vision can also interact at a more basic level, before they each even produce an estimate.

The senses affect one another other in many ways, said Robyn Kim, a former UCLA psychology postdoctoral scholar in Shams' laboratory and lead author of the study.