Academia.eduAcademia.edu

Outline

Title
Abstract
Introduction
Experiment 1 Methods
Procedure
Data Analysis
Participants
Apparatus and Materials
Methods
Results
Discussion
General Discussion
References
All Topics
Psychology
Cognitive Psychology

Drawing sounds: representing tones and chords spatially

Profile image of Juan carlos ArboledaJuan carlos Arboleda

2016, Experimental Brain Research

https://doi.org/10.1007/S00221-016-4747-9
visibility

…

description

14 pages

link

1 file

Abstract
sparkles

AI

Research on the crossmodal correspondences has revealed that seemingly unrelated perceptual information can be matched across the senses in a manner that is consistent across individuals. An interesting extension of this line of research is to study how sensory information biases action. In the present study, we investigated whether different sounds (i.e. tones and piano chords) would bias participants' hand movements in a free movement task. Right-handed participants were instructed to move a computer mouse in order to represent three tones and two chords. They also had to rate each sound in terms of three visual analogue scales (slow-fast, unpleasant-pleasant, and weak-strong). The results demonstrate that tones and chords influence hand movements, with higher-(lower-) pitched sounds giving rise to a significant bias towards upper (lower) locations in space. These results are discussed in terms of the literature on forward models, embodied cognition, crossmodal correspondences, and mental imagery. Potential applications sports and rehabilitation are discussed briefly.

Related papers

Audiovisual correspondence between musical timbre and visual shapes
Mohammad Asif Adeli

Frontiers in human neuroscience, 2014

This article investigates the cross-modal correspondences between musical timbre and shapes. Previously, such features as pitch, loudness, light intensity, visual size, and color characteristics have mostly been used in studies of audio-visual correspondences. Moreover, in most studies, simple stimuli e.g., simple tones have been utilized. In this experiment, 23 musical sounds varying in fundamental frequency and timbre but fixed in loudness were used. Each sound was presented once against colored shapes and once against grayscale shapes. Subjects had to select the visual equivalent of a given sound i.e., its shape, color (or grayscale) and vertical position. This scenario permitted studying the associations between normalized timbre and visual shapes as well as some of the previous findings for more complex stimuli. One hundred and nineteen subjects (31 females and 88 males) participated in the online experiment. Subjects included 36 claimed professional musicians, 47 claimed amate...

View PDFchevron_right
The Impact of Stochastic and Deterministic Sounds on Visual, Tactile and Proprioceptive Modalities
Jocelyn Faubert

Advances in Sound Localization, 2011

View PDFchevron_right
Modifying action sounds influences people’s emotional responses and bodily sensations
Cristiano Porto Klanovicz, Leandro M Tonetto

i-Perception, 2014

View PDFchevron_right
Conditioned Sounds Enhance Visual Processing
Fabrizio Leo

PLoS ONE, 2014

This psychophysics study investigated whether prior auditory conditioning influences how a sound interacts with visual perception. In the conditioning phase, subjects were presented with three pure tones (= conditioned stimuli, CS) that were paired with positive, negative or neutral unconditioned stimuli. As unconditioned reinforcers we employed pictures (highly pleasant, unpleasant and neutral) or monetary outcomes (+50 euro cents, 250 cents, 0 cents). In the subsequent visual selective attention paradigm, subjects were presented with near-threshold Gabors displayed in their left or right hemifield. Critically, the Gabors were presented in synchrony with one of the conditioned sounds. Subjects discriminated whether the Gabors were presented in their left or right hemifields. Participants determined the location more accurately when the Gabors were presented in synchrony with positive relative to neutral sounds irrespective of reinforcer type. Thus, previously rewarded relative to neutral sounds increased the bottom-up salience of the visual Gabors. Our results are the first demonstration that prior auditory conditioning is a potent mechanism to modulate the effect of sounds on visual perception.

View PDFchevron_right
Multidimensional processing of dynamic sounds: more than meets the ear
Eduardo Mercado

Attention, Perception, & Psychophysics, 2011

Strong cross-modal interactions exist between visual and auditory processing. The relative contributions of perceptual versus decision-related processes to such interactions are only beginning to be understood. We used methodological and statistical approaches to control for potential decision-related contributions such as response interference, decisional criterion shift, and strategy selection. Participants were presented with rising-, falling-, and constant-amplitude sounds and were asked to detect change (increase or decrease) in sound amplitude while ignoring an irrelevant visual cue of a disk that grew, shrank, or stayed constant in size. Across two experiments, testing context was manipulated by varying the grouping of visual cues during testing, and cross-modal congruency showed independent perceptual and decision-related effects. Whereas a change in testing context greatly affected criterion shifts, cross-modal effects on perceptual sensitivity remained relatively consistent. In general, participants were more sensitive to increases in sound amplitude and less sensitive to sounds paired with dynamic visual cues. As compared with incongruent visual cues, congruent cues enhanced detection of amplitude decreases, but not increases. These findings suggest that the relative contributions of perceptual and decisional processing and the impacts of these processes on cross-modal interactions can vary significantly depending on asymmetries in within-modal processing, as well as consistencies in cross-modal dynamics. Keywords Cross-modal interactions. Motion in depth. Looming. Audiovisual. Visual capture Successful exploration of the environment often involves evaluating and comparing information from multiple modalities and along multiple dimensions. Cross-modal interactions in perception have been studied using behavioral and physiological measures (Alais, Newell, & Mamassian, 2010; Calvert, Spence, & Stein, 2004). Intersensory conflicts are often introduced in behavioral studies of cross-modal interaction (Soto-Faraco, Spence, Lloyd, & Kingstone, 2004b; Spence, 2011). Participants are asked to attend to signals from a target modality (e.g., audition) while ignoring potentially conflicting signals from an irrelevant modality (e.g., vision). In general, these studies demonstrate that the integration of multimodal stimuli depends on the temporal (

View PDFchevron_right
Processing auditory information: Interaction of two population stereotypes
Enrique Acosta

Journal of Applied Psychology, 1976

View PDFchevron_right
Melodic sound enhances visual awareness of congruent musical notes, but only if you can read music
Randolph Blake

Proceedings of the National Academy of Sciences of the United States of America, 2015

Predictive influences of auditory information on resolution of visual competition were investigated using music, whose visual symbolic notation is familiar only to those with musical training. Results from two experiments using different experimental paradigms revealed that melodic congruence between what is seen and what is heard impacts perceptual dynamics during binocular rivalry. This bisensory interaction was observed only when the musical score was perceptually dominant, not when it was suppressed from awareness, and it was observed only in people who could read music. Results from two ancillary experiments showed that this effect of congruence cannot be explained by differential patterns of eye movements or by differential response sluggishness associated with congruent score/melody combinations. Taken together, these results demonstrate robust audiovisual interaction based on high-level, symbolic representations and its predictive influence on perceptual dynamics during bino...

View PDFchevron_right
Integration of visual and auditory information for hand actions: preliminary evidence for the contribution of natural sounds to grasping
Anna Sedda

Experimental Brain …, 2011

View PDFchevron_right
Crossmodal correspondences between sounds and tastes
Klemens Knöferle

In this article, the rapidly growing body of research that has been published recently on the topic of crossmodal correspondences that involve auditory and gustatory stimuli is critically reviewed. The evidence demonstrates that people reliably match different tastes/flavours to auditory stimuli varying in both their psychoacoustic (e.g., pitch) and musical (e.g., timbre) properties. In order to stimulate progress in this relatively young research field, the present article aims at consolidating prior findings concerning specific auditory gustatory mappings, whereby special attention is given to highlighting a) any conflicts in the existing experimental evidence, and b) any potential caveats with regard to the most appropriate interpretation of prior studies. Next, potential mechanisms underlying auditory-gustatory crossmodal correspondences are discussed. Finally, a number of potentially fruitful avenues for future research are outlined.

View PDFchevron_right
Implicit associations between individual properties of color and sound
Niklas Erben Johansson

2018

We report a series of 22 experiments in which the implicit associations test (IAT) was used to investigate cross-modal correspondences between visual (luminance, hue [R-G, B-Y], saturation) and acoustic (loudness, pitch, formants [F1, F2], spectral centroid, trill) dimensions. Colors were sampled from the perceptually accurate CIE-Lab space, and the complex, vowel-like sounds were created with a formant synthesizer capable of separately manipulating individual acoustic properties. In line with previous reports, the loudness and pitch of acoustic stimuli were associated with both luminance and saturation of the presented colors. However, pitch was associated specifically with color lightness, whereas loudness mapped onto greater visual saliency. Manipulating the spectrum of sounds without modifying their pitch showed that an upward shift of spectral energy was associated with the same visual features (higher luminance and saturation) as higher pitch. In contrast, changing formant frequencies of synthetic vowels while minimizing the accompanying shifts in spectral centroid failed to reveal cross-modal correspondences with color. This may indicate that the commonly reported associations between vowels and colors are mediated by differences in the overall balance of low-and high-frequency energy in the spectrum rather than by vowel identity as such. Surprisingly, the hue of colors with the same luminance and saturation was not associated with any of the tested acoustic features, except for a weak preference to match higher pitch with blue (vs. yellow). We discuss these findings in the context of previous research and consider their implications for sound symbolism in world languages.

View PDFchevron_right

References (90)

  1. Aglioti SM, Pazzaglia M (2010) Representing actions through their sound. Exp Brain Res 206:141-151
  2. Arnott SR, Alain C (2011) The auditory dorsal pathway: orienting vision. Neurosci Biobehav Rev 35:2162-2173
  3. Bakker DR, Martin FH (2014) Musical chords and emotion: major and minor triads are processed for emotion. Cognit Affect Behav Neurosci 15:15-31
  4. Barca L, Pezzulo G (2015) Tracking second thoughts: continuous and discrete revision processes during visual lexical decision. PLoS ONE 10:e0116193
  5. Bergeron V, Lopes D (2009) Hearing and seeing musical expression. Philos Phenomenol Res 78:1-16
  6. Bértolo H (2014) Visual imagery without visual perception: lessons from blind subjects. In: Martins Costa MFPC, Nogueira RN (eds) Second international conference on applications of optics and photonics. International Society for Optics and Photonics, p 92862K
  7. Bolognini N, Rasi F, Coccia M, Làdavas E (2005) Visual search improvement in hemianopic patients after audio-visual stimula- tion. Brain 128:2830-2842
  8. Bood RJ, Nijssen M, van der Kamp J, Roerdink M (2013) The power of auditory-motor synchronization in sports: enhancing running performance by coupling cadence with the right beats. PLoS ONE 8:e70758
  9. Buccino G, Binkofski F, Fink GR, Fadiga L, Fogassi L, Gallese V, Seitz RJ, Zilles K, Rizzolatti G, Freund HJ (2001) Action obser- vation activates premotor and parietal areas in a somatotopic manner: an fMRI study. Eur J Neurosci 13:400-404
  10. Cattaneo Z, Vecchi T, Cornoldi C, Mammarella I, Bonino D, Ricciardi E, Pietrini P (2008) Imagery and spatial processes in blindness and visual impairment. Neurosci Biobehav Rev 32:1346-1360
  11. Chaffin DB, Andersson G, Martin BJ (2006) Occupational biome- chanics. Wiley, Hoboken, NJ
  12. D'Ausilio A, Brunetti R, Delogu F, Santonico C, Belardinelli MO (2010) How and when auditory action effects impair motor per- formance. Exp Brain Res 201:323-330
  13. De Volder AG, Toyama H, Kimura Y, Kiyosawa M, Nakano H, Van- lierde A, Wanet-Defalque MC, Mishina M, Oda K, Ishiwata K, Senda M (2001) Auditory triggered mental imagery of shape involves visual association areas in early blind humans. Neuro- image 14:129-139
  14. DeChaine DR (2002) Affect and embodied understanding in musical experience. Text Perform Q 22:79-98
  15. Deroy O, Fernandez-Prieto I, Navarra J, Spence C (in press) Unrav- elling the paradox of spatial pitch. In Hubbard TL (ed) Spatial biases in perception and cognition. Cambridge University Press, Cambridge (To appear)
  16. Dieter KC, Hu B, Knill DC, Blake R, Tadin D (2013) Kinesthesis can make an invisible hand visible. Psychol Sci 25:66-75
  17. Dimitriou M, Wolpert DM, Franklin DW (2013) The temporal evolu- tion of feedback gains rapidly update to task demands. J Neuro- sci 33:10898-10909
  18. Eitan Z, Timmers R (2010) Beethoven's last piano sonata and those who follow crocodiles: cross-domain mappings of auditory pitch in a musical context. Cognition 114:405-422
  19. Evans KK, Treisman A (2010) Natural cross-modal mappings between visual and auditory features. J Vis 10:1-12
  20. Farah MJ, Hammond KM, Levine DN, Calvanio R (1988) Visual and spatial mental imagery: dissociable systems of representation. Cogn Psychol 20:439-462
  21. Ferri F, Tajadura-Jiménez A, Väljamäe A, Vastano R, Costantini M (2015) Emotion-inducing approaching sounds shape the bound- aries of multisensory peripersonal space. Neuropsychologia 70:468-475
  22. Franklin DW, Wolpert DM (2011) Computational mechanisms of sen- sorimotor control. Neuron 72:425-442
  23. Frassinetti F, Pavani F, Làdavas E (2002) Acoustical vision of neglected stimuli: interaction among spatially converging audio- visual inputs in neglect patients. J Cogn Neurosci 14:62-69
  24. Freeman JB, Ambady N (2010) MouseTracker: software for studying real-time mental processing using a computer mouse-tracking method. Behav Res Methods 42:226-241
  25. Freeman JB, Dale R, Farmer TA (2011) Hand in motion reveals mind in motion. Frontiers Psychol 2:59
  26. Gazzola V, Keysers C (2009) The observation and execution of actions share motor and somatosensory voxels in all tested sub- jects: single-subject analyses of unsmoothed fMRI data. Cereb Cortex 19:1239-1255
  27. Harfield P, Halkon B, Mitchell S, Phillips I, May A (2014) A novel, real-time biomechanical feedback system for use in rowing. Proc Eng 72:126-131
  28. Harris LR, Carnevale MJ (2016) Which direction is up for a high pitch? Multisens Res 29:113-132
  29. Heffner RS, Heffner HE (1992a) Evolution of sound localization in mammals. In: Webster DB, Fay RR, Popper AN (eds) The evolu- tionary biology of hearing. Springer, New York, NY, pp 691-715
  30. Heffner RS, Heffner HE (1992b) Visual factors in sound localization in mammals. J Comp Neurol 317:219-232
  31. Henriques J (2003) Sonic dominance and the reggae sound system session. In: Bull M, Black L (eds) The auditory culture reader. Berg, Oxford, pp 451-480
  32. Holstege G, Huynh HK (2013) Emotions studied by imaging of the human brain: the somatic and emotional motor systems. In: Pfaff DW (ed) Neuroscience in the 21st century: from basic to clinical. Springer, New York, NY, pp 2045-2068
  33. Karwoski TF, Odbert HS, Osgood CE (1942) Studies in synesthetic thinking: II. The role of form in visual responses to music. J Gen Psychol 26:199-222
  34. Keysers C, Kohler E, Umilta MA, Nanetti L, Fogassi L, Gallese V (2003) Audiovisual mirror neurons and action recognition. Exp Brain Res 153:628-636
  35. King AJ, Schnupp JWH, Doubell TP (2001) The shape of ears to come: dynamic coding of auditory space. Trends Cogn Sci 5:261-270
  36. Kohler E, Keysers C, Umiltà MA, Fogassi L, Gallese V, Rizzolatti G (2002) Hearing sounds, understanding actions: action representa- tion in mirror neurons. Science 297:846-848
  37. Lambert S, Sampaio E, Mauss Y, Scheiber C (2004) Blindness and brain plasticity: Contribution of mental imagery? An fMRI study. Cogn Brain Res 20:1-11
  38. Maeda F, Kanai R, Shimojo S (2004) Changing pitch induced visual motion illusion. Curr Biol 14:R990-R991
  39. Maga JA (1974) Influence of color on taste thresholds. Chem Senses Flavor 1:115-119
  40. Makous JC, Friedman RM, Vierck CJ (1995) A critical band filter in touch. J Neurosci 15:2808-2818
  41. Marmolejo-Ramos F, Tian U (2010) A simple method for the detec- tion of outliers. Electron J Appl Stat Anal 2:67-76
  42. Meier BP, Robinson MD (2004) Why the sunny side is up: asso- ciations between affect and vertical position. Psychol Sci 15:243-247
  43. Meier BP, Robinson MD, Caven AJ (2008) Why a Big Mac is good Mac: associations between affect and size. Basic Appl Soc Psy- chol 30:46-55
  44. Melara RD, O'Brien TP (1987) Interaction between synesthetically corresponding dimensions. J Exp Psychol Gen 116:323-336
  45. Miall RC, Wolpert DM (1996) Forward models for physiological motor control. Neural Netw 9:1265-1279
  46. Montoro P, Contreras MJ, Elosúa MJ, Marmolejo-Ramos F (2015) Cross-modal metaphorical mapping of spoken emotion words onto vertical space. Frontiers Psychol 6:1205
  47. Mudd SA (1963) Spatial stereotypes of four dimensions of pure tone. J Exp Psychol 66:347-352
  48. Niedenthal PM, Augustinova M, Rychlowska M (2010) Body and mind: zajonc's (re)introduction of the motor system to emotion and cognition. Emot Rev 2:340-347
  49. Palmer SE, Schloss KB, Xu Z, Prado-León LR (2013) Music-color associations are mediated by emotion. Proc Natl Acad Sci USA 110:8836-8841
  50. Parise CV (2016) Crossmodal correspondences: standing issues and experimental guidelines. Multisens Res 29:7-28
  51. Parise CV, Knorre K, Ernst MO (2014) Natural auditory scene sta- tistics shapes human spatial hearing. Proc Natl Acad Sci USA 111:6104-6108
  52. Parise C, Spence C, Deroy O (2015) Understanding the correspond- ences: introduction to the special issue on crossmodal corre- spondences. Multisens Res 29:1-6
  53. Parker B (2009) Good vibrations: the physics of music. John Hopkins University Press, Baltimore, MD
  54. Penfield W, Rasmussen T (1950) The cerebral cortex of man. Mac- Millan, New York, NY
  55. Phillips-Silver J, Trainor LJ (2005) Feeling the beat: movement influ- ences infants' rhythm perception. Science 308:1430
  56. Phillips-Silver J, Trainor LJ (2007) Hearing what the body feels: audi- tory encoding of rhythmic movement. Cognition 105:533-546
  57. Pratt CC (1930) The spatial character of high and low tones. J Exp Psychol 13:278-285
  58. Press C, Gherri E, Heyes C, Eimer M (2009) Action preparation helps and hinders perception of action. J Cogn Neurosci 22:2198-2211
  59. Ricciardi E, Bonino D, Sani L, Vecchi T, Guazzelli M, Haxby JV, Fadiga L, Pietrini P (2009) Do we really need vision? how blind people "see" the actions of others. J Neurosci 29:9719-9724
  60. Rigas D, Alty J (2005) The rising pitch metaphor: an empirical study. Int J Hum Comput Stud 62:1-20
  61. Rusconi E, Kwan B, Giordano BL, Umiltà C, Butterworth B (2006) Spatial representation of pitch height: the SMARC effect. Cogni- tion 99:113-129
  62. Salgado-Montejo A, Alvarado JA, Velasco C, Salgado CJ, Hasse K, Spence C (2015) The sweetest thing: the influence of angularity, symmetry, and the number of elements on shape-taste matches. Frontiers Psychol 6:1382
  63. Sasaki K, Yamada Y, Miura K (2016) Emotion biases voluntary verti- cal action only with visible cues. Acta Psychol 163:97-106
  64. Schaffert N, Mattes K (2015a) Effects of acoustic feedback training in elite-standard Para-Rowing. J Sports Sci 33:411-418
  65. Schaffert N, Mattes K (2015b) Interactive sonification in row- ing: acoustic feedback for on-water training. IEEE Multimed 22:58-67
  66. Schaffert N, Mattes K, Effenberg AO (2011) An investigation of online acoustic information for elite rowers in on-water training conditions. J Hum Sport Exerc 6:392-405
  67. Scherer KR, Oshinsky JS (1977) Cue utilization in emotion attribu- tion from auditory stimuli. Motiv Emot 1:331-346
  68. Sherman A, Grabowecky M, Suzuki S (2013) Auditory rhythms are systemically associated with spatial-frequency and density infor- mation in visual scenes. Psychon Bull Rev 20:740-746
  69. Sigrist R, Rauter G, Marchal-Crespo L, Riener R, Wolf P (2014) Sonification and haptic feedback in addition to visual feed- back enhances complex motor task learning. Exp Brain Res 233:909-925
  70. Spence C (2011) Crossmodal correspondences: a tutorial review. Atten Percept Psychophys 73:971-995
  71. Spence C, Deroy O (2013a) Crossmodal mental imagery. In: Lacey S, Lawson R (eds) Multisens Imag. Springer, New York, NY, pp 157-183
  72. Spence C, Deroy O (2013b) Taste and shapes: a review of four hypotheses. Histor Sci 25:42-49
  73. Stahl B, Thoshkahna B (2016) Design and evaluation of the effective- ness of a sonification technique for real time heart-rate data. J Multimodal User Interfaces 10:207-219
  74. Stumpf K (1883) Tonpsychologie [Psychology of tones].
  75. S. Hirzel, Leipzig
  76. Tajadura-Jiménez A, Larsson P, Väljamäe A, Västfjäll D, Kleiner M (2010) When room size matters: acoustic influences on emo- tional responses to sounds. Emotion 10:416-442
  77. Tajadura-Jiménez A, Pantelidou G, Rebacz P, Västfjäll D, Tsakiris M (2011) I-space: the effects of emotional valence and source of music on interpersonal distance. PLoS ONE 6:e26083
  78. Thinus-Blanc C, Gaunet F (1997) Representation of space in blind persons: Vision as a spatial sense? Psychol Bull 121:20-42
  79. Tinga AM, Visser-Meily JMA, van der Smagt MJ, van der Stigchel S, van Ee R, Nijboer TCW (2015) Multisensory stimulation to improve low-and higher-level sensory deficits after stroke: a sys- tematic review. Neuropsychol Rev 26:73-91
  80. Treitler L (1982) The early history of music writing in the West. J Am Musicol Soc 35:237-279
  81. Velasco C, Woods AT, Deroy O, Spence C (2015) Hedonic mediation of the crossmodal correspondence between taste and shape. Food Qual Prefer 41:151-158
  82. Vuokowski J, Spence C, Thompson M, Clarke EF (2016) Interaction of sight and sound in the perception and experience of musical performance. Music Psychol 33:457-471
  83. Vuoskoski JK, Thompson MR, Clarke EF, Spence C (2014) Cross- modal interactions in the perception of expressivity in musical performance. Atten Percept Psychophys 76:591-604
  84. Watson L (1971) The omnivorous ape. Coward, McCann, & Geohhe- gan, New York, NY
  85. Wilcox RR (2005) Introduction to robust estimation and hypothesis testing. Elsevier, San Diego, CA
  86. Wilke C, Synofzik M, Lindner A (2012) The valence of action out- comes modulates the perception of one's actions. Conscious Cogn 21:18-29
  87. Wolpert DM, Flanagan JR (2001) Motor prediction. Curr Biol 11:R729-R732
  88. Yohai V (1987) High breakdown-point and high efficiency estimates for regression. Ann Stat 15:642-656
  89. Zajonc RB, Markus H (1984) Affect and cognition: The hard inter- face. In: Izard C, Kagan J, Zajonc RB (eds) Emotion, cogni- tion, and behavior. Cambridge University Press, Cambridge, pp 73-102
  90. Zatorre RJ, Chen JL, Penhune VB (2007) When the brain plays music: auditory-motor interactions in music perception and pro- duction. Nat Rev Neurosci 8:547-558

Related papers

Auditory and Visual Sensations
Peter Cariani

Noise Control Engineering Journal, 2012

View PDFchevron_right
A Study on Crossmodal Correspondence in Sensory Pathways Through Forced Choice Task and Frequency Based Correlation in Sound-Symbolism
Swati Banerjee

Mining Intelligence and Knowledge Exploration, 2017

The crossmodal correspondence in sensory pathways of human can get revealed by subjecting them to forced choice task as in sound-symbolism. Sound-symbolism is a term used for a hypothetical systematic relationship between word and meaning. A well known case in sound symbolism is the Kiki-Bouba phenomenon in which a subject labels a jagged figure as Kiki and rounded figure as Bouba when presented with both figures and words and asked to label the figure with the words. In the current experiment the words for cotton and sword were chosen from foreign languages and the subjects were asked to label the figure with that pair. Majority of subjects labeled the pointed figure with the word for sword for most of the languages. The word for sword had higher frequency components in most languages. The subjects may be associating words with higher frequency components to the jagged figure which implies possible crossmodal correspondence between visual and auditory pathways as was also indicated by neuropsychologists for natural language understanding.

View PDFchevron_right
Spatial Associations for Musical Stimuli: A Piano in the Head
Régine Kolinsky

Journal of Experimental Psychology-human Perception and Performance, 2007

This study was aimed at examining whether pitch height and pitch change are mentally represented along spatial axes. A series of experiments explored, for isolated tones and 2-note intervals, the occurrence of effects analogous to the spatial numerical association of response codes (SNARC) effect. Response device orientation (horizontal vs. vertical), task, and musical expertise of the participants were manipulated. The pitch of isolated tones triggered the automatic activation of a vertical axis independently of musical expertise, but the contour of melodic intervals did not. By contrast, automatic associations with the horizontal axis seemed linked to music training for pitch and, to a lower extent, for intervals. These results, discussed in the light of studies on number representation, provide a new example of the effects of musical expertise on music cognition.

View PDFchevron_right
Does Timbre Modulate Visual Perception? Exploring Crossmodal Interactions
Zachary Wallmark

Music Perception, 2021

Musical timbre is often described using terms from non-auditory senses, mainly vision and touch; but it is not clear whether crossmodality in timbre semantics reflects multisensory processing or simply linguistic convention. If multisensory processing is involved in timbre perception, the mechanism governing the interaction remains unknown. To investigate whether timbres commonly perceived as ‘‘bright-dark’’ facilitate or interfere with visual perception (darkness- brightness), we designed two speeded classification experiments. Participants were presented consecutive images of slightly varying (or the same) brightness along with task-irrelevant auditory primes (‘‘bright’’ or ‘‘dark’’ tones) and asked to quickly identify whether the second image was brighter/darker than the first. Incongruent prime-stimulus combinations produced significantly more response errors compared to congruent combinations but choice reaction time was unaffected. Furthermore, responses in a deceptive identical-image condition indicated subtle semantically congruent response bias. Additionally, in Experiment 2 (which also incorporated a spatial texture task), measures of reaction time (RT) and accuracy were used to construct speed-accuracy tradeoff functions (SATFs) in order to critically compare two hypothesized mechanisms for timbre-based crossmodal interactions, sensory response change vs. shift in response criterion. Results of the SATF analysis are largely consistent with the response criterion hypothesis, although without conclusively ruling out sensory change.

View PDFchevron_right
A Fruity Note: Crossmodal associations between odors and musical notes
Anne-Sylvie Crisinel

Chemical Senses, 2011

Odors are notoriously difficult to describe, but they seem prone to a variety of crossmodal associations. In the present study, we generalize the previously-shown association between odors (from perfumery) and pitch (Belkin et al. 1997) to odors related to food and drink (in this case those associated with wine). We also demonstrate that, to a lesser extent (25% of the odor tested), participants preferentially match specific odors to certain types of instruments. The ratings of the odors along a number of dimensions are used in principal components analysis (PCA) to explore the psychological dimensions underlying the odor-pitch associations. The results demonstrate that both pleasantness and complexity, but not intensity, appear to play a role when choosing a pitch to match an odor. Our results suggest that these features of odor stimuli constitute psychological dimensions that can be consistently matched to auditory features.

View PDFchevron_right

Related topics

  • Psychology
  • Cognitive Psychology
  • Perception
  • Auditory Perception
  • Embodied Cognition
  • Medicine
  • Functional Laterality
  • Acoustic Stimulation
  • Medical and Health Sciences
  • crossmodal correspondences
    • 580 California St., Suite 400
    San Francisco, CA, 94104
    © 2025 Academia. All rights reserved