Edinburgh Napier University

  This paper presents the findings of two exploratory studies. In these studies participants performed a series of image-sound association and detection tasks. The aim of the studies was to investigate the perceived similarity and the effectiveness of two multidimensional mappings, each consisting of three audio-visual associations. The purpose of the mappings is to enable visual control of corpus-based concatenative synthesis. The stimuli in the first study was designed to test the perceived similarity of six audio-visual associations, between the two mappings using three corpora resulting in 18 audio-visual stimuli. The corpora differ in terms of two sound characteristics: harmonic content and continuity. Data analysis revealed no significant differences in the participants' responses between the three corpora, or between the two mappings. However significant differences were revealed between the individual audio-visual association pairs. The second study investigates the effects of the mapping and the corpus in the ability of the participants to detect which image out of three similar images was used to synthesise the audio stimuli. Data analysis revealed significant differences in the ability of the participants to detect the correct image depending on which corpus was used. Less significant was the effect of the mapping in the success rate of the participants' responses.