The Role of Haptic Exploration Techniques in Visual Mental Imagery

Abstract

With the social permeation of digital tools and the subsequent increase in visual, non-tangible media interactions, the gap between the visual information and multisensory or multimodal experiences has grown historically wide. Previous research has demonstrated that the cognitive patterns involved in the formation of visual mental imagery, or photographic mental images, are inextricable from those of non-visual sensory modalities, citing the constructs of vividness, immersion, and sense of presence as evidence of this. While these constructs elucidate the neural processes by which we holistically “feel” the information encoded in visual stimuli, the exact visual elements which contribute to this holistic experience remain unclear. Furthermore, while the heterogeneity of individual capacities for visual mental imagery is widely recognized, the reasons as to why different visual elements are prepotent in different individuals have not yet been fully established. In order to bridge this critical knowledge gap, this study focuses on the bimodal construct of visuo-haptic feedback as it provides an observable measure for how visual attributes are perceived/imagined. Significant visual elements are analyzed through participants’ patterns in haptic exploration (Klatzky, 1991), the results of which are checked by performance in a mental manipulation task (sketching the stimulus object from memory at different angles). These results are then compared to participants’ extroversion levels, building on previous research regarding the inverse correlation between extroversion levels and the perceived degree of vividness in mental imagery (Sas, 2004), thereby linking the multimodal perception of distinct visual elements to cognitive styles. By employing this methodology, this study aims to contribute to a clearer understanding of the multimodal processes involved in the interpretation of visual media, correlate personality styles and the cognition of distinct visual elements, and uncover the cognitive/sensory processes which contribute to visual imagination.