How Affective Properties of Voice Influence Memory and Social Perception

Abstract

Human voice carries precious information about a person. From a brief vocalization to a spoken sentence, listeners rapidly form perceptual judgments of transient affective states such as happiness, as well as perceptual judgments of the more stable social traits such as trustworthiness. In social interactions, sometimes it is not just what we say – but how we say it – that matters. This dissertation sought to better understand how affective properties in voice influence memory and how they subserve social perception. To these ends, I investigated the effect of affective prosody on memory for speech by manipulating both prosody valence and semantic valence, I explored the fundamental dimensions of social perception from voice, and I discussed the relationship of those social dimensions to affective dimensions of voice. In the first chapter, I examined how prosody valence influences memory for speech that varied in semantic valence. Participants listened to narratives spoken in neutral, positive, and negative prosody and recalled as much as they could of the narrative content. Importantly, the arousal level of the affective prosody was controlled across the different prosody valence conditions. Results showed that prosody valence influenced memory for speech content and the effect depended on the relationship between prosody valence and semantic valence. Specifically, congruence between prosody and semantic valence influenced memory. When people were listening to neutral content, affective prosody (either positive or negative) impaired memory. When listening to positive or negative content, incongruent prosody led to better recall. The present research shows that it is not just what you say, but also how you say it that will influence what people remember of your message. In the second chapter, I explored the fundamental dimensions of social perception from voices compared to faces, using a data-driven approach. Participants were encouraged to freely write down anything that came to mind about the voice they heard or the face they saw. Descriptors were classified into categories and the most frequently occurred social trait categories were selected. A separate group of participants rated the voices and faces on the selected social traits. Principal component analyses revealed that female voices were evaluated mostly on three dimensions: attractiveness, trustworthiness, and dominance; whereas male voices were evaluated mostly on two dimensions: social engagement and trustworthiness. For social evaluation of faces, a similar two-dimensional structure of social engagement and trustworthiness was found for both genders. The gender difference in social perception of voice is discussed with respect to gender stereotypes and the role voice pitch played in perceived attractiveness and dominance. This study indicates that both modality (voice vs. face) and gender impact the fundamental dimensions of social perception. Overall, the findings of this dissertation indicate that the affective quality in our voice not only influence how our speech will be remembered but also relate to how we are being socially perceived by others. It would be wise to pay more attention to our tone of voice if we want to make our speech memorable and leave a good impression.