The Role of Affective Consumer Responses in the Understanding of the Gestural Interaction Experience

Gesture interaction became one of the trending topics in human-computer interaction. This new modality is now starting to be used everywhere (Daugherty et al., 2015) and provide individuals a global richer experience than with classical user interfaces (Vanderdonckt & Vatavu, 2018). This is a significant asset when we know that a good customer experience (Lemon & Verhoef, 2016) makes a person five times more likely to recommend a company and more likely to purchase in the future (Yohn, 2019). However, there is currently a great lack of user-centric approaches. The performance of a system is usually measured with usability (Nielsen & Levy, 1994), which includes purely rational dimensions, while it is now accepted that consumers are not necessarily rational decision-makers (Holbrook & Hirschman, 1982). By not taking into account experiential and affective aspects of the interaction, we have an incomplete and biased vision of the performance of such systems, and particularly in the case of gestural interaction (Hassenzahl et al., 2010). Even if different forms of experience exist in the literature in marketing and human-computer interaction, the role of the affective aspects of the experience in the particular case of gestural interaction is still unknown. To address these gaps, this research aims to explore the experience underlying gestural interaction by conducting indepth interviews and observation phases with customers. The contributions of this research are threefold. First, we extend prior research on the gestural interaction experience by observing the key role of affective dimension in consumers’ responses. Then, we shed light to more specific elements such as intuitiveness, perceived control and immersion. Lastly, we show the crucial role of gestural interaction experience in particular touchpoints of the customer journey and consequently on the consumer experience, specifically in the case of impulse buying.