Search

The awareness of the visitor’s role centrality in museums has led to the constant development of techniques and methodologies aimed at improving the visitor’s experience. In this context, Augello et al. suggest the four pillars to enhance the user experience: User Localization, User Understanding, Multimodal Interaction, and Gamification. As part of our research activities, we have focused on the latter two pillars. More specifically, User Localization allows museum curators and staff to understand the user movements in the museum and how much time she spends in front of the artworks. This information can help personalize the user experience to make it more and more satisfying. To collect this information, we believe that rather than using traditional techniques, such as beacons or IR, it is better to leverage deep learning techniques using cameras, thus supporting subsequent analyzes such as an automatic analysis of the user’s face. In particular, analyzing the user’s face allows us to grasp her reactions in front of the artworks and establish whether she may have liked that artwork or not, and if she may want to see similar artworks. The aim of this research is to understand how to use localization systems and automatic face analysis to build a user model that could allow museum staff to customize her experience. The first results obtained regarding an innovative localization system, based on deep learning, show the potential of this technique, which permits the extraction of information from faces through the same cameras. As regards this second part, techniques based on deep neural networks, such as Convolutional and Recurrent Neural Networks, are being tested to identify the best approach to extract implicit feedback from users’ faces without making them fill out long questionnaires that require active participation and steal time from the visit.

Although ubiquitous and fast access to the Internet allows us to admire objects and artworks exhibited worldwide from the comfort of our home, visiting a museum or an exhibition remains an essential experience today. Current technologies can help make that experience even more satisfying. For instance, they can assist the user during the visit, personalizing her experience by suggesting the artworks of her higher interest and providing her with related textual and multimedia content. To this aim, it is necessary to automatically acquire information relating to the active user. In this paper, we show how a deep neural network-based approach can allow us to obtain accurate information for understanding the behavior of the visitor alone or in a group. This information can also be used to identify users similar to the active one to suggest not only personalized itineraries but also possible visiting companions for promoting the museum as a vehicle for social and cultural inclusion.