Curiosity-driven exploration: foundations in neuroscience and computational modeling.

Exploratory behaviors are central to the study of curiosity, since they allow us to infer an otherwise hidden state of being curious from quantitative, experimental observations. Reactive orienting behaviors reflect a fundamental form of curiosity and are driven by the immediate novelty and/or surprise of stimuli. Operant tasks probe curiosity by offering experimental subjects explicit action choices to collect information, in a manner that is either neutral or costly/useful with regards to overall gains/losses. Recently developed computational models of curiosity, based on intrinsically motivated reinforcement learning, describe different expressions of curiosity in a unified theoretical framework. Curiosity refers to the intrinsic desire of humans and animals to explore the unknown, even when there is no apparent reason to do so. Thus far, no single, widely accepted definition or framework for curiosity has emerged, but there is growing consensus that curious behavior is not goal-directed but related to seeking or reacting to information. In this review, we take a phenomenological approach and group behavioral and neurophysiological studies which meet these criteria into three categories according to the type of information seeking observed. We then review recent computational models of curiosity from the field of machine learning and discuss how they enable integrating different types of information seeking into one theoretical framework. Combinations of behavioral and neurophysiological studies along with computational modeling will be instrumental in demystifying the notion of curiosity. [ABSTRACT FROM AUTHOR]