Search

IntroductionUnderstanding how food neophobia affects food experience may help to shift toward sustainable diets. Previous research suggests that individuals with higher food neophobia are more aroused and attentive when observing food-related stimuli. The present study examined whether electrodermal activity (EDA), as index of arousal, relates to food neophobia outside the lab when exposed to a single piece of food.MethodsThe EDA of 153 participants was analyzed as part of a larger experiment conducted at a festival. Participants completed the 10-item Food Neophobia Scale. Subsequently, they saw three lids covering three foods: a hotdog labeled as “meat”, a hotdog labeled as “100% plant-based”, and tofu labeled as “100% plant-based”. Participants lifted the lids consecutively and the area-under-the-curve (AUC) of the skin conductance response (SCR) was captured between 20 s before and 20 s after each food reveal.ResultsWe found a significant positive correlation between food neophobia and AUC of SCR during presentation of the first and second hotdog and a trend for tofu. These correlations remained significant even when only including the SCR data prior to the food reveal (i.e., an anticipatory response).DiscussionThe association between food neophobia and EDA indicates that food neophobic individuals are more aroused upon the presentation of food. We show for the first time that the anticipation of being presented with food already increased arousal for food neophobic individuals. These findings also indicate that EDA can be meaningfully determined using wearables outside the lab, in a relatively uncontrolled setting for single-trial analysis.

Telemeetings such as audiovisual conferences or virtual meetings play an increasingly important role in our professional and private lives. For that reason, system developers and service providers will strive for an optimal experience for the user, while at the same time optimizing technical and financial resources. This leads to the discipline of Quality of Experience (QoE), an active field originating from the telecommunication and multimedia engineering domains, that strives for understanding, measuring, and designing the quality experience with multimedia technology. This paper provides the reader with an entry point to the large and still growing field of QoE of telemeetings, by taking a holistic perspective, considering both technical and non-technical aspects, and by focusing on current and near-future services. Addressing both researchers and practitioners, the paper first provides a comprehensive survey of factors and processes that contribute to the QoE of telemeetings, followed by an overview of relevant state-of-the-art methods for QoE assessment. To embed this knowledge into recent technology developments, the paper continues with an overview of current trends, focusing on the field of eXtended Reality (XR) applications for communication purposes. Given the complexity of telemeeting QoE and the current trends, new challenges for a QoE assessment of telemeetings are identified. To overcome these challenges, the paper presents a novel Profile Template for characterizing telemeetings from the holistic perspective endorsed in this paper.

We examined the effects of an informative pitch and multisensory contexts as potential factors influencing individuals’ experience of tofu with soy sauce and the amount consumed outside the lab. Two hundred and sixteen participants watched one of two pitches (promoting either vegetarian diets or exercise) and were guided into one of three multisensory contexts (‘sustainable’, ‘meat’, or ‘neutral’ theme). Participants rated the aroma and appearance of soy sauce and the taste of tofu dipped in it using the intuitive ‘one touch’ EmojiGrid valence and arousal measuring tool. Our results showed that the ‘meat’ context increased arousal ratings for soy sauce and the tendency to consume more tofu relative to the other contexts. Pitch did not influence affective ratings or amounts consumed. We conclude that the multisensory context has the potential to positively affect peoples’ choices and perceptions of plant-based and sustainable food and promote its consumption.

To monitor adherence to diets and to design and evaluate nutritional interventions, it is essential to obtain objective knowledge about eating behavior. In most research, measures of eating behavior are based on self-reporting, such as 24-h recalls, food records (food diaries) and food frequency questionnaires. Self-reporting is prone to inaccuracies due to inaccurate and subjective recall and other biases. Recording behavior using nonobtrusive technology in daily life would overcome this. Here, we provide an up-to-date systematic overview encompassing all (close-to) publicly or commercially available technologies to automatically record eating behavior in real-life settings. A total of 1328 studies were screened and, after applying defined inclusion and exclusion criteria, 122 studies were included for in-depth evaluation. Technologies in these studies were categorized by what type of eating behavior they measure and which type of sensor technology they use. In general, we found that relatively simple sensors are often used. Depending on the purpose, these are mainly motion sensors, microphones, weight sensors and photo cameras. While several of these technologies are commercially available, there is still a lack of publicly available algorithms that are needed to process and interpret the resulting data. We argue that future work should focus on developing robust algorithms and validating these technologies in real-life settings. Combining technologies (e.g., prompting individuals for self-reports at sensed, opportune moments) is a promising route toward ecologically valid studies of eating behavior.

Abstract How we perceive the world is not solely determined by our experiences at a given moment in time, but also by what we have experienced in our immediate past. Here, we investigated whether such sequential effects influence the affective appraisal of food images. Participants from 16 different countries (N = 1278) watched a randomly presented sequence of 60 different food images and reported their affective appraisal of each image in terms of valence and arousal. For both measures, we conducted an inter-trial analysis, based on whether the rating on the preceding trial(s) was low or high. The analyses showed that valence and arousal ratings for a given food image are both assimilated towards the ratings on the previous trial (i.e., a positive serial dependence). For a given trial, the arousal rating depends on the arousal ratings up to three trials back. For valence, we observed a positive dependence for the immediately preceding trial only, while a negative (repulsive) dependence was present up to four trials back. These inter-trial effects were larger for males than for females, but independent of the participants’ BMI, age, and cultural background. The results of this exploratory study may be relevant for the design of websites of food delivery services and restaurant menus.

In recent years, there has been a significant increase in research on mediated communication via social touch. Previous studies indicated that mediated social touch (MST) can induce similar positive outcomes to interpersonal touch. However, studies investigating the user experience of MST technology predominantly involve brief experiments that are performed in well-controlled laboratory conditions. Hence, it is still unknown how MST affects the relationship and communication between physically separated partners in a romantic relationship, in a naturalistic setting and over a longer period of time. In a longitudinal explorative field study, the effects of MST on social connectedness and longing for touch among geographically separated romantic couples were investigated in a naturalistic setting. For 2 weeks, 17 couples used haptic bracelets, that were connected via the internet, to exchange mediated squeeze-like touch signals. Before and after this period, they reported their feelings of social connectedness and longing for touch through questionnaires. The results show that the use of haptic bracelets (1) enhanced social connectedness among geographically separated couples but (2) did not affect their longing for touch. Interviews conducted at the end of the study were analyzed following the thematic analysis method to generate prominent themes and patterns in using MST technology among participant couples. Two main themes were generated that captured (a) the way the bracelets fostered a positive one-to-one connection between partners and (b) the way in which participants worked around their frustrations with the bracelets. Detailed findings and limitations of this longitudinal field study are further discussed, and suggestions are made for future research.

Cognitive biases can adversely affect human judgment and decision making and should therefore preferably be mitigated, so that we can achieve our goals as effectively as possible. Hence, numerous bias mitigation interventions have been developed and evaluated. However, to be effective in practical situations beyond laboratory conditions, the bias mitigation effects of these interventions should be retained over time and should transfer across contexts. This systematic review provides an overview of the literature on retention and transfer of bias mitigation interventions. A systematic search yielded 52 studies that were eligible for screening. At the end of the selection process, only 12 peer-reviewed studies remained that adequately studied retention over a period of at least 14 days (all 12 studies) or transfer to different tasks and contexts (one study). Eleven of the relevant studies investigated the effects of bias mitigation training using game- or video-based interventions. These 11 studies showed considerable overlap regarding the biases studied, kinds of interventions, and decision-making domains. Most of them indicated that gaming interventions were effective after the retention interval and that games were more effective than video interventions. The study that investigated transfer of bias mitigation training (next to retention) found indications of transfer across contexts. To be effective in practical circumstances, achieved effects of cognitive training should lead to enduring changes in the decision maker's behavior and should generalize toward other task domains or training contexts. Given the small number of overlapping studies, our main conclusion is that there is currently insufficient evidence that bias mitigation interventions will substantially help people to make better decisions in real life conditions. This is in line with recent theoretical insights about the “hard-wired” neural and evolutionary origin of cognitive biases.

Background: In this study we measured the affective appraisal of sounds and video clips using a newly developed graphical self-report tool: the EmojiGrid. The EmojiGrid is a square grid, labeled with emoji that express different degrees of valence and arousal. Users rate the valence and arousal of a given stimulus by simply clicking on the grid. Methods: In Experiment I, observers (N=150, 74 males, mean age=25.2±3.5) used the EmojiGrid to rate their affective appraisal of 77 validated sound clips from nine different semantic categories, covering a large area of the affective space. In Experiment II, observers (N=60, 32 males, mean age=24.5±3.3) used the EmojiGrid to rate their affective appraisal of 50 validated film fragments varying in positive and negative affect (20 positive, 20 negative, 10 neutral). Results: The results of this study show that for both sound and video, the agreement between the mean ratings obtained with the EmojiGrid and those obtained with an alternative and validated affective rating tool in previous studies in the literature, is excellent for valence and good for arousal. Our results also show the typical universal U-shaped relation between mean valence and arousal that is commonly observed for affective sensory stimuli, both for sound and video. Conclusions: We conclude that the EmojiGrid can be used as an affective self-report tool for the assessment of sound and video-evoked emotions.

Emotional state during food consumption is expected to affect food pleasantness. We hypothesize that a negative emotional state reduces food pleasantness and more so for novel foods than for familiar foods because novel foods have not yet been associated with previous emotions. Furthermore, we expect this effect to be stronger when judging the food again from memory without tasting. We induced a positive emotional state in 34 participants by telling them that they earned a monetary bonus and induced a negative emotional state in 35 other participants by subjecting them to a social stress test. After this emotion induction, both groups tasted and rated a (for them) novel soup (sumashi soup) and a familiar soup (vegetable soup). Several explicit and implicit measures of food pleasantness (rated valence, EsSense25, willingness-to-take-home and sip size) indicated that while the negative emotion group did not experience the soups as less pleasant than the positive emotion group, there was an interaction between food familiarity and emotional group. The positive emotion group experienced novel and familiar soups as equally pleasant, whereas the negative emotion group experienced the novel soup as relatively unpleasant and the familiar soup as pleasant. The latter result is consistent with a comforting effect of a familiar taste in a stressful situation. This effect remained in the ratings given 1 week later based on memory and even after retasting. Our results show that emotional state affects food pleasantness differently for novel and familiar foods and that such an effect can be robust.

Telerobotics aims to transfer human manipulation skills and dexterity over an arbitrary distance and at an arbitrary scale to a remote workplace. A telerobotic system that is transparent enables a natural and intuitive interaction. We postulate that embodiment (with three sub-components: sense of ownership, agency, and self-location) of the robotic system leads to optimal perceptual transparency and increases task performance. However, this has not yet been investigated directly. We reason along four premises and present findings from the literature that substantiate each of them: (1) the brain can embody non-bodily objects (e.g., robotic hands), (2) embodiment can be elicited with mediated sensorimotor interaction, (3) embodiment is robust against inconsistencies between the robotic system and the operator's body, and (4) embodiment positively correlates to dexterous task performance. We use the predictive encoding theory as a framework to interpret and discuss the results reported in the literature. Numerous previous studies have shown that it is possible to induce embodiment over a wide range of virtual and real extracorporeal objects (including artificial limbs, avatars, and android robots) through mediated sensorimotor interaction. Also, embodiment can occur for non-human morphologies including for elongated arms and a tail. In accordance with the predictive encoding theory, none of the sensory modalities is critical in establishing ownership, and discrepancies in multisensory signals do not necessarily lead to loss of embodiment. However, large discrepancies in terms of multisensory synchrony or visual likeness can prohibit embodiment from occurring. The literature provides less extensive support for the link between embodiment and (dexterous) task performance. However, data gathered with prosthetic hands do indicate a positive correlation. We conclude that all four premises are supported by direct or indirect evidence in the literature, suggesting that embodiment of a remote manipulator may improve dexterous performance in telerobotics. This warrants further implementation testing of embodiment in telerobotics. We formulate a first set of guidelines to apply embodiment in telerobotics and identify some important research topics.

Reflecting the two main prevailing and opposing views on the nature of emotions, emotional responses to food and beverages are typically measured using either (a) a categorical (lexicon-based) approach where users select or rate the terms that best express their food-related feelings or (b) a dimensional approach where they rate perceived food items along the dimensions of valence and arousal. Relating these two approaches is problematic since a response in terms of valence and arousal is not easily expressed in terms of emotions (like happy or disgusted). In this study, we linked the dimensional approach to a categorical approach by establishing mapping between a set of 25 emotion terms (EsSense25) and the valence–arousal space (via the EmojiGrid graphical response tool), using a set of 20 food images. In two ‘matching’ tasks, the participants first imagined how the food shown in a given image would make them feel and then reported either the emotional terms or the combination of valence and arousal that best described their feelings. In two labeling tasks, the participants first imagined experiencing a given emotion term and then they selected either the foods (images) that appeared capable to elicit that feeling or reported the combination of valence and arousal that best reflected that feeling. By combining (1) the mapping between the emotion terms and the food images with (2) the mapping of the food images to the valence–arousal space, we established (3) an indirect (via the images) mapping of the emotion terms to the valence–arousal space. The results show that the mapping between terms and images was reliable and that the linkages have straightforward and meaningful interpretations. The valence and arousal values that were assigned to the emotion terms through indirect mapping to the valence–arousal space were typically less extreme than those that were assigned through direct mapping.

The affective appraisal of odors is known to depend on their intensity (I), familiarity (F), detection threshold (T), and on the baseline affective state of the observer. However, the exact nature of these relations is still largely unknown. We therefore performed an observer experiment in which participants (N = 52) smelled 40 different odors (varying widely in hedonic valence) and reported the intensity, familiarity and their affective appraisal (valence and arousal: V and A) for each odor. Also, we measured the baseline affective state (valence and arousal: BV and BA) and odor detection threshold of the participants. Analyzing the results for pleasant and unpleasant odors separately, we obtained two models through network analysis. Several relations that have previously been reported in the literature also emerge in both models (the relations between F and I, F and V, I and A; I and V, BV and T). However, there are also relations that do not emerge (between BA and V, BV and I, and T and I) or that appear with a different polarity (the relation between F and A for pleasant odors). Intensity (I) has the largest impact on the affective appraisal of unpleasant odors, while F significantly contributes to the appraisal of pleasant odors. T is only affected by BV and has no effect on other variables. This study is a first step towards an integral study of the affective appraisal of odors through network analysis. Future studies should also include other factors that are known to influence odor appraisal, such as age, gender, personality, and culture.

In this study we evaluate the convergent validity of a new graphical self-report tool (the EmojiGrid) for the affective appraisal of perceived touch events. The EmojiGrid is a square grid labeled with facial icons (emoji) showing different levels of valence and arousal. The EmojiGrid is language independent and efficient (a single click suffices to report both valence and arousal), making it a practical instrument for studies on affective appraisal. We previously showed that participants can intuitively and reliably report their affective appraisal (valence and arousal) of visual, auditory and olfactory stimuli using the EmojiGrid, even without additional (verbal) instructions. However, because touch events can be bidirectional and dynamic, these previous results cannot be generalized to the touch domain. In this study, participants reported their affective appraisal of video clips showing different interpersonal (social) and object-based touch events, using either the validated 9-point SAM (Self-Assessment Mannikin) scale or the EmojiGrid. The valence ratings obtained with the EmojiGrid and the SAM are in excellent agreement. The arousal ratings show good agreement for object-based touch and moderate agreement for social touch. For social touch and at more extreme levels of valence, the EmojiGrid appears more sensitive to arousal than the SAM. We conclude that the EmojiGrid can also serve as a valid and efficient graphical self-report instrument to measure human affective response to a wide range of tactile signals.

Despite of the ongoing interest in the fusion of multi-band images for surveillance applications and a steady stream of publications in this area, there is only a very small number of static registered multi-band test images (and a total lack of dynamic image sequences) publicly available for the development and evaluation of image fusion algorithms. To fill this gap, the TNO Multiband Image Collection provides intensified visual (390â700 nm), near-infrared (700â1000 nm), and longwave infrared (8â12 µm) nighttime imagery of different military and surveillance scenarios, showing different objects and targets (e.g., people, vehicles) in a range of different (e.g., rural, urban) backgrounds. The dataset will be useful for the development of static and dynamic image fusion algorithms, color fusion algorithms, multispectral target detection and recognition algorithms, and dim target detection algorithms. Keywords: Image fusion, Color fusion, False color, Color mapping, Realtime, Fusion, Night vision

We present CROCUFID: a CROss-CUltural Food Image Database that currently contains 840 images, including 479 food images with detailed metadata and 165 images of non-food items. The database includes images of sweet, savory, natural, and processed food from Western and Asian cuisines. To create sufficient variability in valence and arousal we included images of food with different degrees of appetitiveness (fresh, unfamiliar, molded or rotten, spoiled, and partly consumed). We used a standardized photographing protocol, resulting in high resolution images depicting all food items on a standard background (a white plate), seen from a fixed viewing (45°) angle. CROCUFID is freely available under the CC-By Attribution 4.0 International license and hosted on the OSF repository. The advantages of the CROCUFID database over other databases are its (1) free availability, (2) full coverage of the valence – arousal space, (3) use of standardized recording methods, (4) inclusion of multiple cuisines and unfamiliar foods, (5) availability of normative and demographic data, (6) high image quality and (7) capability to support future (e.g., virtual and augmented reality) applications. Individuals from the United Kingdom (N = 266), North-America (N = 275), and Japan (N = 264) provided normative ratings of valence, arousal, perceived healthiness, and desire-to-eat using visual analog scales (VAS). In addition, for each image we computed 17 characteristics that are known to influence affective observer responses (e.g., texture, regularity, complexity, and colorfulness). Significant differences between groups and significant correlations between image characteristics and normative ratings were in accordance with previous research, indicating the validity of CROCUFID. We expect that CROCUFID will facilitate comparability across studies and advance experimental research on the determinants of food-elicited emotions. We plan to extend CROCUFID in the future with images of food from a wide range of different cuisines and with non-food images (for applications in for instance neuro-physiological studies). We invite researchers from all parts of the world to contribute to this effort by creating similar image sets that can be linked to this collection, so that CROCUFID will grow into a truly multicultural food database.

Research on food experience is typically challenged by the way questions are worded. We therefore developed the EmojiGrid: a graphical (language-independent) intuitive self-report tool to measure food-related valence and arousal. In a first experiment participants rated the valence and the arousing quality of 60 food images, using either the EmojiGrid or two independent visual analog scales (VAS). The valence ratings obtained with both tools strongly agree. However, the arousal ratings only agree for pleasant food items, but not for unpleasant ones. Furthermore, the results obtained with the EmojiGrid show the typical universal U-shaped relation between the mean valence and arousal that is commonly observed for a wide range of (visual, auditory, tactile, olfactory) affective stimuli, while the VAS tool yields a positive linear association between valence and arousal. We hypothesized that this disagreement reflects a lack of proper understanding of the arousal concept in the VAS condition. In a second experiment we attempted to clarify the arousal concept by asking participants to rate the valence and intensity of the taste associated with the perceived food items. After this adjustment the VAS and EmojiGrid yielded similar valence and arousal ratings (both showing the universal U-shaped relation between the valence and arousal). A comparison with the results from the first experiment showed that VAS arousal ratings strongly depended on the actual wording used, while EmojiGrid ratings were not affected by the framing of the associated question. This suggests that the EmojiGrid is largely self-explaining and intuitive. To test this hypothesis, we performed a third experiment in which participants rated food images using the EmojiGrid without an associated question, and we compared the results to those of the first two experiments. The EmojiGrid ratings obtained in all three experiments closely agree. We conclude that the EmojiGrid appears to be a valid and intuitive affective self-report tool that does not rely on written instructions and that can efficiently be used to measure food-related emotions.

Human decision-making shows systematic simplifications and deviations from the tenets of rationality (‘heuristics’) that may lead to suboptimal decisional outcomes (‘cognitive biases’). There are currently three prevailing theoretical perspectives on the origin of heuristics and cognitive biases: a cognitive-psychological, an ecological and an evolutionary perspective. However, these perspectives are mainly descriptive and none of them provides an overall explanatory framework for the underlying mechanisms of cognitive biases. To enhance our understanding of cognitive heuristics and biases we propose a neural network framework for cognitive biases, which explains why our brain systematically tends to default to heuristic (‘Type 1’) decision making. We argue that many cognitive biases arise from intrinsic brain mechanisms that are fundamental for the working of biological neural networks. To substantiate our viewpoint, we discern and explain four basic neural network principles: (1) Association, (2) Compatibility, (3) Retainment, and (4) Focus. These principles are inherent to (all) neural networks which were originally optimized to perform concrete biological, perceptual, and motor functions. They form the basis for our inclinations to associate and combine (unrelated) information, to prioritize information that is compatible with our present state (such as knowledge, opinions, and expectations), to retain given information that sometimes could better be ignored, and to focus on dominant information while ignoring relevant information that is not directly activated. The supposed mechanisms are complementary and not mutually exclusive. For different cognitive biases they may all contribute in varying degrees to distortion of information. The present viewpoint not only complements the earlier three viewpoints, but also provides a unifying and binding framework for many cognitive bias phenomena.

Besides sensory characteristics of food, food-evoked emotion is a crucial factor in predicting consumer's food preference and therefore in developing new products. Many measures have been developed to assess food-evoked emotions. The aim of this literature review is (i) to give an exhaustive overview of measures used in current research and (ii) to categorize these methods along measurement level (physiological, behavioral, and cognitive) and emotional processing level (unconscious sensory, perceptual/early cognitive, and conscious/decision making) level. This 3 × 3 categorization may help researchers to compile a set of complementary measures (“toolbox”) for their studies. We included 101 peer-reviewed articles that evaluate consumer's emotions and were published between 1997 and 2016, providing us with 59 different measures. More than 60% of these measures are based on self-reported, subjective ratings and questionnaires (cognitive measurement level) and assess the conscious/decision-making level of emotional processing. This multitude of measures and their overrepresentation in a single category hinders the comparison of results across studies and building a complete multi-faceted picture of food-evoked emotions. We recommend (1) to use widely applied, validated measures only, (2) to refrain from using (highly correlated) measures from the same category but use measures from different categories instead, preferably covering all three emotional processing levels, and (3) to acquire and share simultaneously collected physiological, behavioral, and cognitive datasets to improve the predictive power of food choice and other models.

We tested the hypothesis that, either through training or selection, military personnel is more resilient to stress than civilians, as indicated by several subjective and physiological measures. In addition, we examined the effect of stress on the perceived trustworthiness of faces in these two groups. Stress was induced in 45 civilian participants and 45 army participants through the Sing-a-Song Stress Test (SSST). In this paradigm seven neutral sentences are presented, each followed by a 60-s interval. An eighth sentence asks participants to sing a song aloud after the next (eighth or stress) interval. Participants rated the trustworthiness of five neutral faces, both before and after the SSST. Pupil size, heart rate, and skin conductance were adopted as physiological stress correlates. Stress response was calculated as the difference between the mean values over the last neutral interval and the stress interval. Subjective stress ratings were obtained before and after the SSST. The baseline levels of all physiological and subjective measures were the same in the army and civilian groups, while all measures showed a significant increase following the stressor. However, compared with the civilian group, army participants reported significantly less stress and showed significantly attenuated heart rate and skin conductance responses to the SSST. These results indicate higher stress resilience in the army compared with the civilian group. In addition, we found that perceived facial trustworthiness decreased after presentation of the stressor, suggesting that the effect of a stressor can influence in principle unrelated social judgments based on facial information.

Probing food experience or liking through verbal ratings has its shortcomings. We compare explicit ratings to a range of (neuro)physiological and behavioral measures with respect to their performance in distinguishing drinks associated with different emotional experience. Seventy participants tasted and rated the valence and arousal of eight regular drinks and a “ground truth” high-arousal, low-valence vinegar solution. The discriminative power for distinguishing between the vinegar solution and the regular drinks was highest for sip size, followed by valence ratings, arousal ratings, heart rate, skin conductance level, facial expression of “disgust,” pupil diameter, and Electroencephalogram (EEG) frontal alpha asymmetry. Within the regular drinks, a positive correlation was found between rated arousal and heart rate, and a negative correlation between rated arousal and Heart Rate Variability (HRV). Most physiological measures showed consistent temporal patterns over time following the announcement of the drink and taking a sip. This was consistent over all nine drinks, but the peaks were substantially higher for the vinegar solution than for the regular drinks, likely caused by emotion. Our results indicate that implicit variables have the potential to differentiate between drinks associated with different emotional experiences. In addition, this study gives us insight into the physiological temporal response patterns associated with taking a sip.

We propose a multi-scale image fusion scheme based on guided filtering. Guided filtering can effectively reduce noise while preserving detail boundaries. When applied in an iterative mode, guided filtering selectively eliminates small scale details while restoring larger scale edges. The proposed multi-scale image fusion scheme achieves spatial consistency by using guided filtering both at the decomposition and at the recombination stage of the multi-scale fusion process. First, size-selective iterative guided filtering is applied to decompose the source images into approximation and residual layers at multiple spatial scales. Then, frequency-tuned filtering is used to compute saliency maps at successive spatial scales. Next, at each spatial scale binary weighting maps are obtained as the pixelwise maximum of corresponding source saliency maps. Guided filtering of the binary weighting maps with their corresponding source images as guidance images serves to reduce noise and to restore spatial consistency. The final fused image is obtained as the weighted recombination of the individual residual layers and the mean of the approximation layers at the coarsest spatial scale. Application to multiband visual (intensified) and thermal infrared imagery demonstrates that the proposed method obtains state-of-the-art performance for the fusion of multispectral nightvision images. The method has a simple implementation and is computationally efficient.

Edge preserving filters aim to simplify the representation of images (e.g., by reducing noise or eliminating irrelevant detail) while preserving their most significant edges. These filters are typically nonlinear and locally smooth the image structure while minimizing both blurring and over-sharpening of visually important edges. Here we present the Alternating Guided Filter (AGF) that achieves edge preserving smoothing by combining two recently introduced filters: the Rolling Guided Filter (RGF) and the Smooth and iteratively Restore Filter (SiR). We show that the integration of RGF and SiR in an alternating iterative framework results in a new smoothing operator that preserves significant image edges while effectively eliminating small scale details. The AGF combines the large scale edge and local intensity preserving properties of the RGF with the edge restoring properties of the SiR while eliminating the drawbacks of both previous methods (i.e., edge curvature smoothing by RGF and local intensity reduction and restoration of small scale details near large scale edges by SiR). The AGF is simple to implement and efficient, and produces high-quality results. We demonstrate the effectiveness of AGF on a variety of images, and provide a public code to facilitate future studies.

This study investigated whether personal relevance influences the affective appraisal of a desktop virtual environment (VE) in simulated darkness. In the real world, darkness often evokes thoughts of vulnerability, threat, and danger, and may automatically precipitate emotional responses consonant with those thoughts (fear of darkness). This influences the affective appraisal of a given environment after dark and the way humans behave in that environment in conditions of low lighting. Desktop VEs are increasingly deployed to study the effects of environmental qualities and (architectural or lighting) interventions on human behaviour and feelings of safety. Their (ecological) validity for these purposes depends critically on their ability to correctly address the user’s cognitive and affective experience. Previous studies with desktop (i.e., non-immersive) VEs found that simulated darkness only slightly affects the user’s behavioral and emotional responses to the represented environment, in contrast to the responses observed for immersive VEs. We hypothesize that the desktop VE scenarios used in previous studies less effectively induced emotional and behavioral responses because they lacked personal relevance. In addition, factors like signs of social presence and relatively high levels of ambient lighting may also have limited these responses. In this study, young female volunteers explored either a daytime or a night-time (low ambient light level) version of a desktop VE representing a deserted (no social presence) prototypical Dutch polder landscape. To enhance the personal relevance of the simulation, a fraction of the participants were led to believe that the virtual exploration tour would prepare them for a follow-up tour through the real world counterpart of the VE. The affective appraisal of the VE and the emotional response of the participants were measured through self-report. The results show that the VE was appraised as slightly less pleasant and more arousing in simulated darkness (compared to a daylight) condition, as expected. However, the fictitious follow-up assignment had no emotional effects and did not influence the affective appraisal of the VE. Further research is required to establish the qualities that may enhance the validity of desktop VEs for both etiological (e.g., the effects of signs of darkness on navigation behaviour and fear of crime) and intervention (e.g., effects of street lighting on feelings of safety) research.

How we perceive our environment affects the way we feel and behave. The impressions of our ambient environment are influenced by its entire spectrum of physical characteristics (e.g., luminosity, sound, scents, temperature) in a dynamic and interactive way. The ability to manipulate the sensory aspects of an environment such that people feel comfortable or exhibit a desired behavior is gaining interest and social relevance. Although much is known about the sensory effects of individual environmental characteristics, their combined effects are not a priori evident due to a wide range of non-linear interactions in the processing of sensory cues. As a result, it is currently not known how different environmental characteristics should be combined to effectively induce desired emotional and behavioral effects. To gain more insight into this matter, we performed a literature review on the emotional effects of multisensory stimulation. Although we found some interesting mechanisms, the outcome also reveals that empirical evidence is still scarce and haphazard. To stimulate further discussion and research, we propose a conceptual framework that describes how environmental interventions are likely to affect human emotional responses. This framework leads to some critical research questions that suggest opportunities for further investigation.

The fusion and enhancement of multiband nighttime imagery for surveillance and navigation has been the subject of extensive research for over two decades. Despite the ongoing efforts in this area there is still only a small number of static multiband test images available for the development and evaluation of new image fusion and enhancement methods. Moreover, dynamic multiband imagery is also currently lacking. To fill this gap we present the TRICLOBS dynamic multi-band image data set containing sixteen registered visual (0.4-0.7μm), near-infrared (NIR, 0.7-1.0μm) and long-wave infrared (LWIR, 8-14μm) motion sequences. They represent different military and civilian surveillance scenarios registered in three different scenes. Scenes include (military and civilian) people that are stationary, walking or running, or carrying various objects. Vehicles, foliage, and buildings or other man-made structures are also included in the scenes. This data set is primarily intended for the development and evaluation of image fusion, enhancement and color mapping algorithms for short-range surveillance applications. The imagery was collected during several field trials with our newly developed TRICLOBS (TRI-band Color Low-light OBServation) all-day all-weather surveillance system. This system registers a scene in the Visual, NIR and LWIR part of the electromagnetic spectrum using three optically aligned sensors (two digital image intensifiers and an uncooled long-wave infrared microbolometer). The three sensor signals are mapped to three individual RGB color channels, digitized, and stored as uncompressed RGB (false) color frames. The TRICLOBS data set enables the development and evaluation of (both static and dynamic) image fusion, enhancement and color mapping algorithms. To allow the development of realistic color remapping procedures, the data set also contains color photographs of each of the three scenes. The color statistics derived from these photographs can be used to define color mappings that give the multi-band imagery a realistic color appearance.

This study investigated whether communication via mediated hand pressure during a remotely shared experience (watching an amusing video) can (1) enhance recovery from sadness, (2) enhance the affective quality of the experience, and (3) increase trust towards the communication partner. Thereto participants first watched a sad movie clip to elicit sadness, followed by a funny one to stimulate recovery from sadness. While watching the funny clip they signaled a hypothetical fellow participant every time they felt amused. In the experimental condition the participants responded by pressing a hand-held two-way mediated touch device (a Frebble), which also provided haptic feedback via simulated hand squeezes. In the control condition they responded by pressing a button and they received abstract visual feedback. Objective (heart rate, galvanic skin conductance, number and duration of joystick or Frebble presses) and subjective (questionnaires) data were collected to assess the emotional reactions of the participants. The subjective measurements confirmed that the sad movie successfully induced sadness while the funny movie indeed evoked more positive feelings. Although their ranking agreed with the subjective measurements, the physiological measurements confirmed this conclusion only for the funny movie. The results show that recovery from movie induced sadness, the affective experience of the amusing movie, and trust towards the communication partner did not differ between both experimental conditions. Hence, feedback via mediated hand touching did not enhance either of these factors compared to visual feedback. Further analysis of the data showed that participants scoring low on Extraversion (i.e., persons that are more introvert) or low on Touch Receptivity (i.e., persons who do not like to be touched by others) felt better understood by their communication partner when receiving mediated touch feedback instead of visual feedback, while the opposite was found for participants scoring high on these factors. The implications of these results for further research are discussed, and some suggestions for follow-up experiments are presented.

Previous research has shown that odorants consistently evoke associations with textures and their tactile properties like smoothness and roughness. Also, it has been observed that olfaction can modulate tactile perception. We therefore hypothesized that tactile roughness perception may be biased towards the somatosensory connotation of an ambient odorant. We performed two experiments to test this hypothesis. In the first experiment, we investigated the influence of ambient chemosensory stimuli with different roughness connotations on tactile roughness perception. In addition to a pleasant odor with a connotation of softness (PEA), we also included a trigeminal stimulant with a rough, sharp or prickly connotation (Ethanol). We expected that—compared to a No-odorant control condition—tactile texture perception would be biased towards smoothness in the presence of PEA and towards roughness in the presence of Ethanol. However, our results show no significant interaction between chemosensory stimulation and perceived tactile surface roughness. It could be argued that ambient odors may be less effective in stimulating crossmodal associations, since they are by definition extraneous to the tactile stimuli. In an attempt to optimize the conditions for sensory integration, we therefore performed a second experiment in which the olfactory and tactile stimuli were presented in synchrony and in close spatial proximity. In addition, we included pleasant (Lemon) and unpleasant (Indole) odorants that are known to have the ability to affect tactile perception. We expected that tactile stimuli would be perceived as less rough when simultaneously presented with Lemon or PEA (both associated with softness) than when presented with Ethanol or Indole (odors that can be associated with roughness). Again, we found no significant main effect of chemosensory condition on perceived tactile roughness. We discuss the limitations of this study and we present suggestions for future research.

Previously, we presented two color mapping methods for the application of daytime colors to fused nighttime (e.g., intensified and longwave infrared or thermal (LWIR)) imagery. These mappings not only impart a natural daylight color appearance to multiband nighttime images but also enhance their contrast and the visibility of otherwise obscured details. As a result, it has been shown that these colorizing methods lead to an increased ease of interpretation, better discrimination and identification of materials, faster reaction times and ultimately improved situational awareness. A crucial step in the proposed coloring process is the choice of a suitable color mapping scheme. When both daytime color images and multiband sensor images of the same scene are available, the color mapping can be derived from matching image samples (i.e., by relating color values to sensor output signal intensities in a sample-based approach). When no exact matching reference images are available, the color transformation can be derived from the first-order statistical properties of the reference image and the multiband sensor image. In the current study, we investigated new color fusion schemes that combine the advantages of both methods (i.e., the efficiency and color constancy of the sample-based method with the ability of the statistical method to use the image of a different but somewhat similar scene as a reference image), using the correspondence between multiband sensor values and daytime colors (sample-based method) in a smooth transformation (statistical method). We designed and evaluated three new fusion schemes that focus on (i) a closer match with the daytime luminances; (ii) an improved saliency of hot targets; and (iii) an improved discriminability of materials. We performed both qualitative and quantitative analyses to assess the weak and strong points of all methods.

Desktop virtual environments (VEs) are increasingly deployed to study the effects of environmental qualities and interventions on human behavior and safety related concerns in built environments. For these applications it is essential that users appraise the affective qualities of the VE similar to those of its real world counterpart. Previous studies have shown that factors like simulated lighting, sound and dynamic elements all contribute to the affective appraisal of a desktop VE. Since ambient odor is known to affect the affective appraisal of real environments, and has been shown to increase the sense of presence in immersive VEs, it may also be an effective tool to tune the affective appraisal of desktop VEs. This study investigated if exposure to ambient odor can modulate the affective appraisal of a desktop VE with signs of public disorder.Participants explored a desktop VE representing a suburban neighborhood with signs of public disorder (neglect, vandalism and crime), while being exposed to either room air or subliminal levels of unpleasant (tar) or pleasant (cut grass) ambient odor. Whenever they encountered signs of disorder they reported their safety related concerns and associated affective feelings.Signs of crime in the desktop VE were associated with negative affective feelings and concerns for personal safety and personal property. However, there was no significant difference between reported safety related concerns and affective connotations in the control (no-odor) and in each of the two ambient odor conditions.Ambient odor did not affect safety related concerns and affective connotations associated with signs of disorder in the desktop VE. Thus, semantic congruency between ambient odor and a desktop VE may not be sufficient to influence its affective appraisal, and a more realistic simulation in which simulated objects appear to emit scents may be required to achieve this goal.

Previous research has shown that perceived facial valence is biased toward background valence. Here, we examine whether background dominance also affects perceived facial dominance. In particular, we hypothesized that downward-pointing triangles, which are known to convey threat, would affect perceived facial dominance. Participants judged perceived facial dominance of neutral faces presented overlaid on downward- or upward-pointing background triangles. Our results show that neutral faces are indeed judged more dominant when seen with a downward-pointing triangle in the background. The fact that simple geometric background shapes can affect facial judgments may have important implications for the design and experience of our daily environment and multimedia content.

This study explores the effects of various spatiotemporal dynamic texture characteristics on human emotions. The emotional experience of auditory (eg, music) and haptic repetitive patterns has been studied extensively. In contrast, the emotional experience of visual dynamic textures is still largely unknown, despite their natural ubiquity and increasing use in digital media. Participants watched a set of dynamic textures, representing either water or various different media, and self-reported their emotional experience. Motion complexity was found to have mildly relaxing and nondominant effects. In contrast, motion change complexity was found to be arousing and dominant. The speed of dynamics had arousing, dominant, and unpleasant effects. The amplitude of dynamics was also regarded as unpleasant. The regularity of the dynamics over the textures' area was found to be uninteresting, nondominant, mildly relaxing, and mildly pleasant. The spatial scale of the dynamics had an unpleasant, arousing, and dominant effect, which was larger for textures with diverse content than for water textures. For water textures, the effects of spatial contrast were arousing, dominant, interesting, and mildly unpleasant. None of these effects were observed for textures of diverse content. The current findings are relevant for the design and synthesis of affective multimedia content and for affective scene indexing and retrieval.

Background: In this study we measured the affective appraisal of sounds and video clips using a newly developed graphical self-report tool: the EmojiGrid. The EmojiGrid is a square grid, labeled with emoji that express different degrees of valence and arousal. Users rate the valence and arousal of a given stimulus by simply clicking on the grid. Methods: In Experiment I, observers (N=150, 74 males, mean age=25.2±3.5) used the EmojiGrid to rate their affective appraisal of 77 validated sound clips from nine different semantic categories, covering a large area of the affective space. In Experiment II, observers (N=60, 32 males, mean age=24.5±3.3) used the EmojiGrid to rate their affective appraisal of 50 validated film fragments varying in positive and negative affect (20 positive, 20 negative, 10 neutral). Results: The results of this study show that for both sound and video, the agreement between the mean ratings obtained with the EmojiGrid and those obtained with an alternative and validated affective rating tool in previous studies in the literature, is excellent for valence and good for arousal. Our results also show the typical universal U-shaped relation between mean valence and arousal that is commonly observed for affective sensory stimuli, both for sound and video. Conclusions: We conclude that the EmojiGrid can be used as an affective self-report tool for the assessment of sound and video-evoked emotions.