Search

Multisensory integration (MSI) is the integration by the brain of environmental information acquired through more than one sense. Accurate MSI has been shown to be a key component of successful aging and to be crucial for processes underlying activities of daily living (ADLs). Problems in MSI could prevent older adults (OA) to age in place and live independently. However, there is a need to know how to assess changes in MSI in individuals. This systematic review provides an overview of tests assessing the effect of age on MSI in the healthy elderly population (aged 60 years and older). A literature search was done in Scopus. Articles from the earliest records available to January 20, 2016, were eligible for inclusion if assessing effects of aging on MSI in the healthy elderly population compared to younger adults (YA). These articles were rated for risk of bias with the Newcastle-Ottawa quality assessment. Out of 307 identified research articles, 49 articles were included for final review, describing 69 tests. The review indicated that OA maximize the use of multiple sources of information in comparison to YA (20 studies). In tasks that require more cognitive function, or when participants need to adapt rapidly to a situation, or when a dual task is added to the experiment, OA have problems selecting and integrating information properly as compared to YA (19 studies). Additionally, irrelevant or wrong information (i.e., distractors) has a greater impact on OA than on YA (21 studies). OA failing to weigh sensory information properly, has not been described in previous reviews. Anatomical changes (i.e., reduction of brain volume and differences of brain areas' recruitment) and information processing changes (i.e., general cognitive slowing, inverse effectiveness, larger time window of integration, deficits in attentional control and increased noise at baseline) can only partly explain the differences between OA and YA regarding MSI. Since we have an interest in successful aging and early detection of MSI issues in the elderly population, the identified tests form a good starting point to develop a clinically useful toolkit to assess MSI in healthy OA.

This study investigated whether communication via mediated hand pressure during a remotely shared experience (watching an amusing video) can (1) enhance recovery from sadness, (2) enhance the affective quality of the experience, and (3) increase trust towards the communication partner. Thereto participants first watched a sad movie clip to elicit sadness, followed by a funny one to stimulate recovery from sadness. While watching the funny clip they signaled a hypothetical fellow participant every time they felt amused. In the experimental condition the participants responded by pressing a hand-held two-way mediated touch device (a Frebble), which also provided haptic feedback via simulated hand squeezes. In the control condition they responded by pressing a button and they received abstract visual feedback. Objective (heart rate, galvanic skin conductance, number and duration of joystick or Frebble presses) and subjective (questionnaires) data were collected to assess the emotional reactions of the participants. The subjective measurements confirmed that the sad movie successfully induced sadness while the funny movie indeed evoked more positive feelings. Although their ranking agreed with the subjective measurements, the physiological measurements confirmed this conclusion only for the funny movie. The results show that recovery from movie induced sadness, the affective experience of the amusing movie, and trust towards the communication partner did not differ between both experimental conditions. Hence, feedback via mediated hand touching did not enhance either of these factors compared to visual feedback. Further analysis of the data showed that participants scoring low on Extraversion (i.e., persons that are more introvert) or low on Touch Receptivity (i.e., persons who do not like to be touched by others) felt better understood by their communication partner when receiving mediated touch feedback instead of visual feedback, while the opposite was found for participants scoring high on these factors. The implications of these results for further research are discussed, and some suggestions for follow-up experiments are presented.

Previous research has shown that odorants consistently evoke associations with textures and their tactile properties like smoothness and roughness. Also, it has been observed that olfaction can modulate tactile perception. We therefore hypothesized that tactile roughness perception may be biased towards the somatosensory connotation of an ambient odorant. We performed two experiments to test this hypothesis. In the first experiment, we investigated the influence of ambient chemosensory stimuli with different roughness connotations on tactile roughness perception. In addition to a pleasant odor with a connotation of softness (PEA), we also included a trigeminal stimulant with a rough, sharp or prickly connotation (Ethanol). We expected that-compared to a No-odorant control condition-tactile texture perception would be biased towards smoothness in the presence of PEA and towards roughness in the presence of Ethanol. However, our results show no significant interaction between chemosensory stimulation and perceived tactile surface roughness. It could be argued that ambient odors may be less effective in stimulating crossmodal associations, since they are by definition extraneous to the tactile stimuli. In an attempt to optimize the conditions for sensory integration, we therefore performed a second experiment in which the olfactory and tactile stimuli were presented in synchrony and in close spatial proximity. In addition, we included pleasant (Lemon) and unpleasant (Indole) odorants that are known to have the ability to affect tactile perception. We expected that tactile stimuli would be perceived as less rough when simultaneously presented with Lemon or PEA (both associated with softness) than when presented with Ethanol or Indole (odors that can be associated with roughness). Again, we found no significant main effect of chemosensory condition on perceived tactile roughness. We discuss the limitations of this study and we present suggestions for future research.

Estimating cognitive or affective state from neurophysiological signals and designing applications that make use of this information requires expertise in many disciplines such as neurophysiology, machine learning, experimental psychology, and human factors. This makes it difficult to perform research that is strong in all its aspects as well as to judge a study or application on its merits. On the occasion of the special topic "Using neurophysiological signals that reflect cognitive or affective state" we here summarize often occurring pitfalls and recommendations on how to avoid them, both for authors (researchers) and readers. They relate to defining the state of interest, the neurophysiological processes that are expected to be involved in the state of interest, confounding factors, inadvertently "cheating" with classification analyses, insight on what underlies successful state estimation, and finally, the added value of neurophysiological measures in the context of an application. We hope that this paper will support the community in producing high quality studies and well-validated, useful applications.

While studies exist that compare different physiological variables with respect to their association with mental workload, it is still largely unclear which variables supply the best information about momentary workload of an individual and what is the benefit of combining them. We investigated workload using the n-back task, controlling for body movements and visual input. We recorded EEG, skin conductance, respiration, ECG, pupil size and eye blinks of 14 subjects. Various variables were extracted from these recordings and used as features in individually tuned classification models. Online classification was simulated by using the first part of the data as training set and the last part of the data for testing the models. The results indicate that EEG performs best, followed by eye related measures and peripheral physiology. Combining variables from different sensors did not significantly improve workload assessment over the best performing sensor alone. Best classification accuracy, a little over 90%, was reached for distinguishing between high and low workload on the basis of 2 min segments of EEG and eye related variables. A similar and not significantly different performance of 86% was reached using only EEG from single electrode location Pz.

Gaze-independent event-related potential (ERP) based brain-computer interfaces (BCIs) yield relatively low BCI performance and traditionally employ unimodal stimuli. Bimodal ERP-BCIs may increase BCI performance due to multisensory integration or summation in the brain. An additional advantage of bimodal BCIs may be that the user can choose which modality or modalities to attend to. We studied bimodal, visual-tactile, gaze-independent BCIs and investigated whether or not ERP components' tAUCs and subsequent classification accuracies are increased for (1) bimodal vs. unimodal stimuli; (2) location-congruent vs. location-incongruent bimodal stimuli; and (3) attending to both modalities vs. to either one modality. We observed an enhanced bimodal (compared to unimodal) P300 tAUC, which appeared to be positively affected by location-congruency (p = 0.056) and resulted in higher classification accuracies. Attending either to one or to both modalities of the bimodal location-congruent stimuli resulted in differences between ERP components, but not in classification performance. We conclude that location-congruent bimodal stimuli improve ERP-BCIs, and offer the user the possibility to switch the attended modality without losing performance.

The P300 event-related potential (ERP) can be used to infer whether an observer is looking at a target or not. Common practice in P300 experiments and applications is that observers are asked to fixate their eyes while stimuli are presented. We investigated the possibility to differentiate between single target and nontarget fixations in a target search task involving eye movements by using EEG epochs synchronized to fixation onset (fixation-related potentials: FRPs). Participants systematically scanned search displays consisting of six small Landolt Cs in search of Cs with a particular orientation. After each search display, they indicated whether and where target Cs had been presented. As expected, an FRP component consistent with the P300 reliably distinguished between target and nontarget fixations. It was possible to classify single FRPs into target and nontarget FRPs above chance (on average 62% correct, where 50% would be chance). These results are the first step to practical applications such as covertly monitoring observers' interests and supporting search tasks.

In this study, we investigated a Brain-Computer Interface (BCI) based on EEG responses to vibro-tactile stimuli around the waist. P300 BCIs based on tactile stimuli have the advantage of not taxing the visual or auditory system and of being potentially unnoticeable to other people. A tactile BCI could be especially suitable for patients whose vision or eye movements are impaired. In Experiment 1, we investigated its feasibility and the effect of the number of equally spaced tactors. Whereas a large number of tactors is expected to enhance the P300 amplitude since the target will be less frequent, it could also negatively affect the P300 since it will be difficult to identify the target when tactor density increases. Participants were asked to attend to the vibrations of a target tactor, embedded within a stream of distracters. The number of tactors was two, four or six. We demonstrated the feasibility of a tactile P300 BCI. We did not find a difference in SWLDA classification performance between the different numbers of tactors. In a second set of experiments we reduced the stimulus onset asynchrony (SOA) by shortening the on- and/or off-time of the tactors. The SOA for an optimum performance as measured in our experiments turned out to be close to conventional SOAs of visual P300 BCIs.

This laboratory study examined the possibility of using a car seat instrumented with a tactile display to communicate directional information to a driver. A car seat fitted with an 8 x 8 matrix of vibrators embedded in the seat pan was used to code eight different directions. Localization response time and angular accuracy were examined as a function of stimulus direction, presence of a tactile attention cue, temporal pattern, stimulus layout, age, and gender. The mean absolute angular error was 23 degrees, and both localization accuracy and response times were superior for the back left, backward, and back right directions. Of the various temporal pattern/attention cue combinations examined, results favored the relatively fast patterns consisting of vibration bursts of 125 or 250 ms without a centrally located attention cue over 500 ms bursts that were preceded by an attention cue. Observed age and gender effects were relatively modest, suggesting that using tactile cueing to communicate direction is effective across a wide range of users. In addition, the tactile stimulus was detected by more than 90% of the participants under surprise trial conditions. Overall, these results indicate that the tactile chair provides a promising and robust method of providing directional information.

Objective: We investigated the effectiveness of a tactile torso display as a countermeasure to spatial disorientation (SD) and compared inside-out and outside-in codings., Background: SD is a serious threat to military as well as civilian pilots and aircraft. Considerable effort has been put into SD countermeasures such as training programs and advanced cockpit displays. Tactile displays have been considered a promising technology., Method: Twenty-four participants were assigned to the two coding groups (12 per group and matched for age and gender). We used a rotating chair to build up a state of SD by rotating participants around their yaw axis followed by a sudden stop. During the following recovery phase a random disturbance signal was added to the chair's orientation. Participants actively controlled their orientation and were instructed to maintain a stable orientation., Results: Statistical analyses revealed that recovery from SD was improved with support of the tactile instrument, but tracking performance was reduced. The effects were the same whether the instrument was available full time or during the recovery phase only. There were no differences between outside-in and inside-out coding., Conclusion: The present study demonstrates the potential of tactile cockpit instruments in controlling SD, even in the presence of strong but misleading self-motion information from the vestibular sense., Application: Actual or potential applications of this research include spatial disorientation countermeasures for pilots, divers, and astronauts.

The present study investigates tracking performance with tactile and/or visual presentation of target and cursor. The tactile display consisted of vibrators in a horizontal linear array on the torso, the visual display consisted of dots projected on a horizontal plane surrounding the observer. Both displays presented qualitatively identical information: direction in a horizontal plane. Participants performed two different tracking tasks with target and cursor presented to the same modality (either visual or tactile) or to different modalities (a visual target and a tactile cursor or vice versa). The errors in both cross-modal settings were well predicted by the uni-modal errors. This indicates that no additional costs are involved in cross-modal visual-vibrotactile tracking. Closer inspection of the data reveals that the tactile modality is less suited for processing external disturbances. These results give directions on how to allocate information to the visual and tactile modalities in a multi-modal tracking interface, such as those applied in military cockpits.

For multiple degree-of-freedom (DOF) systems, it is important to determine how accurately operators can control each DOF and what influence perceptual, information processing, and psychomotor components have on performance. Sixteen right-handed male students participated in 2 experiments: 1 involving positioning and 1 involving tracking with 3 translational DOFs. To separate perceptual and psychomotor effects, we used 2 control-display mappings that differed in the coupling of vertical and depth dimensions to the up-down and fore-aft control axes. We observed information processing effects in the positioning task: Initial error correction on the vertical dimension lagged in time behind the horizontal dimension. The depth dimension error correction lagged behind both, which was ascribed to the poorer perceptual information. We observed this perceptual effect also in the tracking experiment: Tracking error along the depth dimension was 3.8 times larger than along the other dimensions. Motor effects were also present, with tracking errors along the up-down axis of the hand controller being 1.1 times larger than along the fore-aft axis. These results indicate that all 3 components contribute to control performance. Actual applications of this research include interface design for remote control and virtual reality.