Search

The occlusion problem remains a crucial challenge in optical flow estimation (OFE). Despite the recent significant progress brought about by deep learning, most existing deep learning OFE methods still struggle to handle occlusions; in particular, those based on two frames cannot correctly handle occlusions because occluded regions have no visual correspondences. However, there is still hope in multi-frame settings, which can potentially mitigate the occlusion issue in OFE. Unfortunately, multi-frame OFE (MOFE) remains underexplored, and the limited studies on it are mainly specially designed for pyramid backbones or else obtain the aligned previous frame's features, such as correlation volume and optical flow, through time-consuming backward flow calculation or non-differentiable forward warping transformation. This study proposes an efficient MOFE framework named SplatFlow to address these shortcomings. SplatFlow introduces the differentiable splatting transformation to align the previous frame's motion feature and designs a Final-to-All embedding method to input the aligned motion feature into the current frame's estimation, thus remodeling the existing two-frame backbones. The proposed SplatFlow is efficient yet more accurate, as it can handle occlusions properly. Extensive experimental evaluations show that SplatFlow substantially outperforms all published methods on the KITTI2015 and Sintel benchmarks. Especially on the Sintel benchmark, SplatFlow achieves errors of 1.12 (clean pass) and 2.07 (final pass), with surprisingly significant 19.4% and 16.2% error reductions, respectively, from the previous best results submitted. The code for SplatFlow is available at https://github.com/wwsource/SplatFlow. [ABSTRACT FROM AUTHOR]

Electroencephalogram (EEG) emotion recognition plays an important role in human–computer interaction. An increasing number of algorithms for emotion recognition have been proposed recently. However, it is still challenging to make efficient use of emotional activity knowledge. In this paper, based on prior knowledge that emotion varies slowly across time, we propose a temporal-difference minimizing neural network (TDMNN) for EEG emotion recognition. We use maximum mean discrepancy (MMD) technology to evaluate the difference in EEG features across time and minimize the difference by a multibranch convolutional recurrent network. State-of-the-art performances are achieved using the proposed method on the SEED, SEED-IV, DEAP and DREAMER datasets, demonstrating the effectiveness of including prior knowledge in EEG emotion recognition. [ABSTRACT FROM AUTHOR]

Accumulating evidence suggests that the brain is highly dynamic; thus, investigation of brain dynamics especially in brain connectivity would provide crucial information that stationary functional connectivity could miss. This study investigated temporal expressions of spatial modes within the default mode network (DMN), salience network (SN) and cognitive control network (CCN) using a reliable data-driven co-activation pattern (CAP) analysis in two independent data sets. We found enhanced CAP-to-CAP transitions of the SN in patients with MDD. Results suggested enhanced flexibility of this network in the patients. By contrast, we also found reduced spatial consistency and persistence of the DMN in the patients, indicating reduced variability and stability in individuals with MDD. In addition, the patients were characterized by prominent activation of mPFC. Moreover, further correlation analysis revealed that persistence and transitions of RCCN were associated with the severity of depression. Our findings suggest that functional connectivity in the patients may not be simply attenuated or potentiated, but just alternating faster or slower among more complex patterns. The aberrant temporal–spatial complexity of intrinsic fluctuations reflects functional diaschisis of resting-state networks as characteristic of patients with MDD. [ABSTRACT FROM AUTHOR]

The human cerebral cortex is vastly expanded relative to nonhuman primates and rodents, leading to a functional orderly topography of brain networks. Here, we show that functional topography may be associated with gene expression heterogeneity. The neocortex exhibits greater heterogeneity in gene expression, with a lower expression of housekeeping genes, a longer mean path length, fewer clusters, and a lower degree of ordering in networks than archicortical and subcortical areas in human, rhesus macaque, and mouse brains. In particular, the cerebellar cortex displays greater heterogeneity in gene expression than cerebellar deep nuclei in the human brain, but not in the mouse brain, corresponding to the emergence of novel functions in the human cerebellar cortex. Moreover, the cortical areas with greater heterogeneity, primarily located in the multimodal association cortex, tend to express genes with higher evolutionary rates and exhibit a higher degree of functional connectivity measured by resting-state fMRI, implying that such a spatial distribution of gene expression may be shaped by evolution and is favourable for the specialization of higher cognitive functions. Together, the cross-species imaging and genetic findings may provide convergent evidence to support the association between the orderly topography of brain function networks and gene expression. Comparative analysis of human, macaque and mouse function and genetic heterogeneity in the brain reveals links between gene expression and orderly topography of functional networks. [ABSTRACT FROM AUTHOR]

The goal of Facial Kinship Verification (FKV) is to automatically determine whether two individuals have a kin relationship or not from their given facial images or videos. It is an emerging and challenging problem that has attracted increasing attention due to its practical applications. Over the past decade, significant progress has been achieved in this new field. Handcrafted features and deep learning techniques have been widely studied in FKV. The goal of this paper is to conduct a comprehensive review of the problem of FKV. We cover different aspects of the research, including problem definition, challenges, applications, benchmark datasets, a taxonomy of existing methods, and state-of-the-art performance. In retrospect of what has been achieved so far, we identify gaps in current research and discuss potential future research directions. [ABSTRACT FROM AUTHOR]

Multi-agent reinforcement learning (MARL) is an important way to realize multi-agent cooperation. But there are still many challenges, including the scalability and the uncertainty of the environment that limit its application. In this paper, we explored to solve those problems through the graph network and the attention mechanism. Finally we succeeded in extending the existing algorithm and obtaining a new algorithm called GAMA. Specifically through the graph network, we made the environment information shared among agents. Meanwhile, the unimportant information was filtered out with the help of the attention mechanism, which helped to improve the communication efficiency. As a result, GAMA obtained the highest mean episode rewards compared to the baselines as well as excellent scalability. The reason why we choose the graph network is that understanding the relationship among agents plays a key role in solving multi-agent problems. And the graph network is very suitable for relational induction bias. Through the integration with the attention mechanism, it was shown that agents could figure out their relationship and focus on the influential environment factors in our experiment. [ABSTRACT FROM AUTHOR]

We present a new approach for 3D human pose estimation from a single image. State-of-the-art methods for 3D pose estimation have focused on predicting a full-body pose of a single person and have not given enough attention to the challenges in application: incompleteness of body pose and existence of multiple persons in image. In this paper, we introduce depth maps to solve these problems. Our approach predicts the depths of human pose over all spatial grids, which supports 3D poses estimation for incomplete or full bodies of multiple persons. The proposed depth maps encode depths of limbs rather than joints. They are more informative and reversibly convertible to depths of joints. The unified network is trained end to end using mixed 2D and 3D annotated samples. The experiments reveal that our algorithm achieves the state of the art on Human3.6M, the largest publicly available 3D pose estimation benchmark. Moreover, qualitative results have been reported to demonstrate the effectiveness of our approach for 3D pose estimation for incomplete human bodies and multiple persons. [ABSTRACT FROM AUTHOR]

This project aims to explore if stronger functional connectivity (FC) exists in the maximal BOLD response of EEG/fMRI analysis when it is concordant with seizure-onset-zone (SOZ). Twenty-six patients with drug-resistant focal epilepsy who had an EEG/fMRI and later underwent stereo-EEG implantation were included. Different types of IEDs were labeled in scalp EEG and IED-related maximal BOLD responses were evaluated separately, each constituting one study. After evaluating concordance between maximal BOLD and SOZ, twenty-seven studies were placed in the concordant group and eight in the discordant group. We evaluated the local connectivity and ipsilaterally distant connectivity difference between the maximal BOLD and the contralateral homotopic region. Significantly stronger local FC was found for the maximal BOLD in the concordant group (p < 0.05, Bonferroni corrected). 52% of the studies in the concordant group and 13% in the discordant group had a significant difference compared to healthy subjects (p < 0.05, uncorrected). The finding suggests that, when concordant with the SOZ, the maximal BOLD is more likely to have stronger local FC compared to its contralateral counterpart. This asymmetry in functional connectivity may help to noninvasively improve the specificity of EEG/fMRI analysis. [ABSTRACT FROM AUTHOR]

Motor imagery (MI) is a mental representation of motor behavior and has been widely used in electroencephalogram based brain–computer interfaces (BCIs). Several studies have demonstrated the efficacy of MI-based BCI-feedback training in post-stroke rehabilitation. However, in the earliest stage of the training, calibration data typically contain insufficient discriminability, resulting in unreliable feedback, which may decrease subjects' motivation and even hinder their training. To improve the performance in the early stages of MI training, a novel hybrid BCI paradigm based on MI and P300 is proposed in this study. In this paradigm, subjects are instructed to imagine writing the Chinese character following the flash order of the desired Chinese character displayed on the screen. The event-related desynchronization/synchronization (ERD/ERS) phenomenon is produced with writing based on one's imagination. Simultaneously, the P300 potential is evoked by the flash of each stroke. Moreover, a fusion method of P300 and MI classification is proposed, in which unreliable P300 classifications are corrected by reliable MI classifications. Twelve healthy naïve MI subjects participated in this study. Results demonstrated that the proposed hybrid BCI paradigm yielded significantly better performance than the single-modality BCI paradigm. The recognition accuracy of the fusion method is significantly higher than that of P300 (p < 0.05) and MI (p < 0.01). Moreover, the training data size can be reduced through fusion of these two modalities. [ABSTRACT FROM AUTHOR]

Many studies reported that ERP-based BCIs can provide communication for some people with amyotrophic lateral sclerosis (ALS). ERP-based BCIs often present characters within a matrix that occupies the center of the visual field. However, several studies have identified some concerns with the matrix-based approach. This approach may lead to fatigue and errors resulting from flashing adjacent stimuli, and is impractical for users who might want to use the BCI in tandem with other software or feedback in the center of the monitor. In this paper, we introduce and validate an alternate ERP-based BCI display approach. By presenting stimuli near the periphery of the display, we reduce the adjacency problem and leave the center of the display available for feedback or other applications. Two ERP-based display approaches were tested on 18 ALS patients to: (1) compare performance between a conventional matrix speller paradigm (Matrix-P, mean visual angle 6°) and a new speller paradigm with peripherally distributed stimuli (Peripheral-P, mean visual angle 8.8°); and (2) assess performance while spelling 42 characters online continuously, without a break. In the Peripheral-P condition, 12 subjects attained higher than 80% feedback accuracy during online performance, and 7 of these subjects obtained higher than 90% accuracy. The experimental results showed that the Peripheral-P condition yielded performance comparable to the conventional Matrix-P condition (p > 0.05) in accuracy and information transfer rate. This paper introduces a new display approach that leaves the center of the monitor open for feedback and/or other display elements, such as movies, games, art, or displays from other AAC software or conventional software tools. [ABSTRACT FROM AUTHOR]

Recently, many auditory BCIs are using beeps as auditory stimuli, while beeps sound unnatural and unpleasant for some people. It is proved that natural sounds make people feel comfortable, decrease fatigue, and improve the performance of auditory BCI systems. Drip drop is a kind of natural sounds that makes humans feel relaxed and comfortable. In this work, three kinds of drip drops were used as stimuli in an auditory-based BCI system to improve the user-friendness of the system. This study explored whether drip drops could be used as stimuli in the auditory BCI system. The auditory BCI paradigm with drip-drop stimuli, which was called the drip-drop paradigm (DP), was compared with the auditory paradigm with beep stimuli, also known as the beep paradigm (BP), in items of event-related potential amplitudes, online accuracies and scores on the likability and difficulty to demonstrate the advantages of DP. DP obtained significantly higher online accuracy and information transfer rate than the BP (p < 0.05, Wilcoxon signed test; p < 0.05, Wilcoxon signed test). Besides, DP obtained higher scores on the likability with no significant difference on the difficulty (p < 0.05, Wilcoxon signed test). The results showed that the drip drops were reliable acoustic materials as stimuli in an auditory BCI system. [ABSTRACT FROM AUTHOR]

Studies on antisocial personality disorder (ASPD) subjects focus on brain functional alterations in relation to antisocial behaviors. Neuroimaging research has identified a number of focal brain regions with abnormal structures or functions in ASPD. However, little is known about the connections among brain regions in terms of inter-regional whole-brain networks in ASPD patients, as well as possible alterations of brain functional topological organization. In this study, we employ resting-state functional magnetic resonance imaging (R-fMRI) to examine functional connectome of 32 ASPD patients and 35 normal controls by using a variety of network properties, including small-worldness, modularity, and connectivity. The small-world analysis reveals that ASPD patients have increased path length and decreased network efficiency, which implies a reduced ability of global integration of whole-brain functions. Modularity analysis suggests ASPD patients have decreased overall modularity, merged network modules, and reduced intra- and inter-module connectivities related to frontal regions. Also, network-based statistics show that an internal sub-network, composed of 16 nodes and 16 edges, is significantly affected in ASPD patients, where brain regions are mostly located in the fronto-parietal control network. These results suggest that ASPD is associated with both reduced brain integration and segregation in topological organization of functional brain networks, particularly in the fronto-parietal control network. These disruptions may contribute to disturbances in behavior and cognition in patients with ASPD. Our findings may provide insights into a deeper understanding of functional brain networks of ASPD. [ABSTRACT FROM AUTHOR]

Parkinson's disease (PD) is one of the most common neurodegenerative disorders in the world. Previous studies have focused on the basal ganglia and cerebral cortices. To date, the cerebellum has not been systematically investigated in patients with PD. In the current study, 45 probable PD patients and 40 age- and gender-matched healthy controls underwent structural magnetic resonance imaging, and we used support vector machines combining with voxel-based morphometry to explore the cerebellar structural changes in the probable PD patients relative to healthy controls. The results revealed that the gray matter alterations were primarily located within the cerebellar Crus I, implying a possible important role of this region in PD. Furthermore, the gray matter alterations in the cerebellum could differentiate the probable PD patients from healthy controls with accuracies of more than 95 % ( p < 0.001, permutation test) via cross-validation, suggesting the potential of analyzing the cerebellum in the clinical diagnosis of PD. [ABSTRACT FROM AUTHOR]

With great progress of space navigation technology, it becomes possible to travel beyond Earth's gravity. So far, it remains unclear whether the human brain can function normally within an environment of microgravity and confinement. Particularly, it is a challenge to figure out some neuroimaging-based markers for rapid screening diagnosis of disrupted brain function in microgravity environment. In this study, a 7-day −6° head down tilt bed rest experiment was used to simulate the microgravity, and twenty healthy male participants underwent resting-state functional magnetic resonance imaging scans at baseline and after the simulated microgravity experiment. We used a multivariate pattern analysis approach to distinguish the brain states with simulated microgravity from normal gravity based on the functional connectivity within the default network, resulting in an accuracy of no less than 85 % via cross-validation. Moreover, most discriminative functional connections were mainly located between the limbic system and cortical areas and were enhanced after simulated microgravity, implying a self-adaption or compensatory enhancement to fulfill the need of complex demand in spatial navigation and motor control functions in microgravity environment. Overall, the findings suggest that the brain states in microgravity are likely different from those in normal gravity and that brain connectome could act as a biomarker to indicate the brain state in microgravity. [ABSTRACT FROM AUTHOR]

We focus on the recognition of human actions in uncontrolled videos that may contain complex temporal structures. It is a difficult problem because of the large intra-class variations in viewpoint, video length, motion pattern, etc. To address these difficulties, we propose a novel system in this paper that represents each action class by hidden temporal models. In this system, we represent the crucial action event per category by a video segment that covers a fixed number of frames and can move temporally within the sequences. To capture the temporal structures, the video segment is described by a temporal pyramid model. To capture large intra-class variations, multiple models are combined using Or operation to represent alternative structures. The index of model and the start frame of segment are both treated as hidden variables. We implement a learning procedure based on the latent SVM method. The proposed approach is tested on two difficult benchmarks: the Olympic Sports and HMDB51 data sets. The experimental results reveal that our system is comparable to the state-of-the-art methods in the literature. [ABSTRACT FROM AUTHOR]

Graph-based image segmentation techniques generally represent the problem in terms of a graph. In this work, we present a novel graph, called the directional nearest neighbor graph. The construction principle of this graph is that each node corresponding to a pixel in the image is connected to a fixed number of nearest neighbors measured by color value and the connected neighbors are distributed in four directions. Compared with the classical grid graph and the nearest neighbor graph, our method can capture low-level texture information using a less-connected edge topology. To test the performance of the proposed method, a comparison with other graph-based methods is carried out on synthetic and real-world images. Results show an improved segmentation for texture objects as well as a lower computational load. [ABSTRACT FROM AUTHOR]

Automatic recognition of the contents of a scene is an important issue in the field of computer vision. Although considerable progress has been made, the complexity of scenes remains an important challenge to computer vision research. Most previous approaches for scene recognition are based on the so-called 'bag of visual words' model, which uses clustering methods to quantize numerous local region descriptors into a codebook. The size of the codebook and the selection of initial clustering centers greatly affect the performance. Furthermore, the large size of the codebook leads to high computational costs and large memory consumption. To overcome these weaknesses, we present an unsupervised natural scene recognition approach that is not based on the 'bag of visual words' model. This approach constructs multiple images of different resolutions and extracts structural and textural features from these images. The structural features are represented by weighted histograms of the gradient orientation descriptor, which is presented in this paper, and the textural features are represented by filter responses of Gabor filters and a Schmid set. We regard the structural and textural features as two independent feature channels, and combine them to realize automatic categorization of scenes using a support vector machine. We then evaluated our approach using three commonly used datasets with various scene categories. Our experiments demonstrate that the weighted histograms of the gradient orientation descriptor outperform the classical scale invariant feature transform descriptor in natural-scene recognition, and our approach achieves good performance with respect to current state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Level set method and Gaussian Mixture model (GMM) are two very valuable tools for natural image segmentation. The former aims to acquire good geometrical continuity of segmentation boundaries, while the latter focuses on analyzing statistical properties of image feature data. Some studies on the integration between them have been reported due to their complementarity in the last 10 years. However, these studies generally supposed that the image-featured data density distribution of every segmented domain is independent with each other and can be separately approximated by Gaussian model or GMM, which conflicts with the fundamental idea of GMM clustering-based image segmentation. To remedy this problem, we give a new insight at image segmentation objective under the combined framework between Bayesian theory and GMM density approximation. Thereby, a novel level set image segmentation method integrated with GMM (GMMLS) is proposed. Then, the theoretical analysis on GMMLS is given, in which some valuable results are demonstrated. At last, several types of natural image segmentation experiments are reported and the corresponding results indicate that GMMLS can obtain better or at least equivalent performance compared with existing relevant methods in almost all cases. [ABSTRACT FROM AUTHOR]