Search

Infrared and visible image fusion aims to obtain an image that can retain the prominent infrared target and the detailed texture information from the source images. Most scale filter-based decomposition methods mainly attempt to extract more detailed features by increasing decomposition layers, and they ignore fully considering the intrinsic properties of the original images and the influence of noises. To solve this issue and obtain better fusion & detection performance, this paper proposes a fusion method via a globalet filter and detail enhancement model to construct a target-aware fusion framework. The globalet filter is first comprehensively considered from three perspectives based on global point-to-point optimization: target brightness, removal of gradients of large-small scales (noise reduction), and preservation of texture details. Mathematically, three limited error measure equations are constructed between the target output and the source image in the shape of the $L_{2}$ -norm and the first-order derivative difference. Next, a weighted average operator and a detail enhancement model are proposed to guide the corresponding sub-layers. This model creates connections between the detail layers and the input images so that the fused detail layer contains the best pixel region from these images to the greatest extent possible after the “maximum absolute” rule. As a result, the fused image is reconstructed by adding the previously obtained sub-images. Extensive experimental results demonstrate that our method outperforms state-of-the-art fusion methods, particularly in highlighting infrared targets, preserving substantial details, and producing average detection accuracy of exceeds 98%.

This research proposes a deep learning method for classifying student behavior in classrooms that follow the professional learning community teaching approach. We collected data on five student activities: hand-raising, interacting, sitting, turning around, and writing. We used the sum of absolute differences (SAD) in the LUV color space to detect scene changes. The K-means algorithm was then applied to select keyframes using the computed SAD. Next, we extracted features using multiple pretrained deep learning models from the convolutional neural network family. The pretrained models considered were InceptionV3, ResNet50V2, VGG16, and EfficientNetB7. We leveraged feature fusion, incorporating optical flow features and data augmentation techniques, to increase the necessary spatial features of selected keyframes. Finally, we classified the students’ behavior using a deep sequence model based on the bidirectional long short-term memory network with an attention mechanism (BiLSTM-AT). The proposed method with the BiLSTM-AT model can recognize behaviors from our dataset with high accuracy, precision, recall, and F1-scores of 0.97, 0.97, and 0.97, respectively. The overall accuracy was 96.67%. This high efficiency demonstrates the potential of the proposed method for classifying student behavior in classrooms.

Aerial images are important for monitoring land cover and land resource management. An aerial imaging source which keeps its position at a higher altitude, and which has a considerable duration of airtime, employs wireless communications for sending images to relevant receivers. An aerial image must be transmitted from the image source to a ground station where it can be stored and analyzed. Due to transmission errors, aerial images which are received from an image transmitter contain distortions which can affect the quality of the images, causing noise, color shifts, and other issues that can impact the accuracy of semantic segmentation and the usefulness of the information contained in the images. Current semantic segmentation methods discard distorted images, which makes the available dataset small or treats them as normal images, which causes poor segmentation results. This paper proposes a deep-learning-based semantic segmentation method for distorted aerial images. For different receivers, distortions occur differently, and by considering the receiver specificness of the distortions, the proposed method was able to grasp the acceptability for a distorted image using semantic segmentation models trained with large aerial image datasets to build a combined model that can effectively segment a distorted aerial image which was received by an analog image receiver. Two combined deep learning models, an approximating model, and a segmentation model were trained combinedly to maximize the segmentation score for distorted images. The results showed that the combined learning method achieves higher intersection-over-union (IoU) scores than the results obtained by using only a segmentation model.

Palm vein recognition has received some considerable attention regarding its use in biometric identification. Palm vein characteristics offer a superior level of security and reliability in personal identification compared to extrinsic methods such as fingerprint, face, and palm print recognition, as vein patterns are difficult to duplicate and do not change throughout one’s lifetime. This study proposes both segmentation and recognition methods to enhance the authentication performance and achieve correct identification using palm vein features. First, we propose a segmentation model based on the U-Net model, enhanced with an attention gate, to effectively segment palm vein patterns. The incorporation of both the attention gate and residual block allows the segmentation model for the learning of the essential features required for specific segmentation tasks. The Hessian-based Jerman filtering method is used for ground-truth labeling. The segmentation model extracts the palm vein patterns and filters out the irrelevant and noisy pixels for the purpose of recognition. The efficient channel attention residual network is trained to learn discriminative features for personal identification using combined margin-based loss functions for palm vein recognition. The channel attention module enhances the useful information and suppresses irrelevant features in the feature maps, which overcomes the problem of rotation, position translation, and scale transformation, as well as improves the recognition rate. The combined loss function used in this study increases the similarity between the intra-class samples and the diversity between inter-class samples. The proposed recognition model achieved 100% accuracy for palm vein recognition and an equal error rate of 0.018 for palm vein verification.

With the industrial demand caused by multi-sensor image fusion, infrared and visible image fusion (IVIF) technology is flourishing. In recent years, scale decomposition methods have led the trend for feature extraction. Such methods, however, have low time efficiency. To address this issue, this paper proposes a simple yet effective IVIF approach via a feature-oriented dual-module complementary. Specifically, we analyze five classical operators comprehensively and construct the spatial gradient capture module (SGCM) and infrared brightness supplement module (IBSM). In the SGCM, three kinds of feature maps are obtained, respectively, by introducing principal component analysis, saliency, and proposing contrast estimation operators considered the relative differences of contrast information covered by the input images. These maps are later reconstructed through pyramidal transformation to obtain the predicted image. The IBSM is then proposed to refine the missing infrared thermal information in the predicted image. Among them, we improve the measurement operators applied to the exposure modalities, namely, the gradient of the grayscale images (2D gradient) and well-exposedness. The former is responsible for extracting fine details, and the latter is meant for locating brightness regions. Experiments performed on public datasets demonstrate that the proposed method outperforms nine state-of-the-art methods in terms of subjective visual and objective indicators.

Abstract Digital imaging devices sometimes capture images with abnormal exposure because of the complex lighting conditions and limited dynamic range of luminance. In this work, a new residual dense U‐Net is proposed to predict the information that has been lost in saturated image areas, to enable abnormal exposure restoration from a single image. Full advantage of the multi‐level features is taken from all the convolution layers in the restoration process. Specifically, the densely connected convolutional layers are used in a contracting encoder net to extract abundant local features. The transition layer and local residual learning after each dense block is then applied to adaptively learn more effectively from prior with present local features. Further, an expanding decoder net with dense layers is used and added with skip connections to preserve low‐level information and existing details. Finally, multiple global residual learning is used to adaptively extract hierarchical features and help train the network. It is shown that such a network can be trained end‐to‐end from abnormal exposure images and outperform the prior best method on image enhancement. Experimental results show that the proposed model can greatly enhance the dynamic range of an abnormal exposure image.

Removing snow particles from an image is a complicated task due to the particles’ shape, size, and color. The latest snow removal methods remove snow from a single image but retain some snow and salt-and-pepper particles. Some approaches, while trying to remove snow from a single image, produce blurry artifacts. In this paper, we solve these problems by designing a network model that consists of a residual generative network, a snow-free image generative network, and a perceptual relativistic discriminative network. In both generative networks, we assign the residual frequency network (ReFNet) as our bottleneck module. Our network model learns to map two relationships. First, the input snowy image is trained to map the snow mask image in the dataset. Then, a retained image resulting from subtraction between an input image and the estimated residual image is concatenated with the input snowy image and mapped to the desired snow-free ground truth. Moreover, we use a perceptual identical-paired adversarial network based on a relativistic discriminative network to make our training results more robust. Our results achieve greater performance than state-of-the-art methods on both synthetic and real-world snowy images.

Automatic and efficient 3D object modeling has become critical in industrial applications. The advancement of deep convolutional neural networks (CNNs) has prompted researchers to use CNNs for learning 3D geometry information directly from images. However, the feature maps directly extracted by CNNs are more suitable for image processing tasks because they contain more deep texture information of the entire 2D image. Compared with this, 3D reconstruction tasks using CNNs demand geometric information about a specific object. The existing architecture mainly tries to infer the geometric structure through texture information, which leads to an uneven distribution of points in the generated point cloud object. To address this problem, we propose a cascade point set generation network (CPSGN) that deforms the predicted object while more effectively inferring the object’s 3D geometric information from the 2D image so that the distribution of the final object becomes more uniform and denser. The CPSGN consists of a point set generation part for producing a basic 3D object and a point deformation part for fine-tuning the basic 3D object. In addition, we designed a projection loss that optimizes the geometry of the model by measuring shape differences from multiple perspectives. Experimental results on different benchmark datasets indicate that the produced point-based model outperforms existing approaches.

In recent years, person re-identification based on deep learning approaches has made great progress and achieved good results. However, many of the latest network design methods, which usually deploy ResNet or SENet as the backbone network, were originally designed for classification tasks. Since the person re-identification task is essentially different from the classification task, the structure of the backbone network should be modified accordingly. In this paper, we propose a retrieval network based on a multi-scale backbone architecture, which is specifically suitable for the person re-identification task. By constructing hierarchical residual-like connections within a single residual block, the model learns multi-scale discriminative features of pedestrian images. Unlike many state-of-the-art methods that use complex network structures and concatenate multi-branch features, our proposed retrieval network is implemented using only global features, simple triplet loss, and softmax with cross-entropy loss. The results of extensive experiments show that the proposed network has stronger fine-grained pedestrian representation ability, leading to performance gains for person re-identification tasks. Our proposed network achieves a rank-1 accuracy of 96.03% on the Market-1501 and 92.11% on DukeMTMC-reID datasets while only using global features.

Human action recognition (HAR) has gained popularity because of its various applications, such as human–object interaction [...]

Face recognition is an important biometric due to its non-intrusive collection of data that can be applied to surveillance systems. However, the human pose is unconstrained under surveillance, and there is only one frontal face available in the gallery in most scenarios, and these two factors challenge face recognition performance. A goodly amount of research has been published to solve this problem. Specially, deep learning-based methods learn generic models between poses to synthesize different poses of a given face, however, generic synthesis models can lose the face identity while warping the face, which deteriorates the discriminative capability of learned features. In this paper, we proposed the deep Siamese networks to learn identity-aware and pose-invariant features, adding contrastive loss to the face synthesis model to preserve the face identity while synthesizing the face. In addition, we trained various face synthesis models with different target poses as supervisory signals, the learned pose-invariant features were incorporated by another Siamese network, resulting in deeper pose-invariant and identity-aware features. The proposed network is free of landmark estimation and face pose, and it is in real time. We tested the proposed algorithm in the FERET and CAS-PEAL datasets, and experimental results demonstrated that our network achieved superior performance to that of recently published algorithms for cross-pose face recognition, especially the 2D deep learning-based algorithms.

Identifying composite sketches with digital face photos is an important and challenging task for law enforcement agencies. It has attracted a wide research interest in the face recognition area. In this paper, we present a novel framework that identifies the photo corresponding to a given composite face sketch. A coupled deep convolutional neural network, named Sketch-Photo Net (SP-Net) is proposed, which is fed with a positive or negative photo-sketch pair. In the proposed SP-Net, the customized VGG-Face network is adopted as base model and is followed by two branches, namely S-Net and P-Net, for sketch and photo, respectively. The S-Net and the P-Net are able to learn discriminative features between the sketches and the photos, regardless of the appearance gap by introducing the concept of elastic learning. In other words, to extract the most important features from the input, the network needs to learn the relevant features along with the irrelevant ones. To do so, higher dimension layers are used after the three 512 layers from VGG-FaceNet. Since the network learns representative features, we decrease the dimension of the layers to produce the most representative features. In addition, contrastive loss is employed to discover the coherent visual structures between sketch and photo. Experimental results on E-PRIP face sketch dataset indicate that the proposed network significantly outperforms the state-of-the-art composite sketch identification methods.

The computer generation of poetry has been studied for more than a decade. Generating poetry on a human level is still a great challenge for the computer-generation process. We present a novel Transformer-XL based on a classical Chinese poetry model that employs a multi-head self-attention mechanism to capture the deeper multiple relationships among Chinese characters. Furthermore, we utilized the segment-level recurrence mechanism to learn longer-term dependency and overcome the context fragmentation problem. To automatically assess the quality of the generated poems, we also built a novel automatic evaluation model that contains a BERT-based module for checking the fluency of sentences and a tone-checker module to evaluate the tone pattern of poems. The poems generated using our model obtained an average score of 9.7 for fluency and 10.0 for tone pattern. Moreover, we visualized the attention mechanism, and it showed that our model learned the tone-pattern rules. All experiment results demonstrate that our poetry generation model can generate high-quality poems.

Deep-learning-based, cross-age face recognition has improved significantly in recent years. However, when using the discriminative method, it is still challenging to extract robust age-invariant features that can reduce the interference caused by age. In this paper, we propose a novel, effective, attention-based feature decomposition model, the age-invariant features extraction network, which can learn more discriminative feature representations and reduce the disturbance caused by aging. Our method uses an efficient channel attention block-based feature decomposition module to extract age-independent identity features from facial representations. Our end-to-end framework learns the age-invariant features directly, which is more convenient and can greatly reduce training complexity compared with existing multi-stage training methods. In addition, we propose a direct sum loss function to reduce the interference of age-related features. Our method achieves a comparable and stable performance. Experimental results demonstrate superior performance on four benchmarked datasets over the state-of-the-art. We obtain the relative improvements of 0.06%, 0.2%, and 2.2% on the cross-age datasets CACD-VS, AgeDB, and CALFW, respectively, and a relative 0.03% improvement on a general dataset LFW.

Research on facial recognition has recently been flourishing, which has led to the introduction of many robust methods. However, since the worldwide outbreak of COVID-19, people have had to regularly wear facial masks, thus making existing face recognition methods less reliable. Although normal face recognition methods are nearly complete, masked face recognition (MFR)—which refers to recognizing the identity of an individual when people wear a facial mask—remains the most challenging topic in this area. To overcome the difficulties involved in MFR, a novel deep learning method based on the convolutional block attention module (CBAM) and angular margin ArcFace loss is proposed. In the method, CBAM is integrated with convolutional neural networks (CNNs) to extract the input image feature maps, particularly of the region around the eyes. Meanwhile, ArcFace is used as a training loss function to optimize the feature embedding and enhance the discriminative feature for MFR. Because of the insufficient availability of masked face images for model training, this study used the data augmentation method to generate masked face images from a common face recognition dataset. The proposed method was evaluated using the well-known masked image version of LFW, AgeDB-30, CFP-FP, and real mask image MFR2 verification datasets. A variety of experiments confirmed that the proposed method offers improvements for MFR compared to the current state-of-the-art methods.

Human action monitoring can be advantageous to remotely monitor the status of patients or elderly person for intelligent healthcare. Human action recognition enables efficient and accurate monitoring of human behaviors, which can exhibit multifaceted complexity attributed to disparities in viewpoints, personality, resolution and motion speed of individuals, etc. The spatial-temporal information plays an important role in the human action recognition. In this paper, we proposed a novel deep learning architecture named as recurrent 3D convolutional neural network (R3D) to extract effective and discriminative spatial-temporal features to be used for action recognition, which enables the capturing of long-range temporal information by aggregating the 3D convolutional network entries to serve as an input to the LSTM (Long Short-Term Memory) architecture. The 3D convolutional network and LSTM are two effective methods for extracting the temporal information. The proposed R3D network integrated these two methods by sharing a shared 3D convolutional network in sliding windows on video streaming to capturing short-term spatial-temporal features into the LSTM. The output features of LSTM encapsulate the longrange spatial-temporal information representing high-level abstraction of the human actions. The proposed algorithm is compared to traditional and the-state-of-the-art and deep learning algorithms. The experimental results demonstrated the effectiveness of the proposed system, which can be used as smart monitoring for remote healthcare.

Person re-identification (re-ID) has been gaining in popularity in the research community owing to its numerous applications and growing importance in the surveillance industry. Recent methods often employ partial features for person re-ID and offer fine-grained information beneficial for person retrieval. In this paper, we focus on learning improved partial discriminative features using a deep convolutional neural architecture, which includes a pyramid spatial pooling module for efficient person feature representation. Furthermore, we propose a multi-task convolutional network that learns both personal attributes and identities in an end-to-end framework. Our approach incorporates partial features and global features for identity and attribute prediction, respectively. Experiments on several large-scale person re-ID benchmark data sets demonstrate the accuracy of our approach. For example, we report rank-1 accuracies of 85.37% (+3.47 %) and 92.81% (+0.51 %) on the DukeMTMC re-ID and Market-1501 data sets, respectively. The proposed method shows encouraging improvements compared with the state-of-the-art methods.

Recent advancements in human-computer interaction research have led to the possibility of emotional communication via brain-computer interface systems for patients with neuropsychiatric disorders or disabilities. In this paper, we efficiently recognize emotional states by analyzing the features of electroencephalography (EEG) signals, which are generated from EEG sensors that noninvasively measure the electrical activity of neurons inside the human brain, and select the optimal combination of these features for recognition. In this paper, the scalp EEG data of 21 healthy subjects (12-14 years old) were recorded using a 14-channel EEG machine while the subjects watched images with four types of emotional stimuli (happy, calm, sad, or scared). After preprocessing, the Hjorth parameters (activity, mobility, and complexity) were used to measure the signal activity of the time series data. We selected the optimal EEG features using a balanced one-way ANOVA after calculating the Hjorth parameters for different frequency ranges. Features selected by this statistical method outperformed univariate and multivariate features. The optimal features were further processed for emotion classification using support vector machine, k-nearest neighbor, linear discriminant analysis, Naive Bayes, random forest, deep learning, and four ensembles methods (bagging, boosting, stacking, and voting). The results show that the proposed method substantially improves the emotion recognition rate with respect to the commonly used spectral power band method.

Semantic segmentation, which refers to pixel-wise classification of an image, is a fundamental topic in computer vision owing to its growing importance in the robot vision and autonomous driving sectors. It provides rich information about objects in the scene such as object boundary, category, and location. Recent methods for semantic segmentation often employ an encoder-decoder structure using deep convolutional neural networks. The encoder part extracts features of the image using several filters and pooling operations, whereas the decoder part gradually recovers the low-resolution feature maps of the encoder into a full input resolution feature map for pixel-wise prediction. However, the encoder-decoder variants for semantic segmentation suffer from severe spatial information loss, caused by pooling operations or stepwise convolutions, and does not consider the context in the scene. In this paper, we propose a novel dense upsampling convolution method based on a guided filter to effectively preserve the spatial information of the image in the network. We further propose a novel local context convolution method that not only covers larger-scale objects in the scene but covers them densely for precise object boundary delineation. Theoretical analyses and experimental results on several benchmark datasets verify the effectiveness of our method. Qualitatively, our approach delineates object boundaries at a level of accuracy that is beyond the current excellent methods. Quantitatively, we report a new record of 82.86% and 81.62% of pixel accuracy on ADE20K and Pascal-Context benchmark datasets, respectively. In comparison with the state-of-the-art methods, the proposed method offers promising improvements.

Human pose estimation is a problem that continues to be one of the greatest challenges in the field of computer vision. While the stacked structure of an hourglass network has enabled substantial progress in human pose estimation and key-point detection areas, it is largely used as a backbone network. However, it also requires a relatively large number of parameters and high computational capacity due to the characteristics of its stacked structure. Accordingly, the present work proposes a more lightweight version of the hourglass network, which also improves the human pose estimation performance. The new hourglass network architecture utilizes several additional skip connections, which improve performance with minimal modifications while still maintaining the number of parameters in the network. Additionally, the size of the convolutional receptive field has a decisive effect in learning to detect features of the full human body. Therefore, we propose a multidilated light residual block, which expands the convolutional receptive field while also reducing the computational load. The proposed residual block is also invariant in scale when using multiple dilations. The well-known MPII and LSP human pose datasets were used to evaluate the performance using the proposed method. A variety of experiments were conducted that confirm that our method is more efficient compared to current state-of-the-art hourglass weight-reduction methods.

Point cloud registration is a key problem in computer vision applications and involves finding a rigid transform from a point cloud into another such that they align together. The iterative closest point (ICP) method is a simple and effective solution that converges to a local optimum. However, despite the fact that point cloud registration or alignment is addressed in learning-based methods, such as PointNetLK, they do not offer good generalizability for point clouds. In this stud, we proposed a learning-based approach that addressed existing problems, such as finding local optima for ICP and achieving minimum generalizability. The proposed model consisted of three main parts: an encoding network, an auxiliary module that weighed the contribution of each input point cloud, and feature alignment to achieve the final transform. The proposed architecture offered greater generalization among the categories. Experiments were performed on ModelNet40 with different configurations and the results indicated that the proposed approach significantly outperformed the state-of-the-art point cloud alignment methods.

Skeleton-based action recognition is a widely used task in action related research because of its clear features and the invariance of human appearances and illumination. Furthermore, it can also effectively improve the robustness of the action recognition. Graph convolutional networks have been implemented on those skeletal data to recognize actions. Recent studies have shown that the graph convolutional neural network works well in the action recognition task using spatial and temporal features of skeleton data. The prevalent methods to extract the spatial and temporal features purely rely on a deep network to learn from primitive 3D position. In this paper, we propose a novel action recognition method applying high-order spatial and temporal features from skeleton data, such as velocity features, acceleration features, and relative distance between 3D joints. Meanwhile, a method of multi-stream feature fusion is adopted to fuse these high-order features we proposed. Extensive experiments on Two large and challenging datasets, NTU-RGBD and NTU-RGBD-120, indicate that our model achieves the state-of-the-art performance.

Clinical and cognitive symptoms domain-based subtyping in schizophrenia (Sz) has been critiqued due to the lack of neurobiological correlates and heterogeneity in symptom scores. We, therefore, present a novel data-driven framework using biclustered independent component analysis to detect subtypes from the reliable and stable gray matter concentration (GMC) of patients with Sz. The developed methodology consists of the following steps: source-based morphometry (SBM) decomposition, selection and sorting of two component loadings, subtype component reconstruction using group information-guided ICA (GIG-ICA). This framework was applied to the top two group discriminative components namely the insula/superior temporal gyrus/inferior frontal gyrus (I-STG-IFG component) and the superior frontal gyrus/middle frontal gyrus/medial frontal gyrus (SFG-MiFG-MFG component) from our previous SBM study, which showed diagnostic group difference and had the highest effect sizes. The aggregated multisite dataset consisted of 382 patients with Sz regressed of age, gender, and site voxelwise. We observed two subtypes (i.e., two different subsets of subjects) each heavily weighted on these two components, respectively. These subsets of subjects were characterized by significant differences in positive and negative syndrome scale (PANSS) positive clinical symptoms (p = 0.005). We also observed an overlapping subtype weighing heavily on both of these components. The PANSS general clinical symptom of this subtype was trend level correlated with the loading coefficients of the SFG-MiFG-MFG component (r = 0.25; p = 0.07). The reconstructed subtype-specific component using GIG-ICA showed variations in voxel regions, when compared to the group component. We observed deviations from mean GMC along with conjunction of features from two components characterizing each deciphered subtype. These inherent variations in GMC among patients with Sz could possibly indicate the need for personalized treatment and targeted drug development.

We propose Identical-pair Adversarial Networks (iPANs) to solve image-to-image translation problems, such as aerial-to-map, edge-to-photo, de-raining, and night-to-daytime. Our iPANs rely mainly on the effectiveness of adversarial loss function and its network architectures. Our iPANs consist of two main networks, an image transformation network T and a discriminative network D. We use U-NET for the transformation network T and a perceptual similarity network, which has two streams of VGG16 that share the same weights for network D. Our proposed adversarial losses play a minimax game against each other based on a real identical-pair and a fake identical-pair distinguished by the discriminative network D; e.g. a discriminative network D considers two inputs as a real pair only when they are identical, otherwise a fake pair. Meanwhile, the transformation network T tries to persuade the discriminator network D that the fake pair is a real pair. We experimented on several problems of image-to-image translation and achieved results that are comparable to those of some existing approaches, such as pix2pix, and PAN.

The exemplar-based method is most frequently used in face sketch synthesis because of its efficiency in representing the nonlinear mapping between face photos and sketches. However, the sketches synthesized by existing exemplar-based methods suffer from block artifacts and blur effects. In addition, most exemplar-based methods ignore the training sketches in the weight representation process. To improve synthesis performance, a novel joint training model is proposed in this paper, taking sketches into consideration. First, we construct the joint training photo and sketch by concatenating the original photo and its sketch with a high-pass filtered image of their corresponding sketch. Then, an offline random sampling strategy is adopted for each test photo patch to select the joint training photo and sketch patches in the neighboring region. Finally, a novel locality constraint is designed to calculate the reconstruction weight, allowing the synthesized sketches to have more detailed information. Extensive experimental results on public datasets show the superiority of the proposed joint training model, both from subjective perceptual and the FaceNet-based face recognition objective evaluation, compared to existing state-of-the-art sketch synthesis methods.

Make and model recognition (MMR) of vehicles plays an important role in automatic vision-based systems. This paper proposes a novel deep learning approach for MMR using the SqueezeNet architecture. The frontal views of vehicle images are first extracted and fed into a deep network for training and testing. The SqueezeNet architecture with bypass connections between the Fire modules, a variant of the vanilla SqueezeNet, is employed for this study, which makes our MMR system more efficient. The experimental results on our collected large-scale vehicle datasets indicate that the proposed model achieves 96.3% recognition rate at the rank-1 level with an economical time slice of 108.8 ms. For inference tasks, the deployed deep model requires less than 5 MB of space and thus has a great viability in real-time applications.

We investigated the signals regularity of electroencephalography (EEG) channels separately and determined the energy of selected frequency waves, such as, δ , θ , α , β , and γ . The goal of this research is to identify the prominent frequency band from selected frequencies. We recorded the EEG signal data of 30 controlled subjects with 18 EEG channels. These subjects are all males with an average age of 24 years. Emotional stimuli related to different emotions were presented to each of selected candidates. EEG data were extracted and further processed for artifact removal, filtering, epoch selection and averaging of the signals. We designed and tested our method for exploring the frequency waves of all EEG channels. We also employed the Hjorth parameters to measure the signal regularity in time and frequency domain. The detailed physiological response of human subjects is also presented in this paper. Our results showed that the energy level of delta wave is mostly high in all cases.

Human computer interaction is a growing field in terms of helping people in their daily life to improve their living. Especially, people with some disability may need an interface which is more appropriate and compatible with their needs. Our research is focused on similar kinds of problems, such as students with some mental disorder or mood disruption problems. To improve their learning process, an intelligent emotion recognition system is essential which has an ability to recognize the current emotional state of the brain. Nowadays, in special schools, instructors are commonly use some conventional methods for managing special students for educational purposes. In this paper, we proposed a novel computer aided method for instructors at special schools where they can teach special students with the support of our system using wearable technologies.

Vehicle analysis involves license-plate recognition (LPR), vehicle-type classification (VTC), and vehicle make and model recognition (MMR). Among these tasks, MMR plays an important complementary role in respect to LPR. In this paper, we propose a novel framework for MMR using local tiled deep networks. The frontal views of vehicle images are first extracted and fed into the local tiled deep networks for training and testing. A local tiled convolutional neural network (LTCNN) is proposed to alter the weight sharing scheme of CNN with local tiled structure. The LTCNN unties the weights of adjacent units and then ties the units k steps from each other within a local map. This architecture provides the translational, rotational, and scale invariance as well as locality. In addition, to further deal with the colour and illumination variation, we applied the histogram oriented gradient (HOG) to the frontal view of images prior to the LTCNN. The experimental results show that our LTCNN framework achieved a 98% accuracy rate in terms of vehicle MMR.

The annotation of functional neuroimaging results for data sharing and reuse is particularly challenging, due to the diversity of terminologies of neuroanatomical structures and cortical parcellation schemes. To address this challenge, we extended the Foundational Model of Anatomy Ontology (FMA) to include cytoarchitectural, Brodmann area labels, and a morphological cortical labeling scheme (e.g., the part of Brodmann area 6 in the left precentral gyrus). This representation was also used to augment the neuroanatomical axis of RadLex, the ontology for clinical imaging. The resulting neuroanatomical ontology contains explicit relationships indicating which brain regions are “part of” which other regions, across cytoarchitectural and morphological labeling schemas. We annotated a large functional neuroimaging dataset with terms from the ontology and applied a reasoning engine to analyze this dataset in conjunction with the ontology, and achieved successful inferences from the most specific level (e.g., how many subjects showed activation in a sub-part of the middle frontal gyrus) to more general (how many activations were found in areas connected via a known white matter tract?). In summary, we have produced a neuroanatomical ontology that harmonizes several different terminologies of neuroanatomical structures and cortical parcellation schemes. This neuranatomical ontology is publicly available as a view of FMA at the Bioportal website at http://rest.bioontology.org/bioportal/ontologies/download/10005. The ontological encoding of anatomic knowledge can be exploited by computer reasoning engines to make inferences about neuroanatomical relationships described in imaging datasets using different terminologies. This approach could ultimately enable knowledge discovery from large, distributed fMRI studies or medical record mining.