Search

In recent years, the contributions of deep learning have had a phenomenal impact on electroencephalography-based brain-computer interfaces. While the decoding accuracy of electroencephalography signals has continued to increase, the process has caused deep learning models to continuously expand in terms of size and computational resource requirements. However, due to their increased size and computational requirements, it has become difficult to embed, store, and execute deep learning models for artificial intelligence of things, cloud-based, or edge devices used in rehabilitation. Hence, this article proposes a novel deep learning-based lightweight model based on attention-inception convolutional neural network and long- short-term memory. The proposed model achieves excellent accuracy on public competition datasets while requiring few parameters and low computational time. Using the BCI competition IV 2a dataset and the high gamma dataset, the proposed model achieved 82.8% and 97.1% accuracies, respectively. [ABSTRACT FROM AUTHOR]

Real-time pedestrian detection is an important task for unmanned driving systems and video surveillance. The existing pedestrian detection methods oftenwork at lowspeed and also fail to detect smaller and densely distributed pedestrians by losing some of their detection accuracy in such cases. Therefore, the proposed algorithm YOLOv2 ("YOU ONLY LOOK ONCE Version 2")-based pedestrian detection (referred to as YOLOv2PD) would be more suitable for detecting smaller and densely distributed pedestrians in real-time complex road scenes. The proposed YOLOv2PD algorithm adopts a Multi-layer Feature Fusion (MLFF) strategy, which helps to improve the model's feature extraction ability. In addition, one repeated convolution layer is removed from the final layer, which in turn reduces the computational complexity without losing any detection accuracy. The proposed algorithm applies the K-means clustering method on the Pascal Voc-2007 + 2012 pedestrian dataset before training to find the optimal anchor boxes. Both the proposed network structure and the loss function are improved to make the model more accurate and faster while detecting smaller pedestrians. Experimental results show that, at 544×544 image resolution, the proposed model achieves 80.7% average precision (AP), which is 2.1% higher than the YOLOv2 Model on the Pascal Voc-2007+2012 pedestrian dataset. Besides, based on the experimental results, the proposed model YOLOv2PD achieves a good trade-off balance between detection accuracy and real-time speed when evaluated on INRIA and Caltech test pedestrian datasets and achieves state-of-the-art detection results. [ABSTRACT FROM AUTHOR]

An electroencephalogram (EEG)-based remote pathology detection system is proposed in this study. The system uses a deep convolutional network consisting of 1D and 2D convolutions. Features from different convolutional layers are fused using a fusion network. Various types of networks are investigated; the types include a multilayer perceptron (MLP) with a varying number of hidden layers, and an autoencoder. Experiments are done using a publicly available EEG signal database that contains two classes: normal and abnormal. The experimental results demonstrate that the proposed system achieves greater than 89% accuracy using the convolutional network followed by the MLP with two hidden layers. The proposed system is also evaluated in a cloud-based framework, and its performance is found to be comparable with the performance obtained using only a local server. [ABSTRACT FROM AUTHOR]

Accurate and efficient text recognition can enhance the accuracy and time efficiency of human–computer interaction and information exchange in noise Metaverse scenarios. For example, it can improve the safety of digital twin-based intelligent transportation and the intelligence of Non-Player Characters. Robust features play a vital role in the performance of scene text recognition models in noise Metaverse situations. To extract robust features, and improve the accuracy and time efficiency of scene text recognition, we propose a Chara cter level text recognition model in Meta verse applications, called MetaChara. It contains two main components: a lightweight text feature extraction module (LightFeature), and a robust character recognition module (RobChara). LightFeature leverages the advantage of global feature aggregation in the primitive representation learning network to handle irregular text images.RobChara incorporates the capability of contrastive learning from the momentum contrast method, improving the robustness of feature extraction in MetaChara. It structures a feature queue for organized storage. By optimizing the similarity of intra-character features and maximizing inter-character differences, it makes the model better adapted to scene text recognition tasks in Metaverse. Experiment results demonstrate that MetaChara is light with 29.14 million parameters and time efficient with an average recognition speed of 1.73 s. It also achieves excellent performance in terms of FLoating-point Operations (FLOPs), registering only 59.60 billion times for each operation. MetaChara achieves an average accuracy of 0.969 for character recognition. We present a case study where MetaChara quickly and accurately recognizes scene texts within the context of autonomous driving in the Metaverse. This demonstrates how MetaChara enhances safety and improves time efficiency for intelligent transportation systems. • Introducing MetaChara, a contrastive learning model for scene text in the Metaverse. • MetaChara blends GCN with MoCo for faster and accurate text recognition. • Our vision strategy boosts noise-free image expansion for character strength. • MetaChara is fast and light with high accuracy in noisy Metaverse scenarios. [ABSTRACT FROM AUTHOR]

Automatically recognizing emotions of patients can be a good facilitator of a connected healthcare framework. It can give automatic feedback to the stakeholders of the healthcare industry about patients' states and satisfaction levels. In this article, we propose an automatic audio-visual emotion recognition system in a connected healthcare framework. The system uses a 2D CNN model for the speech modality and a 3D CNN model for the visual modality. For the speech signal, preprocessing is done to extract the PS-PA feature vector. The features from the two CNN models are blended by two ELM networks. The first ELM is trained with gender-specific data, while the other one is trained with emotion-specific data. The proposed system is evaluated using three databases, and the experiments prove the success of the system. In the healthcare framework, we use edge computing prior to intensive-processing cloud computing. In the edge computing, we realize edge caching, which can store the CNN model parameters and thereby perform the testing fast. [ABSTRACT FROM AUTHOR]

Fruit classification is an important task in many industrial applications. A fruit classification system may be used to help a supermarket cashier identify the fruit species and prices. It may also be used to help people decide whether specific fruit species meet their dietary requirements. In this paper, we propose an efficient framework for fruit classification using deep learning. More specifically, the framework is based on two different deep learning architectures. The first is a proposed light model of six convolutional neural network layers, whereas the second is a fine-tuned visual geometry group-16 pretrained deep learning model. Two color image datasets, one of which is publicly available, are used to evaluate the proposed framework. The first dataset (dataset 1) consists of clear fruit images, whereas the second dataset (dataset 2) contains fruit images that are challenging to classify. Classification accuracies of 99.49% and 99.75% were achieved on dataset 1 for the first and second models, respectively. On dataset 2, the first and second models obtained accuracies of 85.43% and 96.75%, respectively. [ABSTRACT FROM AUTHOR]

A new speech feature extraction technique called moving average multi directional local features (MA-MDLF) is presented in this paper. This method is based on linear regression (LR) and moving average (MA) in the time–frequency plane. Three-point LR is taken along time axis and frequency axis, and 3 points MA is taken along 45° and 135° in the time–frequency plane. The LR captures the voice onset\offset, formant contour, while the moving average captures the dynamics on time–frequency axes which can be seen as voiceprints. The MA-MDLF performance is compared to commonly used speech features in speaker recognition. The comparison is performed in a speaker recognition system (SRS) for three different conditions, namely clean speech, mobile speech, and cross channel. MA-MDLF has shown better performance than the baseline MFCC, RASTA-PLP and LPCC. In clean and mobile speech, MA-MDLF feature performs the best and also in the cross channel task MA-MDLF performed excellent. We also evaluated the MA-MDLF using three speech databases, namely KSU, LDC Babylon and TIMITdatabases, and found that MA-MDLF outperformed the other commonly used features with speech from all the three databases. The first and second databases are for Arabic speech while third is for English speech. [ABSTRACT FROM AUTHOR]

With the increased population in urban areas, it has become a challenge to a service provider to manage big data coming from various users and devices. One of the fastest growing devices used by the citizens of urban areas is the smartphone. Today we can find almost nobody without at least one smartphone in their possession. While talking through a smartphone, surrounding environmental noises are added to the speech of the talker. Sometimes these noises are very annoying to the listeners. In this article, we propose a communication framework for urban mobile big data integrating an urban environment classification system. The proposed system utilizes a deep learning approach to classify environmental noises so that an appropriate noise cancellation algorithm can be used to reduce the effect of noise while having a conversation through smartphones. The proposed framework uses mobile edge computing technology to provide low-latency and efficient transmission. Experimental results demonstrate that the proposed system is very efficient in classifying environmental noises. [ABSTRACT FROM AUTHOR]

With the invention of new communication technologies, new features and facilities are provided in a smart healthcare framework. The features and facilities aim to provide a seamless, easy-to-use, accurate, and real-time healthcare service to clients. As health is a sensitive issue, it should be taken care of with utmost security and caution. This article proposes a new medical image forgery detection system for the healthcare framework to verify that images related to healthcare are not changed or altered. The system works on a noise map of an image, applies a multi-resolution regression filter on the noise map, and feeds the output to support-vector-machine-based and extreme-learning-based classifiers. The noise map is created in an edge computing resource, while the filtering and classification are done in a core cloud computing resource. In this way, the system works seamlessly and in real time. The bandwidth requirement of the proposed system is also reasonable. [ABSTRACT FROM AUTHOR]

Local features for any pattern recognition system are based on the information extracted locally. In this paper, a local feature extraction technique was developed. This feature was extracted in the time–frequency plain by taking the moving average on the diagonal directions of the time–frequency plane. This feature captured the time–frequency events producing a unique pattern for each speaker that can be viewed as a voice print of the speaker. Hence, we referred to this technique as voice print-based local feature. The proposed feature was compared to other features including mel-frequency cepstral coefficient (MFCC) for speaker recognition using two different databases. One of the databases used in the comparison is a subset of an LDC database that consisted of two short sentences uttered by 182 speakers. The proposed feature attained 98.35% recognition rate compared to 96.7% for MFCC using the LDC subset. [ABSTRACT FROM PUBLISHER]

Most existing approaches on aspect-opinion mining focus on the text domain and cannot be applied to social media where the aspects are essentially multimodal and the opinions depend on the specific aspects. To address the problem of multimodal aspect-opinion mining for entities by leveraging multiple cross-collection sources in social media, in this paper we propose a multimodal aspect-opinion model (mmAOM) considering both user-generated photos and textual documents to simultaneously capture correlations between textual and visual modalities, as well as associations between aspects and opinions . By identifying the aspects and the corresponding opinions related to entities, we apply the mmAOM to entity association visualization and multimodal aspect-opinion retrieval. We have conducted extensive experiments on real-world datasets of entities including Flickr photos, Tripadvisor reviews, and news articles. Qualitative and quantitative evaluation results have validated the effectiveness of the multimodal aspect-opinion mining model, and demonstrated the utility of the derived aspects and opinions from mmAOM in applications of entity association visualization and aspect-opinion retrieval. [ABSTRACT FROM PUBLISHER]

Highlights: [•] MPEG-7 audio descriptors are used for voice pathology detection/classification. [•] Fisher ratio is applied to find the most discriminative features. [•] Using only top five features the detection accuracy reached over 99%. [•] Those five features obtained over 93% accuracy for binary classification. [•] Achieved the best detection rate so far in MEEI database (subset). [ABSTRACT FROM AUTHOR]

Copy-move image forgery detection has recently become a very active research topic in blind image forensics. In copy-move image forgery, a region from some image location is copied and pasted to a different location of the same image. Typically, post-processing is applied to better hide the forgery. Using keypoint-based features, such as SIFT features, for detecting copy-move image forgeries has produced promising results. The main idea is detecting duplicated regions in an image by exploiting the similarity between keypoint-based features in these regions. In this paper, we have adopted keypoint-based features for copy-move image forgery detection; however, our emphasis is on accurate and robust localization of duplicated regions. In this context, we are interested in estimating the transformation (e.g., affine) between the copied and pasted regions more accurately as well as extracting these regions as robustly by reducing the number of false positives and negatives. To address these issues, we propose using a more powerful set of keypoint-based features, called MIFT, which shares the properties of SIFT features but also are invariant to mirror reflection transformations. Moreover, we propose refining the affine transformation using an iterative scheme which improves the estimation of the affine transformation parameters by incrementally finding additional keypoint matches. To reduce false positives and negatives when extracting the copied and pasted regions, we propose using 'dense' MIFT features, instead of standard pixel correlation, along with hysteresis thresholding and morphological operations. The proposed approach has been evaluated and compared with competitive approaches through a comprehensive set of experiments using a large dataset of real images (i.e., CASIA v2.0). Our results indicate that our method can detect duplicated regions in copy-move image forgery with higher accuracy, especially when the size of the duplicated region is small. [ABSTRACT FROM AUTHOR]

Gender recognition from facial images plays an important role in biometric applications. Employing Dyadic wavelet Transform (DyWT) and Local Binary Pattern (LBP), we propose a new feature descriptor DyWT-LBP for gender recognition. DyWT is a multi-scale image transformation technique that decomposes an image into a number of sub-bands which separate the features at different scales. DyWT is a kind of translation invariant wavelet transform that has a better potential for detection than Discrete Wavelet Transform (DWT). On the other hand, LBP is a texture descriptor and is known to be the best for representing texture micro-patterns, which play a key role in the discrimination of different objects in an image. For DyWT, we used spline dyadic wavelets (SDW). There exist many types of SDW; we investigated a number of SDWs for finding the best SDW for gender recognition. The dimension of the feature space generated by DyWT-LBP descriptor becomes excessively high. To tackle this problem, we apply a feature subset selection (FSS) technique that not only reduces the number of features significantly but also improves the recognition accuracy. Through a large number of experiments performed on FERET and Multi-PIE databases, we report for DyWT-LBP descriptor the parameter settings, which result in the best accuracy. The proposed system outperforms the stat of the art gender recognition approaches; it achieved a recognition rate of 99.25% and 99.09% on FERET and Multi-PIE databases, respectively. [ABSTRACT FROM AUTHOR]

Automatic high-level feature extraction has become a possibility with the advancement of deep learning, and it has been used to optimize efficiency. Recently, classification methods for Convolutional Neural Network (CNN)-based electroencephalography (EEG) motor imagery have been proposed, and have achieved reasonably high classification accuracy. These approaches, however, use the CNN single convolution scale, whereas the best convolution scale varies from subject to subject. This limits the precision of classification. This paper proposes multibranch CNN models to address this issue by effectively extracting the spatial and temporal features from raw EEG data, where the branches correspond to different filter kernel sizes. The proposed method's promising performance is demonstrated by experimental results on two public datasets, the BCI Competition IV 2a dataset and the High Gamma Dataset (HGD). The results of the technique show a 9.61% improvement in the classification accuracy of multibranch EEGNet (MBEEGNet) from the fixed one-branch EEGNet model, and 2.95% from the variable EEGNet model. In addition, the multibranch ShallowConvNet (MBShallowConvNet) improved the accuracy of a single-scale network by 6.84%. The proposed models outperformed other state-of-the-art EEG motor imagery classification methods. [ABSTRACT FROM AUTHOR]

This paper describes an evaluation of Inhibition/Enhancement (In/En) network for robust automatic speech recognition (ASR). In distinctive phonetic features (DPFs) based speech recognition using neural network, In/En network is needed to discriminate whether the DPFs dynamic patterns of trajectories are convex or concave. The network is used to achieve categorical DPFs movement by enhancing DPFs peak patterns (convex patterns) and inhibiting DPFs dip patterns (concave patterns). We have analyzed the effectiveness of In/En algorithm by incorporating it into a system which consists of three stages: a) Multilayer Neural Networks (MLNs), b) In/En Network and c) Gram-Schmidt (GS) orthogonalization. From the experiments using Japanese Newspaper Article Sentences (JNAS) database in clean and noisy acoustic environments, it is observed that the In/En network plays a significant role on the improvement of phoneme recognition performance. Moreover, In/En network reduces required number of mixture components in Hidden Markov Models (HMMs). [ABSTRACT FROM AUTHOR]

This paper presents a method for articulatory features (AFs) extraction with lower cost. The method comprises two multilayer neural networks (MLNs). The first MLN, MLNLF-DPF, maps local features (LFs) of an input speech signal into discrete AFs and the second MLN, MLNDyn, restricts dynamics of outputted AFs by the MLNLF-DPF. In the experiments on Tohokudai Isolated Spoken-Word Database in clean acoustic environment, the proposed recognizer was found to provide a higher word correct rate (WCR) as well as word accuracy (WA) with fewer mixture components in hidden Markov models (HMMs) in comparison with the other methods investigated. [ABSTRACT FROM AUTHOR]

Infrared and visible image fusion aims to generate informative images by leveraging the distinctive strengths of infrared and visible modalities. These fused images play a crucial role in subsequent downstream tasks, including object detection, recognition, and segmentation. However, complementary information is often difficult to extract. Existing generative adversarial network-based methods generate fused images by modifying the distribution of source images' features to preserve instances and texture details in both infrared and visible images. Nevertheless, these approaches may result in a degradation of the fused image quality when the original image quality is low. Considering the balance of information from different modalities can improve the quality of the fused image. Hence, we introduce CABnet, a Channel Attention dual adversarial Balancing network. CABnet incorporates a channel attention mechanism to capture crucial channel features, thereby, enhancing complementary information. It also employs an adaptive factor to control the mixing distribution of infrared and visible images, which ensures the preservation of instances and texture details during the adversarial process. To enhance efficiency and reduce reliance on manual labeling, our training process adopts a semi-supervised learning strategy. Through qualitative and quantitative experiments across multiple datasets, CABnet surpasses existing state-of-the-art methods in fusion performance, notably achieving a 51.3% enhancement in signal to noise ratio and a 13.4% improvement in standard deviation. • Enhancing feature extraction capability and eliminating redundant features by channel attention module. • Controlling the mixed distribution by adaptive balance module. • Converging faster by an enhanced loss function. [ABSTRACT FROM AUTHOR]

Human activity recognition (HAR) remains a challenging yet crucial problem to address in computer vision. HAR is primarily intended to be used with other technologies, such as the Internet of Things, to assist in healthcare and eldercare. With the development of deep learning, automatic high-level feature extraction has become a possibility and has been used to optimize HAR performance. Furthermore, deep-learning techniques have been applied in various fields for sensor-based HAR. This study introduces a new methodology using convolution neural networks (CNN) with varying kernel dimensions along with bi-directional long short-term memory (BiLSTM) to capture features at various resolutions. The novelty of this research lies in the effective selection of the optimal video representation and in the effective extraction of spatial and temporal features from sensor data using traditional CNN and BiLSTM. Wireless sensor data mining (WISDM) and UCI datasets are used for this proposed methodology in which data are collected through diverse methods, including accelerometers, sensors, and gyroscopes. The results indicate that the proposed scheme is efficient in improving HAR. It was thus found that unlike other available methods, the proposed method improved accuracy, attaining a higher score in the WISDM dataset compared to the UCI dataset (98.53% vs. 97.05%). [ABSTRACT FROM AUTHOR]

• Bio-Inspired novel attention-based inception architecture is proposed that can adapt to different types of spatial contexts using convolution filters of different sizes. The characteristics of each dataset are unique, hence the attention mechanism helps focus on those features to improve classification performance. • The shallow inception model is designed with a two-layer attention mechanism with fewer layers but with a large number of convolution filters that can address the overfitting problem caused by small dataset sizes. • LSTM-based recurrent neural network (RNN) module is proposed to extract temporal features after the inception module is applied. • The proposed model is lightweight with fewer parameters and has less processing time. • The proposed model achieves good performance for both dynamic and static signs and gestures. Bio-inspired deep learning models have revolutionized sign language classification, achieving extraordinary accuracy and human-like video understanding. Recognition and classification of sign language videos in real-time are challenging because the duration and speed of each sign vary for different subjects, the background of videos is dynamic in most cases, and the classification result should be produced in real-time. This study proposes a model based on a convolution neural network (CNN) Inception model with an attention mechanism for extracting spatial features and Bi-LSTM (long short-term memory) for temporal feature extraction. The proposed model is tested on datasets with highly variable characteristics such as different clothing, variable lighting, and variable distance from the camera. Real-time classification achieves significant early detections while achieving performance comparable to the offline operation. The proposed model has fewer parameters, fewer deep learning layers, and requires significantly less processing time than state-of-the-art models. The Inception model with an attention mechanism with two attention blocks [Display omitted] [ABSTRACT FROM AUTHOR]