Search

Mobile Edge Computing (MEC) has emerged as a promising paradigm catering to overwhelming explosions of mobile applications, by offloading the compute-intensive tasks to an MEC network for processing. The surging of deep learning brings new vigor and vitality to shape the prospect of intelligent Internet of Things (IoT), and edge intelligence arises to provision real-time deep neural network (DNN) inference services for users. In this paper, we study a novel delay-aware DNN inference throughput maximization problem by accelerating each DNN inference through jointly exploring DNN partitioning and multi-thread parallelism. Specifically, we consider the problem under both offline and online request arrival settings: a set of DNN inference requests is given in advance, and a sequence of DNN inference requests arrives one by one without the knowledge of future arrivals, respectively. We first show that the defined problems are NP-hard. We then devise a novel constant approximation algorithm for the problem under the offline setting. We also propose an online algorithm with a provable competitive ratio for the problem under the online setting. We finally evaluate the performance of the proposed algorithms through experimental simulations. Experimental results demonstrate that the proposed algorithms are promising.

Today there is a rapid development taking place in phenotyping of plants using non-destructive image based machine vision techniques. Machine vision based plant phenotyping ranges from single plant trait estimation to broad assessment of crop canopy for thousands of plants in the field. Plant phenotyping systems either use single imaging method or integrative approach signifying simultaneous use of some of the imaging techniques like visible red, green and blue (RGB) imaging, thermal imaging, chlorophyll fluorescence imaging (CFIM), hyperspectral imaging, 3-dimensional (3-D) imaging or high resolution volumetric imaging. This paper provides an overview of imaging techniques and their applications in the field of plant phenotyping. This paper presents a comprehensive survey on recent machine vision methods for plant trait estimation and classification. In this paper, information about publicly available datasets is provided for uniform comparison among the state-of-the-art phenotyping methods. This paper also presents future research directions related to the use of deep learning based machine vision algorithms for structural (2-D and 3-D), physiological and temporal trait estimation, and classification studies in plants.

Projection techniques are often used to visualize high-dimensional data, allowing users to better understand the overall structure of multi-dimensional spaces on a 2D screen. Although many such methods exist, comparably little work has been done on generalizable methods of inverse-projection -- the process of mapping the projected points, or more generally, the projection space back to the original high-dimensional space. In this paper we present NNInv, a deep learning technique with the ability to approximate the inverse of any projection or mapping. NNInv learns to reconstruct high-dimensional data from any arbitrary point on a 2D projection space, giving users the ability to interact with the learned high-dimensional representation in a visual analytics system. We provide an analysis of the parameter space of NNInv, and offer guidance in selecting these parameters. We extend validation of the effectiveness of NNInv through a series of quantitative and qualitative analyses. We then demonstrate the method's utility by applying it to three visualization tasks: interactive instance interpolation, classifier agreement, and gradient visualization.

Issuing a disaster certificate, which is used to decide the contents of a victim’s support, requires accuracy and rapidity. However, in Japan at large, issuing of damage certificates has taken a long time in past earthquake disasters. Hence, the government needs a more efficient mechanism for issuing damage certificates. This study developed an estimation system of roof-damaged buildings to obtain an overview of earthquake damage based on aero-photo images using deep learning. To provide speedy estimation, this system utilized the trimming algorithm, which automatically generates roof image data using the location information of building polygons on GIS (Geographic Information System). Consequently, the proposed system can estimate, if a house is covered with a blue sheet with 97.57 % accuracy and also detect whether a house is damaged, with 93.51 % accuracy. It would therefore be worth considering the development of an image recognition model and a method of collecting aero-photo data to operate this system during a real earthquake.

Content Based Image Retrieval is ever growing technology for many applications such as medical, remote sensing, social media search engines and surveillance monitoring, etc,. Representing image content with appropriate features is tedious task using traditional low level feature extraction methods. Deep learning models achieved high precision in classification and object detection algorithms by extracting automated high level feature extraction process. This paper proposed a hybrid feature extraction technique by combining the high level features and low level features to improve the robustness of the feature vector. The proposed model used pre-trained Googlenet model as feature extractor and combined with Gabor multiscale texture features. The final feature vector will be used for retrieving the relevant image data from the large scale image dataset. It has achieved the precision of 91 percent which shows better than state of art methods.

Chatbots are becoming increasingly popular as virtual assistants, many businesses are launching If-This-Then-That programs to help them get started. Such systems, on the other hand, often generate chatbots that are stagnant and difficult to handle. The market for chatbots is increasing. Computer systems are often required, and online connections are improved by allowing users to express their needs, desires, or questions naturally and clearly by speaking, tapping, and talking. They're easy to use, perfect for people of all ages, and have the most detailed responses to questions. The majority of today's education is distributed through e-learning. A chatbot is one of the most powerful ways for students to read, as it can answer questions at any time without the need for human interaction. This chatbot is highly capable of overcoming student uncertainty without the need for human interaction. The IF THIS Then THAT paradigm is used by the bulk of chatbots. Natural language processing and deep learning technologies were used to build this chatbot. As a consequence, the chatbot will comprehend questions at a higher level.

Given the prevalence of depression worldwide and its major impact on society, several studies employed artificial intelligence modelling to automatically detect and assess depression. However, interpretation of these models and cues are rarely discussed in detail in the AI community, but have received increased attention lately. In this study, we aim to analyse the commonly selected features using a proposed framework of several feature selection methods and their effect on the classification results, which will provide an interpretation of the depression detection model. The developed framework aggregates and selects the most promising features for modelling depression detection from 38 feature selection algorithms of different categories. Using three real-world depression datasets, 902 behavioural cues were extracted from speech behaviour, speech prosody, eye movement and head pose. To verify the generalisability of the proposed framework, we applied the entire process to depression datasets individually and when combined. The results from the proposed framework showed that speech behaviour features (e.g. pauses) are the most distinctive features of the depression detection model. From the speech prosody modality, the strongest feature groups were F0, HNR, formants, and MFCC, while for the eye activity modality they were left-right eye movement and gaze direction, and for the head modality it was yaw head movement. Modelling depression detection using the selected features (even though there are only 9 features) outperformed using all features in all the individual and combined datasets. Our feature selection framework did not only provide an interpretation of the model, but was also able to produce a higher accuracy of depression detection with a small number of features in varied datasets. This could help to reduce the processing time needed to extract features and creating the model.

Stock price forecasting is a type of time series problem that forecasts the future price or status of a company on the basis of analysis of time respective values. As the price of stock or company varies with respect to time, its behavior can be analyzed by different machine learning approaches. In this work, methodology is proposed to evaluate the stock position with variation in time using deep learning approach such as recurrent neural network (RNN). This methodology used the technical parameters to evaluate the long term and short-term analysis of any stock or share. This approach also evaluates and gives suggestions to investors either to buy or sell any stock for long term and gives return at very low risk. In this paper work, hybridization of co-relation analysis and deep learning approach for stock price and long-term behavior analysis. The proposed work is termed as time lagged weight optimized RNN (TL-WO-RNN) that is adopted in this work and effectively predict the technical parameters and on the basis of that stock behavior is also predicted. The result analysis was performed on data from different sectors and such as Telecom, Powers, Manufacturing, Finance, Software sectors, etc. The result analysis shows the effectiveness of the TL-WO-RNN algorithm as compared to existing work.

Road surface irregularities not only has an impact on consistency of road, but also affects driver safety, automobile mechanical stability, and fuel consumption. To order to evaluate road roughness and spot potholes, many solutions have been suggested to remotely track the road surface quality. Some of these methods use an embedded mobile accelerometer to take a crowd sensing view to feel the surface of road. Although crowd sensing has various advantages such as ubiquity and less cost, it has some sensitivity to the problems generated by structures made by human, actions of the driver, and characteristics of the road surface which cannot be identified road anomalies. For this purpose, this paper suggests method of deep learning which helps us (a) to identify the different types of surface of road and (b) to differentiate potholes automatically from destabilizations in the crowd sensing-based application sense arising from speed bumps or driver behavior. This paper study and implement different deep learning frameworks in particular: convolutionary neural networks, LSTM networks, and computational structures for reservoirs. The tests were performed using knowledge from the real world, and the results revealed good precision in solving both of the problems.

Emotion recognition based on electroencephalography (EEG) is a significant task in the brain-computer interface field. Recently, many deep learning-based emotion recognition methods are demonstrated to outperform traditional methods. However, it remains challenging to extract discriminative features for EEG emotion recognition, and most methods ignore useful information in channel and time. This paper proposes an attention-based convolutional recurrent neural network (ACRNN) to extract more discriminative features from EEG signals and improve the accuracy of emotion recognition. First, the proposed ACRNN adopts a channel-wise attention mechanism to adaptively assign the weights of different channels, and a CNN is employed to extract the spatial information of encoded EEG signals. Then, to explore the temporal information of EEG signals, extended self-attention is integrated into an RNN to recode the importance based on intrinsic similarity in EEG signals. We conducted extensive experiments on the DEAP and DREAMER databases. The experimental results demonstrate that the proposed ACRNN outperforms state-of-the-art methods.

For face-related applications like face recognition and face verification, face detection is vital. Law enforcement agencies extensively use face recognition applications for the detection of known offenders. Security cameras are increasingly deployed everywhere, and these cameras’ captured data can be used for tagging offenders. This is done by offering the face detection system with input images, which are then processed. Later on, attempts will be done to tag the security cameras’ captured data with this processed input data. Effective utilization of deep learning techniques in order to efficiently carry out this task. Computer vision’s various tasks have been taken over by deep Convolutional Neural Networks (CNNs). Of late, the key paradigms for face detection are the region-based CNN (R-CNN) detection models due to their quicker processing speed and progressively better performance. Nevertheless, the huge feature set causes these models’ to be inefficient for real-time face detection. A Grammatical Evolution (GE) Region-Based Fully Conventional Neural Network (R-FCN) for face detection has been proposed in this work in which the GE heuristics are utilized for tuning the hyper-parameters. The WIDER face dataset and Face Detection Dataset and Benchmark (FDDB) dataset were employed for conducting the assessments. In comparison to other advanced paradigms, the proposed technique’s superiority is evident from the experiments conducted.

The estimation of human pose has been an important field of study for the area of computer vision. Due to its significant applications in various important fields such as human computer interaction, action recognition, video surveillance, threat prediction, etc., it has been in the focus of researchers mainly. Dramatic effects of deep learning techniques have resulted in major steps and important developments in estimating human poses. We are proposing a fast and efficient deep learning approach to detecting 2D pose from a monocular image by multiple persons. The method uses a Vector Field (PVF) part affinity, which learns how well the limbs have been connected. The architectural design provides a global pixel-level interpretation of the number of people by enabling a bottom-up approach that maintains high accuracy in real time. The architecture has been designed to use two branches of a convolutional neural network (CNN) with sequential prediction to understand and reference part locations around each other. Successfully, with the help of the MS COCO human pose dataset, we have implemented our approach.

Deep learning is the extension of machine learning technique which attracted the researchers in the recent past and hence the literature is also limited. This research focused on the need for deep learning for effective prediction process, when compared with few other machine learning models. As a proof of concept, a dataset of students’ performance of a school is analysed by various machine learning models. The performance measurements by all such models are compared with deep learning. These are assessed by two familiar statistical measures, Mean Absolute Error (MAE) and Root Mean Squared Error (RMSE). The error rates are low for deep learning when compared with other models. It is been evident that the result obtained by the deep learning model is more effective than the other machine learning models. The results are improved by reducing the error rates at the rates of 35% by mean absolute error and 48% by root mean squared error.

Machine learning is the sub field of Artificial Intelligence which is responsible for developing a system that has the capability to learn and improve from past experience. There is an increasing demand for the offensive language detection because there is a large code-mixed text. Sometimes there is a possibility of code-mixed texts which are written using native scripts. If the system is getting trained on the data which is created using single language then it will leads to failure because of complexity. Using online communities and social media platforms has become increasing now. Recently many of the researchers have been investigating different ways of identifying the abusive content and developing systems to detect its different types like aggression identification, cyber bullying detection, hate speech identification, offensive language identification and toxic comments identification. Developing mechanisms to detect this is a challenging task. In the last few years, machine learning techniques and deep learning concept has been playing a significant role in the development of solutions for offensive language detection. So, considerable efforts have also been devoted to the development of machine learning techniques and deep learning models which is used to identify toxic content in recent years. First, this survey provides offensive language taxonomy and detection approaches. Then, the article focuses on the offensive language identification and toxic comments identification approaches. This article reviews various models used, type of learning model, languages for which the developed model is applied, other methods or approaches that were compared against this model, dataset used, offensive language detection type and performance metrics. The findings from the review are as well summarized and they urge that further effort is required to get improvement in the current state-of-the art approaches. This article also focuses on future work to improve the offensive language classification and toxic comments identification.

The theme of this study is to provide a detailed description of its recent improvements in image segmentation and lesion classification in disease prognosis. Previous studies have shown that gray-white matter hyperintensities (GWMH) is one of the hallmarks of Nipah encephalitis, which sometimes occurs during the incubation period. Predicting this type of inflammation is a challenging task because it involves some unknown medical risk factors. A typical Magnetic Resonance Imaging (MRIs) is the best non-invasive system to analyze the anatomical structure of the brain. In-depth analysis of the defined pathological structure from isolated MR imaging leads to a reduction in the processing time of the prognostic model. Modern learning techniques such as Machine Learning, Computer Vision, and Deep Learning are the most promising techniques for determining the optimal outcome, computer can able to learn and extract useful information from historical data using various algorithms. Disease prognosis based on deep learning is sophisticated, so it can handle a variety of difficult tasks including image processing, classification, and feature extraction, noise and object detection. Diffusion-weighted imaging (DWI) in MRIs is a clinical prototype that can be used to diagnose brain abnormalities and to evaluate the microscopic architectural and molecular function of human organs or tissues. In this study, we summarize the results of diagnosing Nipah encephalitis using some publicly available brain encephalitis and encephalopathy databases.

It is widely acknowledged that the success of deep learning is built on large-scale training data and tremendous computing power. However, the data and computing power are not always available for many real-world applications. In this paper, we address the machine learning problem where it lacks training data and limits computing power. Specifically, we investigate domain adaptation which is able to transfer knowledge from one labeled source domain to an unlabeled target domain, so that we do not need much training data from the target domain. At the same time, we consider the situation that the running environment is confined, e.g., in edge computing the end device has very limited running resources. Technically, we present the Faster Domain Adaptation (FDA) protocol and further report two paradigms of FDA: early stopping and amid skipping. The former accelerates domain adaptation by multiple early exit points. The latter speeds up the adaptation by wisely skip several amid neural network blocks. Extensive experiments on standard benchmarks verify that our method is able to achieve the comparable and even better accuracy but employ much less computing resources. To the best of our knowledge, there are very few works which investigated accelerating knowledge adaptation in the community.

Hydraulic systems are a class of typical complex nonlinear systems, which have been widely used in manufacturing, metallurgy, energy, and other industries. Nowadays, the intelligent fault diagnosis problem of hydraulic systems has received increasing attention for it can increase operational safety and reliability, reduce maintenance cost, and improve productivity. However, because of the high nonlinear and strong fault concealment, the fault diagnosis of hydraulic systems is still a challenging task. Besides, the data samples collected from the hydraulic system are always in different sampling rates, and the coupling relationship between the components brings difficulties to accurate data acquisition. To solve the above issues, a deep learning model with multirate data samples is proposed in this article, which can extract features from the multirate sampling data automatically without expertise, thus it is more suitable in the industrial situation. Experiment results demonstrate that the proposed method achieves high diagnostic and fault pattern recognition accuracy even when the imbalance degree of sample data is as large as 1:100. Moreover, the proposed method can increase about 10% diagnosis accuracy when compared with some state-of-the-art methods.

Deep learning methods have been very successful at radio frequency fingerprinting tasks, predicting the identity of transmitting devices with high accuracy. We study radio frequency fingerprinting deployments at resource-constrained edge devices. We use structured pruning to jointly train and sparsify neural networks tailored to edge hardware implementations. We compress convolutional layers by a 27.2x factor while incurring a negligible prediction accuracy decrease (less than 1%). We demonstrate the efficacy of our approach over multiple edge hardware platforms, including a Samsung Gallaxy S10 phone and a Xilinx-ZCU104 FPGA. Our method yields significant inference speedups, 11.5x on the FPGA and 3x on the smartphone, as well as high efficiency: the FPGA processing time is 17x smaller than in a V100 GPU. To the best of our knowledge, we are the first to explore the possibility of compressing networks for radio frequency fingerprinting; as such, our experiments can be seen as a means of characterizing the informational capacity associated with this specific learning task.

The advanced metering infrastructure (AMI) in modern networked smart homes brings various advantages. However, smart homes are vulnerable to many cyberattacks, and the most striking one is energy theft. Researchers have developed many countermeasures, fostered by advanced machine learning (ML) techniques. Nevertheless, recent advances are not robust enough in practice, partially due to the vulnerabilities of ML algorithms. In this paper, we present a covert electricity theft strategy through mimicking normal consumption patterns. Such attack is almost impossible to be detected by existing solutions as the manipulated data have little deviation against honest usage records. To address this threat, we initially identify and define two levels of consumption deviations: home-level and interpersonal-level, respectively. Then, we propose a feature extraction method and develop a novel detection model based on deep learning. Extensive experiments show that the presented attack could evade existing mainstream detectors and the proposed countermeasure outperforms existing leading methods.

The staggering innovations and emergence of numerous deep learning (DL) applications have forced researchers to reconsider hardware architecture to accommodate fast and efficient application-specific computations. Applications, such as object detection, image recognition, speech translation, as well as music synthesis and image generation, can be performed with high accuracy at the expense of substantial computational resources using DL. Furthermore, the desire to adopt Industry 4.0 and smart technologies within the Internet of Things infrastructure has initiated several studies to enable on-chip DL capabilities for resource-constrained devices. Specialized DL processors reduce dependence on cloud servers, improve privacy, lessen latency, and mitigate bandwidth congestion. As we reach the limits of shrinking transistors, researchers are exploring various application-specific hardware architectures to meet the performance and efficiency requirements for DL tasks. Over the past few years, several software optimizations and hardware innovations have been proposed to efficiently perform these computations. In this article, we review several DL accelerators, as well as technologies with emerging devices, to highlight their architectural features in application-specific integrated circuit (IC) and field-programmable gate array (FPGA) platforms. Finally, the design considerations for DL hardware in portable applications have been discussed, along with some deductions about the future trends and potential research directions to innovate DL accelerator architectures further. By compiling this review, we expect to help aspiring researchers widen their knowledge in custom hardware architectures for DL. 2012 IEEE. This work was supported in part by the Research University Grant, Universiti Kebangsaan Malaysia, under Grant DPK-2021-001, Grant DIP-2020-004, and Grant MI-2020-002; and in part by the Qatar National Research Foundation (QNRF) under Grant NPRP12s-0227- 190164. Scopus

Many applications utilize sensors on mobile devices and apply deep learning for diverse applications. However, they have rarely enjoyed mainstream adoption due to many different individual conditions users encounter. Individual conditions are characterized by users unique behaviors and different devices they carry, which collectively make sensor inputs different. It is impractical to train countless individual conditions beforehand and we thus argue meta-learning is a great approach in solving this problem. We present MetaSense that leverages seen conditions in training data to adapt to an unseen condition (i.e., the target user). Specifically, we design a meta-learning framework that learns how to adapt to the target via iterative training sessions of adaptation. MetaSense requires very few training examples from the target (e.g., one or two) and thus requires minimal user effort. In addition, we propose a similar condition detector (SCD) that identifies when the unseen condition has similar characteristics to seen conditions and leverages this hint to further improve the accuracy. Our evaluation with 10 different datasets shows that MetaSense improves the accuracy of state-of-the-art transfer learning and meta learning methods by 15% and 11%, respectively. Furthermore, our SCD achieves additional accuracy improvement (e.g., 15% for human activity recognition).

Mobile Deep Learning (MDL) has emerged as a privacy-preserving learning paradigm for mobile devices. This paradigm offers unique features such as privacy preservation, continual learning and low-latency inference to the building of personal mobile sensing applications. However, squeezing Deep Learning to mobile devices is extremely challenging due to resource constraint. Traditional Deep Neural Networks (DNNs) are usually over-parametered, hence incurring huge resource overhead for on-device learning. In this paper, we present a novel on-device deep learning framework named MDLdroidLite that transforms traditional DNNs into resource-efficient model structures for on-device learning. To minimize resource overhead, we propose a novel Release-and-Inhibit Control (RIC) approach based on Model Predictive Control theory to efficiently grow DNNs from tiny to backbone. We also design a gate-based fast adaptation mechanism for channel-level knowledge transformation to quickly adapt new-born neurons with existing neurons, enabling safe parameter adaptation and fast convergence for on-device training. Our evaluations show that MDLdroidLite boosts on-device training on various PMS datasets with 28x to 50x less model parameters, 4x to 10x less floating number operations than the state-of-the-art model structures while keeping the same accuracy level.

Purpose Leukaemia is diagnosed conventionally by observing the peripheral blood and bone marrow smear using a microscope and with the help of advanced laboratory tests. Image processing-based methods, which are simple, fast, and cheap, can be used to detect and classify leukemic cells by processing and analysing images of microscopic smear. The proposed study aims to classify Acute Lymphoblastic Leukaemia (ALL) by Deep Learning (DL) based techniques. Procedures The study used Deep Convolutional Neural Networks (DNNs) to classify ALL according to WHO classification scheme without using any image segmentation and feature extraction that involves intense computations. Images from an online image bank of American Society of Haematology (ASH) were used for the classification. Findings A classification accuracy of 94.12% is achieved by the study in isolating the B-cell and T-cell ALL images using a pretrained CNN AlexNet as well as LeukNet, a custom-made deep learning network designed by the proposed work. The study also compared the classification performances using three different training algorithms. Conclusions The paper detailed the use of DNNs to classify ALL, without using any image segmentation and feature extraction techniques. Classification of ALL into subtypes according to the WHO classification scheme using image processing techniques is not available in literature to the best of the knowledge of the authors. The present study considered the classification of ALL only, and detection of other types of leukemic images can be attempted in future research.

We propose a method for detecting face swapping and other identity manipulations in single images. Face swapping methods, such as DeepFake, manipulate the face region, aiming to adjust the face to the appearance of its context, while leaving the context unchanged. We show that this modus operandi produces discrepancies between the two regions (e.g., Fig. 1). These discrepancies offer exploitable telltale signs of manipulation. Our approach involves two networks: (i) a face identification network that considers the face region bounded by a tight semantic segmentation, and (ii) a context recognition network that considers the face context (e.g., hair, ears, neck). We describe a method which uses the recognition signals from our two networks to detect such discrepancies, providing a complementary detection signal that improves conventional real vs. fake classifiers commonly used for detecting fake images. Our method achieves state of the art results on the FaceForensics++, Celeb-DF-v2, and DFDC benchmarks for face manipulation detection, and even generalizes to detect fakes produced by unseen methods.

The subtleties of human perception, as measured by vision scientists through the use of psychophysics, are important clues to the internal workings of visual recognition. For instance, measured reaction time can indicate whether a visual stimulus is easy for a subject to recognize, or whether it is hard. In this paper, we consider how to incorporate psychophysical measurements of visual perception into the loss function of a deep neural network being trained for a recognition task, under the assumption that such information can enforce consistency with human behavior. As a case study to assess the viability of this approach, we look at the problem of handwritten document transcription. While good progress has been made towards automatically transcribing modern handwriting, significant challenges remain in transcribing historical documents. Here we describe a general enhancement strategy, underpinned by the new loss formulation, which can be applied to the training regime of any deep learning-based document transcription system. Through experimentation, reliable performance improvement is demonstrated for the standard IAM and RIMES datasets for three different network architectures. Further, we go on to show feasibility for our approach on a new dataset of digitized Latin manuscripts, originally produced by scribes in the Cloister of St. Gall in the the 9th century.

It is of great significance to apply deep learning for the early diagnosis of Alzheimer's disease (AD). In this work, a novel tensorizing GAN with high-order pooling is proposed to assess mild cognitive impairment (MCI) and AD. By tensorizing a three-player cooperative game-based framework, the proposed model can benefit from the structural information of the brain. By incorporating the high-order pooling scheme into the classifier, the proposed model can make full use of the second-order statistics of holistic magnetic resonance imaging (MRI). To the best of our knowledge, the proposed Tensor-train, High-order pooling and Semisupervised learning-based GAN (THS-GAN) is the first work to deal with classification on MR images for AD diagnosis. Extensive experimental results on Alzheimer's disease neuroimaging initiative (ADNI) data set are reported to demonstrate that the proposed THS-GAN achieves superior performance compared with existing methods, and to show that both tensor-train and high-order pooling can enhance classification performance. The visualization of generated samples also shows that the proposed model can generate plausible samples for semisupervised learning purpose.

Synthetic aperture radar (SAR) has been widely applied in both civilian and military fields because it provides high-resolution images of the ground target regardless of weather conditions, day or night. In SAR imaging, the separation of moving and stationary targets is of great significance as it is capable of removing the ambiguity stemming from inevitable moving targets in stationary scene imaging and suppressing clutter in moving target imaging. The newly emerged generative adversarial networks (GANs) have great performance in many other signal processing areas; however, they have not been introduced to radar imaging tasks. In this work, we propose a novel shuffle GAN with autoencoder separation method to separate the moving and stationary targets in SAR imagery. The proposed algorithm is based on the independence of well-focused stationary targets and blurred moving targets for creating adversarial constraints. Note that the algorithm operates in a totally unsupervised fashion without requiring a sample set that contains mixed and separated SAR images. Experiments are carried out on synthetic and real SAR data to validate the effectiveness of the proposed method.

The Brain Tumor is one of the most serious scenarios associated with the brain where a cluster of abnormal cells grows in an uncontrolled fashion. The field of image processing has experienced remarkable growth in the area of biomedical applications with the invention of different techniques in deep learning. Brain tumor classification and detection is a subject of prime importance where Convolutional Neural Networks (CNN) find application. But the main drawback of the existing technology is that it is complex with a huge number of parameters contributing to high execution time and high system specifications for implementation. In this paper, a novel architecture for Brain tumor classification and tumor type object detection using the RCNN technique is proposed which has been analyzed using two publicly available datasets from Figshare ( Cheng et al., 2017 ) and Kaggle (2020) . Here we aim to decrease the execution time of a conventional RCNN architecture with the use of a low complex framework and propose a system for brain tumor analysis. We first use a Two Channel CNN, a low complex architecture to classify between Glioma and healthy tumor MRI samples which was successfully done with an accuracy of 98.21 percentage. Later this same architecture is used as the feature extractor of an RCNN to detect the tumor regions of the Glioma MRI sample that has been classified from the previous stage and the tumor region is bounded using bounding boxes. Also, this method has been extended to other two types of tumors Meningioma and Pituitary tumor. The methodology was able to achieve very low execution time as compared with the other existing architectures with an average confidence level of 98.83 percentage.

Long short-term memory (LSTM) neural networks and attention mechanism have been widely used in sentiment representation learning and detection of texts. However, most of the existing deep learning models for text sentiment analysis ignore emotion's modulation effect on sentiment feature extraction, and the attention mechanisms of these deep neural network architectures are based on word- or sentence-level abstractions. Ignoring higher level abstractions may pose a negative effect on learning text sentiment features and further degrade sentiment classification performance. To address this issue, in this article, a novel model named AEC-LSTM is proposed for text sentiment detection, which aims to improve the LSTM network by integrating emotional intelligence (EI) and attention mechanism. Specifically, an emotion-enhanced LSTM, named ELSTM, is first devised by utilizing EI to improve the feature learning ability of LSTM networks, which accomplishes its emotion modulation of learning system via the proposed emotion modulator and emotion estimator. In order to better capture various structure patterns in text sequence, ELSTM is further integrated with other operations, including convolution, pooling, and concatenation. Then, topic-level attention mechanism is proposed to adaptively adjust the weight of text hidden representation. With the introduction of EI and attention mechanism, sentiment representation and classification can be more effectively achieved by utilizing sentiment semantic information hidden in text topic and context. Experiments on real-world data sets show that our approach can improve sentiment classification performance effectively and outperform state-of-the-art deep learning-based methods significantly.

In the medical field, Medical diagnosis using Computed tomography (CT) has become increasingly popular due to their non-invasive approach and quick overall turnaround time. 3D visualization for CT Colonography involves the assessment and diagnosis of a patient to find the presence of cancerous polyps in the colon, by taking a Computed Tomography scan of the patient, and evaluating the reports. This technique reduces evaluation time by allowing the doctors themselves to analyze the CT scans without the need for a radiologist to generate an initial report. This technique also avoids an invasive procedure on the patient. This paper gives the insightsof developing computer aided system for the detection of Polyps in CT Colonography images using the principles of Image Processing and the Deep Learning, specifically an ensemble of Convolutional Neural Networks using GoogleNet architecture and 3D reconstruction of the same. The accuracy achieved by the proposed system for region classification, region 1 polyp detection and region 2 polyp detection are 98.75%, 93.75% and 94.03% respectively, and their F1 Scores are 0.88, 0.82 and 0.84 respectively.

Time-series forecasting is one of the most active research topics in artificial intelligence. Applications in real-world time series should consider two factors for achieving reliable predictions: modeling dynamic dependencies among multiple variables and adjusting the model's intrinsic hyperparameters. A still open gap in that literature is that statistical and ensemble learning approaches systematically present lower predictive performance than deep learning methods. They generally disregard the data sequence aspect entangled with multivariate data represented in more than one time series. Conversely, this work presents a novel neural network architecture for time-series forecasting that combines the power of graph evolution with deep recurrent learning on distinct data distributions; we named our method Recurrent Graph Evolution Neural Network (ReGENN). The idea is to infer multiple multivariate relationships between co-occurring time-series by assuming that the temporal data depends not only on inner variables and intra-temporal relationships (i.e., observations from itself) but also on outer variables and inter-temporal relationships (i.e., observations from other-selves). An extensive set of experiments was conducted comparing ReGENN with dozens of ensemble methods and classical statistical ones, showing sound improvement of up to 64.87% over the competing algorithms. Furthermore, we present an analysis of the intermediate weights arising from ReGENN, showing that by looking at inter and intra-temporal relationships simultaneously, time-series forecasting is majorly improved if paying attention to how multiple multivariate data synchronously evolve., Comment: IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021

Large volumes of video data recorded by the increasing mobile devices and embedded sensors can be leveraged to answer queries of our lives, physical world and our evolving society. Especially, the rapid development of convolutional neural networks (CNNs) in the past few years offers the great advantage for multiple tasks in video analysis. However, adopting running CNNs directly on mobile devices and embedded sensors for video analytics brings heavy burden due to their limited capacity, especially for learning a large volume of data. A promising approach is to outsource the computation-intensive part of CNN to cloud. However, the reveal of data to cloud may cause privacy leakage. In addition, the cloud-assisted approach may also bring some communication efficiency challenges for large volume of data. To address both privacy and efficiency issues, we design a privacy-preserving and computationally efficient framework for mobile video analytics. Extensive experiments demonstrate our proposed system PrivacyEye can effectively protect private information while keep the accuracy of the normal tasks with less than 2% drop, and it saves up to 82.9% execution time and 78.8% energy consumption.

The extensive growth of data in the health domain has increased the utility of Deep Learning in health. Deep learning is a highly advanced successor of artificial neural networks, having powerful computing ability. Due to the availability of fast data storage and hardware parallelism its popularity grows in the last five years. This in article presents a comprehensive literature review of research deploying deep learning medical imaging and medical NLP including tasks, pipelines, and challenges. In this work, we have presented an extensive survey of deep learning architecture deployed in the fields of medical imaging and medical natural language processing. This paper helps in identifying suitable combination of Deep learning, Natural language processing and medical imaging to enhance diagnosis. We have highlighted the major challenges in deploying deep learning in medical imaging and medical natural language processing. All the results are presented in pictorial form. This survey is very helpful for novices working in the area of health informatics.

Visual media, in today’s world, has swept across most forms of day to day communication. In the paradigm of generative modelling, restricted Boltzmann machines (RBMs) are used to solve complex tasks such as feature extraction, neuroimaging, collaborative filtering, radar target cognition, etc. In this paper, we implement an unsupervised Boltzmann machine-based time-aware recommendation system (UBMTR) which detects underlying hidden features in user-movie ratings data in connection with the time at which each rating was made (temporal information). The model takes ratings and time as a dual-input and outputs binary values via the contrastive divergence algorithm which samples from a Monte Carlo Markov Chain. Arguably, there exists a correlation between the content requested and the temporal conditions, which is exactly what our model tries to exploit. There is seldom any work in the field of recommender systems built using Boltzmann machines that incorporate temporal information, which necessitates research in this domain. RBMs are adept at pattern completion to tackle missing values, and can deal with imbalanced datasets and unstructured data by encoding raw data into latent variables. Using RBM, the UBMTR outperforms many earlier made attempts made at recommendation systems done through CF and deep learning or their hybridized models.

This paper presents a new contribution to the recommendation system following E-commerce reviews. Here, it implements a dictionary-based sentiment analysis based on four main steps (i) Pre-processing (ii) Semantic word extraction (iii) Feature Extraction (iv) Classification. Initially, the tweets or reviews extracted from the database are subjected to pre-processing, which involves three processes: stop word removal, stemming, and blank space removal. In the semantic word extraction process, the semantic words from the dictionary will be extracted by matching with the extracted keywords. The next process is the feature extraction, which extracts joint holoentropy and cross holoentropy of all keywords, and further proceeds with conditional holoentropy-based feature selection process. Finally, the selected features are subjected to a classification process using Deep Belief Network (DBN), which is a well-performing deep learning algorithm. Moreover, the main contribution of the paper relies on the improvement of DBN architecture, where the number of hidden neurons and activation function is optimally tuned by a hybrid optimization algorithm termed as Grey linked Chick Update-based Chicken Swarm Optimization (GCU-CSO). At last, the comparative analysis of proposed and conventional algorithms in terms of various performance metrics proves the efficacy of the proposed E-commerce recommendation system.

Diabetic retinopathy (DR) is one of the most common retinal diseases that cause preventable blindness in diabetic patients. The timely screening and grading of retinal images minimize the possibility of vision loss. However, manual screening of retinal images, for detecting micro lesions in the early stages of DR, is time-consuming. This paper proposes an ensemble of deep convolutional neural network (CNN) models for accurate detection and grading of DR using fundus images. Each input image is divided into four patches at the first stage and passed on to pre-trained CNN models (InceptionV3, Xception) for training. The relevant features in the shallow-dense layers of CNN models are utilized as prior knowledge. The integration of shallow and dense layer features helps the model learn the significant information of DR images. At the second stage, a classifier based on artificial neural network is trained using the fused probability vectors of four patches. The results of individual CNN models are combined to generate the final decision in the third stage. This ensemble approach of multi-stage deep learning model improves the overall classification accuracy of diabetic retinopathy grading. Among the five different classification schemes presented in this paper, multistage patch-based deep CNN (MPDCNN), in which local patch-based and holistic details of fundus image are concatenated, provides the best classification accuracy. This ensemble classifier exhibits 96.2 % classification accuracy with fivefold cross-validation.

One truism of deep learning is that the automatic feature engineering (seen in the first layers of those networks) excuses data scientists from performing tedious manual feature engineering prior to running DL. For the specific case of deep learning for defect prediction, we show that that truism is false. Specifically, when we preprocess data with a novel oversampling technique called fuzzy sampling, as part of a larger pipeline called GHOST (Goal-oriented Hyper-parameter Optimization for Scalable Training), then we can do significantly better than the prior DL state of the art in 14/20 defect data sets. Our approach yields state-of-the-art results significantly faster deep learners. These results present a cogent case for the use of oversampling prior to applying deep learning on software defect prediction datasets., Comment: v3, revision 2 (minor revision); submitted to TSE

Objective In this research paper, the brain MRI images are going to classify by considering the excellence of CNN on a public dataset to classify Benign and Malignant tumors. Materials and Methods Deep learning (DL) methods due to good performance in the last few years have become more popular for Image classification. Convolution Neural Network (CNN), with several methods, can extract features without using handcrafted models, and eventually, show better accuracy of classification. The proposed hybrid model combined CNN and support vector machine (SVM) in terms of classification and with threshold-based segmentation in terms of detection. Result The findings of previous studies are based on different models with their accuracy as Rough Extreme Learning Machine (RELM)-94.233%, Deep CNN (DCNN)-95%, Deep Neural Network (DNN) and Discrete Wavelet Autoencoder (DWA)-96%, k-nearest neighbors (kNN)-96.6%, CNN-97.5%. The overall accuracy of the hybrid CNN-SVM is obtained as 98.4959%. Conclusion In today's world, brain cancer is one of the most dangerous diseases with the highest death rate, detection and classification of brain tumors due to abnormal growth of cells, shapes, orientation, and the location is a challengeable task in medical imaging. Magnetic resonance imaging (MRI) is a typical method of medical imaging for brain tumor analysis. Conventional machine learning (ML) techniques categorize brain cancer based on some handicraft property with the radiologist specialist choice. That can lead to failure in the execution and also decrease the effectiveness of an Algorithm. With a brief look came to know that the proposed hybrid model provides more effective and improvement techniques for classification.

With the fast development of various positioning techniques such as Global Position System (GPS), mobile devices and remote sensing, spatio-temporal data has become increasingly available nowadays. Mining valuable knowledge from spatio-temporal data is critically important to many real world applications including human mobility understanding, smart transportation, urban planning, public safety, health care and environmental management. As the number, volume and resolution of spatio-temporal datasets increase rapidly, traditional data mining methods, especially statistics based methods for dealing with such data are becoming overwhelmed. Recently, with the advances of deep learning techniques, deep leaning models such as convolutional neural network (CNN) and recurrent neural network (RNN) have enjoyed considerable success in various machine learning tasks due to their powerful hierarchical feature learning ability in both spatial and temporal domains, and have been widely applied in various spatio-temporal data mining (STDM) tasks such as predictive learning, representation learning, anomaly detection and classification. In this paper, we provide a comprehensive survey on recent progress in applying deep learning techniques for STDM. We first categorize the types of spatio-temporal data and briefly introduce the popular deep learning models that are used in STDM. Then a framework is introduced to show a general pipeline of the utilization of deep learning models for STDM. Next we classify existing literatures based on the types of ST data, the data mining tasks, and the deep learning models, followed by the applications of deep learning for STDM in different domains including transportation, climate science, human mobility, location based social network, crime analysis, and neuroscience. Finally, we conclude the limitations of current research and point out future research directions., Comment: arXiv admin note: text overlap with arXiv:1711.04710 by other authors

With the aggressive and amazing scaling of the feature size of semiconductors, hotspot detection has become a crucial and challenging problem in the generation of optimized mask design for better printability. Machine learning techniques, especially deep learning, have attained notable success on hotspot detection tasks. However, most existing hotspot detectors suffer from suboptimal performance due to two-stage flow and less efficient representations of layout features. What is more, most works can only solve simple benchmarks with apparent hotspot patterns like ICCAD 2012 Contest benchmarks. In this paper, we firstly develop a new end-to-end hotspot detection flow where layout feature embedding and hotspot detection are jointly performed. An attention mechanism-based deep convolutional neural network is exploited as the backbone to learn embeddings for layout features and classify the hotspots simultaneously. Experimental results demonstrate that our framework achieves accuracy improvement over prior arts with fewer false alarms and faster inference speed on much more challenging benchmarks.

With the advent of ‘heavy Artificial Intelligence’ - big data, deep learning, and ubiquitous use of the internet, ethical considerations are widely dealt with in public discussions and governmental bodies. Within Computational Paralinguistics with its manifold topics and possible applications (modelling of long-term, medium-term, and short-term traits and states such as personality, emotion, or speech pathology), we have not yet seen that many contributions. In this article, we try to set the scene by (1) giving a short overview of ethics and privacy, (2) describing the field of Computational Paralinguistics, its history and exemplary use cases, as well as (de-)anonymisation and peculiarities of speech and text data, and (3) proposing rules for good practice in the field, such as choosing the right performance measure, and accounting for representativity and interpretability.

Affective computing is a field of great interest in many computer vision applications, including video surveillance, behaviour analysis, and human-robot interaction. Most of the existing literature has addressed this field by analysing different sets of face features. However, in the last decade, several studies have shown how body movements can play a key role even in emotion recognition. The majority of these experiments on the body are performed by trained actors whose aim is to simulate emotional reactions. These unnatural expressions differ from the more challenging genuine emotions, thus invalidating the obtained results. In this paper, a solution for basic non-acted emotion recognition based on 3D skeleton and Deep Neural Networks (DNNs) is provided. The proposed work introduces three majors contributions. First, unlike the current state-of-the-art in non-acted body affect recognition, where only static or global body features are considered, in this work also temporal local movements performed by subjects in each frame are examined. Second, an original set of global and time-dependent features for body movement description is provided. Third, to the best of out knowledge, this is the first attempt to use deep learning methods for non-acted body affect recognition. Due to the novelty of the topic, only the UCLIC dataset is currently considered the benchmark for comparative tests. On the latter, the proposed method outperforms all the competitors.

( Early Access ) Drug combination therapies have been a viable strategy for the treatment of complex diseases such as cancer due to increased efficacy and reduced side effects. However, experimentally validating all possible combinations for synergistic interaction even with high-throughout screens is intractable due to vast combinatorial search space. Computational techniques can reduce the number of combinations to be evaluated experimentally by prioritizing promising candidates. We present MatchMaker that predicts drug synergy scores using drug chemical structure information and gene expression profiles of cell lines in a deep learning framework. For the first time, our model utilizes the largest known drug combination dataset to date, DrugComb. We compare the performance of MatchMaker with the state-of-the-art models and observe up to 15% correlation and 33% mean squared error (MSE) improvements over the next best method. We investigate the cell types and drug pairs that are relatively harder to predict and present novel candidate pairs. MatchMaker is built and available at https://github.com/tastanlab/matchmaker

Indonesia is a tropical country that experiences forest fires every year. Forest fires occur due to a prolonged summer season. A major effect of forest fires that commonly occurred in Indonesia is the respiratory disorder and the visual impairment caused by the thick smoke that can experienced by the people who lives around the forest. The effects of forest fire fumes when the forest fire spread across the bigger land or area are also experienced by neighboring countries such as Malaysia, Singapore and Brunei Darussalam. Forest is a habitat for many animals, one of which is a bird. Birds are able to communicate with their colonies through sound. The bird sound during communication to the group can be in the form of calls, marriage invitations, and warnings of danger or threat of forest fires. This paper presents bird sounds classification study using one of Deep Learning (DL) algorithms i.e. Convolutional Neural Network (CNN) method. The CNN method is used to classify bird sounds in the two conditions: (1) under normal circumstances or conditions, and (2) under threated or panic condition. The bird sound data used in this study were collected from the local birds in Indonesia. The classification result of two bird sounds based on CNN method achieved up to 96.45%. This paper is a preliminary study for bio inspired early warning of forest fires based on the sound of birds.

This work presents a hybrid and hierarchical deep learning model for midterm load forecasting. The model combines exponential smoothing (ETS), advanced long short-term memory (LSTM), and ensembling. ETS extracts dynamically the main components of each individual time series and enables the model to learn their representation. Multilayer LSTM is equipped with dilated recurrent skip connections and a spatial shortcut path from lower layers to allow the model to better capture long-term seasonal relationships and ensure more efficient training. A common learning procedure for LSTM and ETS, with a penalized pinball loss, leads to simultaneous optimization of data representation and forecasting performance. In addition, ensembling at three levels ensures a powerful regularization. A simulation study performed on the monthly electricity demand time series for 35 European countries confirmed the high performance of the proposed model and its competitiveness with classical models such as ARIMA and ETS as well as state-of-the-art models based on machine learning.

The increase of cyberattacks by using the Internet of Things devices has exposed multiple vulnerabilities not only on devices but also on IoT infrastructures. Now, small devices can affect different networks and their services. This paper presents a review of the vulnerabilities on wireless technologies that gives connectivity to IoT, and the experiences using Software Defined Radio SDR for implementing wireless attacks to IoT technologies are assessed. A systematic literature review was conducted. The types of vulnerabilities and attacks that can affect the wireless technologies that stand the IoT ecosystem and SDR platforms were compared, and recent methods to overcome these attacks were identified. On the IoT reference model, the perception layer was identified as the most vulnerable. Most attacks at this level occur due to hardware limitations, physical device exposure, and heterogeneity of technologies. Future cybersecurity systems based on SDR technologies have notable advantages due to their flexibility and adaptability to new communication technologies and their potential to develop advanced tools. However, the IoT’s cybersecurity challenges are so complex that merging SDR hardware with cognitive and intelligent techniques is highly advisable. Some of these techniques could include deep learning to adapt mitigation systems to rapid technological changes.

Identifying race, which is a major physical feature in humans, is still a challenging task owing much to the lack of a concrete definition of race and the diversity of population across the globe. In this paper, we try to address the problem of race identification of four broad racial groups namely Caucasian, African, Asian and Indian. The newly developed BUPT Equalised Face dataset bearing about 1.3 M images in unrestricted environment is used to train our deep convolution network (R-Net) which achieves a state-of-the-art accuracy of 97%. To test the validity of this model it is evaluated on other datasets namely UTK and CFD. R-Net is also compared with fine-tuned VGG16 model for race estimation. Experimental results prove the robustness of this model for use in unconstrained environments. And finally, Gradient-weighted Class Activation Mapping (Grad-CAM) is applied to get a visual explanation of the deep learning model.

Security issues have resulted in severe damage to the cloud computing environment, adversely affecting the healthy and sustainable development of cloud computing. Intrusion detection is one of the technologies for protecting the cloud computing environment from malicious attacks. However, network traffic in the cloud computing environment is characterized by large scale, high dimensionality, and high redundancy, these characteristics pose serious challenges to the development of cloud intrusion detection systems. Deep learning technology has shown considerable potential for intrusion detection. Therefore, this study aims to use deep learning to extract essential feature representations automatically and realize high detection performance efficiently. An effective stacked contractive autoencoder (SCAE) method is presented for unsupervised feature extraction. By using the SCAE method, better and robust low-dimensional features can be automatically learned from raw network traffic. A novel cloud intrusion detection system is designed on the basis of the SCAE and support vector machine (SVM) classification algorithm. The SCAE+SVM approach combines both deep and shallow learning techniques, and it fully exploits their advantages to significantly reduce the analytical overhead. Experiments show that the proposed SCAE+SVM method achieves higher detection performance compared to three other state-of-the-art methods on two well-known intrusion detection evaluation datasets, namely KDD Cup 99 and NSL-KDD.

Enabling deep learning technology on the mobile web can improve user's experience for achieving web artificial intelligence in various fields. However, heavy DNN models and limited computing resource of the mobile web are now unable to support executing computationally intensive DNNs when deploying in a cloud computing platform. With the help of promising edge computing, we propose a lightweight collaborative deep neural network for the mobile web, named LcDNN, which contributes to three aspects: (1) we design a composite collaborative DNN that reduces the model size, accelerates inference and reduces mobile energy cost by executing a lightweight binary branch on the mobile web. (2) we provide a jointly training method for LcDNN, and implement an energy-efficient inference library for executing the binary branch on the mobile web. (3) to further promote the resource utilization of the edge cloud, we develop a DRL-based online scheduling scheme to obtain an optimal allocation for LcDNN. The experimental results show that LcDNN outperforms the existing approaches for reducing the model size by about 16x to 29x. It also reduces the end-to-end latency and mobile energy cost with an acceptable accuracy, and improves the throughput and resource utilization of the edge cloud.

Imbalanced class distribution is an inherent problem in many real-world classification tasks where the minority class is the class of interest. Many conventional statistical and machine learning classification algorithms are subject to frequency bias, and learning discriminating boundaries between the minority and majority classes could be challenging. To address the class distribution imbalance in deep learning, we propose a class rebalancing strategy based on a class-balanced dynamically weighted loss function where weights are assigned based on the class frequency and predicted probability of ground-truth class. The ability of dynamic weighting scheme to self-adapt its weights depending on the prediction scores allows the model to adjust for instances with varying levels of difficulty resulting in gradient updates driven by hard minority class samples. We further show that the proposed loss function is classification calibrated. Experiments conducted on highly imbalanced data across different applications of cyber intrusion detection (CICIDS2017 data set) and medical imaging (ISIC2019 data set) show robust generalization. Theoretical results supported by superior empirical performance provide justification for the validity of the proposed dynamically weighted balanced (DWB) loss function.