Search

Cross-manifold clustering is a hard topic and many traditional clustering methods fail because of the cross-manifold structures. In this paper, we propose a Multiple Flat Projections Clustering (MFPC) to deal with cross-manifold clustering problems. In our MFPC, the given samples are projected into multiple subspaces to discover the global structures of the implicit manifolds. Thus, the cross-manifold clusters are distinguished from the various projections. Further, our MFPC is extended to nonlinear manifold clustering via kernel tricks to deal with more complex cross-manifold clustering. A series of non-convex matrix optimization problems in MFPC are solved by a proposed recursive algorithm. The synthetic tests show that our MFPC works on the cross-manifold structures well. Moreover, experimental results on the benchmark datasets show the excellent performance of our MFPC compared with some state-of-the-art clustering methods., Comment: 12 pages, 58 figures

This paper attempts at improving the accuracy of Human Action Recognition (HAR) by fusion of depth and inertial sensor data. Firstly, we transform the depth data into Sequential Front view Images(SFI) and fine-tune the pre-trained AlexNet on these images. Then, inertial data is converted into Signal Images (SI) and another convolutional neural network (CNN) is trained on these images. Finally, learned features are extracted from both CNN, fused together to make a shared feature layer, and these features are fed to the classifier. We experiment with two classifiers, namely Support Vector Machines (SVM) and softmax classifier and compare their performances. The recognition accuracies of each modality, depth data alone and sensor data alone are also calculated and compared with fusion based accuracies to highlight the fact that fusion of modalities yields better results than individual modalities. Experimental results on UTD-MHAD and Kinect 2D datasets show that proposed method achieves state of the art results when compared to other recently proposed visual-inertial action recognition methods.

Existing convolution techniques in artificial neural networks suffer from huge computation complexity, while the biological neural network works in a much more powerful yet efficient way. Inspired by the biological plasticity of dendritic topology and synaptic strength, our method, Learnable Heterogeneous Convolution, realizes joint learning of kernel shape and weights, which unifies existing handcrafted convolution techniques in a data-driven way. A model based on our method can converge with structural sparse weights and then be accelerated by devices of high parallelism. In the experiments, our method either reduces VGG16/19 and ResNet34/50 computation by nearly 5x on CIFAR10 and 2x on ImageNet without harming the performance, where the weights are compressed by 10x and 4x respectively; or improves the accuracy by up to 1.0% on CIFAR10 and 0.5% on ImageNet with slightly higher efficiency. The code will be available on www.github.com/Genera1Z/LearnableHeterogeneousConvolution., Published in Neural Networks journal

Omni-directional images are becoming more prevalent for understanding the scene of all directions around a camera, as they provide a much wider field-of-view (FoV) compared to conventional images. In this work, we present a novel approach to represent omni-directional images and suggest how to apply CNNs on the proposed image representation. The proposed image representation method utilizes a spherical polyhedron to reduce distortion introduced inevitably when sampling pixels on a non-Euclidean spherical surface around the camera center. To apply convolution operation on our representation of images, we stack the neighboring pixels on top of each pixel and multiply with trainable parameters. This approach enables us to apply the same CNN architectures used in conventional Euclidean 2D images on our proposed method in a straightforward manner. Compared to the previous work, we additionally compare different designs of kernels that can be applied to our proposed method. We also show that our method outperforms in monocular depth estimation task compared to other state-of-the-art representation methods of omni-directional images. In addition, we propose a novel method to fit bounding ellipses of arbitrary orientation using object detection networks and apply it to an omni-directional real-world human detection dataset.

Compressed sensing (CS) demonstrates significant potential to improve image quality in 3-D millimeter-wave imaging compared with conventional matched filtering (MF). However, existing sparsity-driven 3-D imaging algorithms always suffer from large-scale storage, excessive computational cost, and nontrivial tuning of parameters due to the huge-dimensional matrix-vector multiplication in complicated iterative optimization steps. In this article, we present a novel range migration (RM) kernel-based iterative-shrinkage thresholding network, dubbed as RMIST-Net, by combining the traditional model-based CS method and data-driven deep learning method for near-field 3-D millimeter-wave (mmW) sparse imaging. First, the measurement matrices in ISTA optimization steps are replaced by RM kernels, by which matrix-vector multiplication is converted to the Hadamard product. Then, the modified ISTA optimization is unrolled into a deep hierarchical architecture, in which all parameters are learned automatically instead of manually tuned. Subsequently, 1000 pairs of oracle images with randomly distributed targets and their corresponding echoes are simulated to train the network. A well-trained RMIST-Net produces high-quality 3-D images from range-focused echoes. Finally, we experimentally prove that RMIST-Net is capable process 512 x 512 large-scale imaging tasks within 1 s. Besides, we compare RMIST-Net with other state-of-the-art methods in near-field 3-D imaging applications. Both simulations and real-measured experiments demonstrate that RMIST-Net produces impressive reconstruction performance while maintaining high computational speed compared with conventional and sparse imaging algorithms.

In recent days, facial emotion recognition techniques are considered important and critical due wide array of application domains use facial emotion as part of their workflow and analytics gathering. Reduced recognition rate, inefficient computation and increased time consumption are the major drawbacks of these techniques. To overcome these issues, this paper develops the new facial emotion recognition technique named as Minimal Angular Feature Oriented Neural Network based Emotion Prediction (MAFONN-EP). Initially, the input video sequence are categorized into image frames that are pre-processed by eliminating the noise by using Weighted Median Filtering (WMF) technique and to separate the background and foreground regions using Edge Preserved Background Separation and Foreground Extraction (EPBSFE) technique. Then, the set of texture patterns are extracted based on four key parts such as two eyes, nose and mouth by using the Minimal Angular Deviation (MAD) technique. Particular features are selected by employing the Cuckoo Search based Particle Swarm Optimization (CS-PSO) technique that also reduces the feature dimensionality. Finally, the Weight Based Pointing Kernel Classification (WBPKC) technique is employed for recognizing the emotion. In the experimental results, the performance of the proposed technique is analyzed and compared with different performance measures like accuracy, sensitivity, specificity, precision, recall.

Many data mining workloads are being analyzed in large-scale distributed cloud computing environments which provide nearly infinite resources with diverse hardware configurations. To maintain cost-efficiency in such environments, understanding the characteristics and estimating the overheads of a distributed matrix multiplication task that is a core computation kernel in many machine learning algorithms are essential. This study aims to propose a Matrix Multiplication Performance Estimator on Cloud (MPEC) algorithm. The proposed algorithm predicts the latency incurred when executing distributed matrix multiplication tasks of various input sizes and shapes with diverse instance types and a different number of worker nodes on cloud computing environments. To achieve this goal, we first analyze the characteristics of distributed matrix multiplication tasks. With characteristics generated from qualitative analysis, we propose to apply an ensemble of non-linear regression algorithm to predict the execution time of arbitrary matrix multiplication tasks. Thorough experimental results reveal that the proposed algorithm demonstrates higher accuracy than a state-of-the-art machine learning task performance estimation engine, Ernest, by decreasing the Mean Absolute Percentage Error (MAPE) in half.

An $H$-free editing problem asks whether we can edit at most $k$ edges to make a graph contain no induced copy of the fixed graph $H$. We obtain a polynomial kernel for this problem when $H$ is a diamond. The incompressibility dichotomy for $H$ being a 3-connected graph and the classical complexity dichotomy suggest that except for $H$ being a complete/empty graph, $H$-free editing problems admit polynomial kernels only for a few small graphs $H$. Therefore, we believe that our result is an essential step toward a complete dichotomy on the compressibility of $H$-free editing. Additionally, we give a cubic-vertex kernel for the diamond-free edge deletion problem, which is far simpler than the previous kernel of the same size for the problem.

This article deals with the backstepping design of observer-based compensators for parabolic ODE–PDE–ODE systems. The latter consist of $\boldsymbol {n}$ coupled parabolic PDEs with distinct diffusion coefficients and spatially-varying coefficients, that are bidirectionally coupled to ODEs at both boundaries. The actuation and sensing appears through these ODEs resulting in a challenging control problem. For this setup a systematic backstepping approach is proposed, in order to determine a state feedback controller and an observer. In particular, the state feedback loop and the observer error dynamics are mapped into stable ODE–PDE–ODE cascades by making use of a sequence of transformations. With this, the design can be traced back to the solution of kernel equations already found in the literature as well as initial and boundary value problems, that can be solved numerically. Exponential stability of the closed-loop system is verified, wherein the decay rate can be directly specified in the design. The results of the article are illustrated by the output feedback control of an unstable ODE–PDE–ODE system with two coupled parabolic PDEs.

In this work, we propose, develop, and demonstrate a fundamental and comprehensive theoretical foundation for the first time to formulate and devise known and unknown reciprocal and/or nonreciprocal ferrite components with arbitrarily concurrent multimode features based on the substrate-integrated waveguide (SIW) technology. To achieve such a universal design, a nonreciprocal unit cell (NRUC) is first defined and characterized. Then, an approach is proposed and investigated to deploy a sufficient number of NRUCs next to each other as a kernel multimode SIW topology. The kernel configuration transforms the incident guided-wave into several separate segments, propagating in parallels independently and along each NRUC, which can be manipulated separately. This methodology grants a general solution to forge ferrite components in a multimode configuration, discussed in detail to corroborate the proposed generalized theory. Finally, two multimode components are studied and demonstrated for the first time as application examples for the proposed theory. Namely, a dual-mode circulator and a triple-mode gyrator are prototyped, fabricated, and measured, exhibiting an excellent agreement with simulation results that effectively validate the presented theoretical foundation in this work.

We propose a spherical kernel for efficient graph convolution of 3D point clouds. Our metric-based kernels systematically quantize the local 3D space to identify distinctive geometric relationships in the data. Similar to the regular grid CNN kernels, the spherical kernel maintains translation-invariance and asymmetry properties, where the former guarantees weight sharing among similar local structures in the data and the latter facilitates fine geometric learning. The proposed kernel is applied to graph neural networks without edge-dependent filter generation, making it computationally attractive for large point clouds. In our graph networks, each vertex is associated with a single point location and edges connect the neighborhood points within a defined range. The graph gets coarsened in the network with farthest point sampling. Analogous to the standard CNNs, we define pooling and unpooling operations for our network. We demonstrate the effectiveness of the proposed spherical kernel with graph neural networks for point cloud classification and semantic segmentation using ModelNet, ShapeNet, RueMonge2014, ScanNet and S3DIS datasets. The source code and the trained models can be downloaded from https://github.com/hlei-ziyan/SPH3D-GCN., Comment: Accepted to TPAMI

Deep neural networks achieve remarkable performance in many computer vision tasks. Most state-of-the-art (SOTA) semantic segmentation and object detection approaches reuse neural network architectures designed for image classification as the backbone, commonly pre-trained on ImageNet. However, performance gains can be achieved by designing network architectures specifically for detection and segmentation, as shown by recent neural architecture search (NAS) research for detection and segmentation. One major challenge though is that ImageNet pre-training of the search space representation (a.k.a. super network) or the searched networks incurs huge computational cost. In this paper, we propose a Fast Network Adaptation (FNA++) method, which can adapt both the architecture and parameters of a seed network (e.g. an ImageNet pre-trained network) to become a network with different depths, widths, or kernel sizes via a parameter remapping technique, making it possible to use NAS for segmentation and detection tasks a lot more efficiently. In our experiments, we apply FNA++ on MobileNetV2 to obtain new networks for semantic segmentation, object detection, and human pose estimation that clearly outperform existing networks designed both manually and by NAS. We also implement FNA++ on ResNets and NAS networks, which demonstrates a great generalization ability. The total computation cost of FNA++ is significantly less than SOTA segmentation and detection NAS approaches: 1737x less than DPC, 6.8x less than Auto-DeepLab, and 8.0x less than DetNAS. A series of ablation studies are performed to demonstrate the effectiveness, and detailed analysis is provided for more insights into the working mechanism. Codes are available at https://github.com/JaminFong/FNA., Accepted by TPAMI (extension of arXiv:2001.02525)

Video deblurring is still a challenging low-level vision task since spatio-temporal characteristics across both the spatial and temporal domains are difficult to model. In this article, to model the temporal information, we develop a non-local block which estimates inter-frame similarity and inter-frame difference. Specially, for modeling the spatial characteristics and restoring sharp frame details, we propose a recursive block that iteratively refines feature maps generated at the last iteration. In addition, a novel temporal loss function is introduced to ensure the temporal consistency of generated frames. Experimental results on public datasets demonstrate that our method achieves state-of-the-art performance both quantitatively and qualitatively.

A CMOS image sensor (CIS) that can perform on-chip binary convolution is presented. The CIS can greatly reduce memory usage and computational complexity by directly generating a feature map for a binary neural network. The pixel readout of the CIS is performed in the column-parallel fashion using incremental delta-sigma analog-to-digital converters (ADCs). The CIS operates in two different modes: convolution and normal modes. When the column ADC is working in the convolution mode, it works as a first-order delta-sigma ADC and generates convolved images using a binary kernel. In the normal operation mode, the ADC is switched to a second-order delta-sigma ADC with little hardware modification and used to capture high-quality images. To demonstrate the CIS architecture, a 192 × 128-pixel CIS, which occupies an active die area of 14.44 mm2, is fabricated in a 0.18 μm standard CMOS process. The performance of the CIS is evaluated through measurements and network simulations. In the normal operation mode, the CIS achieves a read noise of 14.79 e-rms and a full-well capacity of 6,420 e- with a resulting dynamic range of 53 dB. The power consumptions of the CIS are 49.2 and 52.5 mW during the normal and convolution modes, respectively.

This article presents a new convolution algorithm: convolution kernel first operated (CKFO), which can solve the problem that the actual calculation is not reduced after pruning the weight of the convolution neural network. According to the convolution algorithm, this article proposes a simulated memristor implementation of a convolutional neural network (CNN). After that, we use the method of ex-situ training to train CNN in Tensorflow and then download the trained parameters to the Simulink system by compiling the conductance value of memristor to test the proposed simulation model. Finally, the effectiveness of the proposed model is verified. In addition, we prune the weights of CNN and retrain it, then adjust the simulation model according to the parameters after being pruned. We are surprised to find that the convolution layer designed according to the new convolution algorithm can apply the results of the pruned weight without any modification to the circuit, which is very cumbersome in other memristor-based CNN because the distribution of the pruned weight is irregular. The parameters are reduced by 75.24% and the number of multiplication operations in the convolution layer was reduced by 30.1%, while the accuracy is just reduced by 0.06%.

Crack detection on concrete bridges is a critical task to ensure bridge safety. However, many cracks on concrete bridges show low contrast and blurry edges in practice, which brings challenges to image-based crack detection. In this article, to improve the detection accuracy of blurred cracks, we propose the HDCB-Net—a deep learning-based network with the hybrid dilated convolutional block (HDCB) for the pixel-level crack detection. Specifically, HDCB is employed to expand the receptive field of the convolution kernel without increasing the computational complexity and to avoid the gridding effect generated by the dilated convolution. Meanwhile, to achieve a reasonable efficiency/accuracy tradeoff, the HDCB-Net only contains a few downsampling stages, which can avoid the loss of blurred crack pixels due to excessive downsampling. Furthermore, a two-stage strategy is proposed to realize the fast crack detection in a massive number of images (more than 100 000) with the high resolution (5120 × 5120 pixels). At the first stage, YOLOv4 is employed to filter out images without cracks and generate coarse region proposals. At the second stage, to achieve refined damage analysis, the HDCB-Net is used to detect pixel-level cracks from the coarse region proposals. The experimental results demonstrate that the proposed HDCB-Net is genetic and able to improve the detection accuracy of blurred cracks, and our two-stage strategy is efficient for fast crack detection. The whole detection process takes only 0.64 s to handle a single image. Additionally, we have established a public dataset, including 150 632 high-resolution images, dedicated to the research of crack detection, which have been released along with this article.

Image deblurring is a classic and important problem in industrial fields, such as aviation photo restoration, object recognition in robotics, and autonomous vehicles. Blurry images in real-world scenarios consist of mixed blurring types, such as a natural motion blurring owing to shaking of the camera. Fast deblurring does not deblur the entire image because it is not the best option. Considering the computational costs, it is also better to have an alternative kernel to deblur different objects at a high-semantic level. To achieve better image restoration quality, it is also beneficial to combine the blurring category location and important structural information in terms of specific artifacts and degree of blurring. The goal of blind image deblurring is to restore sharpness from the unknown blurring kernel of an image. Recent deblurring methods tend to reconstruct prior knowledge, neglecting the influence of blur estimation and visual fidelity on image details and structure. Generative adversarial networks(GANs) have recently been attracting considerable attention from both academia and industry because GAN can perfectly generate new data with the same statistics as the training set. Therefore, this study proposes a generative neural architecture and an edge attention algorithm developed to restore vivid multimedia patches. Joint edge generation and image restoration techniques are designed to solve the low-level multimedia retrieval. This multipath refinement fusion network (MRFNet) can not only perform deblurring of images directly but also individual the frames separately from videos. Ablation experiments validate that our generative adversarial network MRFNet performs better in joint training than in multimodel. Compared to other GAN methods, our two-phase method exhibited state-of-the-art performance in terms of speed and accuracy as well as has a significant visual improvement.

The increasing adoption of smart systems in our daily life has led to the development of new applications with varying performance and energy constraints, and suitable computing architectures need to be developed for these new applications. In this article, we present gem5-X, a system-level simulation framework, based on gem-5, for architectural exploration of heterogeneous many-core systems. To demonstrate the capabilities of gem5-X, real-time video analytics is used as a case-study. It is composed of two kernels, namely, video encoding and image classification using convolutional neural networks (CNNs). First, we explore through gem5-X the benefits of latest 3D high bandwidth memory (HBM2) in different architectural configurations. Then, using a two-step exploration methodology, we develop a new optimized clustered-heterogeneous architecture with HBM2 in gem5-X for video analytics application. In this proposed clustered-heterogeneous architecture, ARMv8 in-order cluster with in-cache computing engine executes the video encoding kernel, giving 20% performance and 54% energy benefits compared to baseline ARM in-order and Out-of-Order systems, respectively. Furthermore, thanks to gem5-X, we conclude that ARM Out-of-Order clusters with HBM2 are the best choice to run visual recognition using CNNs, as they outperform DDR4-based system by up to 30% both in terms of performance and energy savings.

Despite the outstanding performance, deep convolutional neural networks (CNNs) are computationally expensive and contain a large number of redundant parameters, hindering their deployment on resource constrained platforms. To address this issue, many model compression methods have been proposed. However, these methods mainly focus on pruning redundant parameters or designing efficient architectures, the redundancy in convolution kernels has rarely been investigated. In this paper, we find that the contributions of parameters at different locations in the traditional 3 × 3 kernels are not the same, and this distribution varies considerably in different layers. Motivated by this, we propose to use irregular kernels and present a novel approach to implementing convolution with any-shape kernels (ASKs) efficiently. The proposed ASKs are plug-and-play and can be readily embedded into existing CNNs, providing efficient modules for building compact CNNs. Experiments on benchmarks demonstrate the effectiveness of the proposed method. We improve the accuracy of VGG-16 on CIFAR-10 dataset from 93.45% to 94.04% simply by replacing the regular 3 × 3 kernel with cross-shaped kernel, which takes up only about 5 / 9 of the original storage and computing resources. Compared to state-of-the-art model compression methods, our ASKs achieve a better trade-off between accuracy and compression ratio.

The state-of-the-art semantic segmentation solutions usually leverage different receptive fields via multiple parallel branches to handle objects of different sizes. However, employing separate kernels for individual branches may degrade the generalization of the network to objects with different scales, and the computational cost increases with the increase of the number of branches. To tackle this problem, we propose a novel network structure, namely Kernel-Sharing Atrous Convolution (KSAC), where branches with different receptive fields share the same kernel, i.e., let a single kernel ‘see’ the input feature maps more than once with different receptive fields. Experiments conducted on the benchmark PASCAL VOC 2012 dataset show that our proposed sharing strategy can not only boost the network’s generalization and representation abilities but also reduce the computational cost significantly. Specifically, on the validation set, when compared with DeepLabv3+, about 2.7G FLOPs and 12.7G FLOPs are saved for output stride = 16 and 8 respectively. In addition, different from the widely used ASPP structure, our proposed KSAC is able to further improve the mIOU by taking benefit of wider context with larger atrous rates. Finally, our KSAC achieves mIOUs of 88.1%, 45.47% and 80.7% on the PASCAL VOC 2012 test set (Everingham et al., 2009), ADE20K dataset (Zhou et al., 2017) and Cityscapes datasets (Marius et al., 2016), respectively. Our full code will be released on Github: https://github.com/edwardyehuang/iSeg.

This brief addresses understandability of modern machine learning networks with respect to the statistical properties of their convolution layers. It proposes a set of tools for categorizing a convolution layer in terms of kernel property (meanlet, differencelet, or distrotlet) or kernel sequence property (frame spectra and intralayer correlation matrix). These tools are expected to be relevant for determining the generalization capabilities of a convolutional neural network. In particular, this brief highlights that the less frequency penalizing network among AlexNet, GoogleNet, RESNET101, and VGG19 is the more relevant one in terms of solutions for low-level ice-sheet feature enhancement.

Among visual relocalization from a single RGB image, the scene coordinate regression (SCoRe) based on convolutional neural network (CNN) becomes prevailing, however, it is insufficient to extract invariant features under different viewpoints due to fixed geometric structures of CNN. In this letter, we propose a global context-guided spatial feature transformation (SFT) network to learn invariant feature representation for robustness against viewpoint changes. Specifically, global feature extracted from source feature map is regarded as a dynamic convolutional kernel, which is convolved with source feature map for the prediction of transformation parameters. The predicted parameters are used to transform features of multiple viewpoints to a canonical space with the constraint of maximum likelihood-derived loss, and thus viewpoint invariance is achieved. CoordConv is also employed to further improve the discrimination of features on texture-less or repetitive zones. The proposed SFT network can be easily incorporated into the general SCoRe network. To our best knowledge, features are first decoupled from viewpoints explicitly in SCoRe network by the spatial feature transformation network, which achieves a stable and accurate visual relocalization. The experimental results demonstrate the effectiveness of the proposed method in terms of accuracy and efficiency.

Existing most deep learning-based synthetic aperture radar (SAR) ship detectors have huge network scale and big model size. Thus, to solve these defects, we propose a lightweight SAR ship detector “ShipDeNet-20” with 20 convolution layers and < 1 MB (0.82 MB) model size. We use fewer layers and kernels, and depthwise separable convolution (DS-Conv) to ensure ShipDeNet-20’s lightweight attribute. Moreover, we also propose a feature fusion module (FF-Module), a feature enhance module (FE-Module), and a scale share feature pyramid module (SSFP-Module) to compensate for the raw ShipDeNet-20’s accuracy loss. Experimental results on the open SAR ship detection data set (SSDD) reveal that the accuracy and speed of ShipDeNet-20 are both superior to the other nine state-of-the-art object detectors. Finally, detection results on another two wide-region SAR images show ShipDeNet-20’s strong migration ability. ShipDeNet-20 is a novel SAR ship detector, built from scratch, lighter than others by tens even hundreds of times, helpful for real-time SAR application and future hardware transplantation.

Camera-based surveillance systems largely perform an intrusion detection task for sensitive areas. The task may seem trivial but is quite challenging due to environmental changes and object behaviors such as those due to night-time, sunlight, IR camera, camouflage, and static foreground objects, etc. Convolutional neural network based algorithms have shown promise in dealing with these challenges. However, they are exclusively focused on accuracy. This article proposes an efficient supervised foreground detection (SFDNet) algorithm based on atrous deep spatial features. The features are extracted using atrous convolution kernels to enlarge the field-of-view of a kernel mask, thereby encoding rich context features without increasing the number of parameters. The network further benefits from a residual dense block strategy that mixes the mid and high-level features to retain the foreground information lost in low-resolution high-level features. The extracted features are expanded using a novel pyramid upsampling network. The feature maps are upsampled using bilinear interpolation and pass through a 3x3 convolutional kernel. The expanded feature maps are concatenated with the corresponding mid and low-level feature maps from an atrous feature extractor to further refine the expanded feature maps. The SFDNet showed better performance than high-ranked foreground detection algorithms on the three standard databases. The testing demo can be found at https://drive.google.com/file/d/1z_zEj9Yp7GZeM2gSIwYKvSzQlxMAiarw/view?usp=sharing .

Existing nonnegative matrix factorization methods usually focus on learning global structure of the data to construct basis and coefficient matrices , which ignores the local structure that commonly exists among data. To overcome this drawback, in this paper, we propose a new type of nonnegative matrix factorization method, which learns local similarity and clustering in a mutually enhanced way. The learned new representation is more representative in that it better reveals inherent geometric property of the data. Moreover, the new representation is performed in the kernel space, which enhances the capability of the proposed model in discovering nonlinear structures of data. Multiplicative updating rules are developed with theoretical convergence guarantees. Extensive experimental results have confirmed the effectiveness of the proposed model.

The conventional fuzzy C -means (FCM) algorithm is not robust to noise and its rate of convergence is generally impacted by data distribution. Consequently, it is challenging to develop FCM-related algorithms that have good performance and require less computing time. In this article, we elaborate on a comprehensive FCM-related algorithm for image segmentation. To make FCM robust, we first utilize a morphological grayscale reconstruction (MGR) operation to filter observed images before clustering, which guarantees noise-immunity and image detail-preservation. Since real images can generally be approximated by sparse coefficients in a tight wavelet frame system, feature spaces of observed and filtered images can be obtained. Taking such features to be clustered, we investigate an improved FCM model in which a sparse regularization term is introduced into the objective function of FCM. We design a three-step iterative algorithm to solve the sparse regularization-based FCM model, which is constructed by the Lagrangian multiplier method, hard-threshold operator, and normalization operator, respectively. Such an algorithm can not only perform well for image segmentation, but also come with high computational efficiency. To further enhance the segmentation accuracy, we use MGR to filter the label set generated by clustering. Finally, a large number of supporting experiments and comparative studies with other FCM-related algorithms available in the literature are provided. The obtained results for synthetic, medical and color images indicate that the proposed algorithm has good ability for multiphase image segmentation, and performs better than other alternative FCM-related algorithms. Moreover, the proposed algorithm requires less time than most of the existing algorithms.

The intelligent question answering system aims to provide quick and concise feedback on the questions of users. Although the performance of phrase-level and numerous attention models have been improved, the sentence components and position information are not emphasized enough. This article combines Ci-Lin and word2vec to divide all of the words in the question-answer pairs into groups according to the semantics and select one kernel word in each group. The remaining words are common words and realize the semantic mapping mechanism between kernel words and common words. With this Chinese semantic mapping mechanism, the common words in all questions and answers are replaced by the semantic kernel words to realize the normalization of the semantic representation. Meanwhile, based on the bi-directional LSTM model, this article introduces a method of the combination of semantic role labeling and positional context, dividing the sentence into multiple semantic segments according to semantic logic. The weight is given to the neighboring words in the same semantic segment and propose semantic role labeling position attention based on the bi-directional LSTM model (BLSTM-SRLP). The good performance of the BLSTM-SRLP model has been demonstrated in comparative experiments on the food safety field dataset (FS-QA).

Super-resolution methods for facial image via representation learning scheme have become very effective methods due to their efficiency. The key problem for the super-resolution of facial image is to reveal the latent relationship between the low-resolution ( LR ) and the corresponding high-resolution ( HR ) training patch pairs. To simultaneously utilize the contextual information of the target position and the manifold structure of the primitive HR space, in this work, we design a robust context-patch facial image super-resolution scheme via a kernel locality-constrained coupled-layer regression (KLC2LR) scheme to obtain the desired HR version from the acquired LR image. Here, KLC2LR proposes to acquire contextual surrounding patches to represent the target patch and adds an HR layer constraint to compensate the detail information. Additionally, KLC2LR desires to acquire more high-frequency information by searching for nearest neighbors in the HR sample space. We also utilize kernel function to map features in original low-dimensional space into a high-dimensional one to obtain potential nonlinear characteristics. Our compared experiments in the noisy and noiseless cases have verified that our suggested methodology performs better than many existing predominant facial image super-resolution methods.

In the medical field, image segmentation and classification possesses high worth in disease diagnosis and grading. The proposed novel technique segments and classifies histopathology images to find the nuclei pattern for disease diagnosis in four steps: (1) preprocessing, (2) segmentation, (3) feature extraction, and (4) classification. The image is preprocessed through a circular kernel and thresholding followed by curve fitting-based segmentation. The method employs image classification by support vector machine using extraction of bag of visual words and handcrafted features from original and segmented image, respectively. The performance evaluation of the proposed method is done for H&E-stained histopathology images on seven datasets: ADL, Bisque, BreakHis, Nuclei, MoNuSeg 2018, DSB2018, and the proposed MAHIBT. The classification result reports the superiority of the proposed method on the basis of accuracy and area under the characteristics curve, as the existing methods. The weighted cumulation factor confirms the supremacy of the proposed method in segmentation and classification for complex histopathology images.

The training of video fire detection models based on deep learning relies on a large number of positive and negative samples, namely, fire video and scenario video with other disturbances similar to fire. Due to the prohibition of ignition in lots of indoor occasions, the fire video samples in the scene are insufficient. In this paper, a method based on generative adversarial network is proposed to generate flame images which are then migrated into specified scenes, thus increasing fire video samples in those restricted situations. Flame kernel is pre-implanted into the specified scene to keep its characteristics intact. The flame and scene are blended together by adding styling information such as blurry edge and ground reflection. This method overcomes background distortion which is caused by existing multimodal image translation on as a result of information loss and is able to guarantee the diversity of flames in specified scenes and produce perceptually realistic results. Compared with other multimodal image-to-image translation schemes, the FID and LPIPS values of images generated by our method are the highest, reaches 118.4 and 0.1322 respectively. In addition, Unet and the SA-Unet, in which a self-attention mechanism is involved, are used as fire segmenting networks to evaluate the enhancement of the augmented data on improving the accuracy of segmented network. Their F1-scores reaches 0.8905 and 0.9082 respectively after Unet and SA-Unet are trained with GAN-based augmented dataset generated by our model. The F1-scores are second only to 0.9259 and 0.9291 which are obtained when Unet and SA-Unet are trained with real picture serving as augmented dataset.

In recent decades, the introduction of chaos to image encryption has drawn worldwide attention. The permutation-substitution architecture has been widely applied, and chaotic systems are generally employed to produce the required encryption elements. Although many security assessment tests have been conducted, some chaotic image ciphers are being cryptanalyzed. In this article, we evaluate the security of a family of image ciphers whose encryption kernel consists of a bit-level or pixel-level permutation and a bit-wise exclusive OR substitution. After investigating the intrinsic linearity inside the outfitted structures and encryption techniques, we find that each ciphertext-plaintext pair can be represented as a combination of a set of ciphertext-plaintext bases. A chosen-ciphertext attack is proposed to construct the ciphertext-plaintext bases rather than the traditional solution to retrieve equivalent encryption elements. We further reveal that such weakness cannot be remedied by common enhancements such as more chaotic dynamics, complex permutation methods, and random pixel insertion during encryption. In addition, applications of the proposed attack to break 12 ciphers are theoretically presented and experimentally verified.

Convolutional neural networks (CNNs) have demonstrated excellent performance in hyperspectral image (HSI) classification. However, tuning some critical hyperparameters of a CNN—such as the receptive field (RF) size—presents a major challenge due to the presence of features with different scales in HSIs. Contrary to the conventional design of CNNs, which fixes the RF size, it has been proven that the RF size is modulated by the stimulus and hence, depends on the scene being considered. Such a dilemma has been rarely considered in CNN design. In this letter, a new multibranch selective kernel network (MSKNet) is introduced, in which the input image is convolved using different RF sizes to create multiple branches so that the effect of each branch is adjusted by an attention mechanism according to the input contrast. As a result, our newly developed MSKNet is capable of modeling different scales. Our experimental results, conducted on three widely used HSIs, reveal that the MSKNet can outperform state-of-the-art CNNs in the context of HSI classification problems. The source code of our newly developed MSKNet is available from: https://github.com/mhaut/MSKNet-HSI

Pan sharpening is used to fuse a low-resolution multispectral (MS) image and a high-resolution panchromatic (PAN) image to obtain a high-resolution MS image. This article proposes PSMD-Net, an end-to-end pan-sharpening method based on a multi-scale dense network. A shallow feature extraction layer (SFEL) extracts the shallow features from the original images, and these are used as an input to a global dense feature fusion (GDFF) network to learn the global features for image reconstruction. A multiscale dense block (MDB) is designed to fully extract the spatial and spectral information from the shallow features in the GDFF network. In the proposed network, multiple MDBs are stacked to extract rich, multi-scale dense hierarchical features, and a global dense connection (GDC) is designed to allow direct connections from the state of the current MDB to all subsequent MDBs to extract more advanced features. The extracted hierarchical features are sent to the global feature fusion layer (GFFL) to adaptively learn the global features for image reconstruction. Finally, global residual learning (GRL) is adopted to force the network to pay more attention to the changing part of the image. We perform experiments on simulated and real data from WorldView-2 and WorldView-3 satellites. Visual and quantitative assessment results demonstrate that PSMD-Net yields higher-resolution fusion images than the state-of-the-art methods.

The combined-field integral equation (CFIE) usually is less accurate than the electric-field integral equation (EFIE), especially for objects with sharp edges and corners. In this work, we propose a simple but accuracy enhanced CFIE (AE-CFIE) strategy for computing scattering from perfect electric conductor (PEC) objects. AE-CFIE directly replaces the original CFIE on the sharp edges with EFIE. Using the dihedral angle of the meshes to describe the sharpness of the geometric structure, the hybrid criterion of EFIE/CFIE has been studied in detail. This strategy can be implemented conveniently under exiting basis functions and only a few lines of code need to be changed. Due to the inclusion of EFIE, AE-CFIE may sacrifice the convergence speed compared with CFIE. Therefore, we also investigate the performance of AE-CFIE with or without the preconditioner. Numerical examples show the performance of AE-CFIE, and a comprehensively study is made to demonstrate its validity, reliability, and stability.

Deep learning methods have shown their marvel performance on hyperspectral image (HSI) classification tasks. In particular, algorithms based on convolution neural network (CNN) outperformed most of the conventional machine learning-based algorithms and have become the mainstream of the current HSI classification research works. Recently, a newly proposed neural network called capsule network (CapsNet) showed its potential to replace the CNNs in various classification tasks with its amazing performance. In this letter, we proposed a new network architecture based on the CapsNet for HSI classification tasks, called dual-channel capsule network (DCCapsNet). Our DCCapsNet model extracts the features from spectral and spatial domains, respectively, with two separate convolution channels and then concatenates and feeds them into the following capsule layers to classify each of the HSI pixels. The model was trained and validated on four real HSI data sets and achieved high accuracy. We also compared our network with some of the state-of-the-art models and found that our model outperformed these competitor models.

Non-uniformity is a common phenomenon in infrared imaging system, which seriously affects imaging quality. In view of the problems of existing non-uniformity correction of infrared images, such as loss of image details and blurred edge of image, an improved non-uniformity correction method of infrared images based on convolution neural network using long-short connections (LSC-CNN) is proposed. The proposed method designs a long-short connection residual network structure suitable for non-uniformity correction of infrared image.The network depth is increased to fully learn the noise by short connections, image sizes are adjusted to reduce the number of parameters, the long connection is used to solve the problem of image information loss caused by transposed convolution, and a multiply operation is carried out to enhance the contrast of corrected images. Besides, batch normalization is utilized to improve the training speed. The experimental results show that LSC-CNN has excellent performance in non-uniformity correction of infrared images whether qualitative evaluation or quantitative evaluation. LSC-CNN is especially effective in image detail preservation and image edge protection whose average PSNR exceeds 37.5 dB and the average SSIM is greater than 0.98.

Generative adversarial networks (GANs) have shown remarkable effects for various computer vision tasks. Standard convolution plays an important role in the GAN-based model. However, the single type of kernel with a single spatial size limits the learning ability of the model and does not explicitly consider the dependencies among channels. To overcome these issues, this paper proposes a pyramidal convolution attention GAN for image denoising, a model that uses a residual structure with a pyramidal convolution attention block (PyCA) instead of the stacked standard convolution as a generator within the GAN setting. The proposed PyCA considers the channel-wise dependencies while extracting multi-scale features. Besides, we also design a data augmentation method for image denoising. The experimental results show that our model achieves better denoising performance than other competing methods.

Similarity representation plays a central role in increasingly popular anomaly detection techniques, which have been successfully applied in various realistic scenes. Until now, many low-rank representation techniques have been introduced to measure the similarity relations of data; yet, they only concern to minimize reconstruction errors, without involving the structural information of data. Besides, the traditional low-rank representation methods often take nuclear norm as their low-rank constraints, easily yielding a suboptimal solution. To address the problems above, in this article, we propose a novel anomaly detection method, which exploits kernel preserving embedding, as well as the double nuclear norm, to explore the similarity relations of data. Based on the similarity relations, a kind of probability transition matrix is derived, and a tailored random walk is further adopted to reveal anomalies. The proposed method can not only preserve the manifold structural properties of the data, but also alleviate the suboptimal problem. To validate the superiority of our method, extensive experiments with eight popular anomaly detection algorithms were conducted on 12 widely used datasets. The experimental results show that our detection method outperformed the state-of-the-art anomaly detection algorithms in most cases.

In present study, we proposed a general framework based on a convolutional kernel and a variational autoencoder (CVAE) for anomaly detection on both complex image and vector datasets. The main idea is to maximize mutual information (MMI) through regularizing key information as follows: (1) the features between original input and the representation of latent space, (2) that between the first convolutional layer output and the last convolutional layer input, (3) original input and output of the decoder to train the model. Therefore, the proposed CVAE is optimized by combining the representations learned across the three different objectives targeted at MMI on both local and global variables with the original training objective function of Kullback–Leibler divergence distributions. It allowed achieving the additional supervision power for the detection of image and vector data anomalies using convolutional and fully connected layers, respectively. Our proposal CVAE combined by regularizing multiple discriminator spaces to detect anomalies was introduced for the first time as far as we know. To evaluate the reliability of the proposed CVAE-MMI, it was compared with the convolutional autoencoder-based model using the original objective function. Furthermore, the performance of our network was compared over state-of-the-art approaches in distinguishing anomalies concerning both image and vector datasets. The proposed structure outperformed the state-of-the-arts with high and stable area under the curve values.

A major obstacle in digital projector technology is that images projected onto nonideal surfaces with large depth variances can easily become blurred. In this article, present a method to overcome projection defocus for projection surfaces that inevitably have complex shapes and large depth variances. The proposed method has two main advantages over traditional methods. First, an edge-intensification-based defocus compensation algorithm is proposed to manipulate the input image to compensate for its projection defocus before blurring occurs. Unlike previous time-consuming compensation algorithms, the proposed algorithm has very high efficiency, as it is noniterative and open loop. Second, a sinusoidal-projection-based estimation method is proposed to reduce kernel estimation errors on complex surface types. Unlike previous methods limited to specific surface types, the proposed method can provide consistently good kernel estimation results even for discontinuous and textured (nonpure white) projection surfaces. Hence, the proposed method can be applied to a wider range of applicable surfaces. These two contributions are demonstrated through extensive experiments and compared with the state-of-the-art methods. 1 1 The MATALB code and experimental data for this article are available at https://github.com/lpl-code/FastProjDefocusComp .

Many convolutional neural network (CNN) accelerators are proposed to exploit the sparsity of the networks recently to enjoy the benefits of both computation and memory reduction. However, most accelerators cannot exploit the sparsity of both activations and weights. For those works that exploit both sparsity opportunities, they cannot achieve the stable load balance through a static scheduling (SS) strategy, which is vulnerable to the sparsity distribution. In this work, a balanced compressed sparse row format and a dynamic scheduling strategy are proposed to improve the load balance. A set-associate structure is also presented to tradeoff the load balance and hardware resource overhead. We propose SWM to accelerate the CNN inference, which supports both sparse convolution and sparse fully connected (FC) layers. SWM provides Winograd adaptability for large convolution kernels and supports both 16-bit and 8-bit quantized CNNs. Due to the activation sharing, 8-bit processing can achieve theoretically twice the performance of the 16-bit processing with the same sparsity. The architecture is evaluated with VGG16 and ResNet50, which achieves: at most 7.6 TOP/s for sparse-Winograd convolution and three TOP/s for sparse matrix multiplication with 16-bit quantization on Xilinx VCU1525 platform. SWM can process 310/725 images per second for VGG16/ResNet50 with 16-bit quantization. Compared with the state-of-the-art works, our design can achieve at least $1.53 \boldsymbol {\times }$ speedup and $1.8 \boldsymbol {\times }$ energy efficiency improvement.

International audience; Bidirectional Reflectance Distribution Functions (BRDFs) are pivotal to the perceived realism in image synthesis. While measured BRDF datasets are available, reflectance functions are most of the time approximated by analytical formulas for storage efficiency reasons. These approximations are often obtained by minimizing metrics such as L 2 —or weighted quadratic—distances, but these metrics do not usually correlate well with perceptual quality when the BRDF is used in a rendering context, which motivates a perceptual study. The contributions of this paper are threefold. First, we perform a large-scale user study to assess the perceptual quality of 2026 BRDF approximations, resulting in 84138 judgments across 1005 unique participants. We explore this dataset and analyze perceptual scores based on material type and illumination. Second, we assess nine analytical BRDF models in their ability to approximate tabulated BRDFs. Third, we assess several image-based and BRDF-based (Lp, optimal transport and kernel distance) metrics in their ability to approximate perceptual similarity judgments.

The transient features caused by a local fault are of vital importance for bearing fault diagnosis in an intelligent industry. Due to the uncertainty of fault forms and nonstationarity of operating conditions, the fault feature distribution influenced by the physical dynamic response of actual defect is always complex and irregular with morphological differences. This will bring embarrassments for an accurate fault diagnosis. Motivated by this, a convolution sparse self-learning (CSSL) is proposed in this article to accomplish an adaptive feature enhancement. In the view of image sparse processing, the representation for desired morphological structures is promoted by a two-dimensional optimizing approach with manifold sensing. From a randomly selected fragment, the time–frequency manifold learning is first applied to mine the latent structures. The image entropy is then introduced to adaptively output the optimal one as a sensing kernel. Therewith, this kernel is used to operate a shift-invariant sparse analysis on raw time–frequency image. Combining this rebuilt image with the raw phase, an enhanced signal is finally synthesized. In this manner, the desired transient morphology can be automatically mined, which is consistent with the physical dynamic response. Practical defective bearing data are analyzed to illustrate the effectiveness of the proposed method. Specifically, a comparison further illustrates that the proposed CSSL is superior in the morphological transient features enhancement.

As the growing demand on artificial intelligence (AI) Internet-of-Things (IoT) devices, smart vision sensors with energy-efficient computing capability are required. This article presents a low-power and low-voltage dual mode 0.5-V computational CMOS image sensor (C2IS) with array-parallel computing capability for feature extraction using convolution. In the feature extraction mode, by applying the pulsewidth modulation (PWM) pixel and switch-current integration (SCI) circuit, the in-sensor eight-directional matrix-parallel multiply–accumulate (MAC) operation is realized. Furthermore, the analog-domain convolution-on-readout (COR) operation, the programmable $3\times3$ kernel with ±3-bit weights, and the tunable-resolution column-parallel analog-to-digital converter (ADC) (1–8 bit) are implemented to achieve the real-time feature extraction without using additional memory and sacrificing frame rate. In the image capturing mode, the sensor provides the linear-response 8-bit raw image data. The C2IS prototype has been fabricated in the TSMC 0.18- $\mu \text{m}$ standard process technology and verified to demonstrate the raw and feature images at 480 frames/s with a power consumption of 77/ $117~\mu \text{W}$ and the resultant FoM of 9.8/14.8 pJ/pixel/frame, respectively. The prototype sensor is used as a real-time edge feature detection frond-end camera and accompanied with a simplified convolutional neural network (CNN) architecture to demonstrate the hand gesture recognition. The prototype system achieves more than 95% validation accuracy.

The J-integral and stress intensity factor (SIF) are two major parameters in linear elastic fracture mechanics (LEFM) for the fracture criterion. In this paper, we focus on the J-integral of the slant crack and the slant rigid-line inclusion under the remote anti-plane shear. By employing the degenerate kernel, the path independence of J-integral is analytically demonstrated by using the elliptic coordinates. The positive and negative J-integrals are also analytically derived and numerically implemented by using the dual BEM for the crack and the rigid-line inclusion, respectively. It is interesting to find that the J-integral is not an invariant by using different observer systems but is one component of the vector of the first order tensor. Transformation law of the J-integral with respect to different observers is analytically proved and numerically demonstrated. Finally, the tensor property of order one is examined.

Improving the detection accuracy and speed is the prerequisite of multi-object recognition in the complex traffic environment. Despite object detection has made significant advances based on deep neural networks, it remains a challenge to focus on small and occlusion objects. We address this challenge by allowing multiscale fusion. We introduce a cross-layer fusion multi-object detection and recognition algorithm based on Faster R-CNN, an approach that the five-layer structure of VGG16 (Visual Geometry Group) is used to obtain more characteristic information. We implement this idea with lateral embedding the 1×1 convolution kernel, max pooling and deconvolution, in conjunction with weighted balanced multi-class cross entropy loss function and Soft-NMS to control the imbalance between difficult and easy samples. Considering the actual situation in a complex traffic environment, we manually label mixed dataset. On Cityscapes and KITTI datasets, experimental results show that the proposed model achieves better effects than the current mainstream object detection models.

The authors designed an automated framework to segment tumors with various image sequences like T1, T2, and post-processed MRI multimodal images. Contrast-limited adaptive histogram equalization method is used for preprocessing images to enhance the intensity level and view the tumor part clearly. With the combination of kernel possibilistic c means clustering with particle swarm optimization technique, a tumor part is segmented, and morphological filters are applied to remove the unrelated outlier pixels in the segmented image to detect the accurate tumor part. The authors collected various image sequences from online resources like Harvard brain dataset, BRATS, and RIDER, and a few from clinical datasets. Efficiency is ensured by computing various performance metrics like Jaccard Index MSE, PSNR, sensitivity, specificity, accuracy, and computational time. The proposed approach yields 97.06% segmentation accuracy and 98.08% classification accuracy for multimodal images with an average of 5s for all multimodal images.

This work introduces and analyzes new primal and dual-mixed finite element methods for deformable image registration, in which the regularizer has a nontrivial kernel, and constructed under minimal assumptions of the registration model: Lipschitz continuity of the similarity measure and ellipticity of the regularizer on the orthogonal complement of its kernel. The aforementioned singularity of the regularizer suggests to modify the original model by incorporating the additional degrees of freedom arising from its kernel, thus granting ellipticity of the former on the whole solution space. In this way, we are able to prove well-posedness of the resulting extended primal and dual-mixed continuous formulations, as well as of the associated Galerkin schemes. A priori error estimates and corresponding rates of convergence are also established for both discrete methods. Finally, we provide numerical examples confronting our formulations with the standard ones, which prove our finite element methods to be particularly more efficient on the registration of translations and rotations, in addition for the dual-mixed approach to be much more suitable for the quasi-incompressible case, and all the above without losing the flexibility to solve problems arising from more realistic scenarios such as the image registration of the human brain.

Attention mechanism plays an essential role in many tasks such as image classification, object detection, and instance segmentation. However, existing methods typically assigned attention weights to feature maps of the previous layer. The kernels in current layer remained static during the inference stage. To explicitly model the dependency of individual kernel weights on image content at the inference stage, this work proposed attention weight block (AWB) that converts kernels to be steerable to the content in a test image. Specifically, AWB computes a set of on-the-fly coefficients according to the feature maps of the previous layer and applies the coefficients to the kernels in current layers, which makes them steerable. AWB kernels emphasize or suppress the weights of certain kernels depending on the content of input samples and hence significantly improve the feature representation ability of deep neural networks. The proposed AWB is evaluated on various datasets, and experimental results show that steerable kernels in AWB outperformed the state-of-the-art attention approaches when embedded in the architecture for classification, object detection, and semantic segmentation tasks. It outperforms ECA by 1.1% and 1.0% on CIFAR-100 and Tiny ImageNet datasets, respectively, for image classification task; outperforms CornerNet-Lite by 1.5% on COCO2017 dataset for object detection task; and outperforms FCN8s by 1.2% on SBUshadow dataset for semantic segmentation task.

People Counting in images is a worthwhile task as it is widely used for public safety, emergency people planning, intelligent crowd flow, and countless other reasons. Counting the objects manually in images does not make practical sense, since it is very time-consuming, and it never gives accurate results for dense crowded images. In crowded images, as the density of the people increases, object appear to be partially encircling each other. This occlusion problem of objects limits the crowd counting ability of any traditional computer vision model. To overcome this problem, here we addressed a dynamic kernel convolution neural network-linear regression (DKCNN-LR) model for counting the exact number of people in image frames even if crowd is very dense and occlusion problem. The proposed model works in two phases, first a DKCNN model use convolution layers in such a fashion that the kernel weight of each subsequent successive layer is half of its previous convolution layer’s weight. The first three heavy kernel weight layers identify far camera regions (low-level) features, and the later light kernel weight layers help identify near-camera region (high-level) features. Second, a linear regression model is employed to perform parametric regression between the actual people count (ground truth) and the estimated count (predicted values). The performance of the proposed model tested on three challenging and different quality benchmark datasets in terms of MAE, RMSE, Pearson-R and R2. The DKCNN-LR model secured MAE, RMSE on Mall dataset is 1.65, 2.76, on Beijing-BRT 1.43, 1.87 and on SmartCity dataset it is 2.69 and 10.69. These results confirm that the proposed model is quite reliable, effective and robust for real situations.