Search

In this brief, stabilization of Boolean networks (BNs) by flipping a subset of nodes is considered, here we call such action state-flipped control. The state-flipped control implies that the logical variables of certain nodes are flipped from 1 to 0 or 0 to 1 as time flows. Under state-flipped control on certain nodes, a state-flipped-transition matrix is defined to describe the impact on the state transition space. Weak stabilization is first defined and then some criteria are presented to judge the same. An algorithm is proposed to find a stabilizing kernel such that BNs can achieve weak stabilization to the desired state with in-degree more than 0. By defining a reachable set, another approach is proposed to verify weak stabilization, and an algorithm is given to obtain a flip sequence steering an initial state to a given target state. Subsequently, the issue of finding flip sequences to steer BNs from weak stabilization to global stabilization is addressed. In addition, a model-free reinforcement algorithm, namely the Q-learning (QL) algorithm, is developed to find flip sequences to achieve global stabilization. Finally, several numerical examples are given to illustrate the obtained theoretical results.

With the continuously increasing amount of video data, image set classification has recently received widespread attention in the CV & PR community. However, the intra-class diversity and inter-class ambiguity of representations remain an open challenge. To tackle this issue, several methods have been put forward to perform multiple geometry-aware image set modelling and learning. Although the extracted complementary geometric information is beneficial for decision making, the sophisticated computational paradigm (e.g., scatter matrices computation and iterative optimisation) of such algorithms is counterproductive. As a countermeasure, we propose an effective hybrid Riemannian metric learning framework in this paper. Specifically, we design a multiple graph embedding-guided metric learning framework for the sake of fusing these complementary kernel features, obtained via the explicit RKHS embeddings of the Grassmannian manifold, SPD manifold, and Gaussian embedded Riemannian manifold, into a unified subspace for classification. Furthermore, the involved optimisation problem of the developed model can be solved in terms of a series of sub-problems, achieving improved efficiency theoretically and experimentally. Substantial experiments are carried out to evaluate the efficacy of our approach. The experimental results suggest the superiority of it over the state-of-the-art methods.

With the development of Earth observation technology, very-high-resolution (VHR) image has become an important data source of change detection. Nowadays, deep learning methods have achieved conspicuous performance in the change detection of VHR images. Nonetheless, most of the existing change detection models based on deep learning require annotated training samples. In this paper, a novel unsupervised model called kernel principal component analysis (KPCA) convolution is proposed for extracting representative features from multi-temporal VHR images. Based on the KPCA convolution, an unsupervised deep siamese KPCA convolutional mapping network (KPCA-MNet) is designed for binary and multi-class change detection. In the KPCA-MNet, the high-level spatial-spectral feature maps are extracted by a deep siamese network consisting of weight-shared PCA convolution layers. Then, the change information in the feature difference map is mapped into a 2-D polar domain. Finally, the change detection results are generated by threshold segmentation and clustering algorithms. All procedures of KPCA-MNet does not require labeled data. The theoretical analysis and experimental results demonstrate the validity, robustness, and potential of the proposed method in two binary change detection data sets and one multi-class change detection data set.

Spherical images or videos, as typical non-Euclidean data, are usually stored in the form of 2D panoramas obtained through an equirectangular projection, which is neither equal area nor conformal. The distortion caused by the projection limits the performance of vanilla Deep Neural Networks (DNNs) designed for traditional Euclidean data. In this paper, we design a novel Spherical Deep Neural Network (DNN) to deal with the distortion caused by the equirectangular projection. Specifically, we customize a set of components, including a spherical convolution, a spherical pooling, a spherical ConvLSTM cell and a spherical MSE loss, as the replacements of their counterparts in vanilla DNNs for spherical data. The core idea is to change the identical behavior of the conventional operations in vanilla DNNs across different feature patches so that they will be adjusted to the distortion caused by the variance of sampling rate among different feature patches. We demonstrate the effectiveness of our Spherical DNNs for saliency detection and gaze estimation in 360° videos. To facilitate the study of the 360 video saliency detection, we further construct a large-scale 360° video saliency detection dataset. Comprehensive experiments validate the effectiveness of our proposed Spherical DNNs for spherical handwritten digit classification and sport classification, saliency detection and gaze tracking in 360° videos.

This paper attempts at improving the accuracy of Human Action Recognition (HAR) by fusion of depth and inertial sensor data. Firstly, we transform the depth data into Sequential Front view Images(SFI) and fine-tune the pre-trained AlexNet on these images. Then, inertial data is converted into Signal Images (SI) and another convolutional neural network (CNN) is trained on these images. Finally, learned features are extracted from both CNN, fused together to make a shared feature layer, and these features are fed to the classifier. We experiment with two classifiers, namely Support Vector Machines (SVM) and softmax classifier and compare their performances. The recognition accuracies of each modality, depth data alone and sensor data alone are also calculated and compared with fusion based accuracies to highlight the fact that fusion of modalities yields better results than individual modalities. Experimental results on UTD-MHAD and Kinect 2D datasets show that proposed method achieves state of the art results when compared to other recently proposed visual-inertial action recognition methods.

Varying density of point clouds increases the difficulty of 3D detection. In this paper, we present a context-aware dynamic network (CADNet) to capture the variance of density by considering both point context and semantic context. Point-level contexts are generated from original point clouds to enlarge the effective receptive filed. They are extracted around the voxelized pillars based on our extended voxelization method and processed with the context encoder in parallel with the pillar features. With a large perception range, we are able to capture the variance of features for potential objects and generate attentive spatial guidance to help adjust the strengths for different regions. In the region proposal network, considering the limited representation ability of traditional convolution where same kernels are shared among different samples and positions, we propose a decomposable dynamic convolutional layer to adapt to the variance of input features by learning from the local semantic context. It adaptively generates the position-dependent coefficients for multiple fixed kernels and combines them to convolve with local features. Based on our dynamic convolution, we design a dual-path convolution block to further improve the representation ability. We conduct experiments on KITTI dataset and the proposed CADNet has achieved superior performance of 3D detection outperforming SECOND and PointPillars by a large margin at the speed of 30 FPS.

Cross-manifold clustering is a hard topic and many traditional clustering methods fail because of the cross-manifold structures. In this paper, we propose a Multiple Flat Projections Clustering (MFPC) to deal with cross-manifold clustering problems. In our MFPC, the given samples are projected into multiple subspaces to discover the global structures of the implicit manifolds. Thus, the cross-manifold clusters are distinguished from the various projections. Further, our MFPC is extended to nonlinear manifold clustering via kernel tricks to deal with more complex cross-manifold clustering. A series of non-convex matrix optimization problems in MFPC are solved by a proposed recursive algorithm. The synthetic tests show that our MFPC works on the cross-manifold structures well. Moreover, experimental results on the benchmark datasets show the excellent performance of our MFPC compared with some state-of-the-art clustering methods., Comment: 12 pages, 58 figures

Loop-closure detection (LCD), which aims to recognize a previously visited location, is a crucial component of the simultaneous localization and mapping system. In this paper, a novel appearance-based LCD method is presented. In particular, we propose a simple yet surprisingly useful feature matching algorithm for real-time geometrical verification of candidate loop-closures, termed as local relative orientation matching (LRO). It aims to efficiently establish reliable feature correspondences based on preserving local topological structures between the query image and candidate frame. To effectively retrieve candidate loop closures, we introduce the aggregated selective match kernel framework into the LCD task, which can effectively represent images and reduce the quantization noise of the traditional bag-of-words framework. In addition, the SuperPoint neural network is employed to extract reliable interest points and feature descriptors. Extensive experimental results demonstrate that our LRO can significantly improve the LCD performance, and the proposed overall LCD method can achieve much better performance over the current state-of-the-art on six publicly available datasets.

Many applications and contexts including materials science, metallurgy, and solid-state physics, are concerned with the study of the behavior of viscoelastic materials. In addition, any composite or complex construction containing embedded polymers exhibits viscoelastic behavior under static and dynamic stress conditions. Wide types of linear/nonlinear constitutive models have been proposed to define the viscoelastic deformation process of viscoelastic materials in order to explore their mechanical behavior. However, it has been shown that the constitutive relationship in the integer-order of stress-strain available in conventional viscoelastic models may fail in some types of situations and do not match well with empirical evidence. In this paper, a novel mathematical model is provided that uses Caputo-Fabrizio fractional-order derivatives to describe the viscoelastic phenomena and is consistent with thermodynamic principles. The Caputo-Fabrizio kernel has many features, such as nonlocality and non-singularity in addition to the exponential form. The suggested model is used to study the dynamic reactions of an unbounded body with a spherical cavity made of viscoelastic material subjected to time-varying heat. Using the Laplace transform technique, numerical calculations of many physical fields are obtained and explored in depth.

Underwater captured images are usually degraded by low contrast, hazy, and blurry due to absorbing and scattering, which limits their analyses and applications. To address these problems, a red channel prior guided variational framework is proposed based on the complete underwater image formation model (UIFM). Unlike most of the existing methods that only consider the direct transmission and backscattering components, we additionally include forward scattering component into the UIFM. In the proposed variational framework, we successfully incorporate the normalized total variation item and sparse prior knowledge of blur kernel together. In addition, we perform the estimation of blur kernel by varying image resolution in a coarse-to-fine manner to avoid local minima. Moreover, for solving the generated non-smooth optimization problem, we employ the alternating direction method of multipliers (ADMM) to accelerate the whole progress. Experimental results demonstrate that the proposed method has a good performance on dehazing and deblurring. Extensive qualitative and quantitative comparisons further validate its superiority against the other state-of-the-art algorithms. The code is available online at: https://github.com/Hou-Guojia.

Diabetic Foot Ulcer (DFU) is a complication of diabetes that causes lower limb amputation. In this work, a unique stacked parallel convolution layers-based network (DFU_SPNet) is proposed to perform DFU vs. normal skin classification. The main objective of this work is to design an effective CNN-based classification model, along with proper fine-tuning of optimizer settings. DFU_SPNet consists of 3 blocks of parallel convolution layers with multiple kernel sizes, for local and global feature abstractions. The proposed DFU_SPNet, trained using SGD (with momentum) optimizer with 1 e − 2 learning rate on the DFUNet dataset, outperformed the current state-of-the-art results with an AUC of 0.974.

As a lightweight, flexible and high-performance operating system virtualization, containers are used to speedup the big data platform. However, due to the imperfection of resource isolation mechanism and property of shared kernel, the meltdown and spectre attacks can lead to the information leakage of kernel space and co-resident containers. In this paper, a noise-resilient and real-time detection system, named ContainerGuard, is proposed to detect meltdown and spectre attacks in container-based big data platform. ContainerGuard uses a non-intrusive manner to collect lifecycle multivariate time-series performance events data of processes in containers, then uses ensemble of variational autoencoders as generative neural networks to learn the robust representations of normal patterns. Therefore, ContainerGuard meets the urgent need for information protection in container-based big data platform. Our evaluations using real-world datasets show that ContainerGuard achieves excellent detection performance, only introduces about 4.5% of running performance overhead to the platform.

Social Internet of Things is the fusion carrier of social network and Internet of Things. In the social Internet of Things, millions of different intelligent objects connect and communicate with each other, and social data emerge rapidly, which puts forward higher requirements for the rapid dissemination of valuable information. As an important means of information retrieval, the research and application of multi-label classification technology in social Internet of Things environment is relatively few. Characterized by multi-labels cognitive learning, fast response, concept drifting and huge solution space, the multi-label learning becomes more complicated under data stream environment. To overcome the problems above, this paper proposes a multi-label algorithm based on kernel extreme learning machine. In the training phase, both Cholesky matrix decomposition inverse method and matrix block method are deployed on the Spark platform to solve the inverse problem of large matrix, thus improving the modeling efficiency. In addition, incremental learning of the model is realized from the two aspects of example increment and class increment. The online sequential extreme learning machine is improved to adjust the network weights and realize incremental updating of input dimension. Experimental results demonstrate that the proposed method provides an efficient solution to multi-label classification.

Geostatistical modeling, one of the prime motivating applications for exascale computing, is a technique for predicting desired quantities from geographically distributed data, based on statistical models and optimization of parameters. Spatial data are assumed to possess properties of stationarity or non-stationarity via a kernel fitted to a covariance matrix. A primary workhorse of stationary spatial statistics is Gaussian maximum log-likelihood estimation (MLE), whose central data structure is a dense, symmetric positive definite covariance matrix of the dimension of the number of correlated observations. Two essential operations in MLE are the application of the inverse and evaluation of the determinant of the covariance matrix. These can be rendered through the Cholesky decomposition and triangular solution. In this contribution, we reduce the precision of weakly correlated locations to single- or half- precision based on distance. We thus exploit mathematical structure to migrate MLE to a three-precision approximation that takes advantage of contemporary architectures offering BLAS3-like operations in a single instruction that are extremely fast for reduced precision. We illustrate application-expected accuracy worthy of double-precision from a majority half-precision computation, in a context where uniform single-precision is by itself insufficient. In tackling the complexity and imbalance caused by the mixing of three precisions, we deploy the PaRSEC runtime system. PaRSEC delivers on-demand casting of precisions while orchestrating tasks and data movement in a multi-GPU distributed-memory environment within a tile-based Cholesky factorization. Application-expected accuracy is maintained while achieving up to $1.59X$ 1 . 59 X by mixing FP64/FP32 operations on 1536 nodes of HAWK or 4096 nodes of Shaheen II , and up to $2.64X$ 2 . 64 X by mixing FP64/FP32/FP16 operations on 128 nodes of Summit , relative to FP64-only operations. This translates into up to 4.5, 4.7, and 9.1 (mixed) PFlop/s sustained performance, respectively, demonstrating a synergistic combination of exascale architecture, dynamic runtime software, and algorithmic adaptation applied to challenging environmental problems.

Graphics processing units (GPUs) have been a popular choice for the acceleration of iterative tomographic reconstruction algorithms. In this work, two dual-GPU approaches to iterative list-mode reconstruction of PET data are studied. The developed techniques split the image into equal-size cubes and distribute them between the GPUs. Benefiting from caching a plane of the image into the fast shared memory, line-driven forward and back projection kernels are implemented incorporating a shift-invariant point spread function (PSF). In the first approach, lines were individually convolved with the PSF kernel whereas the second approach convolves the reconstructed image with the PSF before the forward projection and after the back projection. The algorithms were implemented using two CUDA-compatible devices with no CPU/GPU data transfer during the reconstruction process. List-mode data obtained from Monte Carlo simulations were used to evaluate the methods. One complete iteration of 107 events on an image of 220 × 220 × 110 voxels is done in less than 510 ms for a non-PSF reconstruction and in 520 to 5220 ms for a PSF reconstruction depending on the PSF and the reconstruction parameters. Dual-GPU memory layout, implementation details, sample images, and performance metrics of the developed algorithms are discussed.

We propose a novel pairwise distance measure between image keypoint sets, for the purpose of large-scale medical image indexing. Our measure generalizes the Jaccard index to account for soft set equivalence (SSE) between keypoint elements, via an adaptive kernel framework modeling uncertainty in keypoint appearance and geometry. A new kernel is proposed to quantify the variability of keypoint geometry in location and scale. Our distance measure may be estimated between $O(N^2)$ image pairs in $O(N~\log~N)$ operations via keypoint indexing. Experiments report the first results for the task of predicting family relationships from medical images, using 1010 T1-weighted MRI brain volumes of 434 families including monozygotic and dizygotic twins, siblings and half-siblings sharing 100%-25% of their polymorphic genes. Soft set equivalence and the keypoint geometry kernel improve upon standard hard set equivalence (HSE) and appearance kernels alone in predicting family relationships. Monozygotic twin identification is near 100%, and three subjects with uncertain genotyping are automatically paired with their self-reported families, the first reported practical application of image-based family identification. Our distance measure can also be used to predict group categories, sex is predicted with an AUC=0.97. Software is provided for efficient fine-grained curation of large, generic image datasets., 10 pages, 8 figures

The binarized neural network (BNN) inference accelerators show great promise in cost-and power-restricted domains. However, the performances of these accelerators are still severely limited by the significant redundancies in BNNs inference. In this brief, we introduce channel-aware sparse accelerator (CAA) to alleviate the performance degradations induced by the redundancies in BNNs while maintaining original accuracies. First, motivated by the observation that the convolution processes of our rebuilt rectangle kernels contain many redundant operations which can be skipped by exploiting the BNN-specific property, we convert the entire original XNOR-popcount convolutions of each neuron into channel-aware-popcount (CAP) operations for all binarized convolutional and fully-connected layers in CAA by employing rectangle kernel simplification strategy and eliminate the unnecessary operations. Meanwhile, these CAP operations can be implemented to directly gain the final output without any extra steps. Furthermore, inspired by our new observations on two specific kinds of properties of the CAP operations, we adopt group pruning approach to save the remaining redundant CAP operations. Experimental results show that our design evaluated on an embedded FPGA achieves 4.2-6.6× inference-speedup, 3.6-5.5× energy-efficiency enhancement, and 1.35× resource-efficiency improvement compared with state-of-the-art works.

This brief presents an event-driven spiking multi-kernel convolution architecture for processing address-event representation (AER) streams from dynamic vision sensor (DVS) chips. The processor architecture is designed based on leaky integrate-and-fire (LIF) neural model, and employs pipeline scheme to accelerate data processing. A new scheme for arranging neuron membrane potentials and kernels in memories is proposed, which enables row-by-row kernel processing and accelerates the multi-kernel convolution computation. An FPGA prototype of the proposed architecture is implemented on a Xilinx Zynq FPGA development board with 100 MHz clock frequency. The proposed processor architecture computes 64 filters with configurable kernel size (from 1x1 to 32x32) on input flow, obtaining the latency of 0.10 us to 10.33 us for convoluting an event, and the energy of 0.12 nJ and 12.08 nJ per event per convolution, respectively.

Fine-grained recognition poses the challenge of discriminating categories with only small subtle visual differences, which can be easily overwhelmed by diverse appearance within categories. Conventional approaches generally locate discriminative parts and then recognize the part-based features. However, we find that tuning the effective receptive field (ERF) of the network to the task plays the key role, which enables significant regions to contribute more to the output. Inspired by the receptive field stimulation mechanism of the visual cortex, we propose a Dynamic Perception framework as a solution. Our framework adapts the ERF by considering the image space and the kernel space simultaneously. In the image space, the Spatial Selective Sampling module is adopted to enlarge informative regions locally. In the kernel space, Spatial Selective Kernel convolution is introduced to adapt different kernel sizes for regions of interest and backgrounds by embedding spatial attention in the multi-path convolution. Extensive experiments on challenging benchmarks, including CUB-200-2011, FGVC-Aircraft, and Stanford Cars, demonstrate that our method yields a performance boost over the state-of-the-art methods.

Existing convolution techniques in artificial neural networks suffer from huge computation complexity, while the biological neural network works in a much more powerful yet efficient way. Inspired by the biological plasticity of dendritic topology and synaptic strength, our method, Learnable Heterogeneous Convolution, realizes joint learning of kernel shape and weights, which unifies existing handcrafted convolution techniques in a data-driven way. A model based on our method can converge with structural sparse weights and then be accelerated by devices of high parallelism. In the experiments, our method either reduces VGG16/19 and ResNet34/50 computation by nearly 5x on CIFAR10 and 2x on ImageNet without harming the performance, where the weights are compressed by 10x and 4x respectively; or improves the accuracy by up to 1.0% on CIFAR10 and 0.5% on ImageNet with slightly higher efficiency. The code will be available on www.github.com/Genera1Z/LearnableHeterogeneousConvolution., Published in Neural Networks journal

Filtering and propagation are two basic operations in image analysis and rendering, and they are also widely used in computer graphics and machine learning. However, the models of filtering and propagation were based on diverse mathematical formulations, which have not been fully understood. This article aims to explore the properties of both filtering and propagation models from a partial differential equation (PDE) learning perspective. We propose a unified PDE learning framework based on nonlinear reaction-diffusion with a guided map, graph Laplacian, and reaction weight. It reveals that: 1) the guided map and reaction weight determines whether the PDE produces filtering or propagation diffusion and 2) the kernel of graph Laplacian controls the diffusion pattern. Based on the proposed PDE framework, we derive the mathematical relations between different models, including learning to diffusion (LTD) model, label propagation, edit propagation, and edge-aware filter. In practical verification, we apply the PDE framework to design diffusion operations with the adaptive kernel to tackle the ill-posed problem of facial intrinsic image analysis (FIIA). A flexible task-aware FIIA system is built to achieve various facial rendering effects, such as face image relighting and delighting, artistic illumination transfer, illumination-aware face swapping, or transfiguring. Qualitative and quantitative experiments show the effectiveness and flexibility of task-aware FIIA and provide new insights on PDE learning for visual analysis and rendering.

This paper considers a variational model for single image blind restoration. By exploiting the fractional-order total variation (FTV) and Lp quasinorm relaxation, a non-convex fractional-order variational model is proposed to restore the image blurred by an unknown blur kernel. A quaternion FTV model is first put forward to exploring more directional information of an image. The new model utilizes non-convex and non-smooth quaternion FTV with Lp quasinorm to constrain the image and L1-norm to constrain the blur kernel, which are unified to a unified regularization framework. Further, an efficient algorithm is proposed to solve the non-convex problem via using the alternating direction minimization. The extensive experiments demonstrate the efficiency and viability of the proposed method and reveal superior performances in comparison with several existing methods.

We build up a decomposition for the flow generated by the heat equation with a real analytic memory kernel. It consists of three components: The first one is of parabolic nature; the second one gathers the hyperbolic component of the dynamics, with null velocity of propagation; the last one exhibits a finite smoothing effect. This decomposition reveals the hybrid parabolic-hyperbolic nature of the flow and clearly illustrates the significant impact of the memory term on the parabolic behavior of the system in the absence of memory terms.

Omni-directional images are becoming more prevalent for understanding the scene of all directions around a camera, as they provide a much wider field-of-view (FoV) compared to conventional images. In this work, we present a novel approach to represent omni-directional images and suggest how to apply CNNs on the proposed image representation. The proposed image representation method utilizes a spherical polyhedron to reduce distortion introduced inevitably when sampling pixels on a non-Euclidean spherical surface around the camera center. To apply convolution operation on our representation of images, we stack the neighboring pixels on top of each pixel and multiply with trainable parameters. This approach enables us to apply the same CNN architectures used in conventional Euclidean 2D images on our proposed method in a straightforward manner. Compared to the previous work, we additionally compare different designs of kernels that can be applied to our proposed method. We also show that our method outperforms in monocular depth estimation task compared to other state-of-the-art representation methods of omni-directional images. In addition, we propose a novel method to fit bounding ellipses of arbitrary orientation using object detection networks and apply it to an omni-directional real-world human detection dataset.

The improved particle swarm optimization algorithm is integrated with variational mode decomposition (VMD) to extract the efficient band-limited intrinsic mode function (BLIMF) of the single and combined power quality events (PQEs). The selected BLIMF of the robust VMD (RVMD) and the privileged Fourier magnitude spectrum (FMS) information are fed to the proposed reduced deep convolutional neural network (RDCNN) for the extraction of the most discriminative unsupervised features. The RVMD-FMS-RDCNN method shows minimum feature overlapping compared with RDCNN and RVMD-RDCNN methods. The feature vector is imported to the novel supervised online kernel random vector functional link network (OKRVFLN) for quick and accurate categorization of complex PQEs. The proposed RVMD-FMS-RDCNN-OKRVFLN method produces excellent recognition capability over RDCNN, RVMD-RDCNN, and RVMD-RDCNN-OKRVFLN methods in noise-free and noisy environments. The unique BLIMF selection, clear detection, descriptive feature extraction, higher learning speed, superior classification accuracy, and robust antinoise performances are considerable importance of the proposed RVMD-FMS-RDCNN-OKRVFLN method. Finally, the proposed method architecture is developed and implemented in a very-high-speed ML506 Virtex-5 FPGA to text, examine, and validate the feasibility, performances, and practicability for online monitoring of the PQEs.

Unsupervised domain adaptation (UDA) has achieved unprecedented success in improving the cross-domain robustness of object detection models. However, existing UDA methods largely ignore the instantaneous data distribution and the sampling strategy during model learning, which could deteriorate the feature representation given large domain shift. In this work, we propose a Self-Guided Adaptation (SGA) model, target at aligning feature representation and transferring object detection models across domains while considering the instantaneous alignment difficulty. The core of SGA is to calculate "hardness" factors for sample pairs indicating domain distance in a kernel space. With the hardness factor, the proposed SGA adaptively indicates the importance of samples and assigns them different constrains. Indicated by hardness factors, Self-Guided Progressive Sampling (SPS) is implemented in an "easy-to-hard" way during model adaptation. Using multi-stage convolutional features, SGA is further aggregated to fully align hierarchical representations of detection models. Extensive experiments on commonly used benchmarks show that SGA improves the state-of-the-art methods with significant margins, while demonstrating the effectiveness on large domain shift.

Mathematical modeling of infectious diseases with non-integer order getting attentions from scientists and researchers day by day. It is obvious that classical models in epidemiology can only be described through a fixed order while models in fractional order derivative are of not fixed order. Having non fixed order the fractional derivative becomes more powerful in modeling real life problems. In the recent era, different novel concepts regarding fractional operators such as the exponential decay and the Mittag–Leffler kernel have been introduced which overcome the limitations of the previous fractional order derivatives. These new operators have been found effective in modeling problems arising in science and engineering. A more recent operator in fractional calculus was introduced that is known as fractal-fractional operator. In this study, we consider this novel approach and apply it to an epidemic model of dengue fever and explore their dynamics. We show some important analysis for the dengue epidemic model in the presence of this new operator. The uniqueness and existence results will be shown. We show the simulation results for the considered model with a novel numerical approach which is not yet considered by anyone for such epidemic model. We obtain results for fractal model when fractional order is one and will have fractional solution when fractal order is one and have when both are present. We show that the fractal-fractional approach is much suitable for an epidemic model rather than fractional operator.

Object detection is an essential task in computer vision. Recently, several convolution neural network (CNN)-based detectors have achieved a great success in natural scenes. However, for optical remote sensing images with a large scale of view, lower proportion of foreground target pixels and drastic differences in object scale present considerable challenges. To address these problems, we propose a novel one-stage detector called the full-scale object detection network (FSoD-Net) which consists of proposed multiscale enhancement network (MSE-Net) backbone cascaded with scale-invariant regression layers (SIRLs). First, MSE-Net provides the multiscale description enhancement by integrated the Laplace kernel with fewer parallel multiscale convolution layers. Second, SIRLs contain three different isolated regression branch layers (i.e., corresponding to small, medium, and large scales), which make default discrete scale bounding boxes (bboxes) cover full-scale object information in regression procedure. A novel specific scale joint loss is also designed that uses the softmax function combined with a strong L₁-norm constraint in each regression branch layer. It can further speed up the convergence and improve the classification scores of predicted bboxes. Finally, extensive experiments are carried on challenge data sets of large-scale dataset for object detection in aerial images (DOTA) and object detection in optical remote sensing images (DIOR) which contain multiple instances from different imaging platforms, and these results demonstrate that FSoD-Net can achieve better performance than other state-of-the-art one-stage detectors, and it can reach a mean average precision (mAP) of 75.33% on DOTA and 71.80% mAP on DIOR, respectively. Especially, the average precision (AP) of tiny object detection can improve 10%-20% approximately.

As the smallest structural unit of feature mapping, the convolution kernel in a deep convolution neural networks (DCNN) convolutional layer is responsible for the input channel features to output channel features. A specific convolution kernel belongs to a specific group from the perspective of the input channel, and it belongs to a specific filter from the perspective of the output channel. If the input and output channels are simultaneously considered in the pruning process, the performance of the pruning model can be further improved. This paper proposes progressive kernel pruning with salient mapping of input-output channels, introduces the concept of input-output channel saliency and defines single-port salient mapping channels and dual-port salient mapping channels. This study demonstrates that single-port salient mapping channels can ensure that each input channel signal has a relatively strong convolution kernel mapped to the output channel, and vice versa. The dual-port salient mapping channel is a channel with high feature mapping abilities from both the input and output directions. Additionally, the average mapping ability measure index is defined, which is used to control the kernel pruning process of the single-port salient mapping channel to switch to the kernel pruning process of the dual-port salient mapping channel. The experimental results and analysis show that the method proposed in this paper can be used to effectively prune a heavyweight model and a lightweight model and can obtain a better accuracy under the conditions of higher compression ratio and acceleration ratios. For example, when VGG-16 is pruned on CIFAR-10, the compression ratio and acceleration ration are 91.00 × and 15.24 × , respectively, and the classification accuracy of the model decreased slightly by 0.22%. When ResNet-101 is pruned on ImageNet, the compression ratio and acceleration ratio are 3.90 × and 3.38 × , respectively, and the classification accuracy of the model decreased slightly by 0.48%. The proposed method is significantly better than state-of-the-art methods.

Synthetic aperture radar (SAR), due to its inherent characteristics, will produce speckle noise, which results in the deterioration of image quality, so the removal of speckle in SAR image is very important for the subsequent high-level image processing. In order to balance the relationship between denoising and texture preservation, we propose a multiscale residual dense dual attention network (MRDDANet) for SAR image denoising. This algorithm can effectively suppress the speckle while fully retaining the texture details of the image. In MRDDANet, shallow features are extracted from the noisy images by multiscale modules with different kernel sizes, and then, the extracted shallow features are mapped to the residual dense dual-attention network to obtain the deep features of SAR image. Finally, the final denoising image is generated through global residual learning. MRDDANet has advantages of both multiscale blocks and residual dense dual attention networks. The dense connection can fully extract features in the image, and the dual-channel attention enables MRDDANet to pay more attention to noise information, which is beneficial to remove noise and keep the details of the original image at the same time. Compared with state-of-the-art algorithms, the results of the experiment indicate that our method not only improves various objective indicators but also shows great advantages in visual effects.

This research article proposes an automatic frame work for detecting COVID -19 at the early stage using chest X-ray image. It is an undeniable fact that coronovirus is a serious disease but the early detection of the virus present in human bodies can save lives. In recent times, there are somany research solutions that have been presented for early detection, but there is still a lack in need of right and even rich technology for its early detection. The proposed deep learning model analysis the pixels of every image and adjudges the presence of virus. The classifier is designed in such a way so that, it automatically detects the virus present in lungs using chest image. This approach uses an image texture analysis technique called granulometric mathematical model. Selected features are heuristically processed for optimization using novel multi scaling deep learning called light weight residual-atrous spatial pyramid pooling (LightRES-ASPP-Unet) Unet model. The proposed deep LightRES-ASPPUnet technique has a higher level of contracting solution by extracting major level of image features. Moreover, the corona virus has been detected using high resolution output. In the framework, atrous spatial pyramid pooling (ASPP) method is employed at its bottom level for incorporating the deep multi scale features in to the discriminative mode. The architectural working starts from the selecting the features from the image using granulometric mathematical model and the selected features are optimized using LightRESASPP- Unet. ASPP in the analysis of images has performed better than the existing Unet model. The proposed algorithm has achieved 99.6% of accuracy in detecting the virus at its early stage. © 2022 Tech Science Press. All rights reserved.

It is well-known that the convolutional neural network (CNN) is an effective method for synthetic aperture radar (SAR) target classification. In the convolutional layer of CNN, convolutional kernels of different sizes can extract different feature information of the target. The small-size kernel can extract the local texture feature information, and the large-size kernel can extract the global contour feature information. Traditional CNN methods usually use fixed-size kernels for convolution, and they generally lose part of the target's feature information, resulting in the inaccurate classification of the SAR targets. This article proposes a novel CNN model based on multikernel-size feature fusion (MKSFF-CNN) for SAR target classification. MKSFF-CNN designs a convolutional methodology with a multichannel parallel topology, it uses convolutional kernels of different sizes to extract the multikernel-size deep features of the SAR target, and then, these features are fused in an optimal way to acquire the lowest loss. Moreover, MKSFF-CNN concatenates the fused features extracted by the convolutional layers of different dimensions to achieve the finest classification. MKSFF-CNN greatly elevates the feature representation completeness of the SAR targets so that more useful feature information can be exploited for SAR target classification. Undoubtedly, MKSFF-CNN can achieve a better classification performance compared with traditional CNN models with a fixed kernel size. The superiority of MKSFF-CNN is validated on the moving and stationary target acquisition and recognition (MSTAR) dataset with the detailed objective and subjective evaluation.

Convolutional neural networks (CNNs) have been widely used in hyperspectral image (HSI) classification. However, a shape-fixed convolution kernel cannot extract appropriate spatial-spectral features. Thus, we propose a novel two-stage classification method based on kernel-guided deformable convolution networks and double-window joint bilateral filter (KDCDWBF) for HSIs. First, according to the calculated similarity map, the shape of the kernel-guided deformable convolution (KDC) is more consistent with the real shape of land covers, so the KDC can extract more pure neighborhood spatial-spectral information. Then, using the piecewise smoothness property of the HSI, a double-window joint bilateral filter (DWJBF) is designed to complete the coarse-to-fine classification stage, which can solve the misclassification problem of single pixels and small regions. Experiments on two HSI datasets demonstrate that the proposed network can achieve better classification performance when compared with other state-of-the-art methods.

Compressed sensing (CS) demonstrates significant potential to improve image quality in 3-D millimeter-wave imaging compared with conventional matched filtering (MF). However, existing sparsity-driven 3-D imaging algorithms always suffer from large-scale storage, excessive computational cost, and nontrivial tuning of parameters due to the huge-dimensional matrix-vector multiplication in complicated iterative optimization steps. In this article, we present a novel range migration (RM) kernel-based iterative-shrinkage thresholding network, dubbed as RMIST-Net, by combining the traditional model-based CS method and data-driven deep learning method for near-field 3-D millimeter-wave (mmW) sparse imaging. First, the measurement matrices in ISTA optimization steps are replaced by RM kernels, by which matrix-vector multiplication is converted to the Hadamard product. Then, the modified ISTA optimization is unrolled into a deep hierarchical architecture, in which all parameters are learned automatically instead of manually tuned. Subsequently, 1000 pairs of oracle images with randomly distributed targets and their corresponding echoes are simulated to train the network. A well-trained RMIST-Net produces high-quality 3-D images from range-focused echoes. Finally, we experimentally prove that RMIST-Net is capable process 512 x 512 large-scale imaging tasks within 1 s. Besides, we compare RMIST-Net with other state-of-the-art methods in near-field 3-D imaging applications. Both simulations and real-measured experiments demonstrate that RMIST-Net produces impressive reconstruction performance while maintaining high computational speed compared with conventional and sparse imaging algorithms.

Object detection is a fundamental part of the interpretation of remote sensing imagery. The one-stage object detector has been adopted into this field because of its high computational efficiency. However, this detector suffers from the misalignment among predefined anchor, object, and feature extracted by standard convolution kernel both in spatial and scale. It limits the further improvement of performance, especially for the long-narrow and multiscale geospatial objects. In this article, the problem is defined as the feature misalignment problem. To deal with this issue, an efficient feature aligned single-shot detector (ASSD) is proposed, which consists of two modules: a novel pseudo anchor proposal module (PAPM) and a flexible context-based feature alignment module (CFAM). The PAPM replaces the regular anchor group with the proposed core anchor and refines it to get aligned locations. It can tackle the spatial misalignment between anchors and their corresponding objects and alleviate the negative/positive imbalance problem. Then, the CFAM adaptively adjusts the sampling points of the convolution kernel and collects the context information according to the aligned core anchor. This plug-and-play module can effectively rectify the misalignment between kernel and objects and extract aligned and robust features. A series of comprehensive experiments are conducted on two large-scale public remote sensing object detection datasets. Experiment results suggest that the proposed method is effective to alleviate the misalignment problem. Compared with the baseline model, the detection accuracy is improved by 8.5% mAP and 11.0% mAP on the challenging benchmark for object detection in optical remote sensing image (DIOR) and a large-scale dataset for object detection in aerial image (DOTA) dataset, respectively. Our best-resulting model achieves the state-of-the-art performance, surpassing other one-stage detectors both on the two datasets at a high detection speed of 21 FPS.

The fusion of infrared and visible images combines the advantages of thermal radiation information in infrared images and texture information in visible images. To preserve salient targets and obtain better fusion results, this paper proposes a novel infrared and visible image fusion method based on dual-kernel side window filtering and detail optimization with S-shaped curve transformation. First, a dual-kernel side window box filter (DSWBF) with adaptive filter kernel size is designed to extract the base and detail layers of the source images. Then, a saliency-based fusion rule is proposed for the base layers to highlight the salient regions of the infrared image. Next, a detail optimization module is constructed based on an S-shaped curve transformation and guided filter to optimize the detail layer of the infrared image. The optimized detail layer is then merged with the detail layer of the visible image to obtain the fused detail layer. Finally, the fusion image is created by reconstructing the fused base and detail layers. Experimental results on the two public datasets indicate that compared with state-of-the-art methods, the proposed method can obtain better fusion performance in terms of subjective and objective evaluations.

Blind pansharpening addresses the problem of generating a high spatial-resolution multi-spectral (HRMS) image given a low spatial-resolution multi-spectral (LRMS) image with the guidance of its associated spatially misaligned high spatial-resolution panchromatic (PAN) image without parametric side information. In this paper, we propose a fast approach to blind pansharpening and achieve state-of-the-art image reconstruction quality. Typical blind pansharpening algorithms are often computationally intensive since the blur kernel and the target HRMS image are often computed using iterative solvers and in an alternating fashion. To achieve fast blind pansharpening, we decouple the solution of the blur kernel and of the HRMS image. First, we estimate the blur kernel by computing the kernel coefficients with minimum total generalized variation that blur a downsampled version of the PAN image to approximate a linear combination of the LRMS image channels. Then, we estimate each channel of the HRMS image using local Laplacian prior to regularize the relationship between each HRMS channel and the PAN image. Solving the HRMS image is accelerated by both parallelizing across the channels and by fast numerical algorithms for each channel. Due to the fast scheme and the powerful priors we used on the blur kernel coefficients (total generalized variation) and on the cross-channel relationship (local Laplacian prior), numerical experiments demonstrate that our algorithm outperforms state-of-the-art model-based counterparts in terms of both computational time and reconstruction quality of the HRMS images., Comment: 17 pages, 47 figures, journal, accepted by IEEE Transactions on Geoscience and Remote Sensing

Integrating compressed sensing (CS) with millimeter-wave (mmW) holography has shown great potential to achieve lightweight onboard hardware, low sampling ratio, and high-speed sensing. However, conventional CS-driven algorithms are always limited by nontrivial adjusting of parameters and excessive computational cost caused by plenty of iterations. To address this problem, we propose a lightweight model-based deep learning framework (LFIST-Net) for mmW 3-D holography, by combining the interpretability of fast iterative shrinkage-thresholding algorithm (FISTA) and tuning-free merit of data-driven deep neural network. First, the single-frequency (SF) holographic imaging technique is integrated into FISTA, which serves as the sensing kernels, to avoid large-scale matrix multiplications. Subsequently, the kernel-based FISTA (KFISTA) is mapped into layer-fixed and parameter-learnable LFIST-Net, whose weights are relaxed to be layer-varied. The updating of key parameters in LFIST-Net, including step sizes, thresholds, and momentum coefficients, are regularized by soft-plus function to ensure the non-negativity and monotonicity. As for 3-D holography implementation, the ``1-D + 2-D'' scheme is adopted, where the matched filtering (MF) and well-trained LFIST-Net are used for range focusing and reconstructions of azimuth slices. Without losing efficiency, the range-focused subechoes are processed parallelly in 3-D cube form. Experiments, including both simulated and measured tests based on a commercial mmW radar, prove that LFIST-Net is capable of reconstructing the imaging scene precisely. In particular, in near-field mmW 3-D holography tests, both numerical and visual results demonstrate LFIST-Net yields compelling reconstruction performance while maintaining high computational speed compared with MF-based, conventional CS-driven, and network-based methods.

In this letter, we introduce MappingConvSeg, a continuous convolution network for semantic segmentation of large-scale point clouds. In particular, a conceptually simple, end-to-end learnable, and continuous convolution operator is proposed for learning spatial correlation of unstructured 3-D point clouds. For each local point set, the unstructured point features are first mapped onto a series of learned kernel points based on the spatial relationship, and the continuous convolution is then applied to capture specific local geometrical patterns. Taking the proposed mapping convolution operation as the building block, a hierarchical network is then built for large-scale point cloud semantic segmentation. Experimental results conducted on two public benchmarks, including Toronto-3D and Stanford large-scale 3-D Indoor Spaces (S3DIS) dataset, demonstrate the superiority of the proposed method.

In recent days, facial emotion recognition techniques are considered important and critical due wide array of application domains use facial emotion as part of their workflow and analytics gathering. Reduced recognition rate, inefficient computation and increased time consumption are the major drawbacks of these techniques. To overcome these issues, this paper develops the new facial emotion recognition technique named as Minimal Angular Feature Oriented Neural Network based Emotion Prediction (MAFONN-EP). Initially, the input video sequence are categorized into image frames that are pre-processed by eliminating the noise by using Weighted Median Filtering (WMF) technique and to separate the background and foreground regions using Edge Preserved Background Separation and Foreground Extraction (EPBSFE) technique. Then, the set of texture patterns are extracted based on four key parts such as two eyes, nose and mouth by using the Minimal Angular Deviation (MAD) technique. Particular features are selected by employing the Cuckoo Search based Particle Swarm Optimization (CS-PSO) technique that also reduces the feature dimensionality. Finally, the Weight Based Pointing Kernel Classification (WBPKC) technique is employed for recognizing the emotion. In the experimental results, the performance of the proposed technique is analyzed and compared with different performance measures like accuracy, sensitivity, specificity, precision, recall.

We devote our interest in this work to investigate the sufficient conditions for the existence, uniqueness, and Ulam-Hyers stability of solutions for a new fractional system in the frame of Atangana-Baleanu-Caputo fractional operator with multi-parameters Mittag-Leffler kernels investigated lately by Abdeljawad (Chaos: An Interdisciplinary J. Nonlinear Sci. Vol. 29, no. 2, (2019): 023102). Moreover, the continuous dependence of solution and $ \delta $-approximate solutions are analyzed to such a system. Our approach is based on Banach's and Schaefer's fixed point theorems and some mathematical techniques. In order to illustrate the validity of our results, an example is given.

Many data mining workloads are being analyzed in large-scale distributed cloud computing environments which provide nearly infinite resources with diverse hardware configurations. To maintain cost-efficiency in such environments, understanding the characteristics and estimating the overheads of a distributed matrix multiplication task that is a core computation kernel in many machine learning algorithms are essential. This study aims to propose a Matrix Multiplication Performance Estimator on Cloud (MPEC) algorithm. The proposed algorithm predicts the latency incurred when executing distributed matrix multiplication tasks of various input sizes and shapes with diverse instance types and a different number of worker nodes on cloud computing environments. To achieve this goal, we first analyze the characteristics of distributed matrix multiplication tasks. With characteristics generated from qualitative analysis, we propose to apply an ensemble of non-linear regression algorithm to predict the execution time of arbitrary matrix multiplication tasks. Thorough experimental results reveal that the proposed algorithm demonstrates higher accuracy than a state-of-the-art machine learning task performance estimation engine, Ernest, by decreasing the Mean Absolute Percentage Error (MAPE) in half.

Inevitable corruption and degeneration make the performance of subsequent high-level semantic tasks in hyperspectral images (HSIs) unsatisfactory. Despite that many denoising methods have been proposed, significant room for improvement still remains. To better suppress noise and preserve the HSI spatial–spectral structure, we propose an attention-based Octave dense network. A separable spectral feature extraction module is introduced to extract the spatial–spectral features consistent with the structure prior. The extracted features are fine-tuned by the attention module in both channel and spatial domains; then, several dense denoising blocks are elaborately employed to focus on noise feature learning; in order to focus on high-frequency features, which usually have more noise information, we introduce the Octave kernel to implement these blocks. Experiments based on simulated and real-world noisy images demonstrate that the proposed method outperforms the existing traditional and learning-based methods in both quantitative evaluations and visual effects, benefiting the subsequent classification task. In addition, the effectiveness of each module is proven by ablation experiments. Our source code is made available at: https://github.com/LbzSteven/AODN.

Hyperspectral images (HSIs) have gained high spectral resolution due to recent advances in spectral imaging technologies. This incurs problems, such as an increased data scale and an increased number of bands for HSIs, which results in a complex correlation between different bands. In the applications of remote sensing and earth observation, ground objects represented by each HSI pixel are composed of physical and chemical non-Euclidean structures, and HSI classification (HIC) is becoming a more challenging task. To solve the above problems, we propose a framework based on a deep attention graph convolutional network (DAGCN). Specifically, we first integrate an attention mechanism into the spectral similarity measurement to aggregate similar spectra. Therefore, we propose a new similarity measurement method, i.e., the mixed measurement of a kernel spectral angle mapper and spectral information divergence (KSAM-SID), to aggregate similar spectra. Considering the non-Euclidean structural characteristics of HSIs, we design deep graph convolutional networks (DeepGCNs) as a feature extraction method to extract deep abstract features and explore the internal relationship between HSI data. Finally, we dynamically update the attention graph adjacency matrix to adapt to the changes in each feature graph. Experiments on three standard HSI data sets, namely, the Indian Pines, Pavia University, and Salinas data sets, demonstrate that the DAGCN outperforms the baselines in terms of various evaluation criteria. For example, on the Indian Pines data set, the overall accuracy of the proposed method achieves 98.61% when the training sample is 10%.

Recently, feature pyramid has been widely exploited in remote sensing detectors, which greatly alleviates the problem arising from scale variation across objects in remote sensing images. However, these object detectors with feature pyramid give insufficient consideration that objects in remote sensing images usually maintain symmetrical shape. To address this issue, we propose an anchor-free-based detector called Centerness-Aware Network (CANet), which could capture the symmetrical shape of objects in remote sensing images. The kernel structure of CANet is a new Centerness-Aware Model (CAM) that contains three components: Multiscale Centerness Descriptor (MSCD), Centerness Detection Head (CDH), and Feature Selective Module (FSM). Considering that symmetrical objects will maintain a rigid appearance around their center region, three components are integrated into the feature pyramid to extract and utilize the features around the center region. More precisely, the MSCD is embedded into the feature pyramid and highlights the center of current objects through the attention mechanism. Guided by the MSCD, the CDH could accurately capture the center of objects by per-pixel prediction. Furthermore, the FSM is connected to the CDH, which guides the CDH to adaptively select the optimal feature level from the pyramidal features. The selected feature level could describe the best semantic information around the center region, which helps the network progressively fit the symmetrical shape of remote sensing objects. Besides, we also design the hybrid loss function to effectively train CAM in the end-to-end way. The experiments show that our network is competitive with some state-of-the-art detection networks.

Currently, the top-performing face detectors use Convolution Neural Networks (CNN). These detectors have high performance but require large computational resources. Hence, it is a challenge to use these for practical applications with limited resources. To address this problem, this research paper proposes a single-stage face detector for practical applications with better accuracy and speed. The proposed face detector is a lightweight and fast CNN-based face detector. To achieve fast detection speed, by swiftly reducing the size of the input feature space through a carefully designed architecture using depth-wise convolution and Receptive Field Blocks (RBF). The high accuracy of the method is achieved by preserving the feature information from the early layers using CReLU (Concatenate Rectified Linear Units) activation in the CNN architecture. To detect faces of different pixel sizes, we used parallel RF blocks, which use multiple filters of different kernel sizes and dilation. The proposed method was tested on benchmark datasets such as WIDER FACE, AFW, and PASCAL faces, and performed well when compared to recent state-of-the-art real-time face detection algorithms.

Learning a unified embedding for utterance-level video attracts significant attention recently due to the rapid development of social media and its broad applications. An utterance normally contains not only spoken language but also the nonverbal behaviors such as facial expressions and vocal patterns. Instead of directly learning utterance embedding based on low-level features, we firstly explore high-level representation for each modality separately via an unimodal representation learning gyroscope structure. In this way, the learnt unimodal representations are more representative and contain more abstract semantic information. In the gyroscope structure, we introduce multi-scale kernel learning, ‘channel expansion’ and ‘channel fusion’ operations to explore high-level features both spatially and channelwise. Another insight of our method lies in that we fuse representations of all modalities to obtain a unified embedding by interpreting fusion procedure as the flow of inter-modality information between various modalities, which is more specialized in terms of the information to be fused and the fusion process. Specifically, considering that each modality carries modality-specific and cross-modality interactions, we innovate to decompose unimodal representations into intra- and inter-modality dynamics using gating mechanism, and further fuse the inter-modality dynamics by passing them from previous modalities to the following one using a recurrent neural fusion architecture. Extensive experiments demonstrate that our method achieves state-of-the-art performance on multiple benchmark datasets.

The depthwise separable convolution is commonly seen in convolutional neural networks (CNNs), and is widely used to reduce the computation overhead of a standard multi-channel 2D convolution. Existing implementations of depthwise separable convolutions target accelerating model training with large batch sizes with a large number of samples to be processed at once. Such approaches are inadequate for small-batch-sized model training and the typical scenario of model inference where the model takes in a few samples at once. This article aims to bridge the gap of optimizing depthwise separable convolutions by targeting the GPU architecture. We achieve this by designing two novel algorithms to improve the column and row reuse of the convolution operation to reduce the number of memory operations performed on the width and the height dimensions of the 2D convolution. Our approach employs a dynamic tile size scheme to adaptively distribute the computational data across GPU threads to improve GPU utilization and to hide the memory access latency. We apply our approach on two GPU platforms: an NVIDIA RTX 2080Ti GPU and an embedded NVIDIA Jetson AGX Xavier GPU, and two data types: 32-bit floating point (FP32) and 8-bit integer (INT8). We compared our approach against cuDNN that is heavily tuned for the NVIDIA GPU architecture. Experimental results show that our approach delivers over $2\times$ 2 × (up to $3\times$ 3 × ) performance improvement over cuDNN. We show that, when using a moderate batch size, our approach averagely reduces the end-to-end training time of MobileNet and EfficientNet by 9.7 and 7.3 percent respectively, and reduces the end-to-end inference time of MobileNet and EfficientNet by 12.2 and 11.6 percent respectively.

Gradual advances have occurred in the design of micro-artificial intelligence for resource-limited hardware. A high-resolution (HR) image reconstruction module is indispensable for video analytics chips or devices at the edge. This paper proposes a low-cost and learning-based interpolation method for HR image reconstruction. The proposed method generates reconstructed pixels by processing reference pixels with optimal weights, which are pre-trained by solving the minimum mean square error problem for real images. To reduce the number of computation units and the usage of learned weights, a cross-directional interpolation architecture, which includes a vertical kernel and a horizontal kernel, is adopted. Moreover, a one-dimensional feature discriminator is proposed to improve the quality of up-scaled images efficiently. The main benefit of the proposed method is that it requires a small number of computation units but can still produce high-quality images. The hardware architecture of the proposed method was implemented on a field-programmable gate array (FPGA) by using Xilinx UltraScale+ ZCU102 and an application-specific integrated circuit (ASIC) by using TSMC’s 0.13-μm technology. On the ASIC, the proposed hardware required only approximately 60K gate counts and 50 KBytes of memory. The experimental results indicate that the average peak signal-to-noise ratio of the up-scaled images reached 35.92 dB for the Set-5 dataset. The throughput of the proposed hardware was at least 1000 Mpixels/s on the FPGA and 1200 Mpixels/s on the ASIC, which indicates that the proposed hardware can handle a target resolution higher than 4K in real time.