Search

Existing convolution techniques in artificial neural networks suffer from huge computation complexity, while the biological neural network works in a much more powerful yet efficient way. Inspired by the biological plasticity of dendritic topology and synaptic strength, our method, Learnable Heterogeneous Convolution, realizes joint learning of kernel shape and weights, which unifies existing handcrafted convolution techniques in a data-driven way. A model based on our method can converge with structural sparse weights and then be accelerated by devices of high parallelism. In the experiments, our method either reduces VGG16/19 and ResNet34/50 computation by nearly 5x on CIFAR10 and 2x on ImageNet without harming the performance, where the weights are compressed by 10x and 4x respectively; or improves the accuracy by up to 1.0% on CIFAR10 and 0.5% on ImageNet with slightly higher efficiency. The code will be available on www.github.com/Genera1Z/LearnableHeterogeneousConvolution., Published in Neural Networks journal

We devote our interest in this work to investigate the sufficient conditions for the existence, uniqueness, and Ulam-Hyers stability of solutions for a new fractional system in the frame of Atangana-Baleanu-Caputo fractional operator with multi-parameters Mittag-Leffler kernels investigated lately by Abdeljawad (Chaos: An Interdisciplinary J. Nonlinear Sci. Vol. 29, no. 2, (2019): 023102). Moreover, the continuous dependence of solution and $ \delta $-approximate solutions are analyzed to such a system. Our approach is based on Banach's and Schaefer's fixed point theorems and some mathematical techniques. In order to illustrate the validity of our results, an example is given.

Deep neural networks (DNNs) and convolutional neural networks (CNNs) have demonstrated greater robustness and accuracy in classifying two-dimensional images and three-dimensional point clouds compared to more traditional machine learning approaches. However, their main drawback is the need for large quantities of semantically labeled training data sets, which are often out of reach for those with resource constraints. In this study, we evaluated the use of simulated 3D point clouds for training a CNN learning algorithm to segment and classify 3D point clouds of real-world urban environments. The simulation involved collecting light detection and ranging (LiDAR) data using a simulated 16 channel laser scanner within the the CARLA (Car Learning to Act) autonomous vehicle gaming environment. We used this labeled data to train the Kernel Point Convolution (KPConv) and KPConv Segmentation Network for Point Clouds (KP-FCNN), which we tested on real-world LiDAR data from the NPM3D benchmark data set. Our results showed that high accuracy can be achieved using data collected in a simulator.

The geometric and color features of agricultural material along with related physical properties are critical to characterize and express its physical quality. The experiments were conducted to classify the physical characteristics like size, shape, color and texture and then workout the relationship between manual observations and using image processing techniques for weight and volume of the four wheat refractions i.e. sound, damaged, shriveled and broken grains of wheat variety PBW 725. A flatbed scanner was used to acquire the images and digital image processing method was used to process the images and output of image analysis was compared with the actual measurements data using digital vernier caliper. A linear relationship was observed between the axial dimensions of refractions between manual measurement and image processing method with R2 in the range of 0.798–0.947. The individual kernel weight and thousand grain weight of the refractions were observed to be in the range of 0.021–0.045 and 12.56–46.32 g respectively. Another linear relationship was found between individual kernel weight and projected area estimated using image processing methodology with R2 in the range of 0.841–0.920. The sphericity of the refractions varied in the range of 0.52–0.71. Analyses of the captured images suggest ellipsoid shape with convex geometry while the same observation was recorded by physical measurements also. A linear relationship was observed between the volume of refractions derived from measured dimensions and calculated from image with R2 in the range of 0.845–0.945. Various color and grey level co-variance matrix texture features were extracted from acquired images using the open-source Python programming language and OpenCV library which can exploit different machine and deep learning algorithms to properly classify these refractions.

In this paper, we consider a fractional-order model of a brushless DC motor. To develop a mathematical model, we use the concept of the Liouville–Caputo noninteger derivative with the Mittag-Lefler kernel. We find that the fractional-order brushless DC motor system exhibits the character of chaos. For the proposed system, we show the largest exponent to be 0.711625. We calculate the equilibrium points of the model and discuss their local stability. We apply an iterative scheme by using the Laplace transform to find a special solution in this case. By taking into account the rule of trapezoidal product integration we develop two iterative methods to find an approximate solution of the system. We also study the existence and uniqueness of solutions. We take into account the numerical solutions for Caputo Liouville product integration and Atangana–Baleanu Caputo product integration. This scheme has an implicit structure. The numerical simulations indicate that the obtained approximate solutions are in excellent agreement with the expected theoretical results.

Non-local Means (NLMs) play essential roles in image denoising, restoration, inpainting, etc., due to its simple theory but effective performance. However, when the noise increases, the denoising accuracy of NLMs decreases significantly. This paper further develop the NLMs-based denoising method to remove noise with less loss of image details. It is realized by embedding an optimal graph edge weights driven NLMs kernel into a multi-layer residual compensation framework. Unlike the patch similarity-based weights in the traditional NLMs filters, the edge weights derived from the optimal graph Laplacian regularization consider (1) the distance between the target pixel and the candidate pixel, (2) the local gradient and (3) the patch similarity. After defining the weights, the graph-based NLMs kernel is then put into a multi-layer framework. The corresponding primal and residual terms at each layer are finally fused with learned weights to recover the image. Experimental results show that our method is effective and robust, especially for piecewise smooth images.

Corona Virus Disease-2019 (COVID-19) is a global pandemic which is spreading briskly across the globe. The gold standard for the diagnosis of COVID-19 is viral nucleic acid detection with real-time polymerase chain reaction (RT-PCR). However, the sensitivity of RT-PCR in the diagnosis of early-stage COVID-19 is less. Recent research works have shown that computed tomography (CT) scans of the chest are effective for the early diagnosis of COVID-19. Convolutional neural networks (CNNs) are proven successful for diagnosing various lung diseases from CT scans. CNNs are composed of multiple layers which represent a hierarchy of features at each level. CNNs require a big number of labeled instances for training from scratch. In medical imaging tasks like the detection of COVID-19 where there is a difficulty in acquiring a large number of labeled CT scans, pre-trained CNNs trained on a huge number of natural images can be employed for extracting features. Feature representation of each CNN varies and an ensemble of features generated from various pre-trained CNNs can increase the diagnosis capability significantly. In this paper, features extracted from an ensemble of 5 different CNNs (MobilenetV2, Shufflenet, Xception, Darknet53 and EfficientnetB0) in combination with kernel support vector machine is used for the diagnosis of COVID-19 from CT scans. The method was tested using a public dataset and it attained an area under the receiver operating characteristic curve of 0.963, accuracy of 0.916, kappa score of 0.8305, F-score of 0.91, sensitivity of 0.917 and positive predictive value of 0.904 in the prediction of COVID-19.

Fractional calculus as was predicted by Leibniz to be an anecdote, has nowadays evolved to become a centre of interest for many researchers from various backgrounds. Their works have made signi…cant contribution to the general topic of fractional calculus and made it to be widely applicable in their respective …elds. This has permitted every researcher to formulate their own topic of interest and lay down their own foundation thereof. As a result, multiple innovative ideas had emerged; which caused signi…cant divisions regarding fractional calculus in the past three years. Some researchers believed fractional di¤erential operators should not have a nonsingular kernel; while others strongly believed that due to the complexity of nature, fractional di¤erential operators can have either singular or non-singular kernels. This contradiction in thoughts had led to the publication of few papers which are against di¤erential operators with non-singular kernels, causing some negative impacts. Thus, publishers and some editors in chief are concerned about the future of fractional calculus; which generally brought confusion among vibrant and innovative young researchers who desire to apply fractional calculus within their respective …elds. Therefore, this work is aimed at developing a mathematical model that could be used to depict the survival of fractional calculus. Six classes are herein considered to construct a mathematical model with six ordinary di¤erential equations. All elementary analysis, equilibrium points, stability analysis of equilibrium points, existence and uniqueness of system solutions are performed. Additionally, a new analysis including strength number that accounts for the accelerative information of nonlinear and linear parts of a given epidemiological model is introduced to better understand the system. This new number is deemed more informative than the well-known reproductive number. Therefore, an analysis of the second derivative of the Lyapunov function as well as an analysis of the second derivative of each class is applied to assess how a wave could be detected. It is strongly believed that this new analysis will particularly open new doors within the …eld of epidemiological modelling; which will aid researchers to better understand the spread of infectious diseases. The stochastic version of the suggested model was also investigated and numerical simulations were performed. The obtained reproductive number, strength number, extinction of criticism together with numerical simulation, revealed that the …eld of fractional calculus will be stable. The present criticism 1 will therefore have no signi…cant e¤ect in the near future. The outcome of this work further predicted that more researchers will join the …eld of fractional calculus, and will utilize both di¤erential operators with singular and non-singular kernels.

In this publication, the Luneberg integrals are revisited and the conditions of the using of such integrals have been recalled. Additivity law of Luneberg's integrals and the link with the Frenel kernel for the propagation are discussed. By means of the definition of the Luneberg's integrals,the propagation of a vectorial electromagnetic field (Hertz potentials) is developped and a new approach of the computation have been proposed based on Zernike polynomials. With this new approch simulations of holograms is illustrated in the case of the digital in-line holography with an opaque disk.

The present article deals with the evaluation of the Hankel transforms involving Bessel matrix functions in the kernel. Moreover, these transforms are associated with products of certain elementary functions and generalized Bessel matrix polynomials. As applications, many useful special cases are discussed. Further, the current results are more general to the previous one. In addition to, these results are yielded to more results in the modern integral transforms with special matrix functions.

This paper presents the free vibration and buckling analyses of functionally graded carbon nanotube-reinforced (FG-CNTR) laminated non-rectangular plates, i.e., quadrilateral and skew plates, using a four-nodded straight-sided transformation method. At first, the related equations of motion and buckling of quadrilateral plate have been given, and then, these equations are transformed from the irregular physical domain into a square computational domain using the geometric transformation formulation via discrete singular convolution (DSC). The discretization of these equations is obtained via two-different regularized kernel, i.e., regularized Shannon’s delta (RSD) and Lagrange-delta sequence (LDS) kernels in conjunctions with the discrete singular convolution numerical integration. Convergence and accuracy of the present DSC transformation are verified via existing literature results for different cases. Detailed numerical solutions are performed, and obtained parametric results are presented to show the effects of carbon nanotube (CNT) volume fraction, CNT distribution pattern, geometry of skew and quadrilateral plate, lamination layup, skew and corner angle, thickness-to-length ratio on the vibration, and buckling analyses of FG-CNTR-laminated composite non-rectangular plates with different boundary conditions. Some detailed results related to critical buckling and frequency of FG-CNTR non-rectangular plates have been reported which can serve as benchmark solutions for future investigations.

This manuscript is devoted to a study of the existence and uniqueness of solutions to a mathematical model addressing the transmission dynamics of the coronavirus-19 infectious disease (COVID-19). The mentioned model is considered with a nonsingular kernel type derivative given by Caputo–Fabrizo with fractional order. For the required results of the existence and uniqueness of solution to the proposed model, Picard’s iterative method is applied. Furthermore, to investigate approximate solutions to the proposed model, we utilize the Laplace transform and Adomian’s decomposition (LADM). Some graphical presentations are given for different fractional orders for various compartments of the model under consideration.

As a fundamental preprocessing technique, remote sensing image motion deblurring is important for visual understanding tasks. Most conventional approaches formulate the image motion deblurring task as a kernel estimation. Because the kernel estimation is a highly ill-posed problem, many priors have been applied to model the images and kernels. Even though these methods have obtained relatively better performances, they are usually time-consuming and not robust for different conditions. To address this problem, we propose a multitask learning mechanism for remote sensing image motion deblurring in this article, which contains an image restoration subtask and an image texture complexity recognition one. First, we consider the image motion deblurring problem as a domain transformation problem, from the blurred domain to a clear one. Specifically, the blurred domain represents the data space consisted of blurring images, and the definition of clear domain is similar. Second, we design a novel weighted attention map loss to enhance the reconstruction capability of the restoration subbranch for difficult local regions. Third, based on the restoration subbranch, a recognition subbranch is incorporated into the framework to guide the deblurring process, which provides the auxiliary texture complexity information to help the optimization of restoration subbranch. Additionally, in order to optimize the proposed network, we construct three large-scale datasets, and each sample in the dataset contains a clear image, a blurred image, and its texture label obtained by corresponding texture complexity. Finally, the experimental results on three constructed datasets demonstrate the robustness and the effectiveness of the proposed method.

Non-convex sparsity-inducing penalties outperform their convex counterparts, but generally sacrifice the cost function convexity. As a middle ground, we propose the sharpening sparse regularizers (SSR) framework to design non-separable non-convex penalties that induce sparsity more effectively than convex penalties such as $\ell _1$ and nuclear norms, but without sacrificing the cost function convexity. The overall problem convexity is preserved by exploiting the data fidelity relative strong convexity. The framework constructs penalties as the difference of convex functions, namely the difference between convex sparsity-inducing penalties and their smoothed versions. We propose a generalized infimal convolution smoothing technique to obtain the smoothed versions. Furthermore, SSR recovers and generalizes several non-convex penalties in the literature as special cases. The SSR framework is applicable to any sparsity regularized least squares ill-posed linear inverse problem. Beyond regularized least squares, the SSR framework can be extended to accommodate Bregman divergence, and other sparsity structures such as low-rankness. The SSR optimization problem can be formulated as a saddle point problem, and solved by a scalable forward-backward splitting algorithm. The effectiveness of the SSR framework is demonstrated by numerical experiments in different applications.

In this paper our group proposes and designs a lightweight convolutional neural network (CNN) for detecting facial emotions in real-time and in bulk to achieve a better classification effect. We verify whether our model is effective by creating a real-time vision system. This system employs multi-task cascaded convolutional networks (MTCNN) to complete face detection and transmit the obtained face coordinates to the facial emotions classification model we designed firstly. Then it accomplishes the task of emotion classification. Multi-task cascaded convolutional networks have a cascade detection feature, one of which can be used alone, thereby reducing the occupation of memory resources. Our expression classification model employs Global Average Pooling to replace the fully connected layer in the traditional deep convolution neural network model. Each channel of the feature map is associated with the corresponding category, eliminating the black box characteristics of the fully connected layer to a certain extent. At the same time, our model marries the residual modules and depth-wise separable convolutions, reducing large quantities of parameters and making the model more portable. Finally, our model is tested on the FER-2013 dataset. It only takes 3.1% of the 16GB memory, that is, only 0.496GB memory is needed to complete the task of classifying facial expressions. Not only can our model be stored in an 872.9 kilobytes file, but also its accuracy has reached 67% on the FER-2013 dataset. And it has good detection and recognition effects on those figures which are out of the dataset.

Reduced-reference stereoscopic image quality assessment (RRSIQA) models evaluate stereoscopic image quality degradation with partial information about the “ideal-quality” reference stereopair. On one hand, sparse representation in recent theoretical studies of visual cognition has been proved to resemble the strategy used to represent natural images in the primary visual cortex. On the other hand, the joint statistics of gradient magnitude (GM) and Laplacian of Gaussian (LOG) features are popularly utilized to form image semantic structures. Motivated by these findings, we present a new RRSIQA metric using gradient sparse representation and structural degradation in this paper. Concretely, the proposed metric is based on two main tasks: the first task extracts the distribution statistics of visual primitives by gradient sparse representation, while the second task measures structural degradation of stereoscopic image due to the presence of distortion by extracting the joint statistics of GM and LOG features. The former, so-called the binocular perceptual visual information (PVI), aims to effectively integrates the gradient map that is sparser than the image itself. Especially, the process of binocular fusion is simulated by using the mutual information of the gradient-based visual primitives between left and right view’s images as binocular cue. Furthermore, the perceptual loss vectors are taken as the differences of binocular perceptual visual information and structural degradation between reference and distorted stereopairs. Finally, the perceptual loss vectors are utilized to calculate the quality score by a prediction function which is trained using kernel ridge regressing (KRR). The experiments are performed on the popular LIVE 3D IQA databases and Waterloo IVC 3D databases, and experimental results show highly competitive performance with the state-of-the-art algorithms. Moreover, in some challenging cases with particular asymmetric distortion types, the proposed metric can achieves the best quality prediction accuracy in LIVE 3D phase II and Waterloo IVC 3D Phase II.

Image feature detection is a key task in computer vision. Scale Invariant Feature Transform (SIFT) is a prevalent and well known algorithm for robust feature detection. However, it is computationally demanding and software implementations are not applicable for real-time performance. In this paper, a versatile and pipelined hardware implementation is proposed, that is capable of computing keypoints and rotation invariant descriptors on-chip. All computations are performed in single precision floating-point format which makes it possible to implement the original algorithm with little alteration. Various rotation resolutions and filter kernel sizes are supported for images of any resolution up to ultra-high definition. For full high definition images, 84 fps can be processed. Ultra high definition images can be processed at 21 fps.

A novel design method using unsigned input is proposed for a high-density and parallel synaptic string architecture capable of bit-wise operation and bit-counting utilizing NAND flash memory. Though the NAND flash memory has a cell string structure, unsigned binary input enables analogue and parallel bit-counting in NAND flash memory, while achieving high accuracy comparable to that of signed input. Adopting unsigned input allows using current sense amplifier as neuron circuit, which reduces burden of peripheral circuits. In addition, the operation scheme for convolution layers is proposed for parallel convolution operation utilizing NAND flash memory. We show that sufficiently low device variation of 7.2 % obtained by applying 1 erase or program pulse achieves accuracy of 98.12 % and 87.12 % for MNIST and CIFAR 10 patterns, respectively.

Matrix completion methods have been widely applied in images recovery and recommendation systems. Most of them are only based on the low-rank characteristics of matrices to predict the missing entries. However, these methods lack consideration of local information. To further improve the performance of matrix completion. In this paper, we propose a novel model based on matrix decompositions and matrix local information. Specifically, we update a number of rank-one matrices, which circumvented the rank estimation in matrix decomposition. And a penalty function is designed to punish singular values without introducing additional parameters. The local information component extracts similar information by an adaptive filter via convolution operation which kernel is obtained by the minimum variance. Finally, we integrate matrix decomposition and local information components via different weights. We apply the proposed method to real-world image datasets and recommendation system datasets. The experimental results demonstrate the proposed model has a lower error and better robustness than several competing matrix completion methods.

Most of the existing image style transfer algorithms transfer the whole image style as a whole. Style feature is a set of correlation matrix based on style image, namely Gram matrix. Each matrix is a global description of the style image. This kind of methods can perform well in the insensitive semantic scenes (such as the style transfer between landscape photos), but in the sensitive semantic scenes (such as the style transfer between portrait photos), the problem of semantic mismatch will be highlighted, such as transferring the background texture of the style image to the foreground of the target image. Although the existing research takes the manually annotated semantic image as an input of the algorithm, and then guides the style transfer based on the semantic information, and finally achieves good results in the style transfer between portraits. But there are still two problems: first, semantic images need to be manually annotated, which costs human resources. In practical applications, large-scale image style transfer is often needed. Second, the details of the synthesized image are fuzzy, and the definition is not enough. We propose an image style transfer algorithm based on semantic segmentation to resolve semantic mismatching in image style transfer. Our algorithm extracts the semantic information of style image and content image automatically through a semantic segmentation network and uses the semantic information to guide the style transfer. Our algorithm builds a semantic segmentation network based on mask R-CNN, introduces semantic information, and then makes style transfer on the patch level, realizes the style transfer between similar objects (consistent semantic information). Experiments on Celeba and Wikiart show that our method could automatically extract the semantic information of style image and content image. Compared with the state-of-art approaches in this field, our method can effectively avoid semantic mismatch in the process of image style transfer. That is, it can maintain semantic consistency in the process of style transfer.

Recently, hyperspectral image classification based on deep learning has achieved considerable attention. Many convolutional neural network classification methods have emerged and exhibited superior classification performance. However, most methods focus on extracting features by using fixed convolution kernels and layer-wise representation, resulting in feature extraction singleness. Additionally, the feature fusion process is rough and simple. Numerous methods get accustomed to fusing different levels of features by stacking modules hierarchically, which ignore the combination of shallow and deep spectral-spatial features. In order to overcome the preceding issues, a novel multiscale dual-branch feature fusion and attention network is proposed. Specifically, we design a multiscale feature extraction (MSFE) module to extract spatial-spectral features at a granular level and expand the range of receptive fields, thereby enhancing the MSFE ability. Subsequently, we develop a dual-branch feature fusion interactive module that integrates the residual connection's feature reuse property and the dense connection's feature exploration capability, obtaining more discriminative features in both spatial and spectral branches. Additionally, we introduce a novel shuffle attention mechanism that allows for adaptive weighting of spatial and spectral features, further improving classification performance. Experimental results on three benchmark datasets demonstrate that our model outperforms other state-of-the-art methods while incurring the lower computational cost.

This paper presents a comprehensive hardware accelerator architecture of YOLOv3-Tiny targeted for low-cost FPGA with a high frame rate, high accuracy, and low latency. The proposed accelerator implements all YOLO layers in hardware including zero pad layer, convolution layer, leaky ReLU layer, batch normalization layer, max-pooling layer, and up-sampling layer. The architecture is built based on data flow and control flow hybrid architecture. The data preparation and computation process work asynchronously using the data flow paradigm, while the overall governing process is controlled by proposed custom instruction set which adopts the principle of control flow paradigm. The principle of General Matrix Multiplication (GEMM) is adopted to compute the convolution process. We designed a GEMM processor using an optimum size of systolic array architecture. The systolic core is small and the overall architecture supports the multicore system, making it scalable to be implemented on larger size FPGAs. We also proposed a hardware architecture for mapping feature maps into matrix form for GEMM convolution which can save on-chip memory space. Lastly, we modified the original YOLO algorithm to further optimize it in our hardware. The modification includes reducing the bit precision to reduce memory space and bandwidth requirement, merging the normalization layer with the convolution layer to reduce arithmetic complexity, and adding a new DLQ layer to keep the bit size small while maintaining the accuracy. Based on the experimental results, our proposed design manages to achieve a frame rate of 8.3 FPS with the throughput of 31.5 GOPS, outperforming the same convolution computation that is performed by Ryzen 5 3600 CPU up to $69.3\times $ in latency. Moreover, our proposed design also has the smallest clock cycle ratio up to $1.75\times $ than other commercial accelerators. The system is useful and suitable for edge computing applications.

Image classification task is an important branch of computer vision. At present, most of the mainstream CNNs are large in size and take up too much computing resources. The quality-price ratio is not satisfying when the heavy CNNs are used in image classification. So, this work proposes a spatial and channel hybrid attention module (Light-Attention module), an ultra-lightweight but efficient attention module. Given an intermediate feature map, the Light-Attention module will firstly derive the most important attention maps of the channels automatically with global stochastic pooling and Multilayer Perceptron (MLP). Then, the residual structure of Light-Attention module helps to repeatedly introduce global information for secondary screening. For better performance, our pioneering Max module and Mean module to extract key spatial features on the basis of previous operations. In the process of extracting the key attention map, LADNet uses the most labor-saving operation and save a lot of network parameters. The Light-Attention module can be seamlessly integrated into any network, while its parameters and required computing resources are both very small. We verified the excellent performance of the Light-Attention module in extracting key image information through ablation experiments on the Cifar-10, Cifar-100 and ImageNet datasets. For the experiment in Cifar-10, the image classification accuracy achieves 98.7%, achieves 96.5% for the Cifar-100, and for the ImageNet, achieves 83.9%. With the Light-Attention module, LADNet reduces the parameters of convolution neural network to 71% of the original. Compared to the other CNNs used in image classification task, the Light-Attention module uses the least parameters but scores most advanced achievements. The experiments proved that the combination of DRN (Dilated Residual Network) and Light-Attention module achieves superior performance.

With the development of deep learning technology, more and more scholars have applied it to hyperspectral image (HSI) classification to improve classification accuracy. However, these deep-learning methods not only take a lot of time in the pre-training phase, but also have relatively limited classification performance when there are fewer labeled samples. In order to improve classification performance while reducing costs, this article proposes a multikernel method based on a local binary pattern and random patches (LBPRP-MK), which integrates a local binary pattern (LBP) and deep learning into a multiple-kernel framework. First, we use LBP and hierarchical convolutional neural networks to extract local textural features and multilayer convolutional features, respectively. The convolution kernel for the convolution operation is obtained from the original image using a random strategy without training. Then, we input local textural features, multilayer convolutional features, and spectral features obtained from the original image into the radial basis function to obtain three kernel functions. Finally, the three kernel functions are merged into a multikernel function according to their optimal weights under the composite kernel strategy. This multikernel function is used as the input for the support vector machine to obtain the classification result map. Experiments show that compared with other HSI classification methods, the proposed method achieves better classification performance on three HSI datasets.

Image haze removal is essential in autonomous driving as the outdoor images captured during unfavorable weather conditions, such as haze or snow, are affected by poor visibility. Much research has been done to overcome image degradation such as low contrast and faded color due to haze. However, in the traditional model, a phenomenon is neglected that several particles simultaneously involved in light acquisition. To address this problem, we propose a novel single image dehazing method based on the spatially adaptive atmospheric point spread function (APSF). We developed a module that estimates the APSF to overcome the limitations of the spatially invariant APSF which used in existing dehazing algorithms. The key factor in the estimation is that road scenes with haze have different statistical characteristic from common hazy images in color and resolution. Furthermore, the APSF on the traffic signs or lights is estimated by generating superpixels to prevent halo artifacts around the sharp edges of the images. We adopted the total variation model as a regularization functional to reduce halo and unnatural artifacts that may occur during deconvolution. The haze-free images from the proposed method tested whether the proposed method can enhance the performance of vision algorithms for autonomous driving. The experimental results demonstrate that the proposed method outperforms state-of-the-art image dehazing methods enhancing the performance of the vision algorithms. Moreover, additional experiments demonstrated the effectiveness of the proposed method for quantitative and qualitative comparison with the state-of-the-art algorithms.

Nighttime light image (NTL) is an effective data for urban built-up area. Due to the shortcomings of low resolution and the blooming effect of NTL, the extraction result of the original NTL image is not ideal. We combined point of interest (POI), road network, and land surface temperature (LST) data to propose the POI, road density, and LST-adjusted NTL Index (PRLANI) to correct the luminance value of the NTL image. The proposed index was verified by NTL images of different developed areas and different spatial resolutions. The threshold segmentation method was used to obtain the result of the built-up area extraction from the original NTL and the image modified by the PRLANI. Then, we generated the vector boundary from the built-up area. Taking Changchun urban area as an example, the kappa coefficient based on PRLANI-VIIRS is 0.802, which is 0.293 higher than of NTL-VIIRS. The kappa coefficient based on PRLANI-LJ is 0.838, which is 0.179 higher than that of NTL-LJ. The results show that: the PRLANI can effectively suppress the blooming effect and improve the overestimation of the built-up area caused by light dimming; the PRLANI can enhance the weak light signal in the original NTL image and compensate for the missing information of the built-up area; and the fusion of road network data improves the extraction accuracy and refines the internal structure of built-up area, especially the NPP/VIIRS. The vector boundary was consistent with the built-up area range of reference data, and the visualization effect was good.

The “cocktail party problem” refers to the ability of human listeners to separate the acoustic signal reaching their ears into its individual components, corresponding to individual sound sources in the environment. Despite this phenomenon appearing trivial for humans, solving the cocktail party problem computationally remains an ambitious challenge. The approach used in this paper takes inspiration from human strategies for separating an acoustic environment into distinct perceptual auditory streams. A series of time-frequency-based features, analogous to those thought to emerge at various stages in the human auditory processing pathway, are derived from biaural auditory inputs. These feature vectors are used as inputs to an unsupervised cluster analysis used to group feature values that are assumed to correspond to the same object. Reconstructed auditory streams are then correlated to the original components used to create the auditory scene. Our model is capable of reconstructing streams that correlate to the original components (r = 0.3-0.7) used to create the complex auditory scene. The success of the reconstructions is largely dependent on the signal-to-noise ratio of the components of the auditory scene.

Almost all image sensors measure only one color per pixel through the color filter array. Missing pixels are estimated using a demosaicing process. For this reason, a demosaiced image leaves a particular trace. When an image is manipulated or tampered, the demosaicing trace can be changed. This change can serve as a basic clue for detecting or localizing image tampering. Demosaicing pattern-based tampering localization algorithms require a re-interpolation process, and the prediction residue between the given image and the re-interpolated image is commonly used to localize tampered regions. However, the prediction residue is not always valid because the demosaicing interpolation kernel cannot be known, which deteriorates the localization performance. This paper presents an effective re-interpolation process using singular value decomposition for an unknown demosaicing method. First, the green channel of the given image is decomposed into four sub-images according to the Bayer pattern. For a small block of each sub-image, the singular value decomposition is performed. The prediction residue is obtained by reconstructing the image block after removing the largest singular value. The feature to localize the forged regions is extracted by the logarithm ratio of the prediction residue variance. The proposed method does not require any statistical model for the extracted feature, because the prediction residue is more accurate than that of conventional methods. We perform intensive experiments for three test datasets and compare the proposed method with state-of-the-art tampering localization methods, the results of which indicate that the proposed scheme outperforms existing approaches.

Decorrelation is one of the main limitations for synthetic aperture radar interferometry. Masking decorrelated pixels is crucial for retrieving information from SAR interferograms. However, for traditional masking methods, manually drawing masks is time-consuming and may be unfeasible when decorrelation areas are with complicated and blurred boundaries. Setting a single coherence threshold is also difficult, if not impossible, to mask out all decorrelated pixels without losing valid phases. Here, we propose a deep-learning segmentation network (Mask Net) based on Selective Kernel Res-Attention UNet, for generating decorrelation masks with applications to TanDEM-X interferograms. We conduct several experiments to determine the training strategy and parameters, including sample size, batch size, loss function, and downsampling scheme, to optimize network performance. Afterwards, we compare the performance of Mask Net with other classical segmentation networks. Our evaluation metrics show that Mask Net outperforms the best performance of other segmentation networks by IoU of 6.32% and F1 Score of 3.97%, respectively. It also possesses the fastest inferring speed, 0.4505 s on sample size of 1024-by-1024 pixels, which is at least ∼50% faster than other segmentation networks. We applied Mask Net to three TanDEM-X interferograms of Ki$\bar{\imath}$lauea crater in Hawaii, metropolitan region of Wuhan, and Muztagata Glacier in China. Our results show that comparing with coherence threshold method, Mask Net can clearly mask out all decorrelation regions, rarely causing loss of valid phases. It also exhibits better segmentation performance than other deep-learning segmentation networks, especially for those complex decorrelation boundaries, with less computational time.

Image fusion operation is beneficial to many applications and is also one of the most common and critical computer vision challenges. The perfect infrared and visible image fusion results should include the important infrared targets while preserving visible textural detail information as much as possible. A novel infrared and visible image fusion framework is proposed for this purpose. In this paper, the proposed fusion network (MIFFuse) is an end-to-end, multi-level-based fusion network for infrared and visible images. The presented approach makes effective use of the intermediate convolution layer’s output features to preserve the primary image fusion information. We also build a cat_block to swap information between two paths to gain more sufficient information during the convolution steps. To reduce the model’s running time even further, the proposed method that reduces the number of feature channels while maintaining the accuracy of the fusion performance. Extensive experiments on the TNO and CVC-14 image fusion datasets show that our MIFFuse outperforms the other methods in terms of both subjective visual effects and quantitative metrics. Furthermore, MIFFuse is approximately twice as fast as the most recent state-of-the-art methods. Our code and models can be found at https://github.com/depeng6/MIFFuse.

Recently, deep learning (DL)-based hyperspectral image classification (HSIC) has attracted substantial attention. Many works based on the convolutional neural network (CNN) model have been certificated to be significantly successful for boosting the performance of HSIC. However, most of these methods extract features by using a fixed convolutional kernel and ignore multiscale features of the ground objects of hyperspectral images (HSIs). Although some recent methods have proposed multiscale feature extraction schemes, more computing and storage resources were consumed. Moreover, when using CNN to implement HSI classification, many methods only use the high-level semantic information extracted from the end of the network, ignoring the edge information extracted from shallow layers of the network. To settle the preceding two issues, a novel HSIC method based on hierarchical shrinkage multiscale network and the hierarchical feature fusion is proposed, with which the newly proposed classification framework can fuse features generated by both of multiscale receptive field and multiple levels. Specifically, multidepth and multiscale residual block (MDMSRB) is constructed by superposition dilated convolution to realize multiscale feature extraction. Furthermore, according to the change of feature size in different stages of the neural networks, we design a hierarchical shrinkage multiscale feature extraction network by pruning MDMSRB to reduce the redundancy of network structure. In addition, to make full use of the features extracted in each stage of the network, the proposed network hierarchically integrates low-level edge features and high-level semantic features effectively. Experimental results demonstrate that the proposed method achieves more competitive performance with a limited computational cost than other state-of-the-art methods.

Infrared (IR) small target detection is challenging because the IR imaging lacks detailed features, weak shape features, and a low signal-to-noise ratio (SNR). The existing small IR target detection methods usually focus on improving their high detective performance without considering the execution time. However, high-speed detection is vital for various applications, such as early warning systems, military surveillance, infrared search and track (IRST), etc. This paper proposes a fast and robust single-frame IR small target detection algorithm with a low computational cost while maintaining excellent detection performance. We propose a layered gradient kernel (LGK) based on the contrast properties of the human visual system (HVS) and model it through a three-layer patch image model. The layered gradient kernel is used to convolute with the input IR frame to obtain its gradient map. The target detection is further performed on the acquired gradient map with an adaptive threshold method. This method is compared with eight representative small target detection algorithms to evaluate the performance. Experimental results demonstrate that the algorithm is fast and suitable for real-time applications, and it is very effective even when the small target size is as small as $2\times 2$ .

Due to the unique feature of the three-dimensional convolution neural network, it is used in image classification. There are some problems such as noise, lack of labeled samples, the tendency to overfitting, a lack of extraction of spectral and spatial features, which has challenged the classification. Among the mentioned problems, the lack of experimental samples is the main problem that has been used to solve the methods in recent years. Among them, convolutional neural network-based algorithms have been proposed as a popular option for hyperspectral image analysis due to their ability to extract useful features and high performance. The traditional convolutional neural network (CNN) based methods mainly use the two-dimensional CNN for feature extraction, which makes the interband correlations of HSIs underutilized. The 3-D-CNN extracts the joint spectral–spatial information representation, but it depends on a more complex model. To address these issues, the report uses a 3-D fast learning block (depthwise separable convolution block and a fast convolution block) followed by a 2-D convolutional neural network was introduced to extract spectral-spatial features. Using a hybrid CNN reduces the complexity of the model compared to using 3-D-CNN alone and can also perform well against noise and a limited number of training samples. In addition, a series of optimization methods including batch normalization, dropout, exponential decay learning rate, and L2 regularization are adopted to alleviate the problem of overfitting and improve the classification results. To test the performance of this hybrid method, it is performed on the Salinas, University Pavia and Indian Pines datasets, and the results are compared with 2-D-CNN and 3-D-CNN deep learning models with the same number of layers.

Due to the need of practical application, multiple sensors are often used for data acquisition, so as to realize the multimodal description of the same object. How to effectively fuse multimodal data has become a challenge problem in different scenarios including remote sensing. Nonsparse multi-Kernel learning has won many successful applications in multimodal data fusion due to the full utilization of multiple Kernels. Most existing models assume that the nonsparse combination of multiple Kernels is infinitely close to a strict binary label matrix during the training process. However, this assumption is very strict so that label fitting has very little freedom. To address this issue, in this article, we develop a novel nonsparse multi-Kernel model for multimodal data fusion. To be specific, we introduce a label softening strategy to soften the binary label matrix which provides more freedom for label fitting. Additionally, we introduce a regularized term based on manifold learning to anti over fitting problems caused by label softening. Experimental results on one synthetic dataset, several UCI multimodal datasets and one multimodal remoting sensor dataset demonstrate the promising performance of the proposed model.

Retinal vessel segmentation is important for analyzing many retinal diseases, where manual segmentation of these vessels is an extensive job. Automatic segmentation of these vessels can help much in the diagnosis of these retinal diseases. Several image processing schemes that are considering the retinal vessel segmentation are lacking in segmentation performance and robustness. Laplacian of Gaussian (LoG) detectors is a popular choice method for edge detection adapted for detecting circular blob detection. Laplacian kernel approximates the second-order derivative to the image, which can be effective in combination with Gaussian smoothing in the form of LoG. The LoG detectors are famous for good border detection in noisy images. Based on their parameter, their behavior can be switched between ridge or blob detection. However, their performance generally falls off for elliptical blobs. A generalized LoG detector was proposed recently to deal with elliptical blob detection. Comparing simulation studies for the second-order ridge detector with other popular ridge detectors provides evidence of its effectiveness. The proposed ridge detector shows promise when applied to detect vessels in real-world retinal images of a publicly available database. The method’s capabilities are evaluated with a comparison of state of the art, and the performances are obtained on the database most used by researchers. The DRIVE,STARE and CHASE_DB1 databases are used for performance evaluation, and we achieved a sensitivity of 0.785, a specificity of 0.967 and an accuracy of 0.952 on the DRIVE database,a sensitivity of 0.788, a specificity of 0.966 and a precision of 0.951 and a sensitivity of 0.787, a specificity of 0.968 and a precision of 0.952 on CHASE_DB1.

Conventional deep-learning-based retrieval models are generally trained under the framework of scene classification with cross-entropy loss, this way focuses only on the output probability corresponding to the label of input samples, while ignoring the predictive information of other categories, which makes the retrieval accuracy susceptible to the intraclass difference of the image samples. And conventional methods often used fixed-size convolution kernels that only consider the local area with fixed sizes, thus largely ignoring the global information. In response to the above problems, this article constructs a triplet nonlocal neural network (T-NLNN) model that combines deep metric learning and nonlocal operation. The proposed T-NLNN follows the three-branch network design, with shared weights in each branch. We evaluate T-NLNN on three public high-resolution remote sensing datasets, and the experimental results suggest that T-NLNN has discriminative feature learning ability and outperforms other existing algorithms. In addition, we propose a dual-anchor triplet loss function to facilitate the utilization of information in the input samples. The experimental results prove that the proposed dual-anchor triplet loss function works better than the traditional triplet loss function on all datasets.

Recently, convolutional neural networks (CNNs) have been used to extract spectral and spatial features of hyperspectral images (HSIs) for hyperspectral image classification (HSIC) because of their excellent performance in extracting and analyzing complex data. However, due to the limited labeled samples and existing mixed pixels, it is difficult to extract features effectively, which further leads to the problem of overfitting of the model. On the other hand, to improve the extraction ability of the CNN, the depth of the model, and the complexity of the convolution kernel often need to be increased. In this article, a sandwich CNN based on spectral feature enhancement (SFE-SCNN) is proposed for HSIC. The proposed method, SFE-SCNN, introduces the spectral feature enhancement operation, which makes the data reflect more discriminative spectral feature details to suppress the interference of mixed pixels. Furthermore, according to the preprocessed data structure features, a lightweight sandwich convolution neural network is proposed. To fully extract the spectral features, the spectral feature re-extraction operation is used for the first time. Experimental results on three real hyperspectral datasets demonstrate that the proposed method achieves better classification performance than other state-of-the-art methods.

In recent years, container technology has caught the attention of the communities by its performance and compactness. Although the design of modern container tools (e.g., Docker and podman) serves as a single-purpose application provider, existing deployed containers still contain extra tools that are unnecessary for a single-purpose process. The existence of unnecessary files and tools is directly proportional to higher security risk. Besides, extraneous files often make the container heavier and slow down its performance. This paper introduces a novel Lightweight Virtualization packaging model for creating profiles for a single-purpose application from an existing multi-purpose container environment, called AppPACK. Specifically, the model can generate a package containing minified versions of images, kernel, and virtual machine profiles from a target application. The experiment results show that AppPACK can provide an image of 1.1 to 37 times smaller in size compared to the original version. The experiment on execution shows that using AppPACK profiles could fasten the booting process from 1.1 to 6 times compared to the non-AppPACK version. The comparison between AppPACK and previous approaches shows that proposed model can provide more compatible and smaller versions in most cases.

Aiming at utilizing artificial neural networks to enhance intelligent filtering for interfered wireless communication signal in harsh environments, a new method named convolutional neural filtering is designed and presented in this paper. This method is based on model-driven deep learning princeple, by analyzing the theoretical connection between the filter model and the convolutional neural layer, it attempts to use one-dimensional convolution kernels to learn a matched or bandpass filter. Moreover, the model introduces a kernel-wise attention mechanism between different convolution kernels to selectively emphasize informative filters. The results show that in terms of interference and noise suppression for received wireless signal, the filtering method has highlighted dynamic adaptability to variation of signals and interference, and it also reveals that the performance is affected by the initialization parameters and the number of convolution kernels. Based on this method an embeddable filtering unit fully based on neural network is provided, which can be easily integrated into a deep learning network targeting such as wireless signal detection and recognition applications, avoiding complex preprocessing for end-to-end wireless signal learning.

Rainy days usually degrade the visual effect of images and videos. At present, most deraining models for single images adopt gradual optimization or elimination to remove rain streaks, but actually with relatively low efficiency in real tasks. An efficient one-stage deraining model, Efficient Transformer Derain Network (ETDNet), is proposed to remove rain streaks in single images efficiently. A new Transformer architecture is designed to provide rich multiple scales and context information, making the model extract features in a coarse-to-fine way. Multiple expansion filters with different expansion rates are embedded to predict the suitable kernel for each pixel of the rainy image in a multi-scale way. A multi-scale Loss Function is introduced to restore features with high-fidelity and detail textures. Experiments on Rain100L, Rain100H, and SPA datasets show that the proposed ETDNet reaches the highest PSNR and SSIM values at the fastest speed compared with other models.

Pitting corrosion of rotating machinery is one of the most common faults in industrial engineering. The convolutional neural network (CNN) is increasingly applied to the fault diagnosis. However, the conventional CNN method will reduce the feature dimension of the collected signal and cause the loss of information during the pooling process. In this paper, a new method based on dilated CNN combined with spatial dropout (DCSD) is proposed to diagnose the early faults of rotating machinery. By filling the convolution kernel, the DCSD method can increase the receptive field of the CNN without increasing the number of parameters while retaining more features of the raw vibration signal of the rotating machine. To avoid the dropout method eliminates the adjacent elements with a strong correlation, the Spatial Dropout method is adopted to reduce the overfitting problem of deep networks. The early pitting gears experiment was designed to verify the DCSD method in this paper. The raw vibration signal data of 6 different healthy states were collected to verify the effectiveness of the method. The experimental results show that the DCSD method proposed can effectively distinguish the different early gears pitting, and the diagnostic accuracy is better than other popular deep learning methods.

Pins are standard fasteners in power transmission lines, and the hidden dangers of pins falling off dramatically affects their safe operation. If a pin is missed, it is called pin defects in this paper. As the pin is a small target and has a complex background, traditional detection algorithms were used to identify pin defects from aerial images which suffer from poor accuracy and low efficiency. This paper proposed a target detection method based on cascaded convolutional neural networks. First, a small-scale shallow full convolutional neural network was used to obtain the region of interest; then, a deeper convolutional neural network conducted target classification and positioning on the obtained region of interest. Next, a nonlinear multilayer perceptron was introduced, the convolution kernel was decomposed, and the multi-scale feature maps were fused. At this point, an angle variable was added to the classification cross-entropy loss function. Multi-task learning and offline hard sample mining strategies were used in the training phase. The proposed method was tested on a self-built pin dataset and the remote sensing image RSOD dataset, and the experimental results proved its effectiveness. Our method can accurately identify pin defects in aerial images, thereby solving the engineering application problem of pin defect detection in transmission lines.

A novel video interpolation network to improve the temporal resolutions of video sequences is proposed in this work. We develop a multi-scale warping module to interpolate intermediate frames robustly for both small and large motions. Specifically, the proposed multi-scale warping module deals with large motions between two consecutive frames using coarse-scale features, while estimating detailed local motions by exploring fine-scale features. To this end, it takes multi-scale features from the encoder and estimates kernel weights and offset vectors for each scale. Finally, it synthesizes multi-scale warping frames and combines them to obtain an intermediate frame. Extensive experimental results demonstrate that the proposed algorithm outperforms state-of-the-art video interpolation algorithms on various benchmark datasets.

In this paper, we analyze the interaction of localized patterns such as traveling wave solutions for reaction-diffusion systems with nonlocal effect in one space dimension. We consider the case that a nonlocal effect is given by the convolution with a suitable integral kernel. At first, we deduce the equation describing the movement of interacting localized patterns in a mathematically rigorous way, assuming that there exists a linearly stable localized solution for general reaction-diffusion systems with nonlocal effect. When the distances between localized patterns are sufficiently large, the motion of localized patterns can be reduced to the equation for the distances between them. Finally, using this equation, we analyze the interaction of front solutions to some nonlocal scalar equation. Under some assumptions, we can show that the front solutions are interacting attractively for a large class of integral kernels.

Although deep learning-based methods have been successfully applied to polarimetric synthetic aperture radar (PolSAR) image classification tasks, most of the available techniques are not suitable to deal with PolSAR data on irregular domains, e.g., superpixel graphs, because they are naturally designed as grid-based architectures in Euclidean space. To overcome this limitation and achieve robust PolSAR image classification, this article proposes the multiscale evolving weighted graph convolutional network, where weighted graphs based on superpixel technique and Wishart-derived distance are constructed to enable efficient handling of graphical PolSAR data representations. In this article, we derive a new architectural design named graph evolving module that combines pairwise latent feature similarity and kernel diffusion to refine the graph structure in each scale. Finally, we propose a graph integration module based on self-attention to perform robust hierarchical feature extraction and learn an optimal linear combination of various scales to exploit effective feature propagation on multiple graphs. We validate the superiority of proposed approach on classification performance with four real-measured datasets and demonstrate significant improvements compared to state-of-the-art methods. Additionally, the proposed method has shown strong generalization capacity across datasets with similar land covers.

Stereo matching is an important research topic in the field of computer vision. It recovers depth information from a pair of color images. Unfortunately, converting multi-dimensional (more than two-dimensional) data into two-dimensional data, such formulations ignore the spatial structure of multi-dimensional images/data. Tensors can be used to describe high-dimensional data structure, which can retain the hidden structure of data, but cannot obtain the deep features that helps to improve the performance of the algorithm. Therefore, it is very important to establish a deep tensor model. In this paper, we propose a two layer tensor form convolutional sparse coding model, which can automatically learn the deep convolutional kernel. Based on the learned two layer convolutional kernels, a two-layer dictionary learning model is established. Then, a new weighted matching cost method is constructed, which combines shallow and deep features. The experimental results on the Middlebury benchmark v2 and Middlebury benchmark v3 show that the proposed two layer tensor convolutional sparse coding is effective for stereo matching.

Traditional optical fiber vibration signal (OFVS) recognition research focuses on signal endpoint detection and feature extraction. These two aspects directly determine the success of OFVS recognition. The traditional method relies on artificially designed features and has a strong pertinence to the classification target, resulting in poor stability and flexibility. In response to the above problems, this paper combines the traditional OFVS recognition ideas (time-frequency analysis and feature extraction) and the characteristics of deep learning automatic learning parameters to construct an end-to-end adaptive filtering convolutional neural network (AF-CNN), which can directly get the classification results through the iterative update of the network. In modeling the original signal, the following steps were taken to make the network interpretable. First, we use a one-dimensional (1-D) convolution on the original OFVS. The convolution kernel can adaptively treat the original signal perform filtering to obtain filtered signals of different frequency bands. Second, using a general convolutional neural network (CNN) to extract the filtered signal features. Finally, a multi-layer perceptron (MLP) is used for classification. This paper compares the AF-CNN network with three traditional pattern recognition methods and proves that the AF-CNN network’s accuracy is better than traditional pattern recognition methods. The average accuracy can reach 96.7%, and it can effectively distinguish OFVS with weaker energy and similar waveforms.

Convolutional Neural Networks (CNNs) revolutionized computer vision and reached the state-of-the-art performance for image processing, object recognition, and video classification. Even though CNN inference is notoriously compute-intensive, as convolutions account for >90% of the total operation tasks, the ability to tradeoff between accuracy, performance, power, and latency to meet target application makes it an open research topic. This paper proposes the Spatial Locality Input Data (SLID) method for computational reuse during the inference stage for a pre-trained network. The method exploits input data spatial locality via skipping partial processing of the multiply-and-accumulate (MAC) operations for adjacent data and equating its value to previously computed ones. SLID improves the throughput of resource-constrained devices (Internet-of-Things, edge devices) and accelerates computations during the inference phase by reducing the number of MAC operations. Such approximate computing schema does not require a similarity quantification step nor any modification for the training stage. The computational data reuse was evaluated on three well-known distinctive CNN structures and data sets with alternating layer selections: LeNet, CIFAR-10, and AlexNet. The computational data reuse method saves up to 34.9%, 49.84%, and 31.5% of MAC operations while reducing the accuracy by 8%, 3.7%, and 5.0% for the three models mentioned earlier, respectively. Besides, the proposed method saves on memory access by eliminating data fetching of skipped inputs. Furthermore, filter size, strides, and padding on the accuracy and savings of operations are analyzed. SLID is the first work to exploit the input spatial locality for savings on CNN convolution operations with minimal accuracy loss and without memory or computational overhead. This makes it a great option to support intelligence at the edge.

Spectral clustering makes use of the spectrum of an input affinity matrix to segment data into disjoint clusters. The performance of spectral clustering depends heavily on the quality of the affinity matrix. Commonly used affinity matrices are constructed by either the Gaussian kernel or the self-expressive model with sparse or low-rank constraints. A technique called diffusion which acts as a post-process has recently shown to improve the quality of the affinity matrix significantly, by taking advantage of the contextual information. In this paper, we propose a variant of the diffusion process, named Self-Supervised Diffusion, which incorporates clustering result as feedback to provide supervisory signals for the diffusion process. The proposed method contains two stages, namely affinity learning with diffusion and spectral clustering. It works in an iterative fashion, where in each iteration the clustering result is utilized to calculate a pseudo-label similarity so that it can aid the affinity learning stage in the next iteration. Extensive experiments on both synthetic and real-world data have demonstrated that the proposed method can learn accurate and robust affinity, and thus achieves superior clustering performance.

Oil spill (OS), as one of the main pollutions in the ocean, is a serious threat to the marine environment. Thus, timely and accurate OS detection (OSD) is necessary for ocean management. In this regard, remote sensing (RS) plays a key role due to multiple advantages over large and remote ocean environments. In this study, a new OSD framework based on a deep learning algorithm was developed for optical RS imagery. The proposed method was based on a multiscale multidimensional residual kernel convolution neural network. The proposed method investigated the deep features by the two-dimensional multiscale residual blocks and, then, utilized them at one-dimensional multiscale residual blocks. In this study, Landsat-5 satellite imagery acquired over the Gulf of Mexico was applied to evaluate the performance of the proposed method. The overall accuracy of the proposed method was more than 95%, and the miss detection and false alarm rates were less than 5%, indicating its high potential for OSD. Moreover, it was observed that the proposed method had better performance compared to other OSD algorithms that were investigated in this study.