Search

This work introduces Orthogonal Lattice Universal Wavelet Unit, a novel trainable wavelet unit to enhance image classification and anomaly detection in convolutional neural networks by reducing information loss during pooling. The unit employs an orthogonal lattice structure, relaxing the zero-at- $\pi $ condition and decreasing the number of trainable wavelet coefficients. This innovation is a key novelty of the work. The unit modifies convolution, pooling, and down-sampling operations. Implemented in residual neural networks with 18 layers, it improved detection accuracy on CIFAR10 (by 2.67%), ImageNet1K (by 1.85%), and the Describable Textures dataset (by 9.52%), showcasing its advantages in detecting detailed features. Similar gains were seen in the implementations for residual neural networks with 34 layers and 50 layers. For anomaly detection in hazelnut images on the MVTec Anomaly Detection dataset, the proposed method achieved a segmentation area under the receiver operating characteristic curve of 97.15% and better anomaly localization. The method excels in detecting detailed features, despite increased trainable parameters from using one-layer fully convolutional networks for feature combination.

Small moving objects at far distance always occupy only one or a few pixels in image and exhibit extremely limited visual features, which bring great challenges to motion detection. Highly evolved visual systems endow flying insects with remarkable ability to pursue tiny mates and prey, providing a good template to develop image processing method for small target motion detection. The insects’ excellent sensitivity to small moving objects is believed to come from a class of specific neurons called small target motion detectors (STMDs). However, existing STMD-based methods often experience performance degradation when coping with complex natural scenes. In this paper, we propose a bio-inspired visual system with spatio-temporal feedback mechanism (called Spatio-Temporal Feedback STMD) to suppress false positive background movement while enhancing system responses to small targets. Specifically, the proposed visual system is composed of two complementary subnetworks and a feedback loop. The first subnetwork is designed to extract spatial and temporal movement patterns of cluttered background by neuronal ensemble coding. The second subnetwork is developed to capture small target motion information where its output and signal from the first subnetwork are integrated together via the feedback loop to filter out background false positives in a recurrent manner. Experimental results demonstrate that the proposed spatio-temporal feedback visual system is more competitive than existing methods in discriminating small moving targets from complex natural environments. [ABSTRACT FROM AUTHOR]

Depression is a mental illness that may be harmful to an individual's health. The detection of mental health disorders in the early stages and a precise diagnosis are critical to avoid social, physiological, or psychological side effects. This work analyzes physiological signals to observe if different depressive states have a noticeable impact on the blood volume pulse (BVP) and the heart rate variability (HRV) response. Although typically, HRV features are calculated from biosignals obtained with contact-based sensors such as wearables, we propose instead a novel scheme that directly extracts them from facial videos, just based on visual information, removing the need for any contact-based device. Our solution is based on a pipeline that is able to extract complete remote photoplethysmography signals (rPPG) in a fully unsupervised manner. We use these rPPG signals to calculate over 60 statistical, geometrical, and physiological features that are further used to train several machine learning regressors to recognize different levels of depression. Experiments on two benchmark datasets indicate that this approach offers comparable results to other audiovisual modalities based on voice or facial expression, potentially complementing them. In addition, the results achieved for the proposed method show promising and solid performance that outperforms hand-engineered methods and is comparable to deep learning-based approaches. [ABSTRACT FROM AUTHOR]

We present our automated real-time socket inspection system capable of detecting an assortment of defects including metallic and liquid staining, loose capacitors and pins, and other debris and foreign material (FM). Our test tools pick and place manufactured units into sockets for electrical testing. Any debris accumulated inside the test sockets will likely damage subsequent units until the defective socket is replaced. To quickly capture in-situ defects and mitigate further damage, we equipped each pick-and-place arm with a new vision system designed to fit within the existing tool. The tight footprint constraints required a highly compact imaging system which resulted in a variety of image artifacts, creating several unique challenges. Our inspection algorithm utilizes a variety of advanced computer vision and machine learning techniques to normalize images, extract and match features, register the images, suppress unwanted artifacts, and detect defects. The detected changes are then sent to a deep learning classifier to further filter between true defects and natural socket deterioration. The flagged socket images can be manually dispositioned by the user and the socket can be sent for repair or cleaning as needed. [ABSTRACT FROM AUTHOR]

The growth of irregular brain cells leads to a disease called brain tumor (BT). It is difficult to predict a patient’s chance of survival due to the low rate and wide range of tumor shapes. Even though it is possible to manually detect cancer, doing so is difficult and time-consuming and runs the risk of producing false-positive results. This can be done via MRI, which is necessary for locating cancer. It is very difficult to reliably identify different illnesses from MRI images for successful therapy via a computer-aided diagnostic system. In the experiment, three openly accessible benchmark datasets were utilized. To perform feature extraction in our proposed method, a CNN model was employed, followed by the application of five machine learning classifiers: Decision tree (DT), Naive Bayes (NB), Adaptive Boosting (AdaBoost), K-nearest neighbor (KNN), and support vector machine (SVM). The outcomes show that the proposed CNN architecture with the KNN classifier performs better than previous CNN models by outperforming other cutting-edge DL models under various classification metrics. Finally, the achieved F1-Score, precision, recall, and accuracy values for the classification and detection of the proposed model were 99.58%, 99.59%, 99.58%, and 99.58%, respectively. For the comparison study, additional Transfer Learning models are utilized. Experimental findings support the strength of the proposed architecture, which has rapidly accelerated and improved the classifications of BTs. The designed method outperforms the body of existing knowledge, demonstrating that it is a quick and precise method for classifying BTs.

Deer antler slices are highly valued in Chinese herbal medicine due to their medicinal properties. However, the current process for classifying these slices is time-consuming and subjective. To overcome this issue, we propose an intelligent classification and recognition model based on the Res2Net architecture. Our neural network utilizes an inverse bottleneck structure to enhance grouped convolution and reduce model parameters and computation time. Additionally, we integrate an improved grouped convolution into the Res2Net model and leverage the efficient channel attention (ECA) mechanism to improve feature extraction. Our model achieves an impressive 97.96% accuracy in classifying deer antler slices and outperforms other related models. This approach can accurately differentiate between different types of deer antler slices and is particularly suitable for small-scale datasets.

This article illustrates the controlling mechanism along with data transmission using optical camera communication in both line-of-sight and non-line-of sight (NLOS) environment. The transmitter of the system is mainly composite of microcontroller (Arduino Mega), DHT sensor, two LEDs, and four push button switches. Here, two LEDs are continuously transmitting the control and data signal. The control signal is operated through push button switches. We have used four different control signals that are transmitted from the LEDs and data transmission by collecting data from the surrounding through the temperature and humidity sensor. The entire transmitter operations are processed and controlled through arduino programming in a microcontroller. The COOK modulation technique is used for this system. For data reception, we have used laptop's camera that is controlled through five different exposure times and operated based on the rolling shutter effect of the camera. A logistic regression algorithm is applied to recognize the data transmitting LED from the normalized intensity of the stripe patterns. The receiver is capable to collect data under certain conditions, if the light source is obstructed by an object through deep neural network. Again, Generative Adversial Network (GAN) is used as a decoder to reduce the BER during moving scenario and NLOS conditions. Different activation functions are used in the GAN model to find the optimum solution for the system. However, we have archived the maximum 2.5 m communication distance with 10-4 BER. The receiver program is designed using Python 3.8.

Intelligent manufacturing raises higher requirements for tool condition monitoring (TCM) in terms of accuracy, robustness, and adaptability. At present, direct methods based on image processing and deep learning have made breakthroughs in TCM. However, some issues, such as image quality, model parameters, and dataset scale in the abovementioned methods, restrict industrial applications of TCM. Regarding the abovementioned issue, the purpose of this article is to propose a lightweight network model based on multiple activation functions to promote the intelligent industrial application of TCM. First, the image quality mechanism caused by complex working conditions is analyzed in industrial environments. Correspondingly, data augmentation is adopted to solve the problem of data scale under the premise of ensuring data quality and richness. Then, the adaptive activation function and the hard version of swish are introduced at the front and second half of the network to avoid information loss and reduce the activation function cost. Finally, a lightweight network based on cloud-edge collaboration for TCM is constructed. The model is iteratively optimized in the cloud and inferenced on the edge embedded device. The accuracy and adaptability of the proposed network are verified by accelerating milling cutter life under multiple working conditions. [ABSTRACT FROM AUTHOR]

Image quality assessment (IQA) is an important task of image processing and has diverse applications, such as image super-resolution reconstruction, image transmission and monitoring systems. This paper proposes a perceptual hashing algorithm with complementary color wavelet transform (CCWT) and compressed sensing (CS) for reduced-reference (RR) IQA. The CCWT is exploited to decompose input color image into different sub-bands. Since the calculation of CCWT uses all color channels without discarding any information, the distortions introduced by digital operations on color channels are preserved in the CCWT sub-bands. The block-based CS is used to extract features from the CCWT sub-bands. As the Euclidean distance between the block-based CS features is slightly influenced by content-preserving operations, perceptual features constructed by Euclidean distances are robust, discriminative and compact. Hash sequence is finally determined by quantifying the perceptual features. Effectiveness of the proposed hashing is verified by various experiments on four open image databases. Experimental results demonstrate that the proposed hashing is superior to some state-of-the-art algorithms in terms of classification and RR IQA application. [ABSTRACT FROM AUTHOR]

In this paper, we conduct an extensive study on the use of pre-trained convolutional neural networks (CNNs) for omnidirectional image quality assessment (IQA). To cope with the lack of available IQA databases, transfer learning from seven pre-trained CNN models is investigated over retraining on standard 2D databases. In addition, we explore the influence of various image representations and training strategies on the model’s performance. A comparison of the use of projected versus radial content, and multichannel CNN versus patch-wise training is also covered. The experimental results on two publicly available databases are used to draw conclusions about which strategy best fits the visual quality prediction and at which computational cost. The analysis shows that retraining CNN models on 2D IQA databases improves the prediction accuracy. The latter and the required computational time are found to be significantly affected by the training strategy. Cross-database evaluations demonstrate that the nature and variety of the content impact the generalization ability of the models. Finally, we show that conclusions coming from other image processing communities may not hold for IQA. The provided discussion shall provide insights and recommendations when using pre-trained CNNs for omnidirectional IQA. [ABSTRACT FROM AUTHOR]

Classification of 3D objects – the selection of a category in which each object belongs – is of great interest in the field of machine learning. Numerous researchers use deep neural networks to address this problem, altering the network architecture and representation of the 3D shape used as an input. To investigate the effectiveness of their approaches, we conduct an extensive survey of existing methods and identify common ideas by which we categorize them into a taxonomy. Second, we evaluate 11 selected classification networks on two 3D object datasets, extending the evaluation to a larger dataset on which most of the selected approaches have not been tested yet. For this, we provide a framework for converting shapes from common 3D mesh formats into formats native to each network, and for training and evaluating different classification approaches on this data. Despite being partially unable to reach the accuracies reported in the original papers, we compare the relative performance of the approaches as well as their performance when changing datasets as the only variable to provide valuable insights into performance on different kinds of data. We make our code available to simplify running training experiments with multiple neural networks with different prerequisites. [ABSTRACT FROM AUTHOR]

Defect pattern detection and classification are challenging for thin-film-transistor liquid-crystal display (TFT-LCD) manufacturing. Limitations of the existing solutions for automatic optical inspection can be traced in part to the lack of a framework within which different existing and new defect patterns can be analyzed, while integrating domain knowledge and effective technologies. This study aims to develop a framework for image-based defect classification that employs the convolution neural networks without using complex and time-consuming image-processing processes in advance. An empirical study was conducted in a leading TFT-LCD manufacturing in Taiwan for validation. The results have shown that the defect patterns can be effectively classified by the proposed convolutional neural networks that outperform the existing approaches such as Support Vector Machine and Random Forest. The developed solution is implemented to effectively support the engineers. [ABSTRACT FROM AUTHOR]

This research aimed to identify objects from a real-time video (30 Hz) stream transmitted from a low-altitude long-endurance (LALE) fixed-wing hybrid vertical takeoff and landing (VTOL) unmanned aerial vehicle (UAV) designed for asset monitoring. Illumination, rotation, and scale variations complicate the video frames created by UAVs’ optical payload, requiring preprocessing. Detection, identification, and classification of different objects (or targets) from a real-time video feed require fast and reliable detection algorithms, models, and computation. The weight restrictions and the onboard power supply only allowed for a Raspberry Pi onboard the UAV, thereby seriously limiting computational power. Image frames were created from the video data after enhancing its quality using image processing algorithms. Then, anchor-based object detection algorithms were applied to the processed and enhanced image frames. The Raspberry Pi at airspeeds of 16–18 m/s requires fast object detection algorithms having minimal computational overhead. A supervised learning technique using a supporting library further augmented the real-time detection at the ground control station (GCS). Four scenarios trained the candidate algorithms, namely: 1) helipad; 2) green area; 3) burned area; and 4) under construction area. The single-shot multibox detector (SSD) aided the supervised learning process during trials utilizing the labeled image datasets. The comparison of detection and identification performance of the SSD with ResNet-50 and ResNet-101 backbones with other methods, such as Faster-RCNN with four backbone architectures, namely: 1) ResNet-101; 2) ResNet-50; 3) deformable ResNet; and 4) FPN in four detection scenarios indicated better precision and reduced detection delays. [ABSTRACT FROM AUTHOR]

Road surface condition can significantly impact the interaction between vehicles and pavement structure, which may even cause high fuel consumption and safety issues of drivers and vehicles. Distributed fiber optic sensing (DFOS) technology is a useful tool to perform continuous and real-time monitoring of traffic and road surface condition. However, it is challenging to process the data for the purpose of road anomaly detection. The study proposed two approaches to detect the road anomaly using DFOS. In the first method, local binary pattern (LBP) histograms were used to extract the features of the images with and without road anomaly, and support vector machine (SVM) combined with principal component analysis (PCA) was adopted as the classifier. The convolutional neural network (CNN) was applied on the binary classification data to analyze the images in the second method. The accuracy and benefits of two methodologies were compared. The vehicle speed was estimated by detecting lines using Hough transform. The feasibility of road anomaly detection using DFOS is proved. [ABSTRACT FROM AUTHOR]

This article proposes a novel framework to characterize the morphological pattern of barely visible impact damage using machine learning. Initially, a sequence of image processing methods is introduced to extract the damage contour, which is then described by a Fourier descriptor-based filter. The uncertainty associated with the damage contour under the same impact energy level is then investigated. A variety of geometric features of the contour are extracted to develop an artificial intelligence model, which effectively groups the tested 100 samples impacted by 5 different impact energy levels with an accuracy of 96%. Predictive polynomial models are finally established to link the impact energy to the three selected features. It is found that the major axis length of the damage has the best prediction performance, with an R2 value up to 0.97. Additionally, impact damage caused by low energy exhibits higher uncertainty than that of high energy, indicating lower predictability. [ABSTRACT FROM AUTHOR]

The papers in this special section focus on advanced machine learning algorithms for biomedical data and image processing. Researchers in machine learning including those working in computer vision, image processing, biomedical analysis, and related fields when tied with experienced clinicians can play a significant role in understanding and working on complex medical data which ultimately improves patient care. Developing a novel machine-learning algorithm specific to medical data is a challenge and need of the hour. Healthcare and biomedical sciences have become data-intensive fields, with a strong need for sophisticated data mining methods to extract the knowledge from the available information. Biomedical data contains several challenges in data analysis, including high dimensionality, class imbalance, and low numbers of samples. Although the current research in this field has shown promising results, several research issues need to be explored as follows. There is a need to explore novel feature selection methods to improve predictive performance along with interpretation and to explore large-scale data in biomedical sciences. [ABSTRACT FROM AUTHOR]

Photo identification is an essential method to identify cetaceans, by using natural marks over their body, and allows experts to acquire straightforward information on these animals. The importance of cetaceans lies in te fact that they play a crucial role in maintaining the healthiness of marine ecosystems, however they are exposed to several anthropogenic stressors, under which they could collapse with extreme consequences on the marine ecosystem functioning. Hence, obtaining new knowledge on their status is extremely urgent for the marine biodiversity conservation. The smart use of technology to automate the individual recognition can speed up the photo identification process, opening the door to large-scale studies that are manually unfeasible. We performed a systematic review on systems based on machine learning and statistical methods for cetacean photo identification, following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses statement. This review highlights that interest has been increasing in recent years and several intelligent systems have been presented. However, there are still some open questions, and further efforts to develop more effective automated systems for cetacean photo identification are recommended.

Underground fire monitoring is an important tool to improve coal mine production safety. In this paper, a multi-source information identification method based on bilinear convolutional neural network (B-CNN) is proposed, which consists of construction of multi-source image acquisition system, B-CNN and integrated decision making based on multi-source B-CNN. Aiming at the problem that Softmax loss function based on the gradient descent in B-CNN is easy falling into the local optimum, an improved Grasshopper Optimization Algorithm (GOA) is proposed to optimally selected two parameters of $\boldsymbol {W}$ and $\boldsymbol {\theta } $ ; the method of initial solution generation based on sine mapping and the method of accepting bad solution with certain probability are respectively adopted. In order to solve high computational complexity in the stage of model training and integrated recognition by multi-source B-CNN, an image feature preprocessing method is proposed in this paper. Several feature vectors of color feature, shape feature and texture feature of the collected image are extracted and used as input vectors of B-CNN to complete model training and integrated recognition. In simulation experiments, firstly, four Benchmark functions are used to verify the performance of the improved GOA; then, by scaling, expanding and rotating the image to simulate the results of image acquisition at multiple positions and angles, different information sources can be formed to complete the integrated recognition by B-CNN. Three performance indexes of Accuracy, Precision and Recall are used to evaluate the simulation result of different comparative models, which show that the proposed method has better recognition effects.

In recent years, remote sensing image processing based on deep learning has been widely applied in many scenes, but the involved deep learning technology requires large-scale labeled data, which has been a practical problem in the remote sensing field. In this study, we proposed a novel data information quality assessment method, called K-nearest neighbor (KNN) distance entropy, to screen the remote sensing images. The evaluation metric was used to assess unlabeled data and assign the pseudolabel, which further constitutes the proposed semisupervised few-shot classification method in this article. The metatask setting was adopted to verify the validity and stability of experimental results. Specifically, the KNN distance entropy metric can be used to distinguish the samples in the core set or boundary set. Experimental results show that the core set samples are more suitable under the few-shot condition, for instance, the metatask average accuracy trained by the core set samples outperforms that by boundary samples by about 18% in the case of 45-ways and 5-shot. The proposed semisupervised few-shot method based on KNN distance entropy achieves significant improvement under different experimental conditions. The visualization of the feature distribution of screened data is shown to provide an intuitive interpretation. This article lays a meaningful foundation for screening and evaluating remote sensing images under few-shot conditions, and inspires the data-efficient few-shot learning based on high-quality data in the remote sensing field.

In the field of image processing, texture features and color are fundamental visual cue with complementary roles. They are used in many applications and in a large variety of areas such as quality control, content-based image retrieval, remote sensing, industrial inspection, surface inspection, object recognition, and medical image analysis. For this purpose, a large number of algorithms have been proposed for texture feature extraction. Some of them are dedicated to gray-scale images while others aim at processing both color and texture. It has been shown that, for many cases, the use of color improves the performance of gray level texture classification. This paper provides a comprehensive survey of the texture feature extraction methods that consider both texture and color information. We propose a categorization of these methods into seven classes, two of them being very recent. For each method, we present the concept, the advantages and drawbacks, and we give examples of application.

In the last decade, there have been many reports on the negative impact of wildfires on various ecosystems. Unfortunately, wildfires have been intensifying as global temperatures, droughts, and other instances of extreme weather events rise around the world. These circumstances are forcing communities to vigorously address the uncontrolled spread of wildfires, where the ultimate goal is the protection of wildlife. At the same time, many disaster prevention and monitoring methods, based on image processing and computer vision, have been developed. In this paper, we present a new unsupervised method based on RGB color space for the early detection of wildfires from still images. From the analysis of existing state-of-the-art methods, it is evident that different methods explore different color spaces for the extraction of flame features. Our motivation was to use only RGB color space and thus eliminate the time-consuming task of color space conversion. The proposed method consists of several new image processing techniques used to efficiently extract flame features. It outperforms the existing methods, where an increase of 3% and 2% is recorded in the F1 score and Matthews correlation coefficient, respectively. Such performance demonstrates the merits of the proposed method for flame segmentation and detection.

Saliency detection by human refers to the ability to identify pertinent information using our perceptive and cognitive capabilities. While human perception is attracted by visual stimuli, our cognitive capability is derived from the inspiration of constructing concepts of reasoning. Saliency detection has gained intensive interest with the aim of resembling human ‘perceptual’ system. However, saliency related to human ‘cognition’, particularly the analysis of complex salient regions (‘cogitating’ process), is yet to be fully exploited. We propose to resemble human cognition, coupled with human perception, to improve saliency detection. We recognize saliency in three phases (‘Seeing’ - ‘Perceiving’ - ‘Cogitating), mimicking human's perceptive and cognitive thinking of an image. In our method, ‘Seeing’ phase is related to human perception, and we formulate the ‘Perceiving’ and ‘Cogitating’ phases related to the human cognition systems via deep neural networks (DNNs) to construct a new module (Cognitive Gate) that enhances the DNN features for saliency detection. To the best of our knowledge, this is the first work that established DNNs to resemble human cognition for saliency detection. In our experiments, our approach outperformed 17 benchmarking DNN methods on six well-recognized datasets, demonstrating that resembling human cognition improves saliency detection. [ABSTRACT FROM AUTHOR]

Deep learning (DL) methods usually need to collect a large amount of labeled data to extract deep features. However, due to the difficulty of obtaining numerous labeled data from synthetic aperture radar (SAR) images, unsupervised feature learning has been focused on SAR image processing. In this article, we propose a 3-D sparse model (3-DSM) to extract deep sparse features from SAR images in an unsupervised way. Concretely, 3-DSM learns the convolution kernels by minimizing the error between the features and the constructed sparse maps, without labeled samples. Thus, the discriminative features can be extracted in an unsupervised way by the learned convolution kernels and are able to capture the main structural information of SAR images. Furthermore, to the best of our knowledge, 3-DSM first specifies the sparsity of convolution kernels, with each convolution kernel exhibiting its independence from the others and the redundancy of convolution kernels being diminishing. It means that each convolution kernel extracts its unique structural features of SAR images. Consequently, in the feature extraction, 3-D sparsities have been specified, including width, height, and depth, with the acquisition of discriminative less-redundant features. The effectiveness of 3-DSM is demonstrated by the feature extraction and segmentation of the simulated and real SAR images. [ABSTRACT FROM AUTHOR]

The automatic extraction of lake water is one of the research hotspots in the field of remote sensing image processing. Due to the small interclass variance between lakes and other ground objects, and the complex texture characteristics of lake boundaries, existing methods often have problems such as over-segmentation and inaccurate boundary segmentation when segmenting lake water bodies. To alleviate these problems, this article designs an end-to-end semantic segmentation network [noise-canceling transformer network (NT-Net)] for the automatic extraction of lake water bodies from remote sensing images. Aiming at the problem of over-segmentation caused by nonlake objects, an interference attenuation module is designed in the network. This module can model the key features that are distinguishable and suitable for segmenting lake water by analyzing the difference in feature representation between lakes and other ground objects, thus suppressing the feature representation of nonlake objects. To more accurately segment the lake boundary, a multilevel transformer module is designed. This module can capture the context association of boundary information and enhance the feature representation of boundary information by using the self-attention mechanism. The comparative experimental results show that, compared with the current mainstream semantic segmentation networks, the method in this article has advantages in extracting lake water bodies comprehensively and coherently. [ABSTRACT FROM AUTHOR]

Change detection for remote sensing images involves detecting regional surface changes of interest between two images taken of the same geographical area but at different times. In image processing, the spatial domain uses grayscale values to describe an image. The frequency is directly related to the spatial change rate, so the frequency domain can be intuitively associated with patterns of intensity variations in the image. These two domains provide different perspectives for image interpretation. Most existing deep-learning-based methods formulate change detection as a pixel-wise binary classification problem and utilize various strategies to extract information in the spatial domain. However, they rarely pay attention to the rich information in the frequency domain. To address this problem, we propose an end-to-end joint frequency-spatial domain network (JFSDNet) to implement remote sensing optical image change detection. Specifically, we introduce frequency information into the change detection to supplement the loss of image details caused by downsampling. In addition, we employ a frequency selection module to adaptively discriminate and choose frequency clues by reducing the complexity of the frequency features. The JFSDNet is applied to two publicly available datasets: the change-detection dataset (CDD) dataset and the LEarning VIsion and Remote sensing Change Detection (LEVIR-CD) dataset. Compared with other methods, both visual interpretation and quantitative assessment confirmed that our proposed method achieved a favorable performance. [ABSTRACT FROM AUTHOR]

As two practical and important image processing tasks, color demosaicking (CDM) and super-resolution (SR) have been studied for decades. However, most literature studies these two tasks independently, ignoring the potential benefits of a joint solution. In this paper, aiming at efficient and effective joint demosaicking and super-resolution (JDSR), a well-designed two-stage convolutional neural network (CNN) architecture is proposed. For the first stage, by making use of the sampling-pattern information, a pattern-aware feature extraction (PFE) module extracts features directly from the Bayer-sampled low-resolution (LR) image, while keeping the resolution of the extracted features the same as the input. For the second stage, a dual-branch feature refinement (DFR) module effectively decomposes the features into two components with different spatial frequencies, on which different learning strategies are applied. On each branch of the DFR module, the feature refinement unit, namely, densely-connected dual-path enhancement blocks (DDEB), establishes a sophisticated nonlinear mapping from the LR space to the high-resolution (HR) space. To achieve strong representational power, two paths of transformations and the channel attention mechanism are adopted in DDEB. Extensive experiments demonstrate that the proposed method is superior to the sequential combination of state-of-the-art (SOTA) CDM and SR methods. Moreover, with much smaller model size, our approach also surpasses other SOTA JDSR methods. [ABSTRACT FROM AUTHOR]

Arbitrary style transfer aims at migrating the style of a reference style painting to a target content image. Existing methods find it challenging to achieve good content fidelity and style migration at the same time. Moreover, they all rely on manually defined content and style, which is of limited universality and robustness. In this paper, we propose to introduce contrastive learning into style transfer, instructing the network to automatically learn to model the structural content and artistic style based on natural contrastive relationships in style transfer. Compared with existing methods, our learned modeling of content and style is more robust and universal. In addition, we further propose instance-wise contrastive style losses and a patch-wise contrastive content loss to guide style transfer. Combining the proposed contrastive losses and two self-reconstruction strategies, we develop a new style transfer framework, which is pluggable and can be flexibly applied to various style transfer modules. Experimental results demonstrate that our method has strong flexibility and synthesizes stylized images with higher quality. [ABSTRACT FROM AUTHOR]

Feature matching, which refers to identifying and then corresponding the same or similar visual pattern from two or more images, is a key technique in any image processing task that requires establishing good correspondences between images. Given potential correspondences (matches) in two scenes, a novel whole-part deep learning framework, termed as Context Structure Representation Network (CSR-Net), is designed to infer the probabilities of arbitrary correspondences being inliers. Traditional approaches commonly build the local relation between correspondences by manually engineered criteria. Different from existing attempts, the main idea of our work is to learn explicitly neighborhood structure of each correspondence, allowing us to formulate the matching problem into a dynamic local structure consensus evaluation in an end-to-end fashion. For this purpose, we propose a permutation-invariant STructure Representation (STR) learning module, which can easily merge different types of networks into a unified architecture to deal with sparse matches directly. By the collaborative use of STR, we introduce a Context-Aware Attention (CAA) mechanism to adaptively re-calibrate structure features via a rotation-invariant context aware encoding and simple feature gating, thus arising the ability of fine-grained patterns recognition. Moreover, to further weaken the cost of establishing reliable correspondences, the CSR-Net is formulated as whole-part consensus learning, where the aim of whole level is compensating rigid transformations. In order to demonstrate our CSR-Net can effectively boost the baselines, we intensively experiment on image matching and other visual tasks. The results of the experiment confirm that the matching performances of CSR-Net have significantly improved over nine state-of-the-art competitors. [ABSTRACT FROM AUTHOR]

3D volumetric image processing has attracted increasing attention in the last decades, in which one major research area is to develop efficient lossless volumetric image compression techniques to better store and transmit such images with massive amount of information. In this work, we propose the first end-to-end optimized learning framework for losslessly compressing 3D volumetric data. Our approach builds upon a hierarchical compression scheme by additionally introducing the intra-slice auxiliary features and estimating the entropy model based on both intra-slice and inter-slice latent priors. Specifically, we first extract the hierarchical intra-slice auxiliary features through multi-scale feature extraction modules. Then, an Intra-slice and Inter-slice Conditional Entropy Coding module is proposed to fuse the intra-slice and inter-slice information from different scales as the context information. Based on such context information, we can predict the distributions for both intra-slice auxiliary features and the slice images. To further improve the lossless compression performance, we also introduce two new gating mechanisms called Intra-Gate and Inter-Gate to generate the optimal feature representations for better information fusion. Eventually, we can produce the bitstream for losslessly compressing volumetric images based on the estimated entropy model. Different from the existing lossless volumetric image codecs, our end-to-end optimized framework jointly learns both intra-slice auxiliary features at different scales for each slice and inter-slice latent features from previously encoded slices for better entropy estimation. The extensive experimental results indicate that our framework outperforms the state-of-the-art hand-crafted lossless volumetric image codecs (e.g., JP3D) and the learning-based lossless image compression method on four volumetric image benchmarks for losslessly compressing both 3D Medical Images and Hyper-Spectral Images. [ABSTRACT FROM AUTHOR]

Cartoonization as a special type of artistic style transfer is a difficult image processing task. The current existing artistic style transfer methods cannot generate satisfactory cartoon-style images due to that artistic style images often have delicate strokes and rich hierarchical color changes while cartoon-style images have smooth surfaces without obvious color changes, and sharp edges. To this end, we propose a cartoon loss based generative adversarial network (CartoonLossGAN) for cartoonization. Particularly, we first reuse the encoder part of the discriminator to build a compact generative adversarial network (GAN) based cartoonization architecture. Then we propose a novel cartoon loss function for the architecture. It can imitate the process of sketching to learn the smooth surface of the cartoon image, and imitate the coloring process to learn the coloring of the cartoon image. Furthermore, we also propose an initialization strategy, which is used in the scenario of reusing the discriminator to make our model training easier and more stable. Extensive experimental results demonstrate that our proposed CartoonLossGAN can generate fantastic cartoon-style images, and outperforms four representative methods. [ABSTRACT FROM AUTHOR]

Deep learning models have shown great potential in remote sensing (RS) image processing and analysis. Nevertheless, there are insufficient labeled samples to train deep networks, which seriously affects the performance of these models. To resolve this contradiction, we propose a generative self-supervised feature learning (S2FL) architecture for multimodal RS image land cover classification. Specifically, multiple complementary observed views are constructed from multimodal RS images, which are employed for following generative self-supervised learning (SSL). The proposed S2FL architecture is capable of extracting high-level meaningful feature representations from multiview data, and this process does not require any labeled information, providing a feasible solution to relieve the urgent need for annotated samples. The learned features are normalized and merged with corresponding spectral information to further improve the discriminative capability of feature representations, and we utilize these fused features for land cover classification. Compared with the existing supervised, semi-supervised, and self-supervised approaches, the proposed generative self-supervised model achieves superior performance in terms of feature learning and land cover classification, especially in the small sample classification case. [ABSTRACT FROM AUTHOR]

Missing data reconstruction is a classical, yet challenging problem in remote sensing (RS) image processing due to the complex atmospheric environment and variability of satellite sensors. Most of the contemporary reconstruction methods either handle only one specific task or require supplementary data, while the single input for multitask reconstruction has not been explored yet. In this article, we propose a novel generative adversarial network-based unified framework for missing RS image reconstruction, which is capable of various reconstruction tasks given only single-source data as input. Specifically, we first propose a mask extraction network (MEN) to obtain a united soft mask, which represents the intrinsic prior under various scenarios and indicates not only location, but also context information. The versatility of mask extraction enables the multitask reconstruction of RS images. Besides, we propose a unified inpainting network (UIN) to repair diverse degraded images. Being specifically tailored for RS images, dilated pyramidal convolutions (DPCs) and an attention fusion mechanism (AFM) are introduced to further improve the feature extraction ability and thus exhaustly leveraging the single-input information. Extensive experiments demonstrate the uncompromising performance of the proposed method against state-of-the-art multiinput methods on diverse missing restoration. Moreover, further exploration shows the potential of the proposed method to utilize joint spatial–spectral–temporal information, which is evaluated to outperform existing competitors on remote sense images. [ABSTRACT FROM AUTHOR]

Frequency response analysis (FRA) suffers from the interpretation of results despite its potential ability to detect faults related to the power transformer windings. This article presents a technique for interpreting frequency responses, which is based on image processing and a deep learning method called graph convolutional neural network (CNN). The proposed procedure transfers frequency responses into 2-D images through a visualization technique. The resulting images are aggregated into a dataset to be used as the CNN input. The proposed technique is applied on frequency responses of two different winding models with short circuit (SC) faults. The SC faults with different intensities are applied on different sections of a simulated ladder model winding and a 20 kV winding of a 1.6 MVA distribution transformer. After determining the frequency response for each faulty case and applying the visualization technique, the precise locating of the SC faults is performed by the CNN. Then, the results are analyzed by performance evaluation metrics. At this stage, the high performance of the CNN in the use of 2-D images instead of the conventional method is observed. Finally, by testing the high impedance SC faults in different sections of the simulated winding model and applying the suggested method step by step, early detection of the SC fault is also performed in this article. It should be noted that the suggested technique, in addition to its accuracy and high detection speed, can be considered as an important step in automatic interpretation of frequency responses for online monitoring of transformers. [ABSTRACT FROM AUTHOR]

In general, image restoration involves mapping from low-quality images to their high-quality counterparts. Such optimal mapping is usually nonlinear and learnable by machine learning. Recently, deep convolutional neural networks have proven promising for such learning processing. It is desirable for an image processing network to support well with three vital tasks, namely: 1) super-resolution; 2) denoising; and 3) deblocking. It is commonly recognized that these tasks have strong correlations, which enable us to design a general framework to support all tasks. In particular, the selection of feature scales is known to significantly impact the performance on these tasks. To this end, we propose the cross-scale residual network to exploit scale-related features among the three tasks. The proposed network can extract spatial features across different scales and establish cross-temporal feature reusage, so as to handle different tasks in a general framework. Our experiments show that the proposed approach outperforms state-of-the-art methods in both quantitative and qualitative evaluations for multiple image restoration tasks. [ABSTRACT FROM AUTHOR]

The intersecting cortical model (ICM), initially designed for image processing, is a special case of the biologically inspired pulse-coupled neural-network (PCNN) models. Although the ICM has been widely used, few studies concern the internal activities and firing conditions of the neuron, which may lead to an invalid model in the application. Furthermore, the lack of theoretical analysis has led to inappropriate parameter settings and consequent limitations on ICM applications. To address this deficiency, we first study the continuous firing condition of ICM neurons to determine the restrictions that exist between network parameters and the input signal. Second, we investigate the neuron pulse period to understand the neural firing mechanism. Third, we derive the relationship between the continuous firing condition and the neural pulse period, and the relationship can prove the validity of the continuous firing condition and the neural pulse period as well. A solid understanding of the neural firing mechanism is helpful in setting appropriate parameters and in providing a theoretical basis for widespread applications to use the ICM model effectively. Extensive experiments of numerical tests with a common image reveal the rationality of our theoretical results. [ABSTRACT FROM AUTHOR]

Joint photographic experts group (JPEG) compression is widely used in image processing and computer vision. Detecting double compressed JPEG images is a common problem in forensics and detecting compressed images with the same quantization matrix remains a challenging task. However, most existing methods were designed for detection in grayscale images and cannot fully use the unique characteristics of color images (such as the relationship between channels and color information). In addition, the performance of existing methods is unsatisfactory for low JPEG quality factors and in cross detection experiments. To solve these problems, we analyze the stability of a color image to obtain the convergence error and transposition error. According to the convergence characteristics of color JPEG images, the continuous compression by the same quantization matrix can make the JPEG image tend to be stable. The final stable state and the convergence process are determined by the number of compressions of the original image. Thus, continuously compressed JPEG images can be regarded as a continuous frame to obtain the convergence error. As the color image converges, its ability to resist interference decreases. To reflect the changes in anti-interference ability, the transposition operation is used to disturb the color JPEG image to obtain the transposition error. In addition, quaternion mapping is used to retain the relationship between continuously compressed JPEG images and enlarge the influence caused by transposition operation. In our experiments on several image databases, the proposed method outperforms existing methods in different settings. [ABSTRACT FROM AUTHOR]

Over the recent years, a number of deep learning approaches are successfully introduced to tackle the problem of image in-painting for achieving better perceptual effects. However, there still exist obvious hole-edge artifacts in these deep learning-based approaches, which need to be rectified before they become useful for practical applications. In this article, we propose an iteration-driven in-painting approach, which combines the deep context model with the backpropagation mechanism to fine-tune the learning-based in-painting process and hence, achieves further improvement over the existing state of the arts. Our iterative approach fine tunes the image generated by a pretrained deep context model via backpropagation using a weighted context loss. Extensive experiments on public available test sets, including the CelebA, Paris Streets, and PASCAL VOC 2012 dataset, show that our proposed method achieves better visual perceptual quality in terms of hole-edge artifacts compared with the state-of-the-art in-painting methods using various context models. [ABSTRACT FROM AUTHOR]

Improving the detection accuracy of pulmonary nodules plays an important role in the diagnosis and early treatment of lung cancer. In this paper, a multiscale aggregation network (MSANet), which integrates spatial and channel information, is proposed for 3D pulmonary nodule detection. MSANet is designed to improve the network's ability to extract information and realize multiscale information fusion. First, multiscale aggregation interaction strategies are used to extract multilevel features and avoid feature fusion interference caused by large resolution differences. These strategies can effectively integrate the contextual information of adjacent resolutions and help to detect different sized nodules. Second, the feature extraction module is designed for efficient channel attention and self-calibrated convolutions (ECA-SC) to enhance the interchannel and local spatial information. ECA-SC also recalibrates the features in the feature extraction process, which can realize adaptive learning of feature weights and enhance the information extraction ability of features. Third, the distribution ranking (DR) loss is introduced as the classification loss function to solve the problem of imbalanced data between positive and negative samples. The proposed MSANet is comprehensively compared with other pulmonary nodule detection networks on the LUNA16 dataset, and a CPM score of 0.920 is obtained. The results show that the sensitivity for detecting pulmonary nodules is improved and that the average number of false-positives is effectively reduced. The proposed method has advantages in pulmonary nodule detection and can effectively assist radiologists in pulmonary nodule detection. [ABSTRACT FROM AUTHOR]

This work proposes an interpretable radiomics approach to differentiate between malignant and benign focal liver lesions (FLLs) on contrast-enhanced ultrasound (CEUS). Although CEUS has shown promise for differential FLLs diagnosis, current clinical assessment is performed only by qualitative analysis of the contrast enhancement patterns. Quantitative analysis is often hampered by the unavoidable presence of motion artifacts and by the complex, spatiotemporal nature of liver contrast enhancement, consisting of multiple, overlapping vascular phases. To fully exploit the wealth of information in CEUS, while coping with these challenges, here we propose combining features extracted by the temporal and spatiotemporal analysis in the arterial phase enhancement with spatial features extracted by texture analysis at different time points. Using the extracted features as input, several machine learning classifiers are optimized to achieve semiautomatic FLLs characterization, for which there is no need for motion compensation and the only manual input required is the location of a suspicious lesion. Clinical validation on 87 FLLs from 72 patients at risk for hepatocellular carcinoma (HCC) showed promising performance, achieving a balanced accuracy of 0.84 in the distinction between benign and malignant lesions. Analysis of feature relevance demonstrates that a combination of spatiotemporal and texture features is needed to achieve the best performance. Interpretation of the most relevant features suggests that aspects related to microvascular perfusion and the microvascular architecture, together with the spatial enhancement characteristics at wash-in and peak enhancement, are important to aid the accurate characterization of FLLs. [ABSTRACT FROM AUTHOR]

Cloud and cloud shadow segmentation is one of the most important issues in remote sensing image processing. Most of the remote sensing images are very complicated. In this work, a dual-branch model composed of transformer and convolution network is proposed to extract semantic and spatial detail information of the image, respectively, to solve the problems of false detection and missed detection. To improve the model’s feature extraction, a mutual guidance module (MGM) is introduced, so that the transformer branch and the convolution branch can guide each other for feature mining. Finally, in view of the problem of rough segmentation boundary, this work uses different features extracted by the transformer branch and the convolution branch for decoding and repairs the rough segmentation boundary in the decoding part to make the segmentation boundary clearer. Experimental results on the Landsat-8, Sentinel-2 data, the public dataset high-resolution cloud cover validation dataset created by researchers at Wuhan University (HRC_WHU), and the public dataset Spatial Procedures for Automated Removal of Cloud and Shadow (SPARCS) demonstrate the effectiveness of our method and its superiority to the existing state-of-the-art cloud and cloud shadow segmentation approaches. [ABSTRACT FROM AUTHOR]

The wire bonding process is one of the most critical processes in semiconductor packaging. The electrical performance and reliability of IC chips must be evaluated by probe testing before wire bonding. In the probe test, the probe usually leaves traces on the surface of the pad. Large probe marks tend to cause a decrease in bond adhesion and so increase the possibility of ball bond lifting from the pads. Shear force is an important measure of the adhesion between the ball and pad. The prediction of the shear force helps to understand the bonding quality in advance. In this study, the six features of the probe marks were extracted by the automatic image recognition method. Using three machine learning techniques (logistic regression, support vector machine, and random forest) based on principal component analysis (PCA), the shear force is estimated based on six features before wire bonding. The results indicate that the proposed PCA-based random forest could identify bad chips with 97.92% accuracy before wire bonding. Unnecessary chips can be found early, and unnecessary wire bonding can be avoided, thereby saving process time and cost. The proposed method would improve the efficiency and quality yield of the semiconductor packaging process. [ABSTRACT FROM AUTHOR]

Spatial failure patterns in wafer defect maps are very useful for root cause analysis, which is essential for yield optimization in the manufacturing process. Especially, the scratch defect type is the most challenging pattern to recognize because the position, shape, size and curvature vary widely from one scratch to another. Discontinuity points within scratches also contribute to the low recognition rate, and such points are often hidden defective dies that become reliability threat. Previous studies on defect pattern identification show that the recognition rate for scratches is among the lowest in all patterns even if the overall accuracy is high. In this paper, we propose a novel scratch pattern recognition method. In this paper, image processing techniques are applied to recognize scratch patterns in wafer maps. The Hough transform is first employed to identify line segments, from which scratches can be reconstructed. In contrast to previous machine-learning based methods, there is no need to train a complicated prediction model so that the computation time is small. The method is validated by using wafer maps from six real products and public data. Experimental results show that the proposed method achieves high recall, precision and accuracy in all cases. [ABSTRACT FROM AUTHOR]

Retrieving 3D shapes based on 2D images is a challenging research topic, due to the significant gap between different domains. Recently, various approaches have been proposed to handle this problem. However, the majority of methods target the cross-domain retrieval task as a pure domain adaptation problem, which focuses on the alignment but ignores the visual relevance between the 2D images and their corresponding 3D shapes. To fundamentally decrease the divergence between different domains, we propose a novel cross-domain learning network (CLN) for 2D image-based 3D shape retrieval task. First, we estimate the pose information from the 2D image to guide the view rendering of 3D shapes, which increases the visual correlations of the cross-domain data to eliminate the divergence between them. Second, we introduce a novel joint learning network, considering both the domain-specific characteristics and the cross-domain interactions for data alignment, which further compensates for the gap between different domains by controlling the distance of intra- and inter-classes. After the metric learning process, discriminative descriptors of images and shapes are generated for the cross-domain retrieval task. To prove the effectiveness and robustness of the proposed method, we conduct extensive experiments on the MI3DOR, SHREC’13, and SHREC’14 datasets. The experimental results demonstrate the superiority of our proposed method, and significant improvements have been achieved compared with state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Remote sensing images, especially hyperspectral images (HSIs), are extremely vulnerable to random noise and stripe noise. As a key aspect of HSI data quality improvement, stripe noise removal has always been a pervasive issue in remote sensing image processing. Convolutional neural networks have been applied for HSI data destriping. However, the existing methods lose the stripe-free component of the original image to a certain extent. These models also ignore the global spatial context of images and the correlation between spatial information and spectral information. Therefore, we propose a novel destriping convolutional network to overcome the problems with the existing methods. Octave convolution is used to extract cross-frequency features, and separate and compress the low-frequency information of the images, while dilation convolution (Dila-Conv) is used to reduce the amount of required calculation and also preserve the key image information. In addition, Dila-Conv can expand the receptive field to obtain multiscale features. Finally, a cross-channel enhanced spatial–spectral feature fusion module is used to acquire and integrate spatial context information and interchannel dependencies on a global scale as auxiliary information so that the network model can learn and pay attention to key feature information, specifically, “what to look for” and “where to look at,” which can facilitate the distinction between stripe and stripe-free components. Experimental results obtained using multiple datasets demonstrated that the proposed method can outperform the existing comparable methods and can produce satisfactory results in terms of visual effects and quantitative evaluation. [ABSTRACT FROM AUTHOR]

Feature matching, which aims to seek good correspondences from an image pair of the same or similar scene, is one of the important studies on digital remote sensing (RS) image processing. However, alongside the common degradation problems, such as geometric distortion, RS images also often face nonlinear radiation distortions, thereby posing more complex matching patterns. To reduce the cost of establishing reliable correspondences, this article proposes a simple, but effective end-to-end hierarchical learning framework, termed interactive generative structure learning network (IGS-Net). The key thinking of our approach is to offer a structure self-generate learning mechanism, called interactive generative structure learning (IGSL) block, for modeling the local context information of potential correspondences. Specifically, IGSL contains two novel operations: adaptive structure-aware representation (ASR) and physical constraint embedding. Besides, we introduce a coarse-to-fine geometry estimation pipeline aligning two sets of feature points to weaken the degree of randomness in matching patterns, thus improving the generalization ability of representation learning. Overall, this differentiable representation learning architecture can be inserted into existing classification models easily for robust outlier detection and removal. In order to demonstrate that our IGS-Net can boost the baselines, we intensively experiment on both single modality and multimodal RS image datasets. The large amounts of experiment results reveal that the matching performances of IGS-Net are significantly improved over eight state-of-the-art competitors. [ABSTRACT FROM AUTHOR]

The profound impact of deep learning and particularly of convolutional neural networks (CNNs) in automatic image processing has been decisive for the progress and evolution of remote sensing (RS) hyperspectral imaging (HSI) processing. Indeed, CNNs have stated themselves as the current state of the art, reaching unparalleled results in HSI classification. However, most CNNs were designed for RGB images, and their direct application to HSI data analysis could lead to nonoptimal solutions. Moreover, CNNs perform classification based on the identification of specific features, neglecting the spatial relationships between different features (i.e., their arrangement) due to pooling techniques. The capsule network (CapsNet) architecture is an attempt to overcome this drawback by nesting several neural layers within a capsule, connected by dynamic routing, both to identify not only the presence of a feature but also its instantiation parameters and to learn the relationships between different features. Although this mechanism improves the data representations, enhancing the classification of HSI data, it still acts as a black box, without control of the most relevant features for classification purposes. Indeed, important features could be discriminated against. In this article, a new multiple attention-guided CapsNet is proposed to improve feature processing for RS-HSIs’ classification, both to improve computational efficiency (in terms of parameters) and increase accuracy. Hence, the most representative visual parts of the images are identified using a detailed feature extractor coupled with attention mechanisms. Extensive experimental results have been obtained on five real datasets, demonstrating the great potential of the proposed method compared to other state-of-the-art classifiers. [ABSTRACT FROM AUTHOR]

An angular clustering method based on image processing is proposed in this article. It is used to identify clusters in 2-D representations of propagation channels. The approach uses operations such as watershed segmentation and is particularly well suited for clustering directional channels obtained by beam steering at millimeter wave. This situation occurs for instance with electronic beam steering using analog antenna arrays during beam-training process or channel modeling measurements using either electronic or mechanical beam steering. In particular, the proposed technique is used here to cluster 2-D power angular spectrum maps. The proposed clustering is unsupervised and is well suited to preserve the shape of clusters by considering the angular connection between neighbor samples, which is useful to obtain more accurate descriptions of channel angular properties. The approach is found to outperform approaches based on K-power-means in terms of accuracy as well as computational resources. The technique is assessed in simulation using the IEEE 802.11ad channel model and in measurement using experiments conducted at 60 GHz in an indoor environment. [ABSTRACT FROM AUTHOR]

Deep convolutional neural networks (CNNs) have emerged as a new paradigm for Mammogram diagnosis. Contemporary CNN-based computer-aided-diagnosis systems (CADs) for breast cancer directly extract latent features from input mammogram image and ignore the importance of morphological features. In this paper, we introduce a novel end-to-end deep learning framework for mammogram image processing, which computes mass segmentation and simultaneously predicts diagnosis results. Specifically, our method is constructed in a dual-path architecture that solves the mapping in a dual-problem manner, with an additional consideration of important shape and boundary knowledge. One path, called the Locality Preserving Learner (LPL), is devoted to hierarchically extracting and exploiting intrinsic features of the input. Whereas the other path, called the Conditional Graph Learner (CGL), focuses on generating geometrical features via modeling pixel-wise image to mask correlations. By integrating the two learners, both the cancer semantics and cancer representations are well learned, and the component learning paths in return complement each other, contributing an improvement to the mass segmentation and cancer classification problem at the same time. In addition, by integrating an automatic detection set-up, the DualCoreNet achieves fully automatic breast cancer diagnosis practically. Experimental results show that in benchmark DDSM dataset, DualCoreNet has outperformed other related works in both segmentation and classification tasks, achieving 92.27% DI coefficient and 0.85 AUC score. In another benchmark INbreast dataset, DualCoreNet achieves the best mammography segmentation (93.69% DI coefficient) and competitive classification performance (0.93 AUC score). [ABSTRACT FROM AUTHOR]

Object detection from remote sensing images (RSIs) is a basic topic in the area of aviation and satellite image processing, which has great effects on geological disaster detection, agricultural planning, and land utilization. However, it is always faced with several severe difficulties. For instance, the scale of the target spans over a very wide range, and the difference between the target size and the image size is huge, as some targets only account for a dozen pixels compared with the RSI of the megapixel level. In this work, an innovative object detection network (GLNet) is proposed for remote sensing imagery. Our approach integrates global context clues extracted by the multiscale perception (MSP) module and local spatial contextual correlations encoded by Clip long short-term memory (Clip-LSTM). These rich semantic features are further exploited to design a self-adapted anchor (SA) module to alleviate the scale variations in RSIs. Extensive experiments are executed on several public easily accessed benchmarks, including DOTA, NWPU VHR-10, and DIOR. Experimental results have demonstrated that our GLNet outperforms numerous latest methods. [ABSTRACT FROM AUTHOR]

Identity preservation is crucial for the practical application of face sketch and photo synthesis, such as law enforcement and entertainment. However, existing methods mainly focus on generating faces with good visual quality, leading the evaluation of identity preservation is partial and insufficient. Besides, it is difficult to simultaneously synthesize photos and sketches that have good visual quality and identity preservation. In this article, we propose to utilize auxiliary deep features extracted by an off-the-shelf face classifier to inject into the synthesis processes, so that the synthesized faces have more identity information. Furthermore, we propose a light interpolated convolutional neural network that has a shared encoder and two face-specialized decoders to simultaneously complete the transformations between sketches and photos. Evaluations on identity preservation and visual quality show our method is superior to existing methods in synthesizing both sketches and photos, and is qualified in practical application. [ABSTRACT FROM AUTHOR]