Search

Describes the above-named upcoming special issue or section. May include topics to be covered or calls for papers. [ABSTRACT FROM AUTHOR]

Machine learning has been widely used in medical image analysis under an assumption that the training and test data are under the same feature distributions. However, medical images from difference devices or the same device with different parameter settings are often contaminated with different amount and types of noises, which violate the above assumption. Therefore, the models trained using data from one device or setting often fail to work for that from another. Moreover, it is very expensive and tedious to label data and re-train models for all different devices or settings. To overcome this noise adaptation issue, it is necessary to leverage on the models trained with data from one device or setting for new data. In this paper, we reformulate this noise adaptation task as an image-to-image translation task such that the noise patterns from the test data are modified to be similar to those from the training data while the contents of the data are unchanged. In this paper, we propose a novel Noise Adaptation Generative Adversarial Network (NAGAN), which contains a generator and two discriminators. The generator aims to map the data from source domain to target domain. Among the two discriminators, one discriminator enforces the generated images to have the same noise patterns as those from the target domain, and the second discriminator enforces the content to be preserved in the generated images. We apply the proposed NAGAN on both optical coherence tomography (OCT) images and ultrasound images. Results show that the method is able to translate the noise style. In addition, we also evaluate our proposed method with segmentation task in OCT and classification task in ultrasound. The experimental results show that the proposed NAGAN improves the analysis outcome. [ABSTRACT FROM AUTHOR]

This paper aims at proposing a novel 2D non-linear phase decomposition of images, which performs the image processing tasks better than the traditional Fourier transformation (linear phase decomposition), but further, it has additional mathematical properties allowing more effective image analysis, including adaptive decomposition components and positive instantaneous phase derivatives. 1D unwinding Blaschke decomposition has recently been proposed and studied. Through factorization it expresses arbitrary 1D signal into an infinite linear combination of Blaschke products. It offers fast converging positive frequency decomposition in the form of rational approximation. However, in the multi-dimensional cases, the usual factorization mechanism does not work. As a consequence, there is no genuine unwinding decomposition for multi-dimensions. In this paper, a 2D partial unwinding decomposition based on algebraic transforms reducing multi-dimensions to the 1D case is proposed and analyzed. The result shows that the fast convergence offers efficient image reconstruction. The tensor type decomposing terms are mutually orthogonal, giving rise to 2D positive frequency decomposition. The comparison results show that the proposed method outperforms the standard greedy algorithm and the most commonly used methods in the Fourier category. An application in watermarking is presented to demonstrate its potential in applications. [ABSTRACT FROM AUTHOR]

Automated scene analysis has been a topic of great interest in computer vision and cognitive science. Recently, with the growth of crowd phenomena in the real world, crowded scene analysis has attracted much attention. However, the visual occlusions and ambiguities in crowded scenes, as well as the complex behaviors and scene semantics, make the analysis a challenging task. In the past few years, an increasing number of works on the crowded scene analysis have been reported, which covered different aspects including crowd motion pattern learning, crowd behavior and activity analyses, and anomaly detection in crowds. This paper surveys the state-of-the-art techniques on this topic. We first provide the background knowledge and the available features related to crowded scenes. Then, existing models, popular algorithms, evaluation protocols, and system performance are provided corresponding to different aspects of the crowded scene analysis. We also outline the available datasets for performance evaluation. Finally, some research problems and promising future directions are presented with discussions. [ABSTRACT FROM PUBLISHER]

Virtual metrology (VM) predicts the actual measurement for ongoing semiconductor process. Optical emission spectroscopy (OES) data is often used to build up VM models, since it has a lot of information on process quality. However, it also includes significant redundant information, so it is important how to select only meaningful features. Deep learning (DL) techniques have been very successful in analyzing image data and it is tempting to apply those techniques to the OES data. In this paper, we propose DL configurations specific to OES data that outperform those used for image analysis. Specifically, our proposed method accounts for variable size data, chamber to chamber differences, condition drift due to accumulation, observation data drift due to an accumulation of deposition on a window, and the effects of maintenance. We evaluated our method on a real, mass-production dataset and compared our results with those obtained by using state-of-the-art image analysis DL techniques in the famous contest, ImageNet large-scale visual recognition challenge (ILSVRC). [ABSTRACT FROM AUTHOR]

Presents the table of contents for this issue of the publication. [ABSTRACT FROM PUBLISHER]

This paper presents a novel computational intelligence vision sensing approach to estimate nutrient content in wheat leaves by analyzing color features of the leaves images captured on field with various lighting conditions. We propose the development of deep sparse extreme learning machines (DSELM) fusion and genetic algorithm (GA) to normalize plant images as well as to reduce color variability due to a variation of sunlight intensities. We also apply the DSELM in image segmentation to differentiate wheat leaves from a complex background. In this paper, four moments of color distribution of the leaves images (mean, variance, skewness, and kurtosis) are extracted and utilized as predictors in the nutrient estimation. We combine a number of DSELMs with committee machine and optimize them using the GA to estimate nitrogen content in wheat leaves. The results have shown the superiority of the proposed method in the term of quality and processing speed in all steps, i.e., color normalization, image segmentation, and nutrient prediction, as compared with other existing methods. Note to Practitioners—In order to support precision farming technology and agricultural automation, it is very essential to estimate the nutrient content in plants to prevent over fertilizing that will harm environment. Furthermore, the existing method to determine nutrient content in leaves is destructive as well as time consuming and requires special expertise to operate expensive devices. A number of methods have been developed to estimate nutrient content in leaves nondestructively based on the color features of the leaves. Most of these methods, however, are conducted in a controlled environment with artificial lighting. Such methods are not practical and need various equipment. We propose a low-cost, simple, and accurate technique to estimate nitrogen content in wheat leaves by analyzing RGB color of the leaves images. In this paper, we found that the developed DSELMs fusion has enabled better performance in normalizing images and is faster than other neural network types, i.e., backpropagation-based multilayer perceptron and original extreme learning machine. In image segmentation step, the established DSELM shows good performance to recognize and distinguish wheat leaves from other undesired background images. Furthermore, the developed weighted DSELMs have demonstrated enhanced ability in estimating nutrient content. [ABSTRACT FROM AUTHOR]

This paper aims at proposing an unsupervised hierarchical nonparametric Bayesian framework for modeling axial data (i.e., observations are axes of direction) that can be partitioned into multiple groups, where each observation within a group is sampled from a mixture of Watson distributions with an infinite number of components that are allowed to be shared across different groups. First, we propose a hierarchical nonparametric Bayesian model for modeling grouped axial data based on the hierarchical Pitman-Yor process mixture model of Watson distributions. Then, we demonstrate that by setting the discount parameters of the proposed model to 0, another hierarchical nonparametric Bayesian model based on hierarchical Dirichlet process can be derived for modeling axial data. To learn the proposed models, we systematically develop a closed-form optimization algorithm based on the collapsed variational Bayes (CVB) inference. Furthermore, to ensure the convergence of the proposed learning algorithm, an annealing mechanism is introduced to the framework of CVB inference, leading to an averaged collapsed variational Bayes inference strategy. The merits of the proposed models for modeling grouped axial data are demonstrated through experiments on both synthetic data and real-world applications involving gene expression data clustering and depth image analysis. [ABSTRACT FROM AUTHOR]

Image segmentation is the foundation of high-level image analysis and image understanding. How to effectively segment an image into regions that are “meaningful” to the human visual perception and ensure that the segmented regions are consistent at different resolutions is still a very challenging issue. Inspired by the idea of the Nonsymmetry and Anti-packing pattern representation Model in the Lab color space (NAMLab) and the “global-first” invariant perceptual theory, in this paper, we propose a novel framework for hierarchical image segmentation. Firstly, by defining the dissimilarity between two pixels in the Lab color space, we propose an NAMLab-based color image representation approach that is more in line with the human visual perception characteristics and can make the image pixels fast and effectively merge into the NAMLab blocks. Then, by defining the dissimilarity between two NAMLab-based regions and iteratively executing NAMLab-based merging algorithm of adjacent regions into larger ones to progressively generate a segmentation dendrogram, we propose a fast NAMLab-based algorithm for hierarchical image segmentation. Finally, the complexities of our proposed NAMLab-based algorithm for hierarchical image segmentation are analyzed in details. The experimental results presented in this paper show that our proposed algorithm when compared with the state-of-the-art algorithms not only can preserve more details of the object boundaries, but also it can better identify the foreground objects with similar color distributions. Also, our proposed algorithm can be executed much faster and takes up less memory and therefore it is a better algorithm for hierarchical image segmentation. [ABSTRACT FROM AUTHOR]

The gold standard for diagnosing lymph node metastasis of papillary thyroid carcinoma is to analyze the whole slide histopathological images (WSIs). Due to the large size of WSIs, recent computer-aided diagnosis approaches adopt the multi-instance learning (MIL) strategy and the key part is how to effectively aggregate the information of different instances (patches). In this paper, a novel transformer-guided framework is proposed to predict lymph node metastasis from WSIs, where we incorporate the transformer mechanism to improve the accuracy from three different aspects. First, we propose an effective transformer-based module for discriminative patch feature extraction, including a lightweight feature extractor with a pruned transformer (Tiny-ViT) and a clustering-based instance selection scheme. Next, we propose a new Transformer-MIL module to capture the relationship of different discriminative patches with sparse distribution on WSIs and better nonlinearly aggregate patch-level features into the slide-level prediction. Considering that the slide-level annotation is relatively limited to training a robust Transformer-MIL, we utilize the pathological relationship between the primary tumor and its lymph node metastasis and develop an effective attention-based mutual knowledge distillation (AMKD) paradigm. Experimental results on our collected WSI dataset demonstrate the efficiency of the proposed Transformer-MIL and attention-based knowledge distillation. Our method outperforms the state-of-the-art methods by over 2.72% in AUC (area under the curve). [ABSTRACT FROM AUTHOR]

Image and video sensors are increasingly being deployed in complex systems due to the rich process information that these sensors can capture. As a result, image data play an important role in process monitoring and control in different application domains such as manufacturing processes, food industries, medical decision-making, and structural health monitoring. Existing process monitoring techniques fail to fully utilize the information of color images due to their complex data characteristics including the high-dimensionality and correlation structure (i.e., temporal, spatial and spectral correlation). This paper proposes a new image-based process monitoring approach that is capable of handling both grayscale and color images. The proposed approach models the high-dimensional structure of the image data with tensors and employs low-rank tensor decomposition techniques to extract important monitoring features monitored using multivariate control charts. In addition, this paper shows the analytical relationships between different low-rank tensor decomposition methods. The performance of the proposed method in quick detection of process changes is evaluated and compared with existing methods through extensive simulations and a case study in a steel tube manufacturing process. [ABSTRACT FROM PUBLISHER]

Total anomalous pulmonary venous connection (TAPVC) is a rare but mortal congenital heart disease in children and can be repaired by surgical operations. However, some patients may suffer from pulmonary venous obstruction (PVO) after surgery with insufficient blood supply, necessitating special follow-up strategy and treatment. Therefore, it is a clinically important yet challenging problem to predict such patients before surgery. In this paper, we address this issue and propose a computational framework to determine the risk factors for postoperative PVO (PPVO) from computed tomography angiography (CTA) images and build the PPVO risk prediction model. From clinical experiences, such risk factors are likely from the left atrium (LA) and pulmonary vein (PV) of the patient. Thus, 3D models of LA and PV are first reconstructed from low-dose CTA images. Then, a feature pool is built by computing different morphological features from 3D models of LA and PV, and the coupling spatial features of LA and PV. Finally, four risk factors are identified from the feature pool using the machine learning techniques, followed by a risk prediction model. As a result, not only PPVO patients can be effectively predicted but also qualitative risk factors reported in the literature can now be quantified. Finally, the risk prediction model is evaluated on two independent clinical datasets from two hospitals. The model can achieve the AUC values of 0.88 and 0.87 respectively, demonstrating its effectiveness in risk prediction. [ABSTRACT FROM AUTHOR]

Segmenting anatomical structures in medical images has been successfully addressed with deep learning methods for a range of applications. However, this success is heavily dependent on the quality of the image that is being segmented. A commonly neglected point in the medical image analysis community is the vast amount of clinical images that have severe image artefacts due to organ motion, movement of the patient and/or image acquisition related issues. In this paper, we discuss the implications of image motion artefacts on cardiac MR segmentation and compare a variety of approaches for jointly correcting for artefacts and segmenting the cardiac cavity. The method is based on our recently developed joint artefact detection and reconstruction method, which reconstructs high quality MR images from k-space using a joint loss function and essentially converts the artefact correction task to an under-sampled image reconstruction task by enforcing a data consistency term. In this paper, we propose to use a segmentation network coupled with this in an end-to-end framework. Our training optimises three different tasks: 1) image artefact detection, 2) artefact correction and 3) image segmentation. We train the reconstruction network to automatically correct for motion-related artefacts using synthetically corrupted cardiac MR k-space data and uncorrected reconstructed images. Using a test set of 500 2D+time cine MR acquisitions from the UK Biobank data set, we achieve demonstrably good image quality and high segmentation accuracy in the presence of synthetic motion artefacts. We showcase better performance compared to various image correction architectures. [ABSTRACT FROM AUTHOR]

The importance of effective surgical wound care cannot never be underestimated. Poorly managing surgical wounds may cause many serious complications. Thus, it raises the necessity to develop a patient-friendly self-care system which can help both patients and medical professionals to ensure the state of the surgical wounds without any special medical equipment. In this paper, a surgical wound assessment system for self-care is proposed. The proposed system is designed to enable patients capture surgical wound images of themselves by using a mobile device and upload these images for analysis. Combining image-processing and machine-learning techniques, the proposed method is composed of four phases. First, images are segmented into superpixels where each superpixel contains the pixels in the similar color distribution. Second, these superpixels corresponding to the skin are identified and the area of connected skin superpixels is derived. Third, surgical wounds will be extracted from this area based on the observation of the texture difference between skin and wounds. Lastly, state and symptoms of surgical wound will be assessed. Extensive experimental results are conducted. With the proposed method, more than 90% state assessment results are correct and more than 91% symptom assessment results consistent with the actual diagnosis. Moreover, case studies are provided to show the advantage and limitation of this system. These results show that this system could perform well in the practical self-care scenario. [ABSTRACT FROM AUTHOR]

As more and more satellites are launched into the space, dynamic estimation of spin satellites has become a critical component of the space situation awareness application. Some explored studies using exterior measurements from different sensors such as optical device and inverse synthetic aperture radar (ISAR) to estimate dynamic parameters of spin satellites. As a single sensor normally provides two-dimensional observation, three-dimensional estimations resulting from these algorithms are strictly related to the prior knowledge of targets characteristics. As a result, it is difficult to expand these methods to other satellites. In order to support the dynamic estimation of most spin satellites, this paper presents a novel dynamic estimation approach which employs synchronized optical-and-radar images. The optical-and-radar fusion strategy has demonstrated its superiority in image analysis field, and breaks down the dynamic estimation of spin satellites into two sub-problems: target attitude estimation and spin parameters estimation. In this work, the proposed algorithm deduces two explicit expressions of target dynamic parameters under the imaging projection model of the joint optical-and-radar observation. Through the particle swarm optimization (PSO), target dynamic parameters are determined in two stages. This paper presents some experiments illustrating the feasibility of the proposed method and subsequent conclusions, which reflect advantages of the joint optical-and-radar observation mode in image interpretation. [ABSTRACT FROM AUTHOR]

Advances in remote sensing hardware have led to a significantly increased capability for high-quality data acquisition, which allows the collection of remotely sensed images with very high spatial, spectral, and radiometric resolution. This trend calls for the development of new techniques to enhance the way that such unprecedented volumes of data are stored, processed, and analyzed. An important approach to deal with massive volumes of information is data compression, related to how data are compressed before their storage or transmission. For instance, hyperspectral images (HSIs) are characterized by hundreds of spectral bands. In this sense, high-performance computing (HPC) and high-throughput computing (HTC) offer interesting alternatives. Particularly, distributed solutions based on cloud computing can manage and store huge amounts of data in fault-tolerant environments, by interconnecting distributed computing nodes so that no specialized hardware is needed. This strategy greatly reduces the processing costs, making the processing of high volumes of remotely sensed data a natural and even cheap solution. In this paper, we present a new cloud-based technique for spectral analysis and compression of HSIs. Specifically, we develop a cloud implementation of a popular deep neural network for non-linear data compression, known as autoencoder (AE). Apache Spark serves as the backbone of our cloud computing environment by connecting the available processing nodes using a master–slave architecture. Our newly developed approach has been tested using two widely available HSI data sets. Experimental results indicate that cloud computing architectures offer an adequate solution for managing big remotely sensed data sets. [ABSTRACT FROM AUTHOR]

With the development of video surveillance in public safety field, there is an increasing research on person re-identification (re-id). In this paper, we address the image-to-video person re-id, in which the probe is an image and the gallery is consists of videos captured by nonoverlapping cameras. Compared with image, video sequence contains more temporal information that can be explored to improve the performance of re-identification system. However, it is challenging to model temporal information in the matching process of image-to-video person re-id. In this paper, we proposed a novel temporally memorized similarity learning neural network for this problem. In specific, the proposed network mainly consisted of two parts, including feature representation sub-network and similarity sub-network. In the first part, we adopted a convolutional neural network (CNN) to extract features from the input image. Given a video sequence of a person, features were first extracted from each its frame by using CNN and further forward to a long shot term memory (LSTM) network to encode the temporal information of video sequence. The outputs of LSTM were concatenated together as the feature vector of video sequences. Finally, the feature vectors of probe image and the video sequence were further forward to the similarity sub-network for distance metric learning. In the proposed framework, the feature representation and the similarity metric learning can be learned and optimized simultaneously. We evaluated the proposed framework on three public person re-id data sets, and the experimental results showed that the proposed approach is effective for the image-to-video person re-id. [ABSTRACT FROM AUTHOR]

Streamer discharges provide efficient means for chemical change in a variety of applications, some of which are ozone generation, water purification, and hydrogen and syngas production. Better understanding of the streamer growth process, and simple physical characteristics, such as size and geometry, could lead to an increase in product yield for these plasma-based applications. This report focuses in particular on the size and geometry of streamers formed in liquid, with an underlying goal to generate hydrogen gas and syngas via plasma-induced hydrocarbon reformation. The examination of size and geometry was performed with a measurement of the streamer length and area, and a fractal analysis, respectively. The fractal analysis used a box counting method to quantify the geometry of the streamer with a metric of complexity called the fractal dimension. This technique was explored for its possible use in characterizing streamers in regards to volume and surface area as such direct measurements can be difficult to obtain. The streamers discussed in this paper were induced under different operating conditions and liquid parameters, namely electrical conductivity of the liquid, voltage polarity, and carbon chain length of the liquid. [ABSTRACT FROM AUTHOR]

Nonnegative matrix factorization (NMF) has been successfully applied in several data mining tasks. Recently, there is an increasing interest in the acceleration of NMF, due to its high cost on large matrices. On the other hand, the privacy issue of NMF over federated data is worthy of attention, since NMF is prevalently applied in image and text analysis which may involve leveraging privacy data (e.g, medical image and record) across several parties (e.g., hospitals). In this paper, we study the acceleration and security problems of distributed NMF. First, we propose a distributed sketched alternating nonnegative least squares(DSANLS) framework for NMF, which utilizes a matrix sketching technique to reduce the size of nonnegative least squares subproblems with a convergence guarantee. For the second problem, we show that DSANLS with modification can be adapted to the security setting, but only for one or limited iterations. Consequently, we propose four efficient distributed NMF methods in both synchronous and asynchronous settings with a security guarantee. We conduct extensive experiments on several real datasets to show the superiority of our proposed methods. The implementation of our methods is available at https://github.com/qianyuqiu79/DSANLS. [ABSTRACT FROM AUTHOR]

The twenty six papers in this special issue focus on the technologies of hyperspectral remote sensing (HRS)and imaging spectroscopy. HRS has emerged as a powerful tool to understand phenomena at local and global scales by virtue of imaging through a diverse range of platforms, including terrestrial in-situ imaging platforms, unmanned and manned aerial vehicles, and satellite platforms. By virtue of imaging over a wide range of spectral wavelengths, it is possible to characterize object specific properties very accurately. As a result, hyperspectral imaging (also known as imaging spectroscopy) has gained popularity for a wide variety of applications, including environment monitoring, precision agriculture, mineralogy, forestry, urban planning, and defense applications. The increased analysis capability comes at a cost?there are a variety of challenges that must be overcome for robust image analysis of such data, including high dimensionality, limited sample size for training supervised models, noise and atmospheric affects, mixed pixels, etc. The papers in this issue represent some of the recent developments in image analysis algorithms and unique applications of hyperspectral imaging data. [ABSTRACT FROM PUBLISHER]

Visual tracking in image streams acquired by static cameras is usually based on change detection and recursive Bayesian estimation, such an approach laying at the core of many practical applications. Yet, the interaction between the change detector and the Bayesian filter is typically designed heuristically. Differently, this paper develops a sound framework to model and implement a bidirectional communication flow between the two processes. In our Bayesian loop, change detection provides well-defined observation likelihood to the recursive filter and the filter prediction provides an informative prior to the change detector, which deploys Bayesian reasoning alike. The loop is developed for the two major variants of Bayesian filters used in tracking, namely the Kalman filter and the particle filter. Experiments on publicly available videos and a novel challenging data set show that the proposed interaction scheme outperforms several state-of-the-art trackers. [ABSTRACT FROM AUTHOR]

In this paper, we discuss and review how combined multiview imagery from satellite to street level can benefit scene analysis. Numerous works exist that merge information from remote sensing and images acquired from the ground for tasks such as object detection, robots guidance, or scene understanding. What makes the combination of overhead and street-level images challenging are the strongly varying viewpoints, the different scales of the images, their illuminations and sensor modality, and time of acquisition. Direct (dense) matching of images on a per-pixel basis is thus often impossible, and one has to resort to alternative strategies that will be discussed in this paper. For such purpose, we review recent works that attempt to combine images taken from the ground and overhead views for purposes like scene registration, reconstruction, or classification. After the theoretical review, we present three recent methods to showcase the interest and potential impact of such fusion on real applications (change detection, image orientation, and tree cataloging), whose logic can then be reused to extend the use of ground-based images in remote sensing and vice versa. Through this review, we advocate that cross fertilization between remote sensing, computer vision, and machine learning is very valuable to make the best of geographic data available from Earth observation sensors and ground imagery. Despite its challenges, we believe that integrating these complementary data sources will lead to major breakthroughs in Big GeoData. It will open new perspectives for this exciting and emerging field. [ABSTRACT FROM PUBLISHER]

This paper proposes a method to detect and evaluate surface and subsurface cracks in aluminum multilayer structures using a planar eddy current testing (ECT) probe and processing an inverse problem algorithm. The proposed excitation method using this ECT probe allows the induction of eddy currents with different orientations on the metal surface without rotating the probe during the scan. An inversion algorithm was applied to evaluate the geometry of the cracks. The main result of this inversion algorithm is the determination of the shape of the cracks using the reconstructed eddy current pattern inside the specimen. These reconstructed patterns give the indication of the length, orientation, and geometry profile of the crack. In this paper, complex geometry cracks at different depths are inspected. Experimental data were obtained around a star crack at different depths in a stack of four aluminum plates where each plate has a thickness equal to 1 mm. The presented work shows that the distance between the sensor and the layer under analysis must be adjusted in the inversion process in order to obtain the best reconstructed images when subsurface cracks are under study. This consideration affects the quality of the resulting images. [ABSTRACT FROM PUBLISHER]

This paper presents a 3-D stacked vision chip featuring in-focal-plane read-out tightly coupled with flexible computing architecture for configurable high-speed image analysis. The chip architecture is based on a scalable standalone structure integrating image sensor on the top tier and processing elements (PEs) plus memories in the bottom tier. By using 3-D stacking partitioning, our prototype benefits from backside illuminated pixels sensitivity, a fully parallel communication between image sensor and PEs for low-latency performances, while leaving enough room in the bottom tier to embed advanced computing features. One scalable structure embeds a $16\times 16$ pixel array (or $64\times 64$ pixels in high-resolution mode), associated with an 8-bit single instruction multiple data (SIMD) processor; fabricated in dual 130-nm 1P6M CMOS process. This paper exhibits a 5500 frames/s and 85 giga operations per second (GOPS)/W in low-resolution mode, with large kernels capabilities through eight directions interpixel communication. Multiflow capability is also demonstrated to execute different programs in different areas of the vision chip. [ABSTRACT FROM AUTHOR]

The classification of hyperspectral images (HSIs) using convolutional neural networks (CNNs) has recently drawn significant attention. However, it is important to address the potential overfitting problems that CNN-based methods suffer when dealing with HSIs. Unlike common natural images, HSIs are essentially three-order tensors which contain two spatial dimensions and one spectral dimension. As a result, exploiting both spatial and spectral information is very important for HSI classification. This paper proposes a new hand-crafted feature extraction method, based on multiscale covariance maps (MCMs), that is specifically aimed at improving the classification of HSIs using CNNs. The proposed method has the following distinctive advantages. First, with the use of covariance maps, the spatial and spectral information of the HSI can be jointly exploited. Each entry in the covariance map stands for the covariance between two different spectral bands within a local spatial window, which can absorb and integrate the two kinds of information (spatial and spectral) in a natural way. Second, by means of our multiscale strategy, each sample can be enhanced with spatial information from different scales, increasing the information conveyed by training samples significantly. To verify the effectiveness of our proposed method, we conduct comprehensive experiments on three widely used hyperspectral data sets, using a classical 2-D CNN (2DCNN) model. Our experimental results demonstrate that the proposed method can indeed increase the robustness of the CNN model. Moreover, the proposed MCMs+2DCNN method exhibits better classification performance than other CNN-based classification strategies and several standard techniques for spectral-spatial classification of HSIs. [ABSTRACT FROM AUTHOR]

Connected component labeling (CCL) is a fundamental image processing problem that has been studied in many platforms, including GPUs. A common approach to CCL performance analysis is studying the total processing time as a function of abstract image features, like the number of connected components or the fraction of foreground pixels, and input data usually includes synthetic images and segmented video datasets. In this paper, we develop on these ideas and propose an evaluation methodology for GPU CCL algorithms based on synthetic image patterns, addressing the nonexistence of a standard and reliable benchmark in the literature. Our methodology, applied on two important algorithms from existing literature, uncovers their data dependency with great detail, and allows us to model their processing time in three real-world video data sets as functions of abstract, high-level image concepts. We also apply our methodology for studying the memory and performance requirements of two strategies for computing connected component properties: an existing memory-hungry approach, and a new memory-preserving strategy. [ABSTRACT FROM AUTHOR]

In this paper, we propose a novel weakly supervised semantic segmentation (WSSS) method that uses image tags as supervision to achieve joint pixel-level localization of the key local structure (KLS) and image-level classification of the aurora images captured by the ground-based optical all-sky imager. First, a patch-scale model (PSM) based on the small-scale structure of aurora is designed to identify the type-specific regions for each training image. Second, a region-scale model is trained with the identified type-specific regions to coarsely localize the KLS from multiple sizes of field of view, based on which the aurora image is classified. Finally, given the predicted image type, the PSM further refines the KLS in a pixel level. By localizing KLS from coarse to fine, the proposed method captures both overall shape with a bottom–up processing and local structure details of aurora in a top–down manner. Extensive experiments on the expert labeled data sets have demonstrated the efficacy of the proposed method in benchmarking with the state-of-the-art WSSS methods. [ABSTRACT FROM AUTHOR]

Selfie photography from the hand-held camera is becoming a popular media type. Although being convenient and flexible, it suffers from low camera motion stability, small field of view, and limited background content. These limitations can annoy users, especially, when touring a place of interest and taking selfie videos. In this paper, we present a novel method to create what we call a BiggerSelfie that deals with these shortcomings. Using a video of the environment that has partial content overlap with the selfie video, we stitch plausible frames selected from the environment video to the original selfie frames and stabilize the composed video content with a portrait-preserving constraint. Using the proposed method, one can easily obtain a stable selfie video with expanded background content by merely capturing some background shots. We show various results and several evaluations to demonstrate the applicability of our method. [ABSTRACT FROM AUTHOR]

This special issue consists of 12 research papers. The papers cover a wide range of important topics including novel imaging, computational modeling, automatic vertebra segmentation, as well as computer assisted intervention. Among the 12 papers, nine are related to medical image analysis and three are in the scope of computer assisted intervention. The papers cover a variety of imaging modalities, include CT, MR, X-ray, ), ultrasound, and multi-modality. [ABSTRACT FROM AUTHOR]

This paper focuses on the thorax disease classification problem in chest X-ray (CXR) images. Different from the generic image classification task, a robust and stable CXR image analysis system should consider the unique characteristics of CXR images. Particularly, it should be able to: 1) automatically focus on the disease-critical regions, which usually are of small sizes; 2) adaptively capture the intrinsic relationships among different disease features and utilize them to boost the multi-label disease recognition rates jointly. In this paper, we propose to learn discriminative features with a two-branch architecture, named ConsultNet, to achieve those two purposes simultaneously. ConsultNet consists of two components. First, an information bottleneck constrained feature selector extracts critical disease-specific features according to the feature importance. Second, a spatial-and-channel encoding based feature integrator enhances the latent semantic dependencies in the feature space. ConsultNet fuses these discriminative features to improve the performance of thorax disease classification in CXRs. Experiments conducted on the ChestX-ray14 and CheXpert dataset demonstrate the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

In this paper, we develop new techniques for monitoring image processes under a fairly general setting with spatially correlated pixels in the image. Monitoring and handling the pixels directly is infeasible due to an extremely high image resolution. To overcome this problem, we suggest control charts that are based on regions of interest. The regions of interest cover the original image which leads to a dimension reduction. Nevertheless, the data are still high-dimensional. We consider residual charts based on the generalized likelihood ratio approach. Existing control statistics typically depend on the inverse of the covariance matrix of the process, involving high computing times and frequently generating instable results in a high-dimensional setting. As a solution of this issue, we suggest two further control charts that can be regarded as modifications of the generalized likelihood ratio statistic. Within an extensive simulation study, we compare the newly proposed control charts using the median run length as a performance criterion. [ABSTRACT FROM AUTHOR]

Fine-grained image analysis (FGIA) is a longstanding and fundamental problem in computer vision and pattern recognition, and underpins a diverse set of real-world applications. The task of FGIA targets analyzing visual objects from subordinate categories, e.g., species of birds or models of cars. The small inter-class and large intra-class variation inherent to fine-grained image analysis makes it a challenging problem. Capitalizing on advances in deep learning, in recent years we have witnessed remarkable progress in deep learning powered FGIA. In this paper we present a systematic survey of these advances, where we attempt to re-define and broaden the field of FGIA by consolidating two fundamental fine-grained research areas – fine-grained image recognition and fine-grained image retrieval. In addition, we also review other key issues of FGIA, such as publicly available benchmark datasets and related domain-specific applications. We conclude by highlighting several research directions and open problems which need further exploration from the community. [ABSTRACT FROM AUTHOR]

Medical image segmentation is fundamental and essential for the analysis of medical images. Although prevalent success has been achieved by convolutional neural networks (CNN), challenges are encountered in the domain of medical image analysis by two aspects: 1) lack of discriminative features to handle similar textures of distinct structures and 2) lack of selective features for potential blurred boundaries in medical images. In this paper, we extend the concept of contrastive learning (CL) to the segmentation task to learn more discriminative representation. Specifically, we propose a novel patch-dragsaw contrastive regularization (PDCR) to perform patch-level tugging and repulsing. In addition, a new structure, namely uncertainty-aware feature re- weighting block (UAFR), is designed to address the potential high uncertainty regions in the feature maps and serves as a better feature re- weighting. Our proposed method achieves state-of-the-art results across 8 public datasets from 6 domains. Besides, the method also demonstrates robustness in the limited-data scenario. The code is publicly available at https://github.com/lzh19961031/PDCR_UAFR-MIShttps://github.com/lzh19961031/PDCR_UAFR-MIS. [ABSTRACT FROM AUTHOR]

Representation learning is the critical task for medical image analysis in computer-aided diagnosis. However, it is challenging to learn discriminative features due to the limited size of the dataset and the lack of labels. In this paper, we propose a stochastic routing normalization and neighborhood embedding framework with application to breast tissue classification by learning discriminative features of probe-based confocal laser endomicroscopy. In order to align the low-level and mid-level of pCLE and histology domain, we firstly build the domain-specific normalization module with stochastic activation strategy considering both depth-wise and feature-wise criterion. For high-level features, the latent centers are learned from the histology domain as the template for feature matching. The proposed method is evaluated on a clinical database with 700 pCLE mosaics. The accuracy of image classification with limited training samples demonstrates that the proposed method can outperform previous works on domain alignment. [ABSTRACT FROM AUTHOR]

Radar backscatter variations that occur because of incidence angle effects constrain the application of Scanning Synthetic Aperture Radar (ScanSAR) data for sea ice monitoring and observations. In this paper, a class-based correction is proposed for normalizing each class in ScanSAR data to a nominal incidence angle. Two tested sea ice synthetic aperture radar (SAR) data sets were acquired: a data set for the Gulf of Saint Lawrence, which was obtained by the RADARSAT-2 satellite, and a data set for the Bohai Sea, which was obtained by the ENVISAT Advanced Synthetic Aperture Radar. An unsupervised classification is performed on each image block prior to normalization, and the incidence angle range of each image block is approximately 5°. Because the distribution of the backscatter coefficients in the azimuth band is discrete and nonlinear, the class-based locally linear mapping (LLM) technique is implemented, based on the assumption that a small quantity of sorted backscatter coefficients is locally linear. This algorithm is a transplantable and easily applied method that requires limited ground data, and it is also a semiautomated technique because nearly all of its parameters can be adaptively determined during the image analysis. The results demonstrate that LLM-corrected ScanSAR images appear to have more detailed textures, and the natural signal variability in the radar data is preserved, which indicates that the LLM produces better results compared with the histogram-based-alike (HIST-alike) technique when correcting the incidence angle in the sea ice SAR data. The results of the data analysis in this paper show that the width of the azimuth band should be selected based on the extent of variation in the incidence angle, and the reference band can be calculated based on the maximum interclass distance principle. The intercomparisons also reveal that the proposed algorithm can improve the accuracy of supervised classifications. [ABSTRACT FROM AUTHOR]

Recent classification-oriented proposals to thematic maps building from hyperspectral images have used both semisupervised approaches and spatial information for correction of spectral classification. Semisupervised approaches enrich the training data set adding similar samples to each class, whereas spatial correction is based on the natural assumption of thematic class spatial compactness. In this paper, we propose and validate the following innovations: 1) a new spectral classifier, which is called anticipative hybrid extreme rotation forest (AHERF); 2) a spatial–spectral semisupervised approach; and 3) a final spatial classification correction step. The novel heterogeneous ensemble learning approach AHERF starts with a model selection phase, using a small subsample of the training data, in order to define a ranking-based selection probability distribution of the classifier architectures that will be used in the ensemble, so that the architecture best adapted to the data domain will be used more frequently to train individual classifiers in the ensemble. After this initial phase, AHERF trains a heterogeneous ensemble applying random rotations to bootstrapped samples of the remaining training data, aiming to obtain diversified and data-domain adapted individual classifiers. The natural assumption that spatially close pixels will most likely have highly correlated values is exploited in two phases of the process pipeline. First, semisupervised label assignment is supported by spectral similarity and spatial proximity. Unsupervised spectral similarity is detected by latent class discovery. In this paper, we use a clustering algorithm (i.e., k-means). Second, maximizing class spatial compactness removes classification errors that appear as speckle noise in the classification image. The whole approach aims to use minimal sets of labeled pixels for training, which we call the seed training data set. Testing results are computed over the entire image ground truth. For comparison, we provide results in several steps: 1) of classification by AHERF and competing classifiers built by semisupervised training and 2) after spatial correction. We validate the approach on several conventional benchmarking images, achieving results which are comparable with state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

This paper is dedicated to the development of a thermal monitoring system for microelectronics based on the attentive vision approach as applied to image sequence analysis using an infrared camera. The attentive vision method implements multiscale image sequence analysis by a spatiotemporal attention operator to detect feature points, which are located inside potential thermal stress regions. The attention operator is a linear aggregation of temporal change and spatial saliency filters. The monitoring process is organized in two hierarchical phases: 1) peripheral and 2) focal. The focal monitoring is mostly carried out through the tracking of stress-relevant feature-point areas and analysis of their spatiotemporal descriptors. The thermal monitoring experiments conducted with wafer-scale integrated circuits have confirmed the reliability of the proposed approach and showed its high potential in image sequence analysis for monitoring purposes. [ABSTRACT FROM PUBLISHER]

Since the SEASAT mission in 1978, satellite-based synthetic aperture radar (SAR) images have been used to study oceanic internal waves. Internal waves become visible in SAR images because their orbital currents modulate the surface roughness. While this leads to an accurate spatial representation of internal wave patterns, the complexity of the imaging mechanism makes it difficult to derive actual currents and internal wave amplitudes from SAR signatures. We demonstrate in this paper how a more robust parameter retrieval is possible with along-track interferometric SAR (InSAR) data that resolve amplitudes and temporal phase changes of the backscattered signal together, the latter of which are directly related to the scatterers' line-of-sight velocities. Our example data set, which was acquired by TerraSAR-X in Dual Receive Antenna mode at Dongsha (South China Sea), exhibits strong signatures of internal waves in the interferogram amplitude and phase. We use a simple internal soliton parameterization and a numerical radar imaging model to find a plausible combination of internal wave parameters, which leads to good agreement between simulated and observed signatures. Testing the sensitivity of radar amplitude and phase signatures to various parameters, we show that along-track InSAR data should generally permit more accurate and less ambiguous internal wave parameter retrievals than conventional SAR images. [ABSTRACT FROM PUBLISHER]

Anemia occurs due to low blood hemoglobin levels, and for its mass screening, both invasive and smart anemia-care techniques are used. However, significant drawbacks include high costs, lack of state-of-the-art facilities, invasive techniques, and lack of smartphone implementation, using additional equipment and non-autonomous functioning, for smart anemia-care techniques. The paper proposes a novel, autonomous, smart anemia-care technique to address these problems in the form of a spectroscopy based blood hemoglobin level monitoring model. This approach uses a smartphone camera as a sensor. It quantifies the hemoglobin level based on the region of interest’s color spectroscopy, which is the conjunctival pallor, extracted autonomously. Anemia is diagnosed if the predicted hemoglobin level is < 11.5 g dL−1. For easy smartphone implementation, the proposed model uses image analysis techniques to estimate the hemoglobin level. Our model reports an accuracy of ±0.32 g dL−1, sensitivity of 89% compared to the actual blood hemoglobin levels (n = 65 participants). Also, the model remains robust to a wide range of illumination and the type of device used. It thereby establishes itself as a reliable and suitable replacement for the blood-based laboratory hemoglobin tests by leveraging the feature of at-home diagnosis of anemia. [ABSTRACT FROM AUTHOR]

In this paper, we focus on improving the proposal classification stage in the object detection task and present implicit negative sub-categorization and sink diversion to lift the performance by strengthening loss function in this stage. First, based on the observation that the “background” class is generally very diverse and thus challenging to be handled as a single indiscriminative class in existing state-of-the-art methods, we propose to divide the background category into multiple implicit sub-categories to explicitly differentiate diverse patterns within it. Second, since the ground truth class inevitably has low-value probability scores for certain images, we propose to add a “sink” class and divert the probabilities of wrong classes to this class when necessary, such that the ground truth label will still have a higher probability than other wrong classes even though it has low probability output. Additionally, we propose to use dilated convolution, which is widely used in the semantic segmentation task, for efficient and valuable context information extraction. Extensive experiments on PASCAL VOC 2007 and 2012 data sets show that our proposed methods based on faster R-CNN implementation can achieve state-of-the-art mAPs, i.e., 84.1%, 82.6%, respectively, and obtain 2.5% improvement on ILSVRC DET compared with that of ResNet. [ABSTRACT FROM PUBLISHER]

This paper provides an optimal high-performance image processing algorithm for a miniaturized independent optical navigation sensor, which combines the functions of a star tracker and a navigation camera. This novel image processing algorithm is capable of extracting two different types of optical navigation measurements from a raw image. The aim is to simultaneously extract stars and target celestial body features with high accuracy and reliability to estimate observer-to-body relative position in subsequent navigation process. This paper presents star and celestial body imaging models and a novel slope edge model. We propose an high-performance algorithm to achieve the synchronous extraction of star and celestial body image features based on the aforementioned models. Double-window variance difference method is proposed to segment and classify stars and edge image regions of a celestial body with strong robustness. The sub-pixel level position of star centroid and celestial body edges are then simultaneously extracted by using the same operator on the basis of the consistency of the derivative distribution of star and celestial body edge profiles. The edge extraction deviation when using the slope edge model is also analyzed and compensated, and the accuracy of the celestial body edge extraction is improved to a higher level. The proposed algorithm has excellent feature extraction performance in terms of qualitative and quantitative measurements. This paper has established a technical foundation for the development of the miniaturized independent optical navigation sensor, which is low cost, light weight and has flexible applicability due to its high accuracy and robustness. [ABSTRACT FROM PUBLISHER]

In this paper, two metrics for measuring image sharpness are presented and used for an autofocus (AF) application. Both measures exploit reorganized discrete cosine transform (DCT) representation. The first metric is a focus measure, which involves optimal high- and middle-frequency coefficients to evaluate relative sharpness. It is robust to noise while remaining sensitive to the best focus position. A psychometric function-based metric is introduced to quantify the focus measure. The second metric is a no-reference blurriness metric, which is used to measure absolute blurriness. It first constructs multiscale DCT edge maps using directional energy information and then determines image blurriness by combining change information in edge structures with image contrast. This metric gives predictions that are closely correlated with subjective perceived scores and shows performance comparable with that of state-of-the-art methods, especially for noisy images. For noisy situations, the two metrics are adjusted adaptively according to the estimated noise level. To prevent the introduction of extra computational load, an efficient noise-level estimation algorithm based on median absolute deviation is presented. This algorithm exploits only the available reorganized DCT coefficients. With the focus and blurriness measures, an AF method for which the two metrics play an important role was developed. Because of their high-quality performance, the realized AF function is able to locate the best focus position swiftly and reliably. [ABSTRACT FROM PUBLISHER]

Surface dielectric barrier discharge (DBD) has recently received growing interest for enormous application potential in various fields. In this paper, the optical observation of surface DBD is conducted by an intensified charge-coupled device camera from top view in the atmospheric air. Spacial and transient characteristics of discharge phenomenon in atmospheric pressure air have been, respectively, visualized with the exposure time of 100 ms and 5 μs. The discharge area is obtained by calculating the number of pixels in MATLAB software at different electrical parameters and different high-voltage electrode configurations with microsecond time scale. Experimental measurements show that the diffuse discharge during the negative-half cycle has good uniformity and stability compared with filamentary discharge during the positive-half cycle. The results offer a new estimated method for the discharge area. [ABSTRACT FROM PUBLISHER]

Ophthalmologists have used fundus images to screen and diagnose eye diseases. However, different equipments and ophthalmologists pose large variations to the quality of fundus images. Low-quality (LQ) degraded fundus images easily lead to uncertainty in clinical screening and generally increase the risk of misdiagnosis. Thus, real fundus image restoration is worth studying. Unfortunately, real clinical benchmark has not been explored for this task so far. In this paper, we investigate the real clinical fundus image restoration problem. Firstly, We establish a clinical dataset, Real Fundus (RF), including 120 low- and high-quality (HQ) image pairs. Then we propose a novel Transformer-based Generative Adversarial Network (RFormer) to restore the real degradation of clinical fundus images. The key component in our network is the Window-based Self-Attention Block (WSAB) which captures non-local self-similarity and long-range dependencies. To produce more visually pleasant results, a Transformer-based discriminator is introduced. Extensive experiments on our clinical benchmark show that the proposed RFormer significantly outperforms the state-of-the-art (SOTA) methods. In addition, experiments of downstream tasks such as vessel segmentation and optic disc/cup detection demonstrate that our proposed RFormer benefits clinical fundus image analysis and applications. [ABSTRACT FROM AUTHOR]

This paper reports the outcomes of the 2014 Data Fusion Contest organized by the Image Analysis and Data Fusion Technical Committee (IADF TC) of the IEEE Geoscience and Remote Sensing Society (IEEE GRSS). As for previous years, the IADF TC organized a data fusion contest aiming at fostering new ideas and solutions for multisource remote sensing studies. In the 2014 edition, participants considered multiresolution and multisensor fusion between optical data acquired at 20-cm resolution and long-wave (thermal) infrared hyperspectral data at 1-m resolution. The Contest was proposed as a double-track competition: one aiming at accurate landcover classification and the other seeking innovation in the fusion of thermal hyperspectral and color data. In this paper, the results obtained by the winners of both tracks are presented and discussed. [ABSTRACT FROM PUBLISHER]

In this paper, a 2-D noncausal Markov model is proposed for passive digital image-splicing detection. Different from the traditional Markov model, the proposed approach models an image as a 2-D noncausal signal and captures the underlying dependencies between the current node and its neighbors. The model parameters are treated as the discriminative features to differentiate the spliced images from the natural ones. We apply the model in the block discrete cosine transformation domain and the discrete Meyer wavelet transform domain, and the cross-domain features are treated as the final discriminative features for classification. The support vector machine which is the most popular classifier used in the image-splicing detection is exploited in our paper for classification. To evaluate the performance of the proposed method, all the experiments are conducted on public image-splicing detection evaluation data sets, and the experimental results have shown that the proposed approach outperforms some state-of-the-art methods. [ABSTRACT FROM PUBLISHER]

The papers in this special section focus on generative adversarial networks in biomedical image computing. The field of biomedical imaging has obtained great progress from Roentgen's original discovery of the X-ray to the current imaging tools, including Magnetic Resonance Imaging (MRI), Positron Emission Tomography (PET), Computed Tomography (CT), and Ultrasound (US). The benefits of using these non-invasive imaging technologies are to assess the current condition of an organ or tissue, which can be used to monitor a patient over time over time for accurate and timely diagnosis and treatment.With the development of imaging technologies, developing advanced artificial intelligence algorithms for automated image analysis has shown the potential to change many aspects of clinical applications within the next decade. Meanwhile, these advanced technologies have also brought new issues and challenges. Thus, there has been a growing demand for biomedical imaging computing to be a component of clinical trials and device improvement. Currently, Generative adversarial networks (GANs) have been attached growing interests in the computer vision community due to their capability of data generation or translation. GAN-based models are able to learn from a set of training data and generate new data with the same characteristics as the training ones, which have also proven to be the state of the art for generating sharp and realistic images. More importantly, GAN has been rapidly applied to many traditional and novel applications in the medical domain, such as image reconstruction, segmentation, diagnosis, synthesis, and so on. Despite GAN substantial progress in these areas, their application to medical image computing still faces challenges and unsolved problems remain. [ABSTRACT FROM AUTHOR]

Based on the analysis and research of potential motion damage attitude judgment methods at present, a method of motion damage attitude acquisition based on three-dimensional image analysis is proposed. In this paper, the three-dimensional image analysis method is introduced to locate the three-dimensional motion damage. The three-dimensional simulation calculation process is used to calibrate the motion limitation, which can effectively avoid the selection of process amount and change amount, and finally realize the information acquisition of the motion damage attitude. By detecting potential motion damage and extracting motion details of athletes in long-term motion environment, a three-dimensional image acquisition and recognition matrix module is established to accurately collect potential motion damage posture through motion categories and parameters. The research shows that this method can extract the details of motion injuries and recognize the three-dimensional sensor tracking. It can express the features of motion injuries better, effectively improve the prevention ability of motion injuries, and provide a guarantee for the safety and health of sports personnel. [ABSTRACT FROM AUTHOR]