Search

Water mapping for satellite imagery has been an active research field for many applications, in particular natural disasters such as floods. Synthetic Aperture Radar (SAR) provides high-resolution imagery without constraints on weather conditions. The single-date SAR approach is less accurate than the multi-temporal approach but can produce results more promptly. This paper proposes novel segmentation schemes that are designed to process both a target superpixel and its surrounding ones for the input for machine learning. Mixture-based Superpixel-Shallow Deit-Ti/XGBoost (MISP-SDT/XGB) schemes are devised to generate, annotate, and classify superpixels, and perform the land/water segmentation of SAR imagery. These schemes are applied to Sentinel-1 SAR data to examine segmentation performances. Single/mask/neighborhood models and single/neighborhood models are introduced in the MISP-SDT scheme and the MISP-XGB scheme, respectively. The effects of the contextual information about the target and its neighbor superpixels are assessed on its segmentation performances. Regarding polarization, it is shown that the VH mode produces more encouraging results than the VV, which is consistent with previous studies. Also, under our MISP-SDT/XGP schemes, the neighborhood models show better performances than FCNN models. Overall, the neighborhood model gives better performances than the single model. Results from attention maps and feature importance scores show that neighbor regions are looked at or used by the algorithms in the neighborhood models. Our findings suggest that under our schemes, the contextual information has positive effects on land/water segmentation. [ABSTRACT FROM AUTHOR]

Automatically segmenting the lesion of clinical data can aid doctors in diagnosis. The key issues with clinical brain tumor segmentation are partial volume effect, bias field, and noise interference. This paper proposes a novel segmentation algorithm based on pathology topological merging to solve the above problems. Here, an improved superpixel technique is used to group the pathology topological blocks, and the vector distance is refined to avoiding the problem of grouping pixels with small similarity near the tumor contour into the same region. Furthermore, meaningful pathology topological blocks are formed, and the entire brain tumor is segmented based on the pathology topological relationship and weight between pathology topological blocks. The proposed method is validated on the BraTS 2015 dataset and 123 patient images with brain tumors from a local hospital, and the mean Dice, Jaccard, Precision, and Recall values are 0.91, 0.92, 0.90, and 0.91, respectively, indicating that the proposed method can efficiently and accurately distinguish brain tumors from other tissues (such as edema). The method can overcome some defects (e.g. partial volume effect, bias field, and noise interference) in medical brain images while having practical clinical application prospects. [ABSTRACT FROM AUTHOR]

New algorithms for merging superpixel regions into segments are proposed. The main idea of merging superpixels is as follows. Firstly, a strategy is used in which a segment is grown from neighboring regions while the conditions for merging are met, and, secondly, when merging regions, the applied information quality measure should not increase. Three algorithms based on the specified strategy are proposed, which differ in the conditions for making a decision on merging superpixels. A computational experiment is carried out on test images. The experiment showed that the proposed algorithms accelerate the segmentation process compared to the procedure used earlier with acceptable losses of information quality measures of the resulting partitions. [ABSTRACT FROM AUTHOR]

Supervised polarimetric synthetic aperture radar (PolSAR) image classification demands a large amount of precisely labeled data. However, such data are difficult to obtain. Therefore, many unsupervised methods have been proposed for unsupervised PolSAR image classification. The classification maps of unsupervised methods contain many high-confidence samples. These samples, which are often ignored, can be used as supervisory information to improve classification performance on PolSAR images. This study proposes a new unsupervised PolSAR image classification framework. The framework combines high-confidence superpixel pseudo-labeled samples and semi-supervised classification methods. The experiments indicated that this framework could achieve higher-level effectiveness in unsupervised PolSAR image classification. First, superpixel segmentation was performed on PolSAR images, and the geometric centers of the superpixels were generated. Second, the classification maps of rotation-domain deep mutual information (RDDMI), an unsupervised PolSAR image classification method, were used as the pseudo-labels of the central points of the superpixels. Finally, the unlabeled samples and the high-confidence pseudo-labeled samples were used to train an excellent semi-supervised method, similarity matching (SimMatch). Experiments on three real PolSAR datasets illustrated that, compared with the excellent RDDMI, the accuracy of the proposed method was increased by 1.70%, 0.99%, and 0.8%. The proposed framework provides significant performance improvements and is an efficient method for improving unsupervised PolSAR image classification. [ABSTRACT FROM AUTHOR]

In recent years, several superpixel-segmentation methods have been developed to efficiently segment noisy images. However, these methods still face challenges such as high computational complexity and poor adaptability. Therefore, this paper develops a novel superpixel-based robust segmentation model including two modules: superpixel generation and superpixel-based image segmentation. In the superpixel generation module, a fuzzy factor containing local spatial information of pixels is introduced into fuzzy possibilistic clustering algorithm with local search. In the superpixel-based segmentation module, a superpixel-based fuzzy C-means algorithm with local spatial information of superpixels is proposed, which nonlinearly combines the membership of superpixels with the membership of their neighboring superpixels. The experimental results demonstrate that the proposed method outperforms existing state-of-the-art segmentation algorithms in terms of evaluation indexes and visual effects. [ABSTRACT FROM AUTHOR]

Graph convolution subspace clustering has been widely used in the field of hyperspectral image (HSI) unsupervised classification due to its ability to aggregate neighborhood information. However, existing methods focus on using graph convolution techniques to design feature extraction functions, ignoring the mutual optimization of the graph convolution operator and the self-expression coefficient matrix, leading to suboptimal clustering results. In addition, these methods directly construct graphs on raw data, which may be easily affected by noises and then degrade the clustering performance, as the constructed topology is not credible for the training procedure. To address these issues, we propose a novel method called spatial-spectral adaptive graph convolutional subspace clustering (S2AGCSC). We employ the reconstruction coefficient matrix to devise a graph convolutional operator with adjacency matrix, which collaboratively computes both the feature representations and coefficient matrix, and the graph-convolutional operator is updated iteratively and adaptively during training. In addition, we harness a combination of spectral and spatial features to introduce additional view information to help learn more robust features and generate more refined superpixels. Experimental validation on three HSI datasets confirms the efficacy of S2AGCSC.

Abstract Background Crop phenotype extraction devices based on multiband narrowband spectral images can effectively detect the physiological and biochemical parameters of crops, which plays a positive role in guiding the development of precision agriculture. Although the narrowband spectral image canopy extraction method is a fundamental algorithm for the development of crop phenotype extraction devices, developing a highly real-time and embedded integrated narrowband spectral image canopy extraction method remains challenging owing to the small difference between the narrowband spectral image canopy and background. Methods This study identified and validated the skewed distribution of leaf color gradation in narrowband spectral images. By introducing kurtosis and skewness feature parameters, a canopy extraction method based on a superpixel skewed color gradation distribution was proposed for narrowband spectral images. In addition, different types of parameter combinations were input to construct two classifier models, and the contribution of the skewed distribution feature parameters to the proposed canopy extraction method was evaluated to confirm the effectiveness of introducing skewed leaf color skewed distribution features. Results Leaf color gradient skewness verification was conducted on 4200 superpixels of different sizes, and 4190 superpixels conformed to the skewness distribution. The intersection over union (IoU) between the soil background and canopy of the expanded leaf color skewed distribution feature parameters was 90.41%, whereas that of the traditional Otsu segmentation algorithm was 77.95%. The canopy extraction method used in this study performed significantly better than the traditional threshold segmentation method, using the same training set, Y1 (without skewed parameters) and Y2 (with skewed parameters) Bayesian classifier models were constructed. After evaluating the segmentation effect of introducing skewed parameters, the average classification accuracies Acc_Y1 of the Y1 model and Acc_Y2 of the Y2 model were 72.02% and 91.76%, respectively, under the same test conditions. This indicates that introducing leaf color gradient skewed parameters can significantly improve the accuracy of Bayesian classifiers for narrowband spectral images of the canopy and soil background. Conclusions The introduction of kurtosis and skewness as leaf color skewness feature parameters can expand the expression of leaf color information in narrowband spectral images. The narrowband spectral image canopy extraction method based on superpixel color skewness distribution features can effectively segment the canopy and soil background in narrowband spectral images, thereby providing a new solution for crop canopy phenotype feature extraction.

The multi-scale representation of remote sensing images is crucial for information extraction, data analysis, and image processing. However, traditional methods such as image pyramid and image filtering often result in the loss of image details, particularly edge information, during the simplification and merging processes at different scales and resolutions. Furthermore, when applied to coastal landforms with rich texture features, such as biologically diverse areas covered with vegetation, these methods struggle to preserve the original texture characteristics. In this study, we propose a new method, multi-scale expression of coastal landforms considering texture features (METF-C), based on computer vision techniques. This method combines superpixel segmentation and texture transfer technology to improve the multi-scale representation of coastal landforms in remote sensing images. First, coastal landform elements are segmented using superpixel technology. Then, global merging is performed by selecting different classes of superpixels, with boundaries smoothed using median filtering and morphological operators. Finally, texture transfer is applied to create a fusion image that maintains both scale and level consistency. Experimental results demonstrate that METF-C outperforms traditional methods by effectively simplifying images while preserving important geomorphic features and maintaining global texture information across multiple scales. This approach offers significant improvements in edge preservation and texture retention, making it a valuable tool for analyzing coastal landforms in remote sensing imagery. [ABSTRACT FROM AUTHOR]

Extracting meaningful structural edges from complex texture images presents a significant challenge. Accurately measuring and differentiating texture information within an image are crucial for efficient texture filtering. While most existing texture filtering methods employ regular rectangular filter windows, the irregularity inherent in textures and structures can limit measurement accuracy, reducing the effectiveness of texture filtering. To address this problem, we propose an edge-aware texture filtering method that integrates superpixels. By employing patch shift, our filter constructs an edge-aware filtering region with superpixels constraint. This region includes pixels with minimal differences and similar texture characteristics. Utilizing the perceptual properties of superpixels for irregular edges enhances texture measurement, thereby improving the quality of texture filtering. Experimental results demonstrate that the proposed method outperforms existing techniques, yielding superior filtering outcomes. The source code is available at: https://github.com/kxZhang1016/EATFS. [ABSTRACT FROM AUTHOR]

Three-dimensional perception in intelligent virtual and augmented reality (VR/AR) and autonomous vehicles (AV) applications is critical and attracting significant attention. The self-supervised monocular depth and ego-motion estimation serves as a more intelligent learning approach that provides the required scene depth and location for 3D perception. However, the existing self-supervised learning methods suffer from scale ambiguity, boundary blur, and imbalanced depth distribution, limiting the practical applications of VR/AR and AV. In this article, we propose a new self-supervised learning framework based on superpixel and normal constraints to address these problems. Specifically, we formulate a novel 3D edge structure consistency loss to alleviate the boundary blur of depth estimation. To address the scale ambiguity of estimated depth and ego-motion, we propose a novel surface normal network for efficient camera height estimation. The surface normal network is composed of a deep fusion module and a full-scale hierarchical feature aggregation module. Meanwhile, to realize the global smoothing and boundary discriminability of the predicted normal map, we introduce a novel fusion loss which is based on the consistency constraints of the normal in edge domains and superpixel regions. Experiments are conducted on several benchmarks, and the results illustrate that the proposed approach outperforms the state-of-the-art methods in depth, ego-motion, and surface normal estimation. [ABSTRACT FROM AUTHOR]

Visual odometry (VO) is a key technology for estimating camera motion from captured images. In this paper, we propose a novel RGB-D visual odometry by constructing and matching features at the superpixel level that represents better adaptability in different environments than state-of-the-art solutions. Superpixels are content-sensitive and perform well in information aggregation. They could thus characterize the complexity of the environment. Firstly, we designed the superpixel-based feature SegPatch and its corresponding 3D representation MapPatch. By using the neighboring information, SegPatch robustly represents its distinctiveness in various environments with different texture densities. Due to the inclusion of depth measurement, the MapPatch constructs the scene structurally. Then, the distance between SegPatches is defined to characterize the regional similarity. We use the graph search method in scale space for searching and matching. As a result, the accuracy and efficiency of matching process are improved. Additionally, we minimize the reprojection error between the matched SegPatches and estimate camera poses through all these correspondences. Our proposed VO is evaluated on the TUM dataset both quantitatively and qualitatively, showing good balance to adapt to the environment under different realistic conditions. [ABSTRACT FROM AUTHOR]

Superpixels, as essential mid-level image representations, have been widely used in computer vision due to their computational efficiency and redundant compression. Compared with traditional superpixel methods, superpixel algorithms based on deep learning frameworks demonstrate significant advantages in segmentation accuracy. However, existing deep learning-based superpixel algorithms suffer from a loss of details due to convolution and upsampling operations in their encoder–decoder structure, which weakens their semantic detection capabilities. To overcome these limitations, we propose a novel superpixel segmentation network based on a multi-attention hybrid network (MAS-Net). MAS-Net is still based on an efficient symmetric encoder–decoder architecture. First, utilizing residual structure based on a parameter-free attention module at the feature encoding stage enhanced the capture of fine-grained features. Second, adoption of a global semantic fusion self-attention module was used at the feature selection stage to reconstruct the feature map. Finally, fusing the channel with the spatial attention mechanism at the feature-decoding stage was undertaken to obtain superpixel segmentation results with enhanced boundary adherence. Experimental results on real-world image datasets demonstrated that the proposed method achieved competitive results in terms of visual quality and metrics, such as ASA and BR-BP, compared with the state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

IntroductionActive learning can significantly decrease the labeling cost of deep learning workflows by prioritizing the limited labeling budget to high-impact data points that have the highest positive impact on model accuracy. Active learning is especially useful for semantic segmentation tasks where we can selectively label only a few high-impact regions within these high-impact images. Most established regional active learning algorithms deploy a static-budget querying strategy where a fixed percentage of regions are queried in each image. A static budget could result in over- or under-labeling images as the number of high-impact regions in each image can vary.MethodsIn this paper, we present a novel dynamic-budget superpixel querying strategy that can query the optimal numbers of high-uncertainty superpixels in an image to improve the querying efficiency of regional active learning algorithms designed for semantic segmentation.ResultsFor two distinct datasets, we show that by allowing a dynamic budget for each image, the active learning algorithm is more effective compared to static-budget querying at the same low total labeling budget. We investigate both low- and high-budget scenarios and the impact of superpixel size on our dynamic active learning scheme. In a low-budget scenario, our dynamic-budget querying outperforms static-budget querying by 5.6% mIoU on a specialized agriculture field image dataset and 2.4% mIoU on Cityscapes.DiscussionThe presented dynamic-budget querying strategy is simple, effective, and can be easily adapted to other regional active learning algorithms to further improve the data efficiency of semantic segmentation tasks.

Foreground segmentation (FS) plays a fundamental and important role in computer vision, but it remains a challenging task in dynamic backgrounds. The supervised method has achieved good results, but the generalization ability needs to be improved. To address this challenge and improve the performance of FS in dynamic scenarios, a simple yet effective method has been proposed that leverages superpixel features and a one-dimensional convolution neural network (1D-CNN) named SPF-CNN. SPF-CNN involves several steps. First, the coined Iterated Robust CUR (IRCUR) is utilized to obtain candidate foregrounds for an image sequence. Simultaneously, the image sequence is segmented using simple linear iterative clustering. Next, the proposed feature extraction approach is applied to the candidate matrix region corresponding to the superpixel block. Finally, the 1D-CNN is trained using the obtained superpixel features. Experimental results demonstrate the effectiveness of SPF-CNN, which also exhibits strong generalization capabilities. The average F1-score reaches 0.83.

Extracting water bodies from synthetic aperture radar (SAR) images plays a crucial role in the management of water resources, flood monitoring, and other applications. Recently, transformer-based models have been extensively utilized in the remote sensing domain. However, due to regular patch-partition and weak inductive bias, transformer-based models face challenges such as edge serration and high data dependency when used for water body extraction from SAR images. To address these challenges, we introduce a new model, the Superpixel-based Transformer (SPT), based on the adaptive characteristic of superpixels and knowledge constraints of the adjacency matrix. (1) To mitigate edge serration, the SPT replaces regular patch partition with superpixel segmentation to fully utilize the internal homogeneity of superpixels. (2) To reduce data dependency, the SPT incorporates a normalized adjacency matrix between superpixels into the Multi-Layer Perceptron (MLP) to impose knowledge constraints. (3) Additionally, to integrate superpixel-level learning from the SPT with pixel-level learning from the CNN, we combine these two deep networks to form SPT-UNet for water body extraction. The results show that our SPT-UNet is competitive compared with other state-of-the-art extraction models, both in terms of quantitative metrics and visual effects. [ABSTRACT FROM AUTHOR]

The most critical aspect of panorama generation is maintaining local semantic consistency. Objects may be projected from different depths in the captured image. When warping the image to a unified canvas, pixels at the semantic boundaries of the different views are significantly misaligned. We propose two lightweight strategies to address this challenge efficiently. First, the original image is segmented as superpixels rather than regular grids to preserve the structure of each cell. We propose effective cost functions to generate the warp matrix for each superpixel. The warp matrix varies progressively for smooth projection, which contributes to a more faithful reconstruction of object structures. Second, to deal with artifacts introduced by stitching, we use a seam line method tailored to superpixels. The algorithm takes into account the feature similarity of neighborhood superpixels, including color difference, structure and entropy. We also consider the semantic information to avoid semantic misalignment. The optimal solution constrained by the cost functions is obtained under a graph model. The resulting stitched images exhibit improved naturalness. Extensive testing on common panorama stitching datasets is performed on the algorithm. Experimental results show that the proposed algorithm effectively mitigates artifacts, preserves the completeness of semantics and produces panoramic images with a subjective quality that is superior to that of alternative methods. [ABSTRACT FROM AUTHOR]

Superpixel-based segmentation has been widely used as a primary prepossessing step to simplify the subsequent image processing tasks. Since determining the number of clusters is subjective and varies based on the type of image, the segmentation algorithm may provide over-segmented or under-segmented superpixels. This paper proposes an image segmentation method to improve the SLIC superpixel by region merging. It aims to improve the segmentation accuracy without defining a precise number of superls. The color difference between superpixels is employed as a homogeneity criterion for the merging process. The Berkeley dataset is used with different quantitative performance metrics to evaluate the proposed model's performance. Results obtained from probabilistic rand index (PRI), boundary recall, and under-segmentation error proved the ability of the proposed algorithm to provide comparable segmentation with a reduced number of clusters. [ABSTRACT FROM AUTHOR]

This study used a watershed algorithm to detect canopy cover in dryland forests. The objective of this study was to determine the best parameters of the watershed segmentation algorithm to obtain information on crown closure from filtered and unfiltered high- and very-high-resolution images. The best performance of each parameter combination of the tolerance value (T), mean value (M), and variance value (V), which is written as C:[T]-[M]-[V], is determined based on the level of accuracy. This study used Pleiades-1B and SPOT-6 images as primary digital data. The results showed that the low-pass filtered Pleiades-1B image showed the best performance with a combination of parameters C6-MF:[10]-[0.7]- [0.3], with an overall accuracy (OA) of 91.0% and an accuracy Kappa (KA) of 83.2%. The low-pass filtered Spot-6 image shows a combination of parameters C7-MF:[10]-[0.8]-[0.2], which has an accuracy of 90.6% OA and 65.4% KA. This study concludes that the filtered image with a low-pass filter always yields more accurate results than the original data (without a filter) for both Pleiades-1B and SPOT-6 images. The very high spatial resolution provides better accuracy than the high spatial resolution. [ABSTRACT FROM AUTHOR]

Abstract Deep neural networks combined with superpixel segmentation have proven to be superior to high-resolution remote sensing image (HRI) classification. Currently, most HRI classification methods that combine deep learning and superpixel segmentation use stacking on multiple scales to extract contextual information from segmented objects. However, this approach does not take into account the contextual dependencies between each segmented object. To solve this problem, a joint superpixel and Transformer (JST) framework is proposed for HRI classification. In JST, HRI is first segmented into superpixel objects as input, and Transformer is used to model the long-range dependencies. The contextual relationship between each input superpixel object is obtained and the class of analyzed objects is output by designing an encoding and decoding Transformer. Additionally, we explore the effect of semantic range on classification accuracy. JST is also tested by using two HRI datasets with overall classification accuracy, average accuracy and Kappa coefficients of 0.79, 0.70, 0.78 and 0.91, 0.85, 0.89, respectively. The effectiveness of the proposed method is compared qualitatively and quantitatively, and the results achieve competitive and consistently better than the benchmark comparison method.

In recent years, many superpixel-based image segmentation algorithms have been presented. However, most of these algorithms face issues of high model complexity and weak robustness. There are two main reasons for this. On the one hand, the traditional superpixel-based image segmentation algorithm constructs two different models to complete the tasks of superpixel generation and superpixel-based image segmentation, which significantly increase the complexity of the algorithm. On the other hand, the segmentation results are largely affected by the characteristics of the generated superpixels, and many superpixel generation algorithms are sensitive to noise, resulting in poor robustness. Therefore, this paper proposes a novel superpixel-based fuzzy C-multiple-means clustering algorithm, which generates superpixels and segments superpixel image in one step to reduce the complexity of the algorithm. Meanwhile, the proposed algorithm introduces local spatial information of pixel, enhancing the robustness of superpixel generation to noise. In addition, the generated superpixels have centroid shift property, further improving the algorithm's detail-preservation ability and robustness. Experimental results show that this algorithm outperforms many state-of-the-art superpixel-related and unrelated fuzzy clustering algorithms in noisy image segmentation, and consumes less time when providing similar segmentation results. Overall, the work in this paper will greatly promote the development of superpixel-based image segmentation theory. [ABSTRACT FROM AUTHOR]

In the military field, camouflage assessment has significant study implications. In this research, a region segmentation algorithm based on superpixels and color clustering is proposed to address the shortcomings of the current methods, which solely focus on the human eye perception mechanism. This algorithm incorporates the attention process of human eyes and cognition. Compared to the rectangular regions used by the bulk of existing algorithms, the segmented regions with uneven borders have more visual meanings. The low-level perception feature set used in this study is also used to develop a novel camouflage assessment metric that takes into account both the irregularity of the attention perception region and the human eye's feature focus. Due to the difficulty in obtaining the camouflage dataset, this research creates the Green Tank dataset and the Green Car dataset using different targets and similar backgrounds. We ran tests on these two datasets and compared it with 16 other well-known algorithms to verify the efficacy of the strategy suggested in this study. The experimental findings demonstrate that the approach suggested in this research has produced the best outcome. [ABSTRACT FROM AUTHOR]

Nearshore ship detection has significant applications in both the military and civilian domains. Compared to synthetic aperture radar (SAR), polarimetric synthetic aperture radar (PolSAR) provides richer information for analyzing the scattering mechanisms of ships and enables better detection of ship targets. However, ships in nearshore areas tend to be highly concentrated, and ship detection is often affected by adjacent strong scattering, resulting in false alarms or missed detections. While the GP-PNF detector performs well in PolSAR ship detection, it cannot obtain satisfactory results in these scenarios, and it also struggles in the presence of azimuthal ambiguity or strong clutter interference. To address these challenges, we propose a nearshore ship detection method named ECD-PNF by integrating superpixel-level GP-PNF and refined polarimetric decomposition. Firstly, polarimetric superpixel segmentation and sea–land segmentation are performed to reduce the influence of land on ship detection. To estimate the sea clutter more accurately, an automatic censoring (AC) mechanism combined with superpixels is used to select the sea clutter superpixels. By utilizing refined eight-component polarimetric decomposition to improve the scattering vector, the physical interpretability of the detector is enhanced. Additionally, the expression of polarimetric coherence is improved to enhance the target clutter ratio (TCR). Finally, this paper combines the third eigenvalue of eigenvalue–eigenvector decomposition to reduce the impact of azimuthal ambiguity. Three spaceborne PolSAR datasets from Radarsat-2 and GF-3 are adopted in the experiments for comparison. The proposed ECD-PNF method achieves the highest figure of merit (FoM) value of 0.980, 1.000, and 1.000 for three datasets, validating the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

The natural terrain (e.g., farm and forest) in temperate zones changes dramatically between seasons due to distinct temperatures and precipitation variations from summer to winter. Moreover, built-up areas vary little in this short period. Therefore, extracting built-up areas via change detection on polarimetric synthetic aperture radar (PolSAR) images is feasible. A common type of PolSAR change detection method is based on hypothesis testing theory. However, in these methods, pixels are selected as the processing units; as a result, these models are computationally complex and poorly maintain the boundaries of built-up areas. In this paper, we innovatively introduce superpixels into the hypothesis test theory and propose a superpixelwise PolSAR change detection method for built-up area extraction. First, we oversegment the PolSAR images into a set of superpixels and derive the probability density function (PDF) of a superpixel's reflectivity on a PolSAR image. Based on this distribution, we present a superpixelwise likelihood-ratio test (LRT) statistic to measure the similarity of two superpixelwise covariance matrices for unsupervised change detection. When actually computing the superpixelwise LRT, the large variation in the areas of the superpixels makes the likelihood functions very complex and estimating the parameters difficult. We further simplify the calculation of the LRT statistic and apply it to built-up area extraction. Compared to the state-of-the-art built-up area extraction methods, our approach provides the best results with overall accuracy values of 91.41%, 92.71%, 93.67% and 93.91% for the four studied areas respectively. In addition, the computational complexity of our method is assessed, and the run time (seconds) of the proposed method in the four study cases is 11.14, 6.32, 5.31, and 4.88, respectively, superior to the values of 66.12, 36.26, 33.65, and 29.67 for DRT, respectively. Code and datasets: https://github.com/SunXJ7/Sentinel-1-Datasets-for-Built-up-Area-Extraction. [ABSTRACT FROM AUTHOR]

Deep neural networks combined with superpixel segmentation have proven to be superior to high-resolution remote sensing image (HRI) classification. Currently, most HRI classification methods that combine deep learning and superpixel segmentation use stacking on multiple scales to extract contextual information from segmented objects. However, this approach does not take into account the contextual dependencies between each segmented object. To solve this problem, a joint superpixel and Transformer (JST) framework is proposed for HRI classification. In JST, HRI is first segmented into superpixel objects as input, and Transformer is used to model the long-range dependencies. The contextual relationship between each input superpixel object is obtained and the class of analyzed objects is output by designing an encoding and decoding Transformer. Additionally, we explore the effect of semantic range on classification accuracy. JST is also tested by using two HRI datasets with overall classification accuracy, average accuracy and Kappa coefficients of 0.79, 0.70, 0.78 and 0.91, 0.85, 0.89, respectively. The effectiveness of the proposed method is compared qualitatively and quantitatively, and the results achieve competitive and consistently better than the benchmark comparison method. [ABSTRACT FROM AUTHOR]

Simple Summary: In this work, we explored computational methods for analyzing a color digital image of a wound and predicting (from the analyzed image) the number of days it will take for the wound to fully heal. We used a hybrid computational approach combining deep neural networks and decision trees, and within this hybrid approach, we explored (and compared the accuracies of) different types of models for predicting the time to heal. More specifically, we explored different models for finding the outline of the wound within the wound image and we proposed a model for computing the proportions of different types of tissues within the wound bed (e.g., fibrin slough, granulation, or necrotic tissue). Our work clarifies what type of model should be used for the computational prediction of wound time-to-healing and establishes that, in order to predict time-to-healing accurately, it is important to incorporate (into the model) data on the proportions of different types in the wound bed. Despite the societal burden of chronic wounds and despite advances in image processing, automated image-based prediction of wound prognosis is not yet in routine clinical practice. While specific tissue types are known to be positive or negative prognostic indicators, image-based wound healing prediction systems that have been demonstrated to date do not (1) use information about the proportions of tissue types within the wound and (2) predict time-to-healing (most predict categorical clinical labels). In this work, we analyzed a unique dataset of time-series images of healing wounds from a controlled study in dogs, as well as human wound images that are annotated for the tissue type composition. In the context of a hybrid-learning approach (neural network segmentation and decision tree regression) for the image-based prediction of time-to-healing, we tested whether explicitly incorporating tissue type-derived features into the model would improve the accuracy for time-to-healing prediction versus not including such features. We tested four deep convolutional encoder–decoder neural network models for wound image segmentation and identified, in the context of both original wound images and an augmented wound image-set, that a SegNet-type network trained on an augmented image set has best segmentation performance. Furthermore, using three different regression algorithms, we evaluated models for predicting wound time-to-healing using features extracted from the four best-performing segmentation models. We found that XGBoost regression using features that are (i) extracted from a SegNet-type network and (ii) reduced using principal components analysis performed the best for time-to-healing prediction. We demonstrated that a neural network model can classify the regions of a wound image as one of four tissue types, and demonstrated that adding features derived from the superpixel classifier improves the performance for healing-time prediction. [ABSTRACT FROM AUTHOR]

To reduce the reliance of existing mainstream vision-based crack detection algorithms on annotated datasets, this paper presents an unsupervised semantic segmentation-based approach. The proposed method first employs the Felzenszwalb-Huttenlocher algorithm for pre-segmenting the image, generating superpixels. Subsequently, a with an autoencoder model is designed to progressively approximate the superpixel segmentation results, and the optimal model is obtained by Bayesian optimization. Through comparative experiments with existing algorithms, it has been demonstrated that the proposed method performance comparable to supervised algorithms, even without the need for labeled data. As a result, the deployment complexity of the algorithm is significantly reduced, while expanding its applicability. [ABSTRACT FROM AUTHOR]

The progressive evolution of the spatial and temporal resolutions of Earth observation satellites has brought multiple benefits to scientific research. The increasing volume of data with higher frequencies and spatial resolutions offers precise and timely information, making it an invaluable tool for environmental analysis and enhanced decision-making. However, this presents a formidable challenge for large-scale environmental analyses and socioeconomic applications based on spatial time series, often compelling researchers to resort to lower-resolution imagery, which can introduce uncertainty and impact results. In response to this, our key contribution is a novel machine learning approach for dense geospatial time series rooted in superpixel segmentation, which serves as a preliminary step in mitigating the high dimensionality of data in large-scale applications. This approach, while effectively reducing dimensionality, preserves valuable information to the maximum extent, thereby substantially enhancing data accuracy and subsequent environmental analyses. This method was empirically applied within the context of a comprehensive case study encompassing the 2002–2022 period with 8-d-frequency-normalized difference vegetation index data at 250-m resolution in an area spanning 43,470 km2. The efficacy of this methodology was assessed through a comparative analysis, comparing our results with those derived from 1000-m-resolution satellite data and an existing superpixel algorithm for time series data. An evaluation of the time-series deviations revealed that using coarser-resolution pixels introduced an error that exceeded that of the proposed algorithm by 25 % and that the proposed methodology outperformed other algorithms by more than 9 %. Notably, this methodological innovation concurrently facilitates the aggregation of pixels sharing similar land-cover classifications, thus mitigating subpixel heterogeneity within the dataset. Further, the proposed methodology, which is used as a preprocessing step, improves the clustering of pixels according to their time series and can enhance large-scale environmental analyses across a wide range of applications.

Brain tumors are life-threatening and are typically identified by experts using imaging modalities like Magnetic Resonance Imaging (MRI), Computed Tomography (CT), and Positron Emission Tomography (PET). However, any error due to human intervention in brain anomaly detection can have devastating consequences. This study proposes a tumor detection algorithm for brain MRI images. Previous research into tumor detection has drawbacks, paving the way for further investigations. A visual attention-based technique for tumor detection is proposed to overcome these drawbacks. Brain tumors have a wide range of intensity, varying from inner matter-alike intensity to skull-alike intensity, making them difficult to threshold. Thus, a unique approach to threshold using entropy has been utilized. An on-center saliency map accurately captures the biological visual attention-focused tumorous region from the original image. Later, a superpixel-based framework has been proposed and used to capture the true structure of the tumor. Finally, it was experimentally shown that the proposed algorithm outperforms the existing algorithms for brain tumor detection.

Constant false alarm rate (CFAR) detector is a common method for ship detection in polarimetric synthetic aperture radar (PolSAR) images. CFAR detectors greatly depend on the clutter modeling that can be easily affected by the contamination caused by both lower- and higher-intensity outliers, such as spilled oil and intensive targets. Traditional CFAR detectors perform detection in a pixel-by-pixel manner, which ignores the spatial information. Both the bias in clutter modeling and the absence of spatial information can degrade the ship target detection performance. In this study, a superpixel-level polarimetric bilateral truncated statistics CFAR detector is proposed to promote the ship target detection performance in complex ocean scenarios. As the preprocessing of the PolSAR image, the superpixel segmentation is conducted based on the multilook polarimetric whitening filter result to select candidate ship target superpixels for bilateral truncation and background clutter modeling. The elliptical truncation is expanded to a complex situation and the relationship between the second moments before and after truncation is derived. The maximum-likelihood estimation estimator of the equivalent number of looks based on the bilateral truncation distribution is derived and compared with other parameter estimators. The influence of the truncation depth on estimator performance is analyzed, according to which the adaptive bilateral truncation method is determined. The Gaussian mixture model and the Parzen window kernel method are compared with the model-based method and utilized for data fitting. The proposed method performs bilateral truncation based on the superpixel segmentation result to provide pure clutter samples for accurate parameter estimation and clutter distribution modeling, reducing time consumption and false alarms. The method is validated efficient on both simulated and measured data from RADARSAT-2.

Open-set segmentation has caught the community’s attention only in the last few years, and it is a growing and active research area with many challenges ahead. To better identify open-set pixels, we address two known issues by improving data representation and ensuring semantic consistency in open-set predictions. First, we present a method called Open Gaussian Mixture of Models (OpenGMM) that allows for multimodal statistical distributions in known class pixels using a Gaussian Mixture of Models instead of unimodal approaches, like Principal Component Analysis. The second approach improved semantic consistency by applying a post-processing technique that uses superpixels to enforce homogeneous regions to have similar predictions, rectifying erroneously classified pixels within these regions and providing better delineation of object borders. We also developed a novel superpixel method called Fusing Superpixels for Improved Semantic Consistency (FuSC) that produced more homogeneous superpixels and enhanced, even more, the open-set segmentation prediction. We applied the proposed approaches to well-known remote sensing datasets with labeled ground truth for semantic segmentation tasks. The proposed methods improved the highest AUROC quantitative results for the International Society for Photogrammetry and Remote Sensing (ISPRS) Vaihingen and Potsdam datasets. Using FuSC, we achieved novel open-set state-of-the-art results for both datasets, improving AUROC results from 0.850 to 0.880 (3.53%) for Vaihingen and 0.764 to 0.797 (4.32%) for Potsdam datasets. The official implementation is available at: https://github.com/iannunes/FuSC.

Using object-based deep learning for the urban land cover classification has become a mainstream method. This study proposed an urban land cover classification method based on segments’ object features, deep features, and spatial association features. The proposed method used the synthetic semivariance function to determine the hyperparameters of the superpixel segmentation and subsequently optimized the image superpixel segmentation result. A convolutional neural network and a graph convolutional neural network were used to obtain segments’ deep features and spatial association features, respectively. The random forest algorithm was used to classify segments based on the multidimension features. The results showed that the image superpixel segmentation results had the significant impact on the classification results. Compared with the pixel-based method, the segment-based methods generally yielded the higher classification accuracy. The strategy of multidimension feature fusion can combine the advantages of each single-dimension feature to improve the classification accuracy. The proposed method utilizing multidimension features was more efficient than traditional methods used for the urban land cover classification. The fusion of segments’ object features, deep features, and spatial association features was the best solution for achieving the urban land cover classification.

Polarimetric synthetic aperture radar (PolSAR) image interpretation is widely used in various fields. Recently, deep learning has made significant progress in PolSAR image classification. Supervised learning (SL) requires a large amount of labeled PolSAR data with high quality to achieve better performance, however, manually labeled data are insufficient. This causes the SL to fail into overfitting and degrades its generalization performance. Furthermore, the scattering confusion problem is also a significant challenge that attracts more attention. To solve these problems, this article proposes a heterogeneous contrastive learning method (HCLNet). It aims to learn high-level representation from unlabeled PolSAR data for few-shot classification according to multifeatures and superpixels. Beyond the conventional CL, HCLNet introduces the heterogeneous architecture for the first time to utilize heterogeneous PolSAR features better. And it develops two easy-to-use plugins to narrow the domain gap between optics and PolSAR, including feature filter and superpixel-based instance discrimination, which the former is used to enhance the complementarity of multifeatures, and the latter is used to increase the diversity of negative samples. Experiments demonstrate the superiority of HCLNet on three widely used PolSAR benchmark datasets compared with state-of-the-art methods. Ablation studies also verify the importance of each component. Besides, this work has implications for how to efficiently utilize the multifeatures of PolSAR data to learn better high-level representation in CL and how to construct networks suitable for PolSAR data better.

Abstract This paper proposes a weakly-supervised structural surface crack detection algorithm that can detect the crack area in an image with low data labeling cost. The algorithm consists of a convolutional neural networks Vgg16-Crack for classification, an improved and optimized class activation map (CAM) algorithm for accurately reflecting the position and distribution of cracks in the image, and a method that combines superpixel segmentation algorithm simple linear iterative clustering (SLIC) with CAM for more accurate semantic segmentation of cracks. In addition, this paper uses Bayesian optimization algorithm to obtain the optimal parameter combination that maximizes the performance of the model. The test results show that the algorithm only requires image-level labeling, which can effectively reduce the labor and material consumption brought by pixel-level labeling while ensuring accuracy.

Despite the success of deep learning in land cover classification, high-resolution (HR) land cover mapping remains challenging due to the time-consuming and labor-intensive process of collecting training samples. Many global land cover products (LCP) can reflect the low-level commonality (LLC) knowledge of land covers, such as basic shape and underlying semantic information. Therefore, we expect to use LCP as weakly supervised information to guide the semantic segmentation of HR images. We regard high-level specialty (HLS) knowledge as HR information unavailable in the LCP. We believe LLC knowledge can gradually evolve into HLS knowledge through self-active learning. Hence, we design a knowledge evolution learning strategy from LLC to HLS knowledge and correspondingly devise a knowledge evolution weakly supervised learning framework (KE-WESUP) based on LCP. KE-WESUP mainly includes three tasks: (1) Abstraction of LLC knowledge. KE-WESUP first adopts a training method based on superpixel to alleviate the inconsistency between LCP and HR images and directly learns the LLC knowledge from LCP according to the feature-fitting capacity of convolutional neural networks. (2) Automatic exploration of HLS knowledge. We propose a dynamic label optimization strategy to obtain a small number of point labels with high confidence and encourage the model to automatically mine HLS knowledge through the knowledge exploration mechanism, which prompts the model to adapt to complexHR scenes. (3) Dynamic interaction of LLC and HLS knowledge. We adopt the consistency regularization method to achieve further optimization and verification of LLC and HLS knowledge. To verify the effectiveness of KE-WESUP, we conduct experiments on USDA National Agriculture Imagery Program (1 m) and GaoFen-2 (1 m) data using WolrdCover (10 m) as labels. The results show that KE-WESUP achieves outstanding results in both experiments, which has significant advantages over existing weakly supervised methods. Therefore, the proposed method has great potential in utilizing the prior information of LCP and is expected to become a new paradigm for large-scale HR land cover classification. [ABSTRACT FROM AUTHOR]

Video object segmentation (VOS) exhibits heavy occlusions, large deformation, and severe motion blur.While many remarkable convolutional neural networks are devoted to the VOS task, they often mis-identify background noise as the target or output coarse object boundaries, due to the failure of mining detail information and high-order correlations of pixels within the whole video. In this work, we propose an edge attention gated graph convolutional network (GCN) for VOS. The seed point initialization and graph construction stages construct a spatio-temporal graph of the video by exploring the spatial intra-frame correlation and the temporal inter-frame correlation of superpixels. The node classification stage identifies foreground superpixels by using an edge attention gated GCN which mines higher-order correlations between superpixels and propagates features among different nodes. The segmentation optimization stage optimizes the classification of foreground superpixels and reduces segmentation errors by using a global appearance model which captures the long-term stable feature of objects. In summary, the key contribution of our framework is twofold: (a) the spatio-temporal graph representation can propagate the seed points of the first frame to subsequent frames and facilitate our framework for the semi-supervised VOS task; and (b) the edge attention gated GCN can learn the importance of each node with respect to both the neighboring nodes and the whole task with a small number of layers. Experiments on Davis 2016 and Davis 2017 datasets show that our framework achieves the excellent performance with only small training samples (45 video sequences). [ABSTRACT FROM AUTHOR]

Weakly supervised semantic segmentation (WSSS) can obtain pseudo-semantic masks through a weaker level of supervised labels, reducing the need for costly pixel-level annotations. However, the general class activation map (CAM)-based pseudo-mask acquisition method suffers from sparse coverage, leading to false positive and false negative regions that reduce accuracy. We propose a WSSS method based on local superpixel transformation that combines superpixel theory and image local information. Our method uses a superpixel local consistency weighted cross-entropy loss to correct erroneous regions and a post-processing method based on the adjacent superpixel affinity matrix (ASAM) to expand false negatives, suppress false positives, and optimize semantic boundaries. Our method achieves 73.5% mIoU on the PASCAL VOC 2012 validation set, which is 2.5% higher than our baseline EPS and 73.9% on the test set, and the ASAM post-processing method is validated on several state-of-the-art methods. If our paper is accepted, our code will be published at https://github.com/JimmyMa99/SPL. [ABSTRACT FROM AUTHOR]

Recently, deep learning methods have been widely studied in the field of polarimetric synthetic aperture radar (PolSAR) ship detection. However, extracting polarimetric and spatial features on the whole PolSAR image will result in high computational complexity. In addition, in the massive data ship detection task, the image to be detected contains a large number of invalid areas, such as land and seawater without ships. Therefore, using ship coarse detection methods to quickly locate the potential areas of ships, that is, ship potential area extraction, is an important prerequisite for PolSAR ship detection. Since existing unsupervised PolSAR ship detection methods based on pixel-level features often rely on fine sea–land segmentation pre-processing and have poor applicability to images with complex backgrounds, in order to solve the abovementioned issue, this paper proposes a PolSAR ship potential area extraction method based on the neighborhood semantic differences of an LDA bag-of-words topic model. Specifically, a polarimetric feature suitable for the scattering diversity condition is selected, and a polarimetric feature map is constructed; the superpixel segmentation method is used to generate the bag of words on the feature map, and latent high-level semantic features are extracted and classified with the improved LDA bag-of-words topic model method to obtain the PolSAR ship potential area extraction result, i.e., the PolSAR ship coarse detection result. The experimental results on the self-established PolSAR dataset validate the effectiveness and demonstrate the superiority of our method. [ABSTRACT FROM AUTHOR]