Search

Deep learning techniques through semantic segmentation networks have been widely used for natural disaster analysis and response. The underlying base of these implementations relies on convolutional neural networks (CNNs) that can accurately and precisely identify and locate the respective areas of interest within satellite imagery or other forms of remote sensing data, thereby assisting in disaster evaluation, rescue planning, and restoration endeavours. Most CNN-based deep-learning models encounter challenges related to the loss of spatial information and insufficient feature representation. This issue can be attributed to their suboptimal design of the layers that capture multiscale-context information and their failure to include optimal semantic information during the pooling procedures. In the early layers of CNNs, the network encodes elementary semantic representations, such as edges and corners, whereas, as the network progresses toward the later layers, it encodes more intricate semantic characteristics, such as complicated geometric shapes. In theory, it is advantageous for a segmentation network to extract features from several levels of semantic representation. This is because segmentation networks generally yield improved results when both simple and intricate feature maps are employed together. This study comprehensively reviews current developments in deep learning methodologies employed to segment remote sensing images associated with natural disasters. Several popular deep learning models, such as SegNet U-Net, FCNs, FCDenseNet, PSPNet, HRNet, and DeepLab, have exhibited notable achievements in various applications, including forest fire delineation, flood mapping, and earthquake damage assessment. These models demonstrate a high level of efficacy in distinguishing between different land cover types, detecting infrastructure that has been compromised or damaged, and identifying regions that are fire-susceptible to further dangers.

Deep learning has achieved efficient success in the field of machine learning, made possible by the emergence of efficient optimization methods such as Stochastic Gradient Descent (SGD) and its variants. Simultaneously, the Information Bottleneck theory (IB) has been studied to train neural networks, aiming to enhance the performance of optimization methods. However, previous works have focused on their specific tasks, and the effect of the IB theory on general deep learning tasks is still unclear. In this study, we introduce a new method inspired by the proximal operator, which sequentially updates the neural network parameters based on the defined bottleneck features between the forward and backward networks. Unlike the conventional proximal-based methods, we consider the second-order gradients of the objective function to achieve better updates for the forward networks. In contrast to SGD-based methods, our approach involves accessing the network’s black box, and incorporating the bottleneck feature update process into the parameter update process. This way, from the perspective of the IB theory, the data is well compressed up to the bottleneck feature, ensuring that the compressed information maintains sufficient mutual information up to the final output. To demonstrate the performance of the proposed approach, we applied the method to various optimizers with several tasks and analyzed the results by training on both the MNIST dataset and CIFAR-10 dataset. We also conducted several ablation studies by modifying the components of the proposed algorithm to further validate its performance.

With the development of deep learning, the performance of image-to-image translation is also increasing. However, most of the image-to-image translation models depend on the implicit method which does not explain why the models alter specific parts of the original input images. In this work, we assume that we can control the extent to which the models translate the input images using an explicit method. We explicitly create masks that will be added to the input images, aiming to highlight the difference between the inputs and the translated images. Since limiting the area of the masks directly affects the shape of the translated images, we can adjust the model through a simple regularization parameter. Our proposed method demonstrates that a simple regularization parameter, which regularizes the generated masks, can control where the model needs to change and remain. Furthermore, by adjusting the degree of the regularization parameter, we can generate diverse translated images from one original image.

Continual learning aims to enable neural networks to learn new tasks without catastrophic forgetting of previously learned knowledge. Orthogonal Gradient Descent algorithms have been proposed as an effective solution to mitigate catastrophic forgetting. However, these algorithms often rely on the Neural Tangent Kernel regime, which imposes limitations on network architecture. In this study, we propose a novel method to construct an orthonormal basis set for orthogonal projection by leveraging Catastrophic Forgetting Loss. In contrast to the conventional gradient-based basis that reflects an update of model within an infinitesimal range, our loss-based basis can account for the variance within two distinct points in the model parameter space, thus overcoming the limitations of the Neural Tangent Kernel regime. We provide both quantitative and qualitative analysis of the proposed method, discussing its advantages over conventional gradient-based baselines. Our approach is extensively evaluated on various model architectures and datasets, demonstrating a significant performance advantage, especially for deep or narrow networks where the Neural Tangent Kernel regime is violated. Furthermore, we offer a mathematical analysis based on higher-order Taylor series to provide theoretical justification. This study introduces a novel theoretical framework and a practical algorithm, potentially inspiring further research in areas such as continual learning, network debugging, and one-pass learning.

Abstract Characterization of tissue architecture promises to deliver insights into development, cell communication, and disease. In silico spatial domain retrieval methods have been developed for spatial transcriptomics (ST) data assuming transcriptional similarity of neighboring barcodes. However, domain retrieval approaches with this assumption cannot work in complex tissues composed of multiple cell types. This task becomes especially challenging in cellular resolution ST methods. We developed Vesalius to decipher tissue anatomy from ST data by applying image processing technology. Vesalius uniquely detected territories composed of multiple cell types and successfully recovered tissue structures in high‐resolution ST data including in mouse brain, embryo, liver, and colon. Utilizing this tissue architecture, Vesalius identified tissue morphology‐specific gene expression and regional specific gene expression changes for astrocytes, interneuron, oligodendrocytes, and entorhinal cells in the mouse brain.

Regularization is essential for avoiding over-fitting to training data in network optimization, leading to better generalization of the trained networks. The label noise provides a strong implicit regularization by replacing the target ground truth labels of training examples by uniform random labels. However, it can cause undesirable misleading gradients due to the large loss associated with incorrect labels. We propose a first-order optimization method (Label-Noised Trim-SGD) that uses the label noise with the example trimming in order to remove the outliers based on the loss. The proposed algorithm is simple yet enables us to impose a large label-noise and obtain a better regularization effect than the original methods. The quantitative analysis is performed by comparing the behavior of the label noise, the example trimming, and the proposed algorithm. We also present empirical results that demonstrate the effectiveness of our algorithm using the major benchmarks and the fundamental networks, where our method has successfully outperformed the state-of-the-art optimization methods.

Eliminating the undesirable features is crucial to computer vision applications since undesirable features degrade the visibility of images. For that purpose, denoising, dehazing and deraining have been actively studied in both traditional model-based approaches and modern deep learning methods. However, elimination of hair in dermoscopic images has not received sufficient attention despite its significance and potential. Meanwhile, hair removal algorithms remain within the classical morphological methodologies, while only a few attempts apply the latest data-driven techniques. Hair is desired to be removed in dermoscopy applications because it causes undesired effects such as occlusions in lesion areas. However, removing hair is challenging because of its inherent complex structure and variations. In this paper, we propose a new unsupervised algorithm for hair removal and evaluate it on a real-world melanoma dataset. The proposed method eliminates hair from dermoscopic images by inducing a reconstructed distribution of images with hair to resemble a hairless distribution using generative adversarial learning. In the generative adversarial learning framework, hair features are characterized with a coarse-grained label simply via a binary classifier. At the same time, the important features of the lesions are preserved by minimizing $L_{1}$ -norm reconstruction loss based on Laplace noise assumption. The qualitative evaluation of the hair-removed results show that the proposed algorithm is robust to hair variations, and the quantitative results demonstrate that applying our hair removal algorithm considerably improves the performance of melanoma classification, outperforming the benchmarks.

We present an adaptive regularization algorithm that can be effectively applied to the optimization problem in deep learning framework. Our regularization algorithm aims to take into account the fitness of data to the current state of model in the determination of regularity to achieve better generalization. The degree of regularization at each element in the target space of the neural network architecture is determined based on the residual at each optimization iteration in an adaptive way. Our adaptive regularization algorithm is designed to apply a diffusion process driven by the heat equation with spatially varying diffusivity depending on the probability density function following a certain distribution of residual. Our data-driven regularity is imposed by adaptively smoothing a simplified objective function in which the explicit regularization term is omitted in an alternating manner between the evaluation of residual and the determination of the degree of its regularity. The effectiveness of our algorithm is empirically demonstrated by the numerical experiments in the application of image classification problems, indicating that our algorithm outperforms other commonly used optimization algorithms in terms of generalization using popular deep learning models and benchmark datasets.

Regularization in the optimization of deep neural networks is often critical to avoid undesirable over-fitting leading to better generalization of model. One of the most popular regularization algorithms is to impose L2 penalty on the model parameters resulting in the decay of parameters, called weight-decay, and the decay rate is generally constant to all the model parameters in the course of optimization. In contrast to the previous approach based on the constant rate of weight-decay, we propose to consider the residual that measures dissimilarity between the current state of model and observations in the determination of the weight-decay for each parameter in an adaptive way, called adaptive weight-decay (AdaDecay) where the gradient norms are normalized within each layer and the degree of regularization for each parameter is determined in proportional to the magnitude of its gradient using the sigmoid function. We empirically demonstrate the effectiveness of AdaDecay in comparison to the state-of-the-art optimization algorithms using popular benchmark datasets: MNIST, Fashion-MNIST, and CIFAR-10 with conventional neural network models ranging from shallow to deep. The quantitative evaluation of our proposed algorithm indicates that AdaDecay improves generalization leading to better accuracy across all the datasets and models.

The reduction of unnecessary details is important in a variety of imaging tasks. Image denoising can be generally formulated as a diffusion process that iteratively suppresses undesirable image features with high variance. We propose a recursive diffusion process that simultaneously computes the local geometrical property of the image features and determines the size and shape of the diffusion kernel, leading to an anisotropic scale-space. In the construction of the proposed anisotropic scale-space, image features due to undesirable noise are suppressed while significant geometrical image features such as edges and corners are preserved across the scale-space. The diffusion kernels are iteratively determined based on the local geometrical properties of the image features. We demonstrate the effectiveness and robustness of the proposed algorithm in the detection of curvilinear features using simple yet effective synthetic and real images. The algorithm is quantitatively evaluated based on the identification of fissures in lung CT imagery. The experimental results indicate that the proposed algorithm can be used for the detection of linear or curvilinear structures in a variety of images ranging from satellite to medical images.

Isocontour mapping is efficient for extracting meaningful information from a biomedical image in a topographic analysis. Isocontour extraction from real world medical images is difficult due to noise and other factors. As such, adaptive selection of contour generation parameters is needed. This paper proposes an algorithm for generating an adaptive contour map that is spatially adjusted. It is based on the modified active contour model, which imposes successive spatial constraints on the image domain. The adaptability of the proposed algorithm is governed by the energy term of the model. This work focuses on mammograms and the analysis of their intensity. Our algorithm employs the Mumford-Shah energy functional, which considers an image’s intensity distribution. In mammograms, the brighter regions generally contain significant information. Our approach exploits this characteristic to address the initialization and local optimum problems of the active contour model. Our algorithm starts from the darkest region; therefore, local optima encountered during the evolution of contours are populated in less important regions, and the important brighter regions are reserved for later stages. For an unrestricted initial contour, our algorithm adopts an existing technique without re-initialization. To assess its effectiveness and robustness, the proposed algorithm was tested on a set of mammograms.