Search

In recent times, there has been notable and swift advancement in the field of image dehazing. Several deep learning techniques have demonstrated remarkable proficiency in resolving homogeneous dehazing issues. Nonetheless, the current dehazing approaches are generally formulated to deal with homogeneous haze, which is often undermined in real-world scenarios due to the uncertain haze dispersion. In this paper, we propose a dehazing model named RepDehazeNet by combining a structurally Reparameterization Encoder-Decoder subnet and a Full-Resolution Attention subnet. To be specific, the structural reparameterization idea is introduced into the encoder–decoder subnet to strengthen the feature extraction of dehazed images and improve the feature extraction speed. RepDehazeNet is compared with seven SOTA models on different datasets in terms of PSNR, SSIM, parameter quantity, and inference time. Compared to the DW-GAN model, the proposed RepDehazeNet model reduces the number of parameters by 2.7 million, and improves the inference speed by 90.3%, while achieving a higher PSNR of 0.5 dB on the NH-Haze2021 dataset. The experimental results demonstrate that the proposed RepDehazeNet model can effectively improve the real-time performance, accuracy of dehazing synthesized and nonhomogeneous haze images. [Display omitted] • Structural reparameterization dehazenet: outstanding performance, faster speed. • Replacing Tanh with ReLU leads to better results. • Transfer learning addresses the problem of insufficient samples. • Dual subnets method proves highly effective in datasets of different scales. [ABSTRACT FROM AUTHOR]

Image segmentation is one of the most classical computer vision tasks. Segmentation tasks yield a set of classes attributed to individual pixels instead of sparsely predicted images or patches, such as in classification or detection tasks. However, creating annotation sets for pixelwise tasks is a very costly task, often requiring hours for labeling single samples in images with multiple classes of objects. In this context, unsupervised learning can be leveraged either to expedite the annotation procedure and/or to guide the segmentation algorithms altogether without the need for manual annotations. Classical unsupervised segmentation methods leveraged techniques from areas as graph theory, image processing, clustering or supervised classifiers in order to achieve "shallow" pixelwise classification. These techniques usually aim to achieve superpixel over-segmentations by grouping similar pixels that should pertain to the same object. Modern deep unsupervised approaches for image segmentation aimed to group pixels in a data-driven way by using the capabilities of deep architectures to process unstructured data such as images. Later, self-supervised learning bypassed the need for labels via pretext tasks, compelling deep architectures to learn more generic features capable of enhancing downstream tasks, including segmentation. The generalized representations produced by unsupervised models have propelled the recent progress in self-supervised, few- and zero-shot learning and even general-purpose foundational models in computer vision, yielding state-of-the-art results across diverse tasks and datasets. This paper provides an overview of unsupervised and generalizable approaches for image segmentation, introduces key concepts and terminology, and discusses the main aspects of state-of-the-art methods. Additionally, we highlight prominent applications in various domains such as remote sensing, medical imaging, and geology. Finally, we discuss trends and future directions for state-of-the-art unsupervised image segmentation. [Display omitted] [ABSTRACT FROM AUTHOR]

Foundation models, such as ULIP-2 (Xue et al., 2023) recently projected forward the field of 3D deep learning. These models are trained with significantly more data and show superior representation learning capacity in many downstream tasks like 3D shape classification and few-shot part segmentation. A particular characteristic of the recent 3D foundation models is that they are typically multi-modal , and involve image (2D) as well as caption (text) branches. This leads to an intricate interplay that benefits all modalities. At the same time, the nature of the 3D encoders alone, involved in these foundation models is not well-understood. Specifically, there is little analysis on the utility of both pre-trained 3D features provided by these models, or their capacity to adapt to new downstream 3D data. Furthermore, existing studies typically focus on label-oriented downstream tasks, such as shape classification, and ignore other critical applications, such as 3D content-based object retrieval. In this paper, we fill this gap and show, for the first time, how 3D foundation models can be leveraged for strong 3D-to-3D retrieval performance on seven different datasets, on par with state-of-the-art view-based architectures. We evaluate both the pre-trained foundation models, as well as their fine-tuned versions using downstream data. We compare supervised fine-tuning using classification labels against two self-supervised label-free fine-tuning methods. Importantly, we introduce and describe a methodology for fine-tuning, as we found this to be crucial to make transfer learning from 3D foundation models work in a stable manner. [Display omitted] • Self-supervised 3D object retrieval is competitive with supervised learning on the same data distribution. • Self-supervised 3D object retrieval outperforms supervised learning out-of-distribution. • Object alignment is highly impactful on the out-of-distribution performance of retrieval systems. • Non-contrastive siamese networks are a practical option in 3D deep learning. [ABSTRACT FROM AUTHOR]

Text-conditioned image synthesis methods such as DALLE-2, IMAGEN, and Stable Diffusion are gaining strong attention from deep learning and art communities recently. Meanwhile, Image-to-Image (Img2Img) synthesis applications that emerged from the pioneering Neural Style Transfer (NST) approach have swiftly transitioned towards the feed-forward Automatic Style Transfer (AST) methods, due to numerous constraints inherent in the former method, including inconsistent synthesis outcomes and sluggish optimization-based synthesis process. However, NST holds significant potential yet remains relatively underexplored within this research domain. In this paper, we revisited the original NST method and uncovered its potential to attain image quality comparable to the AST synthesis methods across a diverse range of artistic styles. We propose a two-stage Feature-guided Style Transfer (FeaST) which consists (a) pre-stylization step called Sketching to address the poor initialization issue, and (b) Finetuning to guide the synthesis process based on high-frequency (HF) and low-frequency (LF) guidance channels. By addressing the issues of inconsistent synthesis and slow convergence inherent in the original method, FeaST unlocks the full capabilities of NST and significantly enhances its efficiency. [Display omitted] • We revisit the Neural Style Transfer (NST) framework for high-fidelity art generation. • We proposed a feature-guided reconstruction based on separated guidance channels for high and low-frequency feature regions to solve slow convergence issues. • We proposed a pre-stylization step called Sketching which improved the image synthesis time by a six-order magnitude. • We experimented a set of optimal weighting factors in feature-guided reconstruction for consistent results over wide variety of styles and contents. [ABSTRACT FROM AUTHOR]

In order to facilitate the robust and precise 3D vessel shape extraction and quantification from in-vivo Magnetic Resonance Imaging (MRI), this paper presents a novel multi-scale Knowledge Transfer Vision Transformer (i.e., KT-ViT) for 3D vessel shape segmentation. First, it uniquely integrates convolutional embeddings with transformer in a U-net architecture, which simultaneously responds to local receptive fields with convolution layers and global contexts with transformer encoders in a multi-scale fashion. Therefore, it intrinsically enriches local vessel feature and simultaneously promotes global connectivity and continuity for a more accurate and reliable vessel shape segmentation. Furthermore, to enable using relatively low-resolution (LR) images to segment fine scale vessel shapes, a novel knowledge transfer network is designed to explore the inter-dependencies of data and automatically transfer the knowledge gained from high-resolution (HR) data to the low-resolution handling network at multiple levels, including the multi-scale feature levels and the decision level, through an integration of multi-level loss functions. The modeling capability of fine-scale vessel shape data distribution, possessed by the HR image transformer network, can be transferred to the LR image transformer to enhance its knowledge for fine vessel shape segmentation. Extensive experimental results on public image datasets have demonstrated that our method outperforms all other state-of-the-art deep learning methods. • A novel knowledge transfer transformer for segmenting fine-scale 3D vasculature. • A unique vision transformer integrating both local convolutions and global encodings. • A U-shape multi-scale processing through the hierarchical convolutional transformer. • Knowledge transfer through transformers to explore the resolution inter-dependencies. [ABSTRACT FROM AUTHOR]

To display images on panels and screens of different dimensions, there is a need to create algorithms that help adjust them to desired sizes through image retargeting (IR). In this sense, choosing the right algorithms seems to be a challenge. From this perspective, this paper was to propose a method for the assessment of retargeted images using deep learning (DL) models. To this end, the Conditional Random Fields as Recurrent Neural Networks (CRF-RNN) approach was applied to find image pixels in terms of importance and, the You-Only-Look-Once (YOLO) model was employed to identify semantic objects of images. As well, image patch similarity was computed based on Siamese Neural Network (SNN), and output image distortion was recognized by the CNN model. With reference to three well-known databases, viz., MIT RetargetMe, NRID and CUHK, the assessment results of the proposed algorithm demonstrated its superior usage as compared to the existing ones. [Display omitted] • Operating an R-CNN model to extract image importance maps. • Using a pre-trained YOLO to extract regions containing important image objects. • Training SNN to compute corresponding patch-similarity of original and retargeted images. • Training regression-based CNN to measure distortion of retargeted images based on pixel displacement patterns. • Merging measures using Gaussian Process Regression (GPR) and calculating the final measure. [ABSTRACT FROM AUTHOR]

Point cloud registration aligns point clouds through transformation or deformation. The existing advanced registration methods perform poorly in registering under various perturbations, especially for large rotations and small overlaps. Moreover, these methods cannot achieve rigid and non-rigid registrations simultaneously. In this paper, we propose a two-step generative optimization (GO) method to achieve registration, which alleviates the constraints of transformation representation and completely circumvents the matching of correspondences, making it suitable for rigid registration with large rotations and small overlaps, as well as for non-rigid registration. GO achieves registration by generating a pseudo point cloud approximating the target point cloud. The pseudo point cloud is iteratively deformed to approach the target point cloud through an estimation step (ES) and an updating step (US). The ES step generates initial pseudo point clouds in three different views and estimates their deformation via the extracted features decoding the similarity between the pseudo point clouds and the target point cloud. The US step calculates regressors to update the current estimations, generating new pseudo point clouds closer to the target point cloud. The final pseudo point clouds generated are averaged to attain the registration result. We evaluate the performance of GO for rigid and non-rigid registrations through the comparison with seven deep learning-based methods on eight datasets, consisting of both 3D and 2D data and ranging from synthetic to real scenes. Our experimental results demonstrate the high robustness and stability of GO in rigid registrations and illustrate the feasibility and high accuracy for non-rigid registrations. [Display omitted] Given a source point cloud S and a target point cloud M, GO aims to register S to M by iteratively generating different pseudo point clouds P t S, P t F, P t O, and making each generated pseudo point cloud approach the target model M to mimic the deformation of the source model S. The registration result S T is the average of the generated pseudo point clouds at the final iteration. Specifically, the GO iteration alternates between performing an estimation (ES) step and an updating (US) step. The former aims to estimate the coordinates of the newly generated pseudo point clouds in different views via the extracted feature of the existing pseudo point clouds. The latter intends to update the existing pseudo point clouds for generating new pseudo point clouds by making the current estimation approach the target point cloud. The generated pseudo point clouds are then used to estimate the coordinates of new pseudo point clouds in the next ES step. GO terminates the iteration once the training error is below a threshold. And the generated pseudo point clouds are averaged to attain the final registration result simultaneously. • A novel two-step generative optimization (GO) is proposed for rigid and non-rigid registration. • GO outperforms both traditional and deep learning methods in robustness and accuracy. • The generation of pseudo models expedites the convergence of GO. [ABSTRACT FROM AUTHOR]

Point cloud registration precision degrades when the scene contains symmetric structures or repeated patches. Also, the common operation of continuous downsampling in existing registration methods causes the loss of detailed information and leads to further registration precision degradation. In this paper, we propose a high precision point cloud registration network (HPNet) to learn discriminative features by considering both multi-scale information and position information. We first propose a multi-scale feature extraction module similar to the feature pyramid that allows the extracted features to contain multi-scale information. Then, an information interaction is performed on the features to learn global contextual information by using Transformer with a hybrid position encoding, which takes into account both absolute and relative positions of points. Finally, the features obtained from the information interaction module are directly used to predict the point correspondences. Comprehensive experiments on 3DMatch, 3DLoMatch, ModelNet, and ModelLoNet datasets demonstrate the state-of-the-art performance of the proposed method and the implementation of HPNet in SLAM shows its effectiveness in real application. [Display omitted] • Point cloud registration loses detail information during continuous downsampling. • Feature fusion of low-level detail information and high-level semantic information. • Hybrid position encoding consider both absolute and relative positions. positions. • Point cloud registration algorithms applied to SLAM. [ABSTRACT FROM AUTHOR]