Search

Crack detection is integral in civil infrastructure maintenance, with automated robots for detailed inspections and repairs becoming increasingly common. Ensuring fast and accurate crack detection for autonomous vehicles is crucial for safe road navigation. In these fields, existing detection models demonstrate impressive performance. However, they are primarily optimized for clear weather and struggle with occlusions and brightness variations in adverse weather conditions. These problems affect automated robots and autonomous vehicle navigation that must operate reliably in diverse environmental conditions. To address this problem, we propose Auxcoformer, designed for robust crack detection in adverse weather conditions. Considering the image degradation caused by adverse weather conditions, Auxcoformer incorporates an auxiliary restoration network. This network efficiently restores damaged crack details, ensuring the primary detection network obtains better quality features. The proposed approach uses a non-local patch-based 3D transform technique, emphasizing the characteristics of cracks and making them more distinguishable. Considering the connectivity of cracks, we also introduce contrastive patch loss for precise localization. Then, we demonstrate the performance of Auxcoformer, comparing it with other detection models through experiments.

Abstract Most single‐image super‐resolution (SR) models suffer from the degradation of image restoration performance when restoring a high‐resolution (HR) image from a low‐resolution (LR) image downscaled using an unknown blur kernel. The spatially invariant blur kernel estimators have been proposed to predict the blur kernel to address this issue. Nevertheless, the spatially variant blur exists in the real‐world; thus, these blur kernel estimators are unsuitable for real‐world applications. Although the spatially variant blur kernel estimators have been proposed, SR models still suffer from performance degradation because these estimators do neither consider the consistency between surrounding blur kernels nor refine non‐parametric blur kernels as parameters. To address this problem, the authors propose a multiregression spatially variant blur kernel estimation based on inter‐kernel consistency. The proposed estimator consists of three parts: non‐parametric regression, an inter‐kernel consistency block, and parametric regression. Specifically, it predicts spatially variant blur kernels while considering the inter‐kernel consistency between nearby blur kernels. Our source codes with pretrained models are available on https://github.com/alsgur0720/multiregression.

Recently, several arbitrary-scale models have been proposed for single-image super-resolution. Furthermore, the importance of arbitrary-scale single image super-resolution is emphasized for applications such as satellite image processing, HR display, and video-based surveillance. However, the baseline integer-scale model must be retrained to fit the existing network, and the learning speed is slow. This paper proposes a network to solve these problems, processing super-resolution by restoring the high-frequency information lost in the remaining arbitrary-scale while maintaining the baseline integer scale. The proposed network extends an integer-scaled image to an arbitrary-scale target in the discrete cosine transform spectral domain. We also modulate the high-frequency restoration weights of the depthwise multi-head attention to use memory efficiently. Finally, we demonstrate the performance through experiments with existing state-of-the-art models and their flexibility through integration with existing integer-scale models in terms of peak signal-to-noise ratio (PSNR) and similarity index measure (SSIM) scores. This means that the proposed network restores high-resolution (HR) images appropriately by improving the image sharpness of low-resolution (LR) images.

Character recognition in a single image is a technology utilized in various sensor platforms, such as smart parking and text-to-speech systems, and numerous studies are being conducted to improve its performance by experimenting with novel approaches. However, when low-quality images were inputted to a character recognition neural network for recognition, a difference in the resolution of the training image and low-quality image results in poor accuracy. To resolve this problem, this study proposes a collaborative trainable mechanism that integrates a global image feature extraction-based super-resolution neural network with a character recognition neural network. This collaborative trainable mechanism helps the character recognizer to be robust to inputs with varying quality in the real world. The alternative collaborative learning and character recognition performance test was conducted using the license plate image dataset among various character images, and the effectiveness of the proposed algorithm was verified using a performance test.

Abstract Although developments in deep learning have resulted in considerable perormance enhancements in super‐resolution (SR), they have also caused substantial increases in computational costs and memory requirements. Thus, various compression techniques, such as quantisation, pruning, and knowledge distillation (KD), for single SR models have been introduced. However, multiple SR models are required in the real world to robustly reconstruct low‐resolution (LR) images of varying input sizes. Because of the limited resources, storing multiple models is impossible for mobile devices and embedded systems. In this letter, we propose a multi‐scale SR network using weight‐sharing method to effectively eliminate redundant parameters. To train our multi‐scale SR network and mitigate SR performance degradation due to knowledge confusion, we divide backpropagation into two stages. Furthermore, we propose a compression framework that distils shared knowledge within a multi‐scale SR network. We achieve a compression rate of 94% from storing multiple scales of single SR models, while only compromising 0.3 dB on average in terms of peak signal‐to‐noise ratio (PSNR).

In real‐world intelligent transportation systems, accuracy in vehicle license plate detection and recognition is considered quite critical. Many algorithms have been proposed for still images, but their accuracy on actual videos is not satisfactory. This stems from several problematic conditions in videos, such as vehicle motion blur, variety in viewpoints, outliers, and the lack of publicly available video datasets. In this study, we focus on these challenges and propose a license plate detection and recognition scheme for videos based on a temporal matching prior network. Specifically, to improve the robustness of detection and recognition accuracy in the presence of motion blur and outliers, forward and bidirectional matching priors between consecutive frames are properly combined with layer structures specifically designed for plate detection. We also built our own video dataset for the deep training of the proposed network. During network training, we perform data augmentation based on image rotation to increase robustness regarding the various viewpoints in videos.

Although real-image emotion recognition has been developed in several studies, an acceptable accuracy level has not been achieved in portrait drawings. This paper proposes a portrait emotion recognition framework based on independent component analysis (ICA) and latent codes to overcome the performance degradation problem in drawings. This framework employs latent code extracted through a generative adversarial network (GAN)-based encoder. It learns independently from factors that interfere with expression recognition, such as color, small occlusion, and various face angles. It is robust against environmental factors since it filters latent code by adding an emotion-relevant code extractor to extract only information related to facial expressions from the latent code. In addition, an image is generated by changing the latent code to the direction of the eigenvector for each emotion obtained through the ICA method. Since only the position of the latent code related to the facial expression is changed, there is little external change and the expression changes in the desired direction. This technique is helpful for qualitative and quantitative emotional recognition learning. The experimental results reveal that the proposed model performs better than the existing models, and the latent editing used in this process suggests a novel manipulation method through ICA. Moreover, the proposed framework can be applied for various portrait emotion applications from recognition to manipulation, such as automation of emotional subtitle production for the visually impaired, understanding the emotions of objects in famous classic artwork, and animation production assistance.

Surface defect detection systems, which have advanced beyond conventional defect detection methods, lower the risk of accidents and increase working efficiency and productivity. Most fault detection techniques demand extra tools, such as ultrasonic sensors or lasers. With the advancements, these techniques can be examined without additional tools. We propose a morphological attention ensemble learning for surface defect detection called OASIS-Net, which can detect defects of three kinds (crack, efflorescence, and spalling) at the bounding box level. Based on the morphological analysis of each defect, OASIS-Net offers specialized loss functions for each defect that can be examined. Specifically, high-frequency image augmentation, connectivity attention, and penalty areas are used to detect cracks. It also compares the colors of the sensing objects and analyzes the image histogram peaks to improve the efflorescence-verification accuracy. Analyzing the ratio of the major and minor axes of the spalling through morphological comparison reveals that the spalling-detection accuracy improved. Defect images are challenging to obtain due to their properties. We labeled some data provided by AI hub and some concrete crack datasets and used them as custom datasets. Finally, an ensemble learning technique based on multi-task classification is suggested to learn and apply the specialized loss of each class to the model. For the custom dataset, the accuracy of the crack detection increased by 5%, the accuracy of the efflorescence detection increased by 4.4%, and the accuracy of the spalling detection increased by 6.6%. The experimental results reveal that the proposed network outperforms the previous state-of-the-art methods.

Object recognition is conducted using red, green, and blue (RGB) images in object recognition studies. However, RGB images in low-light environments or environments where other objects occlude the target objects cause poor object recognition performance. In contrast, infrared (IR) images provide acceptable object recognition performance in these environments because they detect IR waves rather than visible illumination. In this paper, we propose an inter- and intra-weighted cross-fusion network (Infusion-Net), which improves object recognition performance by combining the strengths of the RGB-IR image pairs. Infusion-Net connects dual object detection models using a high-frequency (HF) assistant (HFA) to combine the advantages of RGB-IR images. To extract HF components, the HFA transforms input images into a discrete cosine transform domain. The extracted HF components are weighted via pretrained inter- and intra-weights for feature-domain cross-fusion. The inter-weighted fused features are transmitted to each other’s networks to complement the limitations of each modality. The intra-weighted features are also used to enhance any insufficient HF components of the target objects. Thus, the experimental results present the superiority of the proposed network and present improved performance of the multispectral object recognition task.

Scene text detection and recognition, such as automatic license plate recognition, is a technology utilized in various applications. Although numerous studies have been conducted to improve recognition accuracy, accuracy decreases when low-quality legacy license plate images are input into a recognition module due to low image quality and a lack of resolution. To obtain better recognition accuracy, this study proposes a high-frequency augmented license plate recognition model in which the super-resolution module and the license plate recognition module are integrated and trained collaboratively via a proposed gradual end-to-end learning-based optimization. To optimally train our model, we propose a holistic feature extraction method that effectively prevents generating grid patterns from the super-resolved image during the training process. Moreover, to exploit high-frequency information that affects the performance of license plate recognition, we propose a license plate recognition module based on high-frequency augmentation. Furthermore, we propose a gradual end-to-end learning process based on weight freezing with three steps. Our three-step methodological approach can properly optimize each module to provide robust recognition performance. The experimental results show that our model is superior to existing approaches in low-quality legacy conditions on UFPR and Greek vehicle datasets.

Object detection and recognition are crucial in the field of computer vision and are an active area of research. However, in actual object recognition processes, recognition accuracy is often degraded due to resolution mismatches between training and test image data. To solve this problem, we designed and developed an integrated object recognition and super-resolution framework by proposing an image super-resolution technique that improves object recognition accuracy. In detail, we collected a number of license plate training images through web-crawling and artificial data generation, and the image super-resolution artificial neural network was trained by defining an objective function to be robust to image flips. To verify the performance of the proposed algorithm, we experimented with the trained image super-resolution and recognition on representative test images and confirmed that the proposed super-resolution technique improves the accuracy of character recognition. For character recognition with the 4× magnification, the proposed method remarkably increased the mean average precision by 49.94% compared to the existing state-of-the-art method.

Images captured by cameras in closed-circuit televisions and black boxes in cities have low or poor quality owing to lens distortion and optical blur. Moreover, actual images acquired through imaging sensors of cameras such as charge-coupled devices and complementary metal-oxide-semiconductors generally include noise with spatial-variant characteristics that follow Poisson distributions. If compression is directly applied to an image with such spatial-variant sensor noises at the transmitting end, complex and difficult noises called compressed Poisson noises occur at the receiving end. The super-high-definition imaging technology based on deep neural networks improves the image resolution as well as effectively removes the undesired compressed Poisson noises that may occur during real image acquisition and compression as well as in transmission and reception systems. This solution of using deep neural networks at the receiving end to solve the image degradation problem can be used in the intelligent image analysis platform that performs accurate image processing and analysis using high-definition images obtained from various camera sources such as closed-circuit televisions and black boxes. In this review article, we investigate the current state-of-the-art super-high-definition imaging techniques in terms of image denoising for removing the compressed Poisson noises as well as super-resolution based on the deep neural networks.

In recent years, deep learning has dominated medical image segmentation. Encoder-decoder architectures, such as U-Net, can be used in state-of-the-art models with powerful designs that are achieved by implementing skip connections that propagate local information from an encoder path to a decoder path to retrieve detailed spatial information lost by pooling operations. Despite their effectiveness for segmentation, these naïve skip connections still have some disadvantages. First, multi-scale skip connections tend to use unnecessary information and computational sources, where likable low-level encoder features are repeatedly used at multiple scales. Second, the contextual information of the low-level encoder feature is insufficient, leading to poor performance for pixel-wise recognition when concatenating with the corresponding high-level decoder feature. In this study, we propose a novel spatial-channel attention gate that addresses the limitations of plain skip connections. This can be easily integrated into an encoder-decoder network to effectively improve the performance of the image segmentation task. Comprehensive results reveal that our spatial-channel attention gate remarkably enhances the segmentation capability of the U-Net architecture with a minimal computational overhead added. The experimental results show that our proposed method outperforms the conventional deep networks in term of Dice score, which achieves 71.72%.

In real image coding systems, block-based coding is often applied on images contaminated by camera sensor noises such as Poisson noises, which cause complicated types of noises called compressed Poisson noises. Although many restoration methods have recently been proposed for compressed images, they do not provide satisfactory performance on the challenging compressed Poisson noises. This is mainly due to (i) inaccurate modeling regarding the image degradation, (ii) the signal-dependent noise property, and (iii) the lack of analysis on intercorrelation distortion. In this paper, we focused on the challenging issues in practical image coding systems and propose a compressed Poisson noise reduction scheme based on a secondary domain intercorrelation enhanced network. Specifically, we introduced a compressed Poisson noise corruption model and combined the secondary domain intercorrelation prior with a deep neural network especially designed for signal-dependent compression noise reduction. Experimental results showed that the proposed network is superior to the existing state-of-the-art restoration alternatives on classical images, the LIVE1 dataset, and the SIDD dataset.

Physiological signals are widely used in the recognition of affective status. Recording of such physiological signals involves elicitation of emotions through different stimuli including video-based stimulus. Considering that the same stimulus videos often induce different emotions in different individuals, emotion recognition in such a scenario requires consideration of the individual differences in the consumption of the stimulus content. With this as our goal, we present a Physiological dataset for Multimodal Emotion Recognition (PhyMER) for studying emotion through physiological response with personality as a context. The PhyMER dataset consists of electroencephalogram (EEG), electrodermal activity (EDA), blood volume pulse (BVP), and skin temperature along with the personality traits of 30 participants. We collected the video-based stimulus dataset for emotion elicitation and developed a web-based annotation tool for labeling felt emotions. We compared the stimulus labels and the self-annotation of felt emotions labeled during physiological data recording. Correlation among personalities was analyzed to study the impact of personality on the intensity of emotions in arousal and valence dimensions. Finally, we proposed a baseline model for the classification of emotions using physiological signals. The dataset is publicly available to the academic community for analysis of affective states and the development of emotion recognition models.