Search

The motion of an object or camera platform makes the acquired image blurred. This degradation is a major reason to obtain a poor-quality image from an imaging sensor. Therefore, developing an efficient deep-learning-based image processing method to remove the blur artifact is desirable. Deep learning has recently demonstrated significant efficacy in image deblurring, primarily through convolutional neural networks (CNNs) and Transformers. However, the limited receptive fields of CNNs restrict their ability to capture long-range structural dependencies. In contrast, Transformers excel at modeling these dependencies, but they are computationally expensive for high-resolution inputs and lack the appropriate inductive bias. To overcome these challenges, we propose an Efficient Hybrid Network (EHNet) that employs CNN encoders for local feature extraction and Transformer decoders with a dual-attention module to capture spatial and channel-wise dependencies. This synergy facilitates the acquisition of rich contextual information for high-quality image deblurring. Additionally, we introduce the Simple Feature-Embedding Module (SFEM) to replace the pointwise and depthwise convolutions to generate simplified embedding features in the self-attention mechanism. This innovation substantially reduces computational complexity and memory usage while maintaining overall performance. Finally, through comprehensive experiments, our compact model yields promising quantitative and qualitative results for image deblurring on various benchmark datasets. [ABSTRACT FROM AUTHOR]

Advances in deep learning and computer vision techniques have made impacts in the field of remote sensing, enabling efficient data analysis for applications such as land cover classification and change detection. Convolutional neural networks (CNNs) and transformer architectures have been utilized in visual perception algorithms due to their effectiveness in analyzing local features and global context. In this paper, we propose a hybrid transformer architecture that consists of a CNN-based encoder and transformer-based decoder. We propose a feature adjustment module that refines the multiscale feature maps extracted from an EfficientNet backbone network. The adjusted feature maps are integrated into the transformer-based decoder to perform the semantic segmentation of the remote sensing images. This paper refers to the proposed encoder–decoder architecture as a semantic feature adjustment network (SFA-Net). To demonstrate the effectiveness of the SFA-Net, experiments were thoroughly conducted with four public benchmark datasets, including the UAVid, ISPRS Potsdam, ISPRS Vaihingen, and LoveDA datasets. The proposed model achieved state-of-the-art accuracy on the UAVid, ISPRS Vaihingen, and LoveDA datasets for the segmentation of the remote sensing images. On the ISPRS Potsdam dataset, our method achieved comparable accuracy to the latest model while reducing the number of trainable parameters from 113.8 M to 10.7 M. [ABSTRACT FROM AUTHOR]

Plants are widely grown around the world and have high economic benefits. plant leaf diseases not only negatively affect the healthy growth and development of plants, but also have a negative impact on the environment. While traditional manual methods of identifying plant pests and diseases are costly, inefficient and inaccurate, computer vision technologies can avoid these drawbacks and also achieve shorter control times and associated cost reductions. The focusing mechanism of Transformer-based models(such as Visual Transformer) improves image interpretability and enhances the achievements of convolutional neural network (CNN) in image recognition, but Visual Transformer(ViT) performs poorly on small and medium-sized datasets. Therefore, in this paper, we propose a new hybrid architecture named FOTCA, which uses Transformer architecture based on adaptive Fourier Neural Operators(AFNO) to extract the global features in advance, and further down sampling by convolutional kernel to extract local features in a hybrid manner. To avoid the poor performance of Transformer-based architecture on small datasets, we adopt the idea of migration learning to make the model have good scientific generalization on OOD (Out-of-Distribution) samples to improve the model's overall understanding of images. In further experiments, Focal loss and hybrid architecture can greatly improve the convergence speed and recognition accuracy of the model in ablation experiments compared with traditional models. The model proposed in this paper has the best performance with an average recognition accuracy of 99.8% and an F1-score of 0.9931. It is sufficient for deployment in plant leaf disease image recognition. [ABSTRACT FROM AUTHOR]

Blurring is one of the main degradation factors in image degradation, so image deblurring is of great interest as a fundamental problem in low-level computer vision. Because of the limited receptive field, traditional CNNs lack global fuzzy region modeling, and do not make full use of rich context information between features. Recently, a transformer-based neural network structure has performed well in natural language tasks, inspiring rapid development in the field of defuzzification. Therefore, in this paper, a hybrid architecture based on CNN and transformers is used for image deblurring. Specifically, we first extract the shallow features of the blurred images using a cross-layer feature fusion block that emphasizes the contextual information of each feature extraction layer. Secondly, an efficient transformer module for extracting deep features is designed, which fully aggregates feature information at medium and long distances using vertical and horizontal intra- and inter-strip attention layers, and a dual gating mechanism is used as a feedforward neural network, which effectively reduces redundant features. Finally, the cross-layer feature fusion block is used to complement the feature information to obtain the deblurred image. Extensive experimental results on publicly available benchmark datasets GoPro, HIDE, and the real dataset RealBlur show that the proposed method outperforms the current mainstream deblurring algorithms and recovers the edge contours and texture details of the images more clearly. [ABSTRACT FROM AUTHOR]

Rice is a staple food for over half of the global population, but it faces significant yield losses: up to 52% due to leaf blast disease and brown spot diseases, respectively. This study aimed at proposing a hybrid architecture, namely ResViT-Rice, by taking advantage of both CNN and transformer for accurate detection of leaf blast and brown spot diseases. We employed ResNet as the backbone network to establish a detection model and introduced the encoder component from the transformer architecture. The convolutional block attention module was also integrated to ResViT-Rice to further enhance the feature-extraction ability. We processed 1648 training and 104 testing images for two diseases and the healthy class. To verify the effectiveness of the proposed ResViT-Rice, we conducted comparative evaluation with popular deep learning models. The experimental result suggested that ResViT-Rice achieved promising results in the rice disease-detection task, with the highest accuracy reaching 0.9904. The corresponding precision, recall, and F1-score were all over 0.96, with an AUC of up to 0.9987, and the corresponding loss rate was 0.0042. In conclusion, the proposed ResViT-Rice can better extract features of different rice diseases, thereby providing a more accurate and robust classification output. [ABSTRACT FROM AUTHOR]

The increasing frequency of skin tumour across the globe and their timely diagnosis is one of the most promising research directions in the healthcare domain. The most important cause behind the skin cancer mortalities is delayed detection. Early detection followed by adequate treatment may enhance the chances of human survival to a great extent. However, extracting the features from the tumour images for the possible detection of skin cancer is not a trivial task. Numerous deep learning models are extensively employed for the efficient features' extraction for skin cancer detection but literature demonstrate further scope of improvements in various performance measures. In this paper, we propose a hybrid deep Convolutional Neural Network architecture inspired from pretrained architectures for the skin cancer detection by incorporating three major heuristics viz. usage of multiple smaller sized convolutional filters instead of using a single larger filter homogeneously and consistently across the entire model, utilization of skip or residual connections to mitigate the vanishing gradient problem in the deeper model, and learning rate annealing by introducing cyclic learning rate. Experimental results performed on HAM10000 dataset observed an improvement in various performance measures and faster model convergence to a significant extent in comparison with the state-of-the-arts. [ABSTRACT FROM AUTHOR]

Context provides additional information to determine the actual emotional state of a person as part of a scene. Existing works on emotion recognition in context focused only on the features extracted from the entire image and the target's body. In this work, we propose a comprehensive multi-cue based emotion recognition framework that incorporates the context, using a hybrid architecture comprised of four separate deep convolutional neural networks and a novel feature fusion mechanism. Each deep network presented in our proposed approach effectively learns the emotion-related features from the facial, body pose, non-target subject and entire image, individually. Experiments on Emotic, an in-painted and a newly constructed EmoSec datasets show that our proposed emotion recognition framework is promising when compared to existing methods in terms of accurately classifying emotions. Comparison with the state-of-the-art deep networks and off-the-shelf fusion techniques demonstrates that our network showed an improved performance. Furthermore, the performance evaluation of our proposed approach on all three datasets confirms that the contextual information influences the emotional state of a human. [ABSTRACT FROM AUTHOR]

Facial expression is an important aspect to recognize emotions between humans. However, this task remains difficult for machines. Several approaches have been developed aiming at strengthening the machine and endowing it, with the ability to decipher and read people's emotions from their faces in order to interact more intelligently. In this context, many deep learning (DL) approaches have been applied due to their outstanding recognition Accuracy. Aiming to gain better performance for facial expression recognition (FER) systems, we propose a hybrid DL architecture based on convolutional neural network and Stacked AutoEncoder. The main idea of this work is to combine the two feature vectors generated by each of these architectures and feed the resulting vector to a meaningful neural network. The architecture of the latter leads to learn each feature by dedicating a set of neurons for each component of the vector before combining them all together in the last layer. The publicly available four facial expression Datasets: Japanse Female Facial Expression (JAFFE), Extended Chon-Kanade (CK+), Facial Expression Recognition 2013 (FER2013) and AffectNet, were used during this research for both training and testing. The experimental results of our proposed architecture are comparable to or better than the relevant state-of-the-art methods in term of Accuracy, Recall, Precision and F-measure. We noted that our proposed approach obtains the best accuracy of 98.65% on the CK+, 95.78% on the JAFFE, 63.14% on the AffectNet and 80.02% on the FER2013 Datasets. [ABSTRACT FROM AUTHOR]

Infrared video colorization can significantly improve perceptual quality by predicting reasonable colors and restoring vivid details, especially in harsh environments. However, as an inconspicuous computer vision task, there is no specialized method. Besides, directly applying current grayscale colorization methods may generate structurally obscure and temporally inconsistent frames. In this paper, we design an infrared video colorization network CPNet aims to generate visually plausible and spatial–temporal consistent colorized videos. To achieve this, a feature fusion module and a hierarchical colorizer are designed to learn the importance of each consecutive frame and the local and global correlation of the integrated features, respectively. In addition, to consolidate the temporal consistency at a fine-grained level, we further introduce a composite loss function to narrow the distance between high-level feature representations while retaining pixel-wise correspondence. Moreover, a new metric named Mean Temporal Variation Similarity (MTVS) is proposed for effectively evaluating the degree of video continuity. Comprehensive experiments conducted on the KAIST dataset demonstrate the superiority of CPNet to produce more authentic colorized videos than state-of-the-art colorization methods. In terms of quantitative comparison, CPNet achieves improvements of at least 0.89 dB on PSNR, 0.016 on SSIM, and a significant promotion on MTVS. In addition, experiments conducted on the DAVIS dataset also prove the applicability of CPNet for grayscale video colorization task. • To the best of our knowledge, CPNet is the first study which dedicated to infrared video colorization and will attract more researchers' attention to this field. • Extensive experimental results verify that CPNet not only demonstrates the ability to predict reasonable colors but retains temporal consistency. • An effective and computationally friendly metric MTVS is introduced for evaluating video continuity, which will serve as the benchmark evaluation protocol. [ABSTRACT FROM AUTHOR]

Video text in real-world scenes often carries rich high-level semantic information and plays an ever-increasingly important role in the content-based video analysis and retrieval. Therefore, the scene video text detection and tracking are important prerequisites of numerous multimedia applications. However, the performance of most existing tracking methods is not satisfactory due to frequent mis-detections, unexpected camera motion and similar appearances between text regions. To address these problems, we propose a new video text tracking approach based on hybrid deep text detection and layout constraint. Firstly, a deep text detection network that combines the advantages of object detection and semantic segmentation in a hybrid way is proposed to locate possible text candidates in individual frames. Then, text trajectories are derived from consecutive frames with a novel data association method, which effectively exploits the layout constraint of text regions in large camera motion. By utilizing the layout constraint, the ambiguities caused by similar text regions are effectively reduced. We conduct experiments on four benchmark datasets, i.e., ICDAR 2015, MSRA-TD 500, USTB-SV1K and Minetto, to evaluate the proposed method. The experimental results demonstrate the effectiveness and superiority of the proposed approach. [ABSTRACT FROM AUTHOR]

Semantic segmentation and depth estimation are two important tasks in computer vision, and many methods have been developed to tackle them. Commonly these two tasks are addressed independently, but recently the idea of merging these two problems into a sole framework has been studied under the assumption that integrating two highly correlated tasks may benefit each other to improve the estimation accuracy. In this paper, depth estimation and semantic segmentation are jointly addressed using a single RGB input image under a unified convolutional neural network. We analyze two different architectures to evaluate which features are more relevant when shared by the two tasks and which features should be kept separated to achieve a mutual improvement. Likewise, our approaches are evaluated under two different scenarios designed to review our results versus single-task and multi-task methods. Qualitative and quantitative experiments demonstrate that the performance of our methodology outperforms the state of the art on single-task approaches, while obtaining competitive results compared with other multi-task methods.

• A new proposition of hybrid architecture for breast cancer detection, • Diagnosis for cancer detection without the need for region of interest (ROI) information, • Breast cancer diagnosis with high classification accuracy, • The proposed method is easy and high performance. In this study, mammography images are classified as normal, benign, and malignant using the Mammographic Image Analysis Society (MIAS) and INbreast datasets. After the preprocessing of each image, the processed images are given as input to two different end-to-end deep networks. The first network contains only a Convolutional Neural Network (CNN), while the second network is a hybrid structure that includes both the CNN and Bidirectional Long Short Term Memories (BiLSTM). The classification accuracy obtained using the first and second hybrid architectures is 97.60% and 98.56% for the MIAS dataset, respectively. In addition, experiments performed for the INbreast dataset at the study's end prove the proposed method's effectiveness. These results are comparable to those obtained in previous popular studies. The proposed study contributes to previous studies in terms of preprocessing steps, deep network design, and high diagnostic accuracy. [ABSTRACT FROM AUTHOR]

Convolutional neural networks (CNNs) have recently achieved impressive performances in image processing tasks such as image classification and object recognition. However, CNNs typically have a large number of parameters, leading to their requirement of a large number of training samples to extract spatial features. To address these limitations, we propose a lightweight ScatterNet with the learnable weight matrix and sparse transformation such as scale transformation and translation to learn sparse filters. This filter based on ScatterNet uses He initialization algorithm and learns from input images which are viewed as two-directional sequential data in the initial stage of model training. A Strip-Recurrent module sweeps both horizontally and vertically across the image to compress feature matrices. Then, ScatterNet decomposes the above feature matrices as a learned mixture of different harmonic functions to integrate the spectral analysis into CNNs. Finally, we combine the sequential and spectral features to build our hybrid architectures to complete image classification and segmentation. These architectures can obtain good classification accuracy on both small and large training datasets. Our proposed method is evaluated at both layer and network levels on five widely-used benchmark datasets: MNIST, CIFAR-10, CIFAR-100, Small NORB and Tiny ImageNet. We also study other small sample problems such as medical image segmentation and image classification based on few-shot learning. Experiments show that our proposed layer and hybrid model achieves better accuracy for small sample training. [ABSTRACT FROM AUTHOR]

Semantic segmentation and depth estimation are two important tasks in computer vision, and many methods have been developed to tackle them. Commonly these two tasks are addressed independently, but recently the idea of merging these two problems into a sole framework has been studied under the assumption that integrating two highly correlated tasks may benefit each other to improve the estimation accuracy. In this paper, depth estimation and semantic segmentation are jointly addressed using a single RGB input image under a unified convolutional neural network. We analyze two different architectures to evaluate which features are more relevant when shared by the two tasks and which features should be kept separated to achieve a mutual improvement. Likewise, our approaches are evaluated under two different scenarios designed to review our results versus single-task and multi-task methods. Qualitative and quantitative experiments demonstrate that the performance of our methodology outperforms the state of the art on single-task approaches, while obtaining competitive results compared with other multi-task methods. [ABSTRACT FROM AUTHOR]