Search

Unsupervised anomaly detection techniques, which operate without prior knowledge of anomalies, have garnered significant attention in industrial inspection due to their adaptability and generalization. Therefore, knowledge-based computer vision techniques have been broadly applied to identify unusual image patterns. However, real-time industrial applications present challenges such as limited anomalous samples, inadequate defect knowledge, and complex background textures. These factors lead to difficulties in accurately identifying defect regions, and conventional auto-encoder networks often struggle to overcome these issues.To address these limitations, we propose a multi-scale feature reconstruction (MSFR) network specifically designed for domain shift scenarios. Our approach employs a pyramidal vision transformer network (PVTN) to reconstruct multi-scale feature maps, capturing discriminative features at various scales. Additionally, a pre-trained module extracts multi-level features at the same scale, and a dedicated feature matching module enhances accuracy by improving the alignment probability between features. The MSFR strategy surpasses conventional auto-encoders by filtering pixel-level information at multiple depths. Empirical evaluations were conducted using benchmark datasets such as MVTec AD and AeBAD-S. Furthermore, an extensive ablation study demonstrates the effectiveness and viability of the proposed MSFR approach for industrial anomaly detection tasks. The experimental results show that the proposed model significantly outperforms recent approaches, making it highly suitable for real-world industrial applications, particularly in manufacturing. [Display omitted] • MSFR is introduced for industrial anomaly detection to handle domain shifts generalization. • A multi-scale features network is developed to efficiently detect defects. • A pyramidal network is proposed to cover the limitations of traditional autoencoders. • Comprehensive experiments are generated over AeBAD-S and MVTec IAD benchmark datasets. [ABSTRACT FROM AUTHOR]

The generation of a large human-labelled facial expression dataset is challenging due to ambiguity in labelling the facial expression class, and annotation cost. However, facial expression recognition (FER) systems demand discriminative feature representation, and require many training samples to establish stronger decision boundaries. Recently, FER approaches have used data augmentation techniques to increase the number of training samples for model generation. However, these augmented samples are derived from existing training data, and therefore have limitations for developing an accurate FER system. To achieve meaningful facial expression representations, we introduce an augmentation technique based on deep learning and genetic algorithms for FER. The proposed approach exploits the hypothesis that augmenting the feature-set is better than augmenting the visual data for FER. By evaluating this relationship, we discovered that the genetic evolution of discriminative features for facial expression is significant in developing a robust FER approach. In this approach, facial expression samples are generated from RGB visual data from videos considering human face frames as regions of interest. The face detected frames are further processed to extract key-frames within particular intervals. Later, these key-frames are convolved through a deep convolutional network for feature generation. A genetic algorithm's fitness function is gauged to select optimal genetically evolved deep facial expression receptive fields to represent virtual facial expressions. The extended facial expression information is evaluated through an extreme learning machine classifier. The proposed technique has been evaluated on five diverse datasets i.e. JAFFE, CK+, FER2013, AffectNet and our application-specific Instructor Facial Expression (IFEV) dataset. Experimentation results and analysis show the promising accuracy and significance of the proposed technique on all these datasets. [ABSTRACT FROM AUTHOR]

The resource-limited nature of underwater vision equipment leads to poor, otherwise low-resolution information affecting the downstream underwater robotics and ocean engineering tasks. Underwater Image Enhancement (UIE) methods have emerged, particularly Super-Resolution (SR), to tackle the aforementioned challenge by restoring the corresponding low-resolution image to a high-quality counterpart. Vision Transformers (ViTs) have recently been employed for SR tasks thanks to their superior performance over mainstream convolution neural networks. The success of ViTs is largely due to their self-attention mechanism; however, they may encounter challenges in dealing with severe and unpredictable degradation in underwater imaging. In contrast, Multi-scale ViTs (MViTs) variants such as the Swin transformers have overcome that challenge by preserving long-range dependencies over multi-scale feature hierarchies through evolving channel capacity. MViTs tend to induce spatial efficiency through classical down-sampling, such as average pooling over key/values, which results in an inevitable loss of high-frequency components. To address this lack, in the current work, we propose a novel algorithm, SwinWave-SR, for efficient and accurate multi-scale SR for underwater images. Our proposed algorithm is based on Swin transformer consisting of a wavelet block to restrict the information drop by downsampling in an invertible fashion. Consequently, the key components are preserved to assist self-attention learning while reducing its computational cost simultaneously. To further complement it, we explore a prominent compression regime, namely the Lottery Ticket Hypothesis (LTH), to discover a lightweight sub-network with competitive performance to its original model by reducing computational costs up to 70.44%. Overall, SwinWave-SR improves peak signal-to-noise ratio (PSNR) by 0.95 dB ∼ 2.23 dB compared to the state-of-the-art SwinIR while reducing the number of parameters by 29.56% and the calculation cost by 18.734%. Experimental results show that the proposed SwinWave-SR method outperforms the state-of-the-art SR methods on four benchmark underwater datasets and significantly improves PSNR and structural similarity index (SSIM). • A new wavelet block is introduced to preserve details in the downsampling process. • We propose a multi-scale enhancement framework for underwater images. • Using self-attention to learn over keys and values with loss-less downsampling. • For SwinWave-SR, we identify winning input tickets in order to induce computational costs. [ABSTRACT FROM AUTHOR]

The estimation of high-quality underwater images is an important step towards the development of computer vision systems in marine environments. This fundamental step contains numerous computer vision and robotics applications including marine exploration, robotics manipulation, navigation, object detection, tracking, and sea life monitoring. However, this pre-processing step becomes more challenging in the presence of back-scattering of underwater particles and attenuation issues, which lead to the formation of hazy underwater images. Vision transformers have recently demonstrated outstanding performance in many computer vision applications. Window-based Transformers (WT) show promising enhancement performance by computing self-attention within non-overlapping local windows. WT has been identified as an essential component in improving representation capabilities; however, it has received less attention in improving the performance of Underwater Image Enhancement (UIE). Therefore, we propose a novel end-to-end Underwater window-based Transformer Generative Adversarial Network (UwTGAN). Our proposed algorithm consists of two main components, including a transformer generator that generates a restored underwater image and a transformer discriminator that classifies the generated underwater image. Both components are equipped with Window-based Self-Attention Blocks (WSABs), which maximize efficiency by limiting self-attention computation to non-overlapping local windows and provide relatively low computational costs. WSAB-based transformer generators and discriminators are trained end-to-end. We formulated an efficient loss function to ensure that the variables are closely integrated. Extensive experimental evaluations are performed on four independent underwater image datasets. Our results demonstrate that the proposed UwTGAN algorithm has outperformed several state-of-the-art UIE methods in terms of both quantitative and qualitative metrics by a significant margin. [ABSTRACT FROM AUTHOR]

The classification of gigapixel Whole Slide Images (WSIs) is an important task in the emerging area of computational pathology. There has been a surge of interest in deep learning models for WSI classification with clinical applications such as cancer detection or prediction of cellular mutations. Most supervised methods require expensive and labor-intensive manual annotations by expert pathologists. Weakly supervised Multiple Instance Learning (MIL) methods have recently demonstrated excellent performance; however, they still require large-scale slide-level labeled training datasets that require a careful inspection of each slide by an expert pathologist. In this work, we propose a fully unsupervised WSI classification algorithm based on mutual transformer learning. The instances (i.e., patches) from gigapixel WSIs are transformed into a latent space and then inverse-transformed to the original space. Using the transformation loss, pseudo labels are generated and cleaned using a transformer label cleaner. The proposed transformer-based pseudo-label generator and cleaner modules mutually train each other iteratively in an unsupervised manner. A discriminative learning mechanism is introduced to improve normal versus cancerous instance labeling. In addition to the unsupervised learning, we demonstrate the effectiveness of the proposed framework for weakly supervised learning and cancer subtype classification as downstream analysis. Extensive experiments on four publicly available datasets show better performance of the proposed algorithm compared to the existing state-of-the-art methods., Competing Interests: Declaration of competing interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (Copyright © 2024 Elsevier B.V. All rights reserved.)

Analysis of the 3-D texture is indispensable for various tasks, such as retrieval, segmentation, classification, and inspection of sculptures, knit fabrics, and biological tissues. A 3-D texture represents a locally repeated surface variation (SV) that is independent of the overall shape of the surface and can be determined using the local neighborhood and its characteristics. Existing methods mostly employ computer vision techniques that analyze a 3-D mesh globally, derive features, and then utilize them for classification or retrieval tasks. While several traditional and learning-based methods have been proposed in the literature, only a few have addressed 3-D texture analysis, and none have considered unsupervised schemes so far. This article proposes an original framework for the unsupervised segmentation of 3-D texture on the mesh manifold. The problem is approached as a binary surface segmentation task, where the mesh surface is partitioned into textured and nontextured regions without prior annotation. The proposed method comprises a mutual transformer-based system consisting of a label generator (LG) and a label cleaner (LC). Both models take geometric image representations of the surface mesh facets and label them as texture or nontexture using an iterative mutual learning scheme. Extensive experiments on three publicly available datasets with diverse texture patterns demonstrate that the proposed framework outperforms standard and state-of-the-art unsupervised techniques and performs reasonably well compared to supervised methods.

Drone-person tracking in uniform appearance crowds poses unique challenges due to the difficulty in distinguishing individuals with similar attire and multi-scale variations. To address this issue and facilitate the development of effective tracking algorithms, we present a novel dataset named D-PTUAC (Drone-Person Tracking in Uniform Appearance Crowd). The dataset comprises 138 sequences comprising over 121 K frames, each manually annotated with bounding boxes and attributes. During dataset creation, we carefully consider 18 challenging attributes encompassing a wide range of viewpoints and scene complexities. These attributes are annotated to facilitate the analysis of performance based on specific attributes. Extensive experiments are conducted using 44 state-of-the-art (SOTA) trackers, and the performance gap between the visual object trackers on existing benchmarks compared to our proposed dataset demonstrate the need for a dedicated end-to-end aerial visual object tracker that accounts the inherent properties of aerial environment., (© 2024. The Author(s).)

Accurate classification of nuclei communities is an important step towards timely treating the cancer spread. Graph theory provides an elegant way to represent and analyze nuclei communities within the histopathological landscape in order to perform tissue phenotyping and tumor profiling tasks. Many researchers have worked on recognizing nuclei regions within the histology images in order to grade cancerous progression. However, due to the high structural similarities between nuclei communities, defining a model that can accurately differentiate between nuclei pathological patterns still needs to be solved. To surmount this challenge, we present a novel approach, dubbed neural graph refinement, that enhances the capabilities of existing models to perform nuclei recognition tasks by employing graph representational learning and broadcasting processes. Based on the physical interaction of the nuclei, we first construct a fully connected graph in which nodes represent nuclei and adjacent nodes are connected to each other via an undirected edge. For each edge and node pair, appearance and geometric features are computed and are then utilized for generating the neural graph embeddings. These embeddings are used for diffusing contextual information to the neighboring nodes, all along a path traversing the whole graph to infer global information over an entire nuclei network and predict pathologically meaningful communities. Through rigorous evaluation of the proposed scheme across four public datasets, we showcase that learning such communities through neural graph refinement produces better results that outperform state-of-the-art methods.

Tissue phenotyping is a fundamental step in computational pathology for the analysis of tumor micro-environment in whole slide images (WSIs). Automatic tissue phenotyping in whole slide images (WSIs) of colorectal cancer (CRC) assists pathologists in better cancer grading and prognostication. In this paper, we propose a novel algorithm for the identification of distinct tissue components in colon cancer histology images by blending a comprehensive learning system with deep features extraction in the current work. Firstly, we extracted the features from the pre-trained VGG19 network which are then transformed into mapped features space for nodes enhancement generation. Utilizing both mapped features and enhancement nodes, the proposed algorithm classifies seven distinct tissue components including stroma, tumor, complex stroma, necrotic, normal benign, lymphocytes, and smooth muscle. To validate our proposed model, the experiments are performed on two publicly available colorectal cancer histology datasets. We showcase that our approach achieves a remarkable performance boost surpassing existing state-of-the-art methods by (1.3% AvTP, 2% F1) and (7% AvTP, 6% F1) on CRCD-1, and CRCD-2, respectively., (© 2023. The Author(s) under exclusive licence to Society for Imaging Informatics in Medicine.)

Accurate and robust visual object tracking is one of the most challenging and fundamental computer vision problems. It entails estimating the trajectory of the target in an image sequence, given only its initial location, and segmentation, or its rough approximation in the form of a bounding box. Discriminative Correlation Filters (DCFs) and deep Siamese Networks (SNs) have emerged as dominating tracking paradigms, which have led to significant progress. Following the rapid evolution of visual object tracking in the last decade, this survey presents a systematic and thorough review of more than 90 DCFs and Siamese trackers, based on results in nine tracking benchmarks. First, we present the background theory of both the DCF and Siamese tracking core formulations. Then, we distinguish and comprehensively review the shared as well as specific open research challenges in both these tracking paradigms. Furthermore, we thoroughly analyze the performance of DCF and Siamese trackers on nine benchmarks, covering different experimental aspects of visual tracking: datasets, evaluation metrics, performance, and speed comparisons. We finish the survey by presenting recommendations and suggestions for distinguished open challenges based on our analysis.

Hypertensive retinopathy (HR) refers to changes in the morphological diameter of the retinal vessels due to persistent high blood pressure. Early detection of such changes helps in preventing blindness or even death due to stroke. These changes can be quantified by computing the arteriovenous ratio and the tortuosity severity in the retinal vasculature. This paper presents a decision support system for detecting and grading HR using morphometric analysis of retinal vasculature, particularly measuring the arteriovenous ratio (AVR) and retinal vessel tortuosity. In the first step, the retinal blood vessels are segmented and classified as arteries and veins. Then, the width of arteries and veins is measured within the region of interest around the optic disk. Next, a new iterative method is proposed to compute the AVR from the caliber measurements of arteries and veins using Parr-Hubbard and Knudtson methods. Moreover, the retinal vessel tortuosity severity index is computed for each image using 14 tortuosity severity metrics. In the end, a hybrid decision support system is proposed for the detection and grading of HR using AVR and tortuosity severity index. Furthermore, we present a new publicly available retinal vessel morphometry (RVM) dataset to evaluate the proposed methodology. The RVM dataset contains 504 retinal images with pixel-level annotations for vessel segmentation, artery/vein classification, and optic disk localization. The image-level labels for vessel tortuosity index and HR grade are also available. The proposed methods of iterative AVR measurement, tortuosity index, and HR grading are evaluated using the new RVM dataset. The results indicate that the proposed method gives superior performance than existing methods. The presented methodology is a novel advancement in automated detection and grading of HR, which can potentially be used as a clinical decision support system., (© 2021. The Author(s) under exclusive licence to Society for Imaging Informatics in Medicine.)

Conventional neural networks have been demonstrated to be a powerful framework for background subtraction in video acquired by static cameras. Indeed, the well-known Self-Organizing Background Subtraction (SOBS) method and its variants based on neural networks have long been the leading methods on the large-scale CDnet 2012 dataset during a long time. Convolutional neural networks, which are used in deep learning, have been recently and excessively employed for background initialization, foreground detection, and deep learned features. The top background subtraction methods currently used in CDnet 2014 are based on deep neural networks, and have demonstrated a large performance improvement in comparison to conventional unsupervised approaches based on multi-feature or multi-cue strategies. Furthermore, since the seminal work of Braham and Van Droogenbroeck in 2016, a large number of studies on convolutional neural networks applied to background subtraction have been published, and a continual gain of performance has been achieved. In this context, we provide the first review of deep neural network concepts in background subtraction for novices and experts in order to analyze this success and to provide further directions. To do so, we first surveyed the background initialization and background subtraction methods based on deep neural networks concepts, and also deep learned features. We then discuss the adequacy of deep neural networks for the task of background subtraction. Finally, experimental results are presented for the CDnet 2014 dataset., (Copyright © 2019 Elsevier Ltd. All rights reserved.)

Moving object detection is a fundamental step in various computer vision applications. Robust Principal Component Analysis (RPCA) based methods have often been employed for this task. However, the performance of these methods deteriorates in the presence of dynamic background scenes, camera jitter, camouflaged moving objects, and/or variations in illumination. It is because of an underlying assumption that the elements in the sparse component are mutually independent, and thus the spatiotemporal structure of the moving objects is lost. To address this issue, we propose a spatiotemporal structured sparse RPCA algorithm for moving objects detection, where we impose spatial and temporal regularization on the sparse component in the form of graph Laplacians. Each Laplacian corresponds to a multi-feature graph constructed over superpixels in the input matrix. We enforce the sparse component to act as eigenvectors of the spatial and temporal graph Laplacians while minimizing the RPCA objective function. These constraints incorporate a spatiotemporal subspace structure within the sparse component. Thus, we obtain a novel objective function for separating moving objects in the presence of complex backgrounds. The proposed objective function is solved using a linearized alternating direction method of multipliers based batch optimization. Moreover, we also propose an online optimization algorithm for real-time applications. We evaluated both the batch and online solutions using six publicly available datasets that included most of the aforementioned challenges. Our experiments demonstrated the superior performance of the proposed algorithms compared with the current state-of-the-art methods.