Search

To enable fast reliable feature matching or tracking in scenes, features need to be discrete and meaningful, and hence edge or corner features, commonly called interest points are often used for this purpose. Experimental research has illustrated that biological vision systems use neuronal circuits to extract particular features such as edges or corners from visual scenes. Inspired by this biological behaviour, this paper proposes a biologically inspired spiking neural network for the purpose of image feature extraction. Standard digital images are processed and converted to spikes in a manner similar to the processing that transforms light into spikes in the retina. Using a hierarchical spiking network, various types of biologically inspired receptive fields are used to extract progressively complex image features. The performance of the network is assessed by examining the repeatability of extracted features with visual results presented using both synthetic and real images. [ABSTRACT FROM AUTHOR]

Highlights • Research based on the study of the retina, particularly the modelling of ganglion cells. • Artificial white noise used as input, both full field and checkerboard flicker. • Alternative models to the standard linear-nonlinear model are presented. • Performance increase indicated for various machine learning methods. Abstract Artificial vision using computational models that can mimic biological vision is an area of ongoing research. One of the main themes within this research is the study of the retina and in particular, retinal ganglion cells which are responsible for encoding the visual stimuli. A common approach to modelling the internal processes of retinal ganglion cells is the use of a linear – non-linear cascade model, which models the cell's response using a linear filter followed by a static non-linearity. However, the resulting model is generally restrictive as it is often a poor estimator of the neuron's response. In this paper we present an alternative to the linear – non-linear model by modelling retinal ganglion cells using a number of machine learning techniques which have a proven track record for learning complex non-linearities in many different domains. A comparison of the model predicted spike rate shows that the machine learning models perform better than the standard linear – non-linear approach in the case of temporal white noise stimuli. [ABSTRACT FROM AUTHOR]

Deep learning methods for salient object detection (SOD) have been studied actively and promisingly. However, it is still challenging for the studies with two aspects. The first one is a single type of label from the network to convey limit information, which leads to the poor generalization ability of the network. The second one is the difficulty to improve the accuracy and detect details of target. To address these challenges, we develop a novel approach via joint weakly supervised, unsupervised and supervised learning for SOD (WUSL–SOD), which differs from existing methods just based on ground-truth or other sparse labels. Specifically, to optimize the objective of the image, the unsupervised learning module (ULM) is designed to generate coarse saliency feature and suppress background noises via attention guiding mechanism. Then, we propose the weakly supervised learning module (WLM) based on scribbles for producing relatively accurate saliency feature. Note that this structure is used to enhance the details and remedy the deficiency of scribbles in WLM. For further refining information from the ULM and WLM, we propose a supervised learning module (SLM), which is not only applied to process and refine information from the ULM and WLM, but also enhance the image details and capture the entire target area. Furthermore, we also exchange information between the SLM and the WLM to obtain more accurate saliency maps. Extensive experiments on five datasets demonstrate that the proposed approach can effectively outperform the state-of-the-art approaches and achieve real-time. [ABSTRACT FROM AUTHOR]

For one-shot segmentation, prototype learning is extensively used. However, using only one RGB prototype to represent all information in the support image may lead to ambiguities. To this end, we propose a one-shot segmentation network based on multi-modal prototype learning that uses depth information to complement RGB information. Specifically, we propose a multi-modal fusion and refinement block (MFRB) and multi-modal prototype learning block (MPLB). MFRB fuses RGB and depth features to generate multi-modal features and refined depth features, which are used by MPLB, to generate multi-modal information prototypes, depth information prototypes, and global information prototypes. Furthermore, we introduce self-attention to capture global context information in RGB and depth images. By integrating self-attention, MFRB, and MPLB, we propose the multi-modal prototype learning network (MMPL-Net), which adapts to the ambiguity of visual information in the scene. Finally, we construct a one-shot RGB-D segmentation dataset called OSS-RGB-D-5 i . Experiments using OSS-RGB-D-5 i show that our proposed method outperforms several state-of-the-art techniques with fewer labeled images and generalizes well to previously unseen objects. [ABSTRACT FROM AUTHOR]

Glaciers have been retreating over the last century as a result of climate change, particularly in the Arctic, causing sea levels to rise, affecting coastal communities and potentially changing global weather and climate patterns. In this study, we mapped 2203 glaciers in Novaya Zemlya (Russian Arctic), Penny Ice Cap (Baffin Island), Disko Island (Qeqertarsuaq, Greenland) and part of Kenai (Alaska), using Object-Based Image Analysis (OBIA) applied to multispectral Landsat satellite imagery in Google Earth Engine (GEE) to quantify the glacier area changes over three decades. Between 1985–89 and 2019–21, the results show that the overall glacier area loss in Novaya Zemlya is 1319 ± 419 km² (5.7% of area), 452 ± 227 km2 (6.6%) for Penny Ice Cap, 457 ± 168 km² (23.6%) in Disko Island and 196 ± 84 km2 (25.7%) in Kenai. A total of seventy-three glaciers have disappeared completely, including sixty-nine on Disko Island, three in Novaya Zemlya and one in Kenai. [ABSTRACT FROM AUTHOR]

In this paper, a data assimilation network is proposed to tackle the challenges of domain generalization for person re-identification (ReID). Most of the existing research efforts only focus on single-dataset issues, and the trained models are difficult to generalize to unseen scenarios. This paper presents a distinctive idea to improve the generality of the model by assimilating three types of images: style-variant images, misaligned images and unlabeled images. The latter two are often ignored in the previous domain generalization ReID studies. In this paper, a non-local convolutional block attention module is designed for assimilating the misaligned images, and an attention adversary network is introduced to correct it. A progressive augmented memory is designed for assimilating the unlabeled images by progressive learning. Moreover, we propose an attention adversary difference loss for attention correction, and a labeling-guide discriminative embedding loss for progressive learning. Rather than designing a specific feature extractor that is robust to style shift as in most previous domain generalization work, we propose a data assimilation meta-learning procedure to train the proposed network, so that it learns to assimilate style-variant images. It is worth mentioning that we add an unlabeled augmented dataset to the source domain to tackle the domain generalization ReID tasks. Extensive experiments demonstrate that our approach significantly outperforms the state-of-the-art domain generalization methods. [ABSTRACT FROM AUTHOR]

Visual place recognition (VPR) in environments subject to extreme appearance variation due to changing weather, illumination or seasons is a challenging task. Recent works have shown that features learned from CNNs can achieve promising performance. However, most of the existing methods concentrate so much on the image itself that they neglect the architecture of the network, especially different filters that may carry more meaningful information. In this paper, we develop a learnable feature map filtering (FMF) module constrained by triplet loss to re-calibrate the weight of the individual feature map. In this way, specific feature maps that encode invariant characteristics of location are extracted. Moreover, to make full use of the rich global mutual information that resides in the sample set, we propose an influence-based graph attention network (IB-GAT) with a verification subnet to better incorporate the relations among samples during the training process. Different from conventional GAT approaches, IB-GAT enables feature nodes to attend over the influence of other nodes instead of the original feature. Thus refined features with more discriminative power could be generated. Extensive experiments have been conducted on six public VPR datasets with varying appearances. Ablation analysis verifies the potential efficacy of the FMF module and the IB-GAT components. The experimental results also demonstrate that the proposed methods can achieve better performance than the current state of the art. [ABSTRACT FROM AUTHOR]

Metal defect inspection is critical for maintaining product quality and ensuring production safety. However, the vast majority of existing defect segmentation methods rely heavily on large-scale datasets that only cater to specific defects, making them unsuitable for the industrial sector, where training samples are often limited. To address these challenges, we propose a bilateral guidance network for one-shot metal defect segmentation that leverages the perceptual consistency of background regions within industrial images to distinguish foreground and background regions. Our model uses an interactive feature reweighting scheme that models the inter- and self-dependence of foreground and background feature maps, enabling us to build robust pixel-level correspondences. Our proposed method demonstrates good domain adaptability and accurately segments defects in multiple materials, such as steel, leather, and carpet, among others. Additionally, we have incorporated a multi-scale receptive field encoder to enhance the model's ability to perceive objects of varying scales, providing a comprehensive solution for industrial defect segmentation. Experimental results indicate that our proposed method has the potential to be effective in a variety of real-world applications where defects may not be immediately visible or where large amounts of labeled data are not readily available. With only one shot, our method achieves the state-of-the-art performance of 41.62% mIoU and 70.30% MPA on the Defect-3 i dataset. [ABSTRACT FROM AUTHOR]

Person re-identification plays an important role in searching for a specific person in a camera network with non-overlapping cameras. The most critical problem for re-identification is feature representation. In this paper, a multi-level cross-view consistent feature learning framework is proposed for person re-identification. First, local deep, LOMO and SIFT features are extracted to form multi-level features. Specifically, local features from the lower and higher layers of a convolutional neural network (CNN) are extracted, these features complement each other as they extract apparent and semantic properties. Second, an ID-based cross-view multi-level dictionary learning (IDB-CMDL) is carried out to obtain sparse and discriminant feature representation. Third, a cross-view consistent word learning is performed to get the cross-view consistent BoVW histograms from sparse feature representation. Finally, a multi-level metric learning fuses multiple BoVW histograms, and learns the sample distance in the subspace for ranking. Experiments on the public CUHK03, Market1501, and DukeMTMC-ReID datasets show results that are superior to many state-of-the-art methods for person re-identification. [ABSTRACT FROM AUTHOR]

Autism spectrum disorder is an umbrella term for a group of neurodevelopmental disorders that is associated with impairments to social interaction, communication, and behaviour. Typically, autism spectrum disorder is first detected with a screening tool (e.g. modified checklist for autism in toddlers). However, the interpretation of autism spectrum disorder behavioural symptoms varies across cultures: the sensitivity of modified checklist for autism in toddlers is as low as 25 per cent in Sri Lanka. A culturally sensitive screening tool called pictorial autism assessment schedule has overcome this problem. Low- and middle-income countries have a shortage of mental health specialists, which is a key barrier for obtaining an early autism spectrum disorder diagnosis. Early identification of autism spectrum disorder enables intervention before atypical patterns of behaviour and brain function become established. This article proposes a culturally sensitive autism spectrum disorder screening mobile application. The proposed application embeds an intelligent machine learning model and uses a clinically validated symptom checklist to monitor and detect autism spectrum disorder in low- and middle-income countries for the first time. Machine learning models were trained on clinical pictorial autism assessment schedule data and their predictive performance was evaluated, which demonstrated that the random forest was the optimal classifier (area under the receiver operating characteristic (0.98)) for embedding into the mobile screening tool. In addition, feature selection demonstrated that many pictorial autism assessment schedule questions are redundant and can be removed to optimise the screening process. [ABSTRACT FROM AUTHOR]

Weakly supervised salient object detection (SOD) is a challenging task and has drawn much attention from several research perspectives, it has revealed two problems while driving the rapid development of saliency detection. (1) Large divergence in the characteristics of saliency regions in terms of location, shape and size makes them difficult to recognize. (2) The properties of convolutional neural networks dictate that it is insensitive to various transformations, which will lead to hardly balance the application of various disturbances. To tackle these limitations, this paper proposes a novel seminar learning framework with consistent transformation ensembling (SLF-CT) for scribble supervised SOD. The framework consists of the teacher–student model and the student–student model for segmenting the salient objects. Specifically, we first design a cross attention guided network (CAGNet) as a baseline model for saliency prediction. Then we assign CAGNet to the teacher–student model, where the teacher network is based on the exponential moving average and guides the training of the student network. Moreover, we adopt multiple pseudo labels to transfer the information among students from different conditions. To further enhance the regularization of the network, a consistency transformation mechanism is also incorporated, which encourages the saliency prediction and input image of the network to be consistent. The experimental results demonstrate that the proposed approach performs favorably comparable with the state-of-the-art weakly supervised methods. As far as we know, the proposed approach is the first application of seminar learning in the SOD area. [ABSTRACT FROM AUTHOR]

Salient object detection (SOD) is a challenging and fundamental research in computer vision and image processing. Since the cost of pixel-level annotations is high, scribble annotations are usually used as weak supervisions. However, scribble annotations are too sparse and always located inside the objects with lacking annotations close to the semantic boundaries, which can't make confident predictions. To alleviate these issues, we propose a novel and effective scribble-based weakly supervised approach for SOD, named complementary characteristics fusion network (CCFNet). To be more specific, we design an edge fusion module (EFM) by taking account of local and high-level semantic information to equip our model, which would be beneficial to enhance the power of aggregating edge information. Then to achieve the complementary role of different features, a series of feature correlation modules (FCMs) are employed to strengthen the localization information and details learning. This is based on low-level, high-level global and edge information, which will complement each other to obtain relatively complete salient regions. Alternatively, to encourage the network to learn structural information and further improve the results of saliency maps in foreground and background, we propose a self-supervised salient detection (SSD) loss. Extensive experiments using five benchmark datasets demonstrate that our proposed approach performs favorably against the state-of-the-art weakly supervised algorithms, and even surpasses the performance of those fully supervised. • We propose an edge fusion module using the local and high-level semantic information. • Feature correlation module is employed to make full use of the complementary different features. • We propose a self-supervised salient detection loss to learn structural information. • The proposed method performs competitive against the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Most of existing salient object detection models are based on fully convolutional network (FCN), which learn multi-scale/level semantic information through convolutional layers to obtain high-quality predicted saliency maps. However, convolution is locally interactive, it is difficult to capture remote dependencies, and FCN-based methods suffer from coarse object boundaries. In this paper, to solve these problems, we propose a novel transformer framework for salient object detection (named TF-SOD), which mainly consists of the encoder part of the FCN, fusion module (FM), transformer module (TM) and feature decoder module (FDM). Specifically, FM is a bridge connecting the encoder and TM and provides some foresight for the non-local interaction of TM. Besides, FDM can efficiently decode the non-local features output by TM and achieve deep fusion with local features. This architecture enables the network to achieve a close integration of local and non-local interactions, making information complementary to each other, deeply mining the associated information between features. Furthermore, we also propose a novel edge reinforcement learning strategy, which can effectively suppress edge blurring from local and global aspects by means of powerful network architecture. Extensive experiments using five datasets demonstrate that the proposed method outperforms 19 state-of-the-art methods [ABSTRACT FROM AUTHOR]

Retail shoplifting is one of the most prevalent forms of theft and has accounted for over one billion GBP in losses for UK retailers in 2018. An automated approach to detecting behaviours associated with shoplifting using surveillance footage could help reduce these losses. Until recently, most state-of-the-art vision-based approaches to this problem have relied heavily on the use of black box deep learning models. While these models have been shown to achieve very high accuracy, this lack of understanding on how decisions are made raises concerns about potential bias in the models. This limits the ability of retailers to implement these solutions, as several high-profile legal cases have recently ruled that evidence taken from these black box methods is inadmissible in court. There is an urgent need to develop models which can achieve high accuracy while providing the necessary transparency. One way to alleviate this problem is through the use of social signal processing to add a layer of understanding in the development of transparent models for this task. To this end, we present a social signal processing model for the problem of shoplifting prediction which has been trained and validated using a novel dataset of manually annotated shoplifting videos. The resulting model provides a high degree of understanding and achieves accuracy comparable with current state of the art black box methods. [ABSTRACT FROM AUTHOR]

The role of 5G-IoT has become indispensable in smart applications and it plays a crucial part in e-health applications. E-health applications require intelligent schemes and architectures to overcome the security threats against the sensitive data of patients. The information in e-healthcare applications is stored in the cloud which is vulnerable to security attacks. However, with deep learning techniques, these attacks can be detected, which needs hybrid models. In this article, a new deep learning model (CNN-DMA) is proposed to detect malware attacks based on a classifier—Convolution Neural Network (CNN). The model uses three layers, i.e., Dense, Dropout, and Flatten. Batch sizes of 64, 20 epoch, and 25 classes are used to train the network. An input image of 32 × 32 × 1 is used for the initial convolutional layer. Results are retrieved on the Malimg dataset where 25 families of malware are fed as input and our model has detected is Alueron.gen!J malware. The proposed model CNN-DMA is 99% accurate and it is validated with state-of-the-art techniques. [ABSTRACT FROM AUTHOR]

Although methods based on the fully convolutional neural networks (FCNs) have shown strong advantages in the field of salient object detection, the existing methods still have two challenging issues: insufficient multi-level feature fusion ability and boundary blur. To overcome these issues, we propose a novel salient object detection method based on a multi-feature fusion cross network (denoted MFC-Net). Firstly, to overcome the issue of insufficient multi-level feature fusion ability, inspired by the connection mode of human brain neurons, we propose a novel cross network framework, combined with contextual feature transfer modules (CFTMs) to integrate, enhance and transmit multi-level feature information in an iterative manner. Secondly, to address the issue of blurred boundaries, we effectively enhance the edge features of saliency map by a simple edge enhancement strategy. Thirdly, to reduce the loss of information caused by the saliency map generated by multi-level feature fusion, we use feature fusion modules (FFMs) to learn contextual feature information from multiple angles and then output the resulting saliency map. Finally, a hybrid loss function fully supervises the network at the pixel and object level, optimizing the network performance. The proposed MFC-Net has been evaluated using five benchmark datasets. The performance evaluation demonstrates that the proposed method outperforms other state-of-the-art methods, which proves the superiority of MFC-Net approach. [ABSTRACT FROM AUTHOR]

Underwater images are important carriers and forms of underwater information, playing a vital role in exploring and utilizing marine resources. However, underwater images have characteristics of low contrast and blurred details because of the absorption and scattering of light. In recent years, deep learning has been widely used in underwater image enhancement and restoration because of its powerful feature learning capabilities, but there are still shortcomings in detailed enhancement. To address the problem, this paper proposes a deep supervised residual dense network (DS_RD_Net), which is used to better learn the mapping relationship between clear in-air images and synthetic underwater degraded images. DS_RD_Net first uses residual dense blocks to extract features to enhance feature utilization; then, it adds residual path blocks between the encoder and decoder to reduce the semantic differences between the low-level features and high-level features; finally, it employs a deep supervision mechanism to guide network training to improve gradient propagation. Experiments results (PSNR was 36.2, SSIM was 96.5%, and UCIQE was 0.53) demonstrated that the proposed method can fully retain the local details of the image while performing color restoration and defogging compared with other image enhancement methods, achieving good qualitative and quantitative effects. [ABSTRACT FROM AUTHOR]

Place recognition is an essential component to address the problem of visual navigation and SLAM. The long-term place recognition is challenging as the environment exhibits significant variations across different times of the days, months, and seasons. In this paper, we view appearance changes as multiple domains and propose a Feature Disentanglement Network (FDNet) based on a convolutional auto-encoder and adversarial learning to extract two independent deep features — content and appearance. In our network, the content feature is learned which only retains the content information of images through the competition with the discriminators and content encoder. Besides, we utilize the triplets loss to make the appearance feature encode the appearance information. The generated content features are directly used to measure the similarity of images without dimensionality reduction operations. We use datasets that contain extreme appearance changes to carry out experiments, which show how meaningful recall at 100% precision can be achieved by our proposed method where existing state-of-art approaches often get worse performance. • A novel Feature Disentanglement Network (FDNet) for vPR. • Appearance and content factors are disentangled through adversarial training. • Content features provide place recognition robust to extreme condition changes. • Disentangled features are appearance-invariant and show discriminative ability. [ABSTRACT FROM AUTHOR]

• A novel densely connection inception convolutional neural network based on U-Net architecture is proposed for medical image segmentation tasks. • The modified Inception-res module combining inception architecture and residual connection is used to make the proposed network deeper and wider. • The densely connection is used in the network to avoid gradient vanishing or redundant computation during network training. • Apply the proposed network to CT and MRI medical segmentation tasks and make evaluation with other segmentation methods. Convolutional neural networks (CNNs) play an important role in the field of medical image segmentation. Among many kinds of CNNs, the U-net architecture is one of the most famous fully convolutional network architectures for medical semantic segmentation tasks. Recent work shows that the U-net network can be substantially deeper thus resulting in improved performance on segmentation tasks. Though adding more layers directly into network is a popular way to make a network deeper, it may lead to gradient vanishing or redundant computation during training. A novel CNN architecture is proposed that integrates the Inception-Res module and densely connecting convolutional module into the U-net architecture. The proposed network model consists of the following parts: firstly, the Inception-Res block is designed to increase the width of the network by replacing the standard convolutional layers; secondly, the Dense-Inception block is designed to extract features and make the network more deep without additional parameters; thirdly, the down-sampling block is adopted to reduce the size of feature maps to accelerate learning and the up-sampling block is used to resize the feature maps. The proposed model is tested on images of blood vessel segmentations from retina images, the lung segmentation of CT Data from the benchmark Kaggle datasets and the MRI scan brain tumor segmentation datasets from MICCAI BraTS 2017. The experimental results show that the proposed method can provide better performance on these two tasks compared with the state-of-the-art algorithms. The results reach an average Dice score of 0.9857 in the lung segmentation. For the blood vessel segmentation, the results reach an average Dice score of 0.9582. For the brain tumor segmentation, the results reach an average Dice score of 0.9867. The experiments highlighted that combining the inception module with dense connections in the U-Net architecture is a promising approach for semantic medical image segmentation. [ABSTRACT FROM AUTHOR]