Search

Estimating the size of animal populations plays an important role in evidence‐based conservation and management. Some methods for estimating population size rely on animals being individually identifiable. Traditionally, this has been done by marking physically captured animals, but increasingly, animals with distinctive natural markings are surveyed noninvasively using cameras. Animal reidentification from photographs is usually done manually, which is expensive, laborious, and requires considerable skill. An alternative is to develop computer vision methods that can support or replace the manual identification task. We developed an automated approach using deep learning to identify whether a pair of photographs is of the same individual or not. The core of the approach is a similarity learning network that uses paired convolutional neural networks with a triplet loss function to summarize image pairs and decide whether they are from the same individual. Prior to the main matching step, two additional convolutional neural networks perform image segmentation, cropping the animal object within the image, and orientation prediction, deciding which side of the animal was photographed. We applied the approach to four species, with images of the same individual often spanning several years: systematic surveys of bottlenose dolphins (Tursiops truncatus, 2008–2019) and harbor seals (Phoca vitulina, 2015–2019), a citizen science dataset of western leopard toads (Sclerophrys pantherina, unknown dates), and a publicly available repository of humpback whale images (Megaptera novaeangliae, unknown dates). For these species, our best‐performing models were able to identify whether a pair of images were from the same individual or different individuals in 95.8%, 94.6%, 88.2%, and 83.8% of the cases, respectively. We found that triplet loss functions outperformed binary cross‐entropy loss functions and that data augmentation and additional manual curation of training data provided small but consistent improvements in performance. These results demonstrate the potential of deep learning to replace or, more likely, support and facilitate manual individual identification efforts. [ABSTRACT FROM AUTHOR]

In recent years, deep learning-based bearing fault diagnosis methods have made significant achievements. However, these methods only work with single faults and cannot diagnose compound faults because compound fault data is often unavailable in practice. To address this problem, this paper proposes a zero-shot learning-based bearing fault diagnosis method for compound defects. The proposed method utilizes an autoencoder network to observe the attributes of single faults and then estimates the attributes of compound faults. Afterward, a mapping from the data space to the attribute space is established to predict the attribute output of the data. The attribute output is then compared with prior attributes to determine the type of bearing fault. Verification experiments were conducted on HUST bearing dataset. The experimental results showed that the proposed method achieved a high accuracy of 75.64 % in diagnosing compound bearing faults. [ABSTRACT FROM AUTHOR]

How to effectively utilize unlabeled data for training is a key problem in Semi-Supervised Learning (SSL). Existing SSL methods often consider the unlabeled data whose predictions are beyond a fixed threshold (e.g., 0.95) and discard those less than 0.95. We argue that these discarded data have a large proportion, are of hard sample, and will benefit the model training if used properly. In this article, we propose a novel method to take full advantage of the unlabeled data, termed DTS-SimL, which includes two core designs: Dual-Threshold Screening and Similarity Learning. Except for the fixed threshold, DTS-SimL extracts another class-adaptive threshold from the labeled data. Such a class-adaptive threshold can screen many unlabeled data whose predictions are lower than 0.95 but over the extracted one for model training. On the other hand, we design a new similar loss to perform similarity learning for all the highly similar unlabeled data, which can further mine the valuable information from the unlabeled data. Finally, for more effective training of DTS-SimL, we construct an overall loss function by assigning four different losses to four different types of data. Extensive experiments are conducted on five benchmark datasets, including CIFAR-10, CIFAR-100, SVHN, Mini-ImageNet, and DomainNet-Real. Experimental results show that the proposed DTS-SimL achieves state-of-the-art classification accuracy. The code is publicly available at https://github.com/GZHU-DVL/DTS-SimL. [ABSTRACT FROM AUTHOR]

Feature selection is a basic and important step in real applications, such as face recognition and image segmentation. In this paper, we propose a new weakly supervised multi-view feature selection method by utilizing pairwise constraints, i.e., the pairwise constraint-guided multi-view feature selection (PCFS for short) method. In this method, linear projections of all views and a consistent similarity graph with pairwise constraints are jointly optimized to learning discriminative projections. Meanwhile, the l 2 , 0 -norm-based row sparsity constraint is imposed on the concatenation of projections for discriminative feature selection. Then, an iterative algorithm with theoretically guaranteed convergence is developed for the optimization of PCFS. The performance of the proposed PCFS method was evaluated by comprehensive experiments on six benchmark datasets and applications on cancer clustering. The experimental results demonstrate that PCFS exhibited competitive performance in feature selection in comparison with related models. [ABSTRACT FROM AUTHOR]

Fashion image retrieval is one of the important services of e-commerce platforms, and it is also the basis of various fashion-related AI applications. Studies have shown that in a multi-modal environment (images + attribute labels), embedding items into specific attribute spaces can support more fine-grained similarity measures, which is especially suitable for fashion retrieval tasks. In this paper, we propose an attention-based attribute-guided similarity learning network (AttnFashion) for fashion image retrieval. The core of this network is an attribute-guided spatial attention module and an attribute-guided channel attention module, which correspond to the mapping between attributes and image regions, and the mapping between attributes and high-level image semantics, respectively. To make these two modules interact deeply, we design a parallel structure that allows them to share attribute embeddings and guide each other to extract specific features, which also helps to reduce the network parameters of the attention modules. An adaptive feature fusion strategy is proposed to synthesise the features extracted by the two modules. Extensive experiments show that the proposed AttnFashion performs better than current competitive networks in the field of fine-grained attribute-based fashion retrieval. [ABSTRACT FROM AUTHOR]

Abstract Estimating the size of animal populations plays an important role in evidence‐based conservation and management. Some methods for estimating population size rely on animals being individually identifiable. Traditionally, this has been done by marking physically captured animals, but increasingly, animals with distinctive natural markings are surveyed noninvasively using cameras. Animal reidentification from photographs is usually done manually, which is expensive, laborious, and requires considerable skill. An alternative is to develop computer vision methods that can support or replace the manual identification task. We developed an automated approach using deep learning to identify whether a pair of photographs is of the same individual or not. The core of the approach is a similarity learning network that uses paired convolutional neural networks with a triplet loss function to summarize image pairs and decide whether they are from the same individual. Prior to the main matching step, two additional convolutional neural networks perform image segmentation, cropping the animal object within the image, and orientation prediction, deciding which side of the animal was photographed. We applied the approach to four species, with images of the same individual often spanning several years: systematic surveys of bottlenose dolphins (Tursiops truncatus, 2008–2019) and harbor seals (Phoca vitulina, 2015–2019), a citizen science dataset of western leopard toads (Sclerophrys pantherina, unknown dates), and a publicly available repository of humpback whale images (Megaptera novaeangliae, unknown dates). For these species, our best‐performing models were able to identify whether a pair of images were from the same individual or different individuals in 95.8%, 94.6%, 88.2%, and 83.8% of the cases, respectively. We found that triplet loss functions outperformed binary cross‐entropy loss functions and that data augmentation and additional manual curation of training data provided small but consistent improvements in performance. These results demonstrate the potential of deep learning to replace or, more likely, support and facilitate manual individual identification efforts.

This article studies deep unsupervised hashing, which has attracted increasing attention in large-scale image retrieval. The majority of recent approaches usually reconstruct semantic similarity information, which then guides the hash code learning. However, they still fail to achieve satisfactory performance in reality for two reasons. On the one hand, without accurate supervised information, these methods usually fail to produce independent and robust hash codes with semantics information well preserved, which may hinder effective image retrieval. On the other hand, due to discrete constraints, how to effectively optimize the hashing network in an end-to-end manner with small quantization errors remains a problem. To address these difficulties, we propose a novel unsupervised hashing method called HARR to learn discriminative and high-quality hash codes. To comprehensively explore semantic similarity structure, HARR adopts the Winner-Take-All hash to model the similarity structure. Then similarity-preserving hash codes are learned under the reliable guidance of the reconstructed similarity structure. Additionally, we improve the quality of hash codes by a bit correlation reduction module, which forces the cross-correlation matrix between a batch of hash codes under different augmentations to approach the identity matrix. In this way, the generated hash bits are expected to be invariant to disturbances with minimal redundancy, which can be further interpreted as an instantiation of the information bottleneck principle. Finally, for effective hashing network training, we minimize the cosine distances between real-value network outputs and their binary codes for small quantization errors. Extensive experiments demonstrate the effectiveness of our proposed HARR. [ABSTRACT FROM AUTHOR]

Pressurized fluid pipelines are among the most crucial components in industrial settings. Operating under high pressure leads to pipeline susceptibility to cracking, rupture, and significant damage. Monitoring the condition and predicting the remaining useful life before the failure of pressurized pipes are essential for informed and timely maintenance decisions. In this work, we propose a novel method for predicting the remaining useful life of pressurized pipelines based on acoustic emission monitoring and similarity-based learning. Specifically, acoustic emission sensors are deployed to record acoustic emission events caused by cracks in the pipeline. A pipeline health indicator is proposed based on accumulated events detected through a constant false alarm rate signal detector. Leveraging the historical run-to-fail trajectories of the health indicator, a similarity measure is introduced to predict the remaining pipeline life. This method computes the similarity between the current health indicator trajectory and past trajectories based on a Euclidean distance in the proposed derivative convolutional domain. Trajectory similarity determines the remaining lifetime similarity, which is weighted using data-adaptive fuzzy rules to estimate the current remaining useful life. Elaborate experimental validations are conducted on a custom pressurized pipeline system in a laboratory setting. Experimental results demonstrate the high efficacy of the proposed method in predicting the remaining useful life of the pipeline, surpassing other commonly used methods in both accuracy and certainty.

The attention-based models are widely used for time series data. However, due to the quadratic complexity of attention regarding input sequence length, the application of Transformers is limited by high resource demands. Moreover, their modifications for industrial time series need to be robust to missing or noisy values, which complicates the expansion of their application horizon. To cope with these issues, we introduce the class of efficient Transformers named Regularized Transformers (Reguformers). We implement the regularization technique inspired by the dropout ideas to improve robustness and reduce computational expenses without significantly modifying the pipeline. The focus in our experiments is on oil&gas data. For well-interval similarity task, our best Reguformer configuration reaches ROC AUC 0.97, which is comparable to Informer (0.978) and outperforms baselines: the previous LSTM model (0.934), the classical Transformer model (0.967), and three recent most promising modifications of the original Transformer, namely, Performer (0.949), LRformer (0.955), and DropDim (0.777). We also conduct the corresponding experiments on three additional datasets from different domains and obtain superior results. The increase in the quality of the best Reguformer relative to Transformer for different datasets varies from 3.7% to 9.6%, while the increase range relative to Informer is wider: from 1.7% to 18.4%.

Scene-level change detection (SLCD) can provide semantic change information at image level, thus it is of great significance for monitoring land use changes. Supervised SLCD approaches tend to outperform unsupervised ones. However, the existing supervised methods rely on postclassification, which often results in unsatisfactory performance due to classification error accumulation. We therefore formulate SLCD as a similarity learning task, and propose a scene similarity learning network (SSLN) for land use SLCD. To be specific, SSLN is a two-branch network with ResNet as the backbone for feature extraction, where the global feature difference and the multiscale local feature fusion modules are considered in order to better mine the temporal correlation between bitemporal scenes. Then, the trained SSLN is further exploited to obtain the similarity of scene pairs for determining the similarity threshold via threshold traversal algorithm. Finally, the land use scenes are categorized into changed or unchanged by comparing scene similarity with the threshold. Experimental results on the publicly available MtS-WH dataset and our newly released land use scene change detection dataset show that the proposed approach achieves better performance than comparison methods, indicating that our approach is a simple yet effective solution to land use SLCD.

Abstract Cartography is a technique for creating maps, which are graphical representations of spatial information. Traditional cartography involves the creation of geographical data, such as locations of countries, geographical features of mountains, rivers, and oceans, and celestial objects. However, cartography has recently been utilized to display various data, such as antigenic signatures, graphically. Hence, it is natural to consider a new cartography that can flexibly deal with various data types. This study proposes a model of Siamese topological neural networks consisting of a pair of hierarchical neural networks, each with a low-dimensional internal layer for creating context-flexible maps. The proposed Siamese topological neural network transfers high-dimensional data with various contexts into their low-dimensional spatial representations on a map that humans can use to gain insights from the data. Here, it is enough to define a metric of difference between an arbitrary pair of data instances for training the proposed neural network. As the metric can be arbitrarily defined, the proposed neural network realizes context-flexible cartography useful for visual data analysis. This paper applies the proposed network for visualizing various demographic data.

High dimension, low sample size (HDLSS) problems are numerous among real-world applications of machine learning. From medical images to text processing, traditional machine learning algorithms are usually unsuccessful in learning the best possible concept from such data. In a previous work, we proposed a dissimilarity-based approach for multi-view classification, the random forest dissimilarity, that perfoms state-of-the-art results for such problems. In this work, we transpose the core principle of this approach to solving HDLSS classification problems, by using the RF similarity measure as a learned precomputed SVM kernel (RFSVM). We show that such a learned similarity measure is particularly suited and accurate for this classification context. Experiments conducted on 40 public HDLSS classification datasets, supported by rigorous statistical analyses, show that the RFSVM method outperforms existing methods for the majority of HDLSS problems and remains at the same time very competitive for low or non-HDLSS problems. [ABSTRACT FROM AUTHOR]

Cartography is a technique for creating maps, which are graphical representations of spatial information. Traditional cartography involves the creation of geographical data, such as locations of countries, geographical features of mountains, rivers, and oceans, and celestial objects. However, cartography has recently been utilized to display various data, such as antigenic signatures, graphically. Hence, it is natural to consider a new cartography that can flexibly deal with various data types. This study proposes a model of Siamese topological neural networks consisting of a pair of hierarchical neural networks, each with a low-dimensional internal layer for creating context-flexible maps. The proposed Siamese topological neural network transfers high-dimensional data with various contexts into their low-dimensional spatial representations on a map that humans can use to gain insights from the data. Here, it is enough to define a metric of difference between an arbitrary pair of data instances for training the proposed neural network. As the metric can be arbitrarily defined, the proposed neural network realizes context-flexible cartography useful for visual data analysis. This paper applies the proposed network for visualizing various demographic data. [ABSTRACT FROM AUTHOR]

Defect detection has achieved state-of-the-art results in both localisation and classification of various types of defects, manufacturing domains is no exception to this. Just like in many areas of computer vision there is an assume of very high-quality datasets that have been verified by domain experts, however labelling such data has become an increasing problem as we require greater quantities of it. Within defect detection the variability and composite nature of defect characteristics makes this a time-consuming and interaction-heavy task with great amount of expert effort. We propose a new acquisition function based on the similarity of defect properties for refining labels over time by showing the expert only the most required to be labelled. We also explore different ways in which the expert labels defects and how we should feed these new refinements back into the model for utilising new knowledge in an effortful way. We achieve this with a graphical interface that provides additional information as data gets refined into a dense segmentation, allowing for decision-making with uncertain areas of the image. [ABSTRACT FROM AUTHOR]

Multi-view clustering is extensively applied to practical applications. The similarity matrix acquired by most of the existing approaches is obtained by using original multi-view data. However, when dealing with data in a nonlinear subspace, the results of existing methods are not satisfactory. In addition, the existing methods cannot solve the high order relevance of multi-view data. To solve these problems, we present a novel multi-view subspace similarity learning method, MSSLt-SVD, on the basis of tensor singular value decomposition (t-SVD). First, we map each view of the data to the Hilbert space through a Gaussian kernel and then minimize the reconstruction error of the obtained kernel matrix. Second, we use the t-SVD-based tensor nuclear norm (TNN) instead of the matrix-kernel norm as the regularization term to capture the high-order relevance of multi-view data. Then, we incorporate these two steps into a framework and design the corresponding goal function, which can be solved by using the augmented lagrange multiplier (ALM) method. Experiments on some datasets show that the performance of MSSLt-SVD algorithm is better than some representative ones. [ABSTRACT FROM AUTHOR]

Feature selection is a basic and important step in real applications, such as face recognition and image segmentation. In this paper, we propose a new weakly supervised multi-view feature selection method by utilizing pairwise constraints, i.e., the pairwise constraint-guided multi-view feature selection (PCFS for short) method. In this method, linear projections of all views and a consistent similarity graph with pairwise constraints are jointly optimized to learning discriminative projections. Meanwhile, the l2,0-norm-based row sparsity constraint is imposed on the concatenation of projections for discriminative feature selection. Then, an iterative algorithm with theoretically guaranteed convergence is developed for the optimization of PCFS. The performance of the proposed PCFS method was evaluated by comprehensive experiments on six benchmark datasets and applications on cancer clustering. The experimental results demonstrate that PCFS exhibited competitive performance in feature selection in comparison with related models.

Diabetes affects roughly 537 million people, and is predicted to reach 783 million by 2045. Diabetes Foot Ulcer (DFU) is a major complication associated with diabetes and can lead to lower limb amputation. The rapid evolution of diabetic foot ulcers (DFUs) necessitates immediate intervention to prevent the severe consequences of amputation and related complications. Continuous and meticulous patient monitoring for individuals with diabetic foot ulcers (DFU) is crucial and is currently carried out by medical practitioners on a daily basis. This research article introduces DFU-Helper, a novel framework that employs a Siamese Neural Network (SNN) for accurate and objective assessment of the progression of diabetic foot ulcers (DFUs) over time. DFU-Helper provides healthcare professionals with a comprehensive visual and numerical representation in terms of the similarity distance of the disease, considering five distinct disease conditions: none, infection, ischemia, both (presence of ischemia and infection), and healthy. The SNN achieves the best Macro F1-score of 0.6455 on the test dataset when applying pseudo-labeling with a pseudo-threshold set to 0.9. The SNN is used in the process of creating anchors for each class using feature vectors. When a patient initially consults a healthcare professional, an image is transmitted to the model, which computes the distances from each class anchor point. It generates a comprehensive table with corresponding figures and a visually intuitive radar chart. In subsequent visits, another image is captured and fed into the model alongside the initial image. DFU-Helper then plots both images and presents the distances from the class anchor points. Our proposed system represents a significant advancement in the application of deep learning for the longitudinal assessment of DFU. To the best of our knowledge, no existing tool harnesses deep learning for DFU follow-up in a comparable manner. [ABSTRACT FROM AUTHOR]

Many people wonder, when they look at fashion models on social media or on television, whether they could look like them by wearing similar products. Furthermore, many people suffer when they sometimes find fashion models in e-commerce, and they want to obtain similar products, but after clicking on the fashion model, they receive unwanted products or products for the opposite gender. To address these issues, in our work, we built a multi-task learning and gender-aware fashion recommendation system (MLGFRS). The proposed MLGFRS can increase the revenue of the e-commerce fashion market. Moreover, we realized that people are accustomed to clicking on that part of the fashion model, which includes the product they want to obtain. Therefore, we classified the query image into many cropped products to detect the user's click. What makes this paper novel is that we contributed to improving the efficiency performance by detecting the gender from the query image to reduce the retrieving time. Second, we effectively improved the quality of results by retrieving similarities for each object in the query image to recommend the most relevant products. The MLGFRS consists of four components: gender detection, object detection, similarity generation, and recommendation results. The MLGFRS achieves better performance compared to the state-of-the-art baselines. [ABSTRACT FROM AUTHOR]

• We propose a fast and differentiable approximation of DTW by comparing two architectures. • Our method achieves at least the same level of accuracy as other DTW main approximations with higher computational efficiency. • It can be used to learn in an end-to-end setting on long time series by proposing generative models of EEGs. Dynamic Time Warping (DTW) is a widely used algorithm for measuring similarities between two time series. It is especially valuable in a wide variety of applications, such as clustering, anomaly detection, classification, or video segmentation, where the time series have different timescales, are irregularly sampled, or are shifted. However, it is not prone to be considered as a loss function in an end-to-end learning framework because of its non-differentiability and its quadratic temporal complexity. While differentiable variants of DTW have been introduced by the community, they still present some drawbacks: computing the distance is still expensive, and this similarity tends to blur some differences in the time series. In this paper, we propose a fast and differentiable approximation of DTW by comparing two architectures: the first one aims to learn an embedding in which the Euclidean distance mimics the DTW, and the second one to directly predict the DTW output value using regression. We build the former by training a siamese neural network to regress the DTW value between two time series. Depending on the nature of the activation function, this approximation naturally supports differentiation, and it is efficient to compute. We show, in a time series retrieval context on EEG datasets, that our methods achieve at least the same level of accuracy as other DTW main approximations with higher computational efficiency. We also show that it can be used to learn in an end-to-end setting on long time series by proposing generative models of EEGs. [ABSTRACT FROM AUTHOR]

As AIS data play an increasingly important role in intelligent shipping and shipping regulation, research on AIS trajectories has attracted more attention. Effective measurement is a critical issue in AIS trajectory research. It directly impacts downstream research areas such as anomaly detection, trajectory clustering, and trajectory prediction. However, the extremely time-consuming and labor-intensive traditional pairwise methods for calculating different types of distances between trajectories hinders the large-scale application and further analysis of AIS data. To tackle these issues, we introduce AISim—a metric learning framework that utilizes heterogeneous graph neural networks. This framework includes a spatial pre-training graph and a hierarchical heterogeneous graph, which incorporate spatial and sequential dependency to extract latent features from vessel trajectories. This approach enhances the model's ability to capture a more accurate representation of the trajectories and approximate various similarity measurements. Extensive experiments on multiple real trajectory datasets have verified the effectiveness and generality of the proposed framework. AISim outperforms advanced learning-based models by 5% to 66% on the HR10 metric in top-k search tasks. The experimental results demonstrate that the proposed framework facilitates research on AIS trajectory similarity learning, thereby promoting the development of AIS trajectory analysis. [ABSTRACT FROM AUTHOR]

The representation learning problem in the oil and gas industry aims to construct a model that provides a representation based on logging data for a well interval. Previous attempts are mainly supervised and focus on similarity task, which estimates closeness between intervals. We desire to build informative representations without using supervised (labeled) data. One of the possible approaches is self-supervised learning (SSL). In contrast to the supervised paradigm, this one requires little or no labels for the data. Nowadays, most SSL approaches are either contrastive or noncontrastive. Contrastive methods make representations of similar (positive) objects closer and distancing different (negative) ones. Due to possible wrong marking of positive and negative pairs, these methods can provide an inferior performance. Noncontrastive methods do not rely on such labeling and are widespread in computer vision. They learn using only pairs of similar objects that are easier to identify in logging data. We are the first to introduce noncontrastive SSL for well-logging data. In particular, we exploit Bootstrap Your Own Latent (BYOL) and Barlow Twins methods that avoid using negative pairs and focus only on matching positive pairs. The crucial part of these methods is an augmentation strategy. Our augmentation strategies and adaption of BYOL and Barlow Twins together allow us to achieve superior quality on clusterization and mostly the best performance on different classification tasks. Our results prove the usefulness of the proposed noncontrastive self-supervised approaches for representation learning and interval similarity in particular. [ABSTRACT FROM AUTHOR]

The learning similarity between users and points of interests (POIs) is an important function in location-based social networks (LBSN), which could primarily benefit multiple location-based services, especially in terms of POI recommendation. As one of the well-known recommender technologies, Collaborative Filtering (CF) has been employed to a great extent in literature, due to its simplicity and interpretability. However, it is facing a great challenge in generating accurate similarities between users or items, because of data sparsity. Traditional similarity measures which rely on explicit user feedback (e.g., rating) are not applicable for implicit feedback (e.g., check-ins). In this study, we propose multi-criteria user–user and POI–POI similarity measures, based on the dominance concept. In this regard, we incorporate geographical, temporal, social, preferential and textual criteria into the similarity measures of CF. We are interested in measuring POI similarity from a location perspective, by taking into account the influence of the dominance concept on multiple dimensions of POIs. To evaluate the effectiveness of our method, a series of experiments are conducted with a large-scale real dataset, collected from the Foursquare of two cities in terms of POI recommendation. Experimental results revealed that the proposed method significantly outperforms the existing state-of-the-art alternatives. A further experiment demonstrates the superiority of the proposed method in alleviating sparsity and handling the cold-start problem affecting POI recommendation. [ABSTRACT FROM AUTHOR]

Learning the similarity between fashion items is essential for many fashion-related tasks. Most methods based on global or local image similarity cannot meet the fine-grained retrieval requirements related to attributes. We are the first to clearly distinguish the concepts of attribute name and their values and divide fashion retrieval tasks that combine images and text into: attribute-guided retrieval and attribute-manipulated retrieval. We propose a hierarchical attribute-aware embedding network (HAEN) that takes images and attributes as input, learns multiple attribute-specific embedding spaces, and measures fine-grained similarity in the corresponding spaces. It can accurately map different attributes to the corresponding areas of the image, thereby facilitating the feature fusion of two different modalities of text and image, including enhancement and replacement. Then on this basis, we propose three attribute-manipulated similarity learning methods, HAEN_Avg, HAEN_Rec, and HAEN_Cmb. With comprehensive validation on two real-world fashion datasets, we demonstrate that our methods can effectively leverage semantic knowledge to improve image retrieval performance, including attribute-guided and attribute-manipulated retrieval tasks. [ABSTRACT FROM AUTHOR]

Due to practical demands and substantial potential benefits, there is growing interest in fashion image retrieval with attribute manipulation. For example, if a user wants a product similar to a query image and has the attribute "3/4 sleeves" instead of "short sleeves" he can modify the query image by entering text. Unlike general items, fashion items are rich in categories and attributes, and some items with different attributes have only very subtle differences in vision. Moreover, the visual appearance of fashion items changes dramatically under different conditions, such as lighting, viewing angle, and occlusion. These pose challenges to the fashion retrieval task. Therefore, we consider learning an attribute-specific space for each attribute to obtain discriminative features. In this paper, we propose a dual attention composition network for image retrieval with manipulation, which addresses two important issues, where to focus and how to modify. The dual attention module aims to capture fine-grained image-text alignment through corresponding spatial and channel attention and then satisfy multi-modal composition through corresponding affine transformation. The TIRG-based semantic composition module combines the query image's attention features and the manipulation text's embedding features to obtain a synthetic representation close to the target image. Meanwhile, we investigate the semantic hierarchy of attributes and propose a hierarchical encoding method, which can preserve the associations between attributes for efficient feature learning. Extensive experiments conducted on three multi-modal fashion-related retrieval datasets demonstrate the superiority of our network. [ABSTRACT FROM AUTHOR]

Predictive modeling in buildings is a key task for the optimal management of building energy. Relevant building operational data are a prerequisite for such task, notably when deep learning is used. However, building operational data are not always available, such is the case in newly built, newly renovated, or even not yet built buildings. To address this problem, we propose a deep similarity learning approach to recommend relevant training data to a target building solely by using a minimal contextual description on it. Contextual descriptions are modeled as user queries. We further propose to ensemble most used machine learning algorithms in the context of predictive modeling. This contributes to the genericity of the proposed methodology. Experimental evaluations show that our methodology offers a generic methodology for cross-building predictive modeling and achieves good generalization performance. [ABSTRACT FROM AUTHOR]

Driven by multi-omics data, some multi-view clustering algorithms have been successfully applied to cancer subtypes prediction, aiming to identify subtypes with biometric differences in the same cancer, thereby improving the clinical prognosis of patients and designing personalized treatment plan. Due to the fact that the number of patients in omics data is much smaller than the number of genes, multi-view spectral clustering based on similarity learning has been widely developed. However, these algorithms still suffer some problems, such as over-reliance on the quality of pre-defined similarity matrices for clustering results, inability to reasonably handle noise and redundant information in high-dimensional omics data, ignoring complementary information between omics data, etc. This paper proposes multi-view spectral clustering with latent representation learning (MSCLRL) method to alleviate the above problems. First, MSCLRL generates a corresponding low-dimensional latent representation for each omics data, which can effectively retain the unique information of each omics and improve the robustness and accuracy of the similarity matrix. Second, the obtained latent representations are assigned appropriate weights by MSCLRL, and global similarity learning is performed to generate an integrated similarity matrix. Third, the integrated similarity matrix is used to feed back and update the low-dimensional representation of each omics. Finally, the final integrated similarity matrix is used for clustering. In 10 benchmark multi-omics datasets and 2 separate cancer case studies, the experiments confirmed that the proposed method obtained statistically and biologically meaningful cancer subtypes. [ABSTRACT FROM AUTHOR]

Clustering approaches based on similarity learning have achieved good results, but they still have the following problems: (1) these approaches generally learn similar expressions on the original data, thereby disregarding the nonlinear structure of the data; (2) these methods generally do not consider the consistency and high-order relevance among multi-view data; and (3) these approaches generally use the learned similarity matrix for clustering, usually not achieving the optimal effect. To resolve the above issues, we present a new approach referred to as consensus similarity learning based on tensor nuclear norm. First, to address the first problem, we map the data of each view to the Hilbert space to discover the nonlinear structure of the data. Second, to address the second problem, we introduce the tensor nuclear norm to constrain the regularization term, and then, the consistency and high-order relevance among multi-view data can be captured. Third, to address the third problem, i.e., to obtain a better clustering effect, we learn a clustering indicator matrix in the kernel space instead of a similarity matrix for clustering by using a consensus representation term. Last, we incorporate these three steps into a unified framework and design the corresponding goal function. In addition, experimental outcomes on some datasets show that our algorithm is superior to certain representative approaches. [ABSTRACT FROM AUTHOR]

Accurate clustering of oil wells is important for lithological studies and hydrocarbon production processes. Traditional methods for solving this problem require large amount of expert work, including careful analysis of high-dimensional datasets. Modern approaches to automatizing clustering are mostly based on deep neural networks (DNN) with complex architecture, which require significant training time and lack interpretability. This paper analyses methods based on simple similarity heuristics, which stem from reasonable assumptions on data generating process and can be interpreted in terms of statistics and geometry. For dataset labeled by experts, clustering is performed by means of computed heuristics, and the quality of algorithm is measured by Adjusted Rand Index (ARI). Results thus obtained turn out to be comparable to those of modern DNN models (0.41 vs 0.37), but the computation time is reduced significantly. For some types of heuristics physical interpretation is suggested and approaches to obtaining geological insights are studied. [ABSTRACT FROM AUTHOR]

Industrial materials images are an important application domain for content-based image retrieval. Users need to quickly search databases for images that exhibit similar appearance, properties, and/or features to reduce analysis turnaround time and cost. The images in this study are 2D images of millimeter-scale rock samples acquired at micrometer resolution with light microscopy or extracted from 3D micro-CT scans. Labeled rock images are expensive and time-consuming to acquire and thus are typically only available in the tens of thousands. Training a high-capacity deep learning (DL) model from scratch is therefore not practicable due to data paucity. To overcome this "few-shot learning" challenge, we propose leveraging pretrained common DL models in conjunction with transfer learning. The "similarity" of industrial materials images is subjective and assessed by human experts based on both visual appearance and physical qualities. We have emulated this human-driven assessment process via a physics-informed neural network including metadata and physical measurements in the loss function. We present a novel DL architecture that combines Siamese neural networks with a loss function that integrates classification and regression terms. The networks are trained with both image and metadata similarity (classification), and with metadata prediction (regression). For efficient inference, we use a highly compressed image feature representation, computed offline once, to search the database for images similar to a query image. Numerical experiments demonstrate superior retrieval performance of our new architecture compared with other DL and custom-featurebased approaches. [ABSTRACT FROM AUTHOR]

As a popular technique for swapping faces with someone else's in images or videos through deep neural networks, deepfake causes a serious threat to the security of multimedia content today. However, because counterfeit images are usually compressed when propagating over the Internet, and because the compression factor used is unknown, most of the existing deepfake detection models have poor robustness for the detection of compressed images with unknown compression factors. To solve this problem, we notice that an image has a high similarity with its compressed image based on symmetry, and this similarity is not easily affected by the compression factor, so this similarity feature can be used as an important clue for compressed deepfake detection. A TCNSC (Two-branch Convolutional Networks with Similarity and Classifier) method that combines compression factor independence is proposed in this paper. The TCNSC method learns two feature representations from the deepfake image, i.e., similarity of the image and its compressed counterpart and authenticity of the deepfake image. A joint training strategy is then utilized for feature extraction, in which the similarity characteristics are obtained by similarity learning while obtaining authenticity characteristics, so the proposed TCNSC model is trained for robust feature learning. Experimental results on the FaceForensics++ (FF++) dataset show that the proposed method significantly outperforms all competing methods under three compression settings of high-quality (HQ), medium-quality (MQ), and low-quality (LQ). For the LQ, MQ, and HQ settings, TCNSC achieves 91.8%, 93.4%, and 95.3% in accuracy, and outperforms the state-of-art method (Xception-RAW) by 16.9%, 10.1%, and 4.1%, respectively. [ABSTRACT FROM AUTHOR]

Diabetes affects roughly 537 million people, and is predicted to reach 783 million by 2045. Diabetes Foot Ulcer (DFU) is a major complication associated with diabetes and can lead to lower limb amputation. The rapid evolution of diabetic foot ulcers (DFUs) necessitates immediate intervention to prevent the severe consequences of amputation and related complications. Continuous and meticulous patient monitoring for individuals with diabetic foot ulcers (DFU) is crucial and is currently carried out by medical practitioners on a daily basis. This research article introduces DFU-Helper, a novel framework that employs a Siamese Neural Network (SNN) for accurate and objective assessment of the progression of diabetic foot ulcers (DFUs) over time. DFU-Helper provides healthcare professionals with a comprehensive visual and numerical representation in terms of the similarity distance of the disease, considering five distinct disease conditions: none, infection, ischemia, both (presence of ischemia and infection), and healthy. The SNN achieves the best Macro F1-score of 0.6455 on the test dataset when applying pseudo-labeling with a pseudo-threshold set to 0.9. The SNN is used in the process of creating anchors for each class using feature vectors. When a patient initially consults a healthcare professional, an image is transmitted to the model, which computes the distances from each class anchor point. It generates a comprehensive table with corresponding figures and a visually intuitive radar chart. In subsequent visits, another image is captured and fed into the model alongside the initial image. DFU-Helper then plots both images and presents the distances from the class anchor points. Our proposed system represents a significant advancement in the application of deep learning for the longitudinal assessment of DFU. To the best of our knowledge, no existing tool harnesses deep learning for DFU follow-up in a comparable manner.

Deep metric learning (DML) is a cornerstone of many computer vision applications. It aims at learning a mapping from the input domain to an embedding space, where semantically similar objects are located nearby and dissimilar objects far from another. The target similarity on the training data is defined by user in form of ground-truth class labels. However, while the embedding space learns to mimic the user-provided similarity on the training data, it should also generalize to novel categories not seen during training. Besides user-provided groundtruth training labels, a lot of additional visual factors (such as viewpoint changes or shape peculiarities) exist and imply different notions of similarity between objects, affecting the generalization on the images unseen during training. However, existing approaches usually directly learn a single embedding space on all available training data, struggling to encode all different types of relationships, and do not generalize well. We propose to build a more expressive representation by jointly splitting the embedding space and the data hierarchically into smaller sub-parts. We successively focus on smaller subsets of the training data, reducing its variance and learning a different embedding subspace for each data subset. Moreover, the subspaces are learned jointly to cover not only the intricacies, but the breadth of the data as well. Only after that, we build the final embedding from the subspaces in the conquering stage. The proposed algorithm acts as a transparent wrapper that can be placed around arbitrary existing DML methods. Our approach significantly improves upon the state-of-the-art on image retrieval, clustering, and re-identification tasks evaluated using CUB200-2011, CARS196, Stanford Online Products, In-shop Clothes, and PKU VehicleID datasets. [ABSTRACT FROM AUTHOR]

As a special case in content-based image retrieval, fabric retrieval has high potential application value in many fields. However, fabric retrieval has higher requirements for results, which makes it difficult for common retrieval methods to be directly applied to fabric retrieval. It is also a challenging issue with several obstacles: variety and complexity of fabric appearance, and high requirements for retrieval accuracy. To address this issue, this paper presents a novel method for fabric image retrieval based on soft similarity and pairwise learning. First, a soft similarity between two fabric images is defined to describe their relationship. Then, a convolutional neural network with compact structure and cross-domain connections is designed to learn the fabric image representation. Finally, listwise learning is introduced to train the convolutional neural network model and hash function. The generated hash codes are used to index the fabric image. The experiments are conducted on a wool fabric dataset. The experimental results show that the newly proposed method has a greater improvement than our previous work. [ABSTRACT FROM AUTHOR]

The type of neural networks widely used in artificial intelligence applications mixes its computation and memory modules in neuron weights and activities. The previously learned information are stored in network weights. When dealing with complex data, e.g., those possessing diverse content or containing long-sequences, some information stored in the weights can be altered drastically or wiped as the training goes, but they are not necessarily unimportant. External memory is a recent technique proposed to prevent from forgetting significant previously learned information. In this work, we aim at taking advantage of this recent technique to advance the similarity learning task that is critical in many real-world artificial intelligence applications. We propose suitable external memory design supported by extended attention mechanism. Two different kinds of memory modules are proposed so that the similarity learning process can dynamically shift focus over a wide range of diverse content contained by the training data. Effectiveness of the proposed method is demonstrated through evaluations based on different image retrieval tasks and compared against various state-of-the-art algorithms in the field. [ABSTRACT FROM AUTHOR]

Crowdsourcing learning (Bonald and Combes 2016; Dawid and Skene, J R Stat Soc: Series C (Appl Stat), 28(1):20–28 1979; Karger et al. 2011; Li et al, IEEE Trans Knowl Data Eng, 28(9):2296–2319 2016; Liu et al. 2012; Schlagwein and Bjorn-Andersen, J Assoc Inform Syst, 15(11):3 2014; Zhang et al. 2014) plays an increasingly important role in the era of big data (Liu et al., IEEE Trans Syst Man Cybern: Syst, 48(12): 451–2461, 2017; Zhang et al. 2014) due to its ability to easily solve large-scale data annotations (Musen et al., J Amer Med Informs Assoc, 22(6):1148–1152 2015). However, in the process of crowdsourcing learning, the uneven knowledge level of workers often leads to low accuracy of the label after marking, which brings difficulties to the subsequent processing (Edwards and Teddy 2013) and analysis of crowdsourcing data. In order to solve this problem, this paper proposes a two-step learning crowdsourced data classification algorithm, which optimizes the original label data by simultaneously considering the two issues of different worker abilities and the similarity between crowdsourced data (Kasikci et al. 2013) samples, so as to get more accurate label data. The two-step learning algorithm mainly includes two steps. Firstly, the worker's ability to label different samples is obtained by constructing and training the worker's ability model, and then the similarity between samples is calculated by the cosine measurement method (Muflikhah and Baharudin 2009), and finally the original label data is optimized by combining the above two results. The experimental results also show that the two-step learning classification algorithm proposed in this article has achieved better experimental results than the comparison algorithm. [ABSTRACT FROM AUTHOR]