Search

In many practical applications data used for training a machine learning model and the deployment data does not always preserve the same distribution. Transfer learning and, in particular, domain adaptation allows to overcome this issue, by adapting the source model to a new target data distribution and therefore generalizing the knowledge from source to target domain. In this work, we present a method that makes the adaptation process more transparent by providing two complementary explanation mechanisms. The first mechanism explains how the source and target distributions are aligned in the latent space of the domain adaptation model. The second mechanism provides descriptive explanations on how the decision boundary changes in the adapted model with respect to the source model. Along with a description of a method, we also provide initial results obtained on publicly available, real-life dataset. © The Author(s) 2024., Funding: The paper is funded from the XPM project funded by the National Science Centre, Poland under the CHIST-ERA programme (NCN UMO2020/02/Y/ST6/00070) and the Swedish Research Council under grant CHIST-ERA19-XAI-012 and by a grant from the Priority Research Area (DigiWorld) under the Strategic Programme Excellence Initiative at Jagiellonian University.

Data-driven methods have been gaining increasing attention; however, along with the benefits they offer, they also present several challenges, particularly concerning data availability, accessibility, and heterogeneity, the three factors that have shaped the development of this thesis. Data availability is the primary consideration when employing data-driven methodologies. Suppose we consider a system for which we aim to develop a Machine Learning (ML) model. Gathering labeled samples, particularly in the context of real-world problem-solving, consistently poses challenges. While collecting raw data may be feasible in certain situations, the process of labeling them is often difficult, leading to a shortage of labeled data. However, historical (outdated) data or labeled data may occasionally be available from different yet related systems. A feasible approach would be to leverage data from different but related sources to assist in situations in which data is scarce. The challenge with this approach is that data collected from various sources may exhibit statistical differences even if they have the same features, i.e., data heterogeneity. Data heterogeneity impacts the performance of ML models. This issue arises because conventional machine learning algorithms assume what’s known as the IID (Independently and Identically Distributed) assumption; training and test data come from the same underlying distribution and are independent and identically sampled. The IID assumption may not hold when data comes from different sources and can result in a trained model performing less effectively when used in another system or context. In such situations, Domain Adaptation (DA) is a solution. DA enhances the performance of ML models by minimizing the distribution distance between samples originating from diverse resources. Several factors come into play within the DA context, each necessitating distinct DA methods. In this thesis, we conduct an investigation and propose DA met

Intelligent fault diagnosis (IFD) based on deep learning methods has shown excellent performance, however, the fact that their implementation requires massive amount of data and lack of sufficient labeled data, limits their real-world application. In this paper, we propose a two-step technique to extract fault discriminative features using unlabeled and a limited number of labeled samples for classification. To this end, we first train an Autoencoder (AE) using unlabeled samples to extract a set of potentially useful features for classification purpose and consecutively, a Contrastive Learning-based post-training is applied to make use of limited available labeled samples to improve the feature set discriminability. Our Experiments—on SEU bearing dataset—show that unsupervised feature learning using AEs improves classification performance. In addition, we demonstrate the effectiveness of the employment of contrastive learning to perform the post-training process; this strategy outperforms Cross-Entropy based post-training in limited labeled information cases. © The Author(s) 2023.

Federated Learning (FL) is a setting where different clients collaboratively train a Machine Learning model in a privacy-preserving manner, i.e., without the requirement to share data. Given the importance of security and privacy in real-world applications, FL is gaining popularity in many areas, including predictive maintenance. For example, it allows independent companies to construct a model collaboratively. However, since different companies operate in different environments, their working conditions may differ, resulting in heterogeneity among their data distributions. This paper considers the fault identification problem and simulates different scenarios of data heterogeneity. Such a setting remains challenging for popular FL algorithms, and thus we demonstrate the considerations to be taken into account when designing federated predictive maintenance solutions.

The performance versus interpretability trade-off has been well-established in the literature for many years in the context of machine learning models. This paper demonstrates its twin, namely the data-centric performance versus interpretability trade-off. In a case study of bearing fault diagnosis, we found that substituting the original acceleration signal with a demodulated version offers a higher level of interpretability, but it comes at the cost of significantly lower classification performance. We demonstrate these results on two different datasets and across four different machine learning algorithms. Our results suggest that “there is no free lunch,” i.e., the contradictory relationship between interpretability and performance should be considered earlier in the analysis process than it is typically done in the literature today; in other words, already in the preprocessing and feature extraction step. © 2023, The Author(s), under exclusive license to Springer Nature Switzerland AG.

Recently and due to the advances in sensor technology and Internet-of-Things, the operation of machinery can be monitored, using a higher number of sources and modalities. In this study, we demonstrate that Multi-Modal Translation is capable of transferring knowledge from a modality with higher level of applicability (more usefulness to solve an specific task) but lower level of accessibility (how easy and affordable it is to collect information from this modality) to another one with higher level of accessibility but lower level of applicability. Unlike the fusion of multiple modalities which requires all of the modalities to be available during the deployment stage, our proposed method depends only on the more accessible one; which results in the reduction of the costs regarding instrumentation equipment. The presented case study demonstrates that by the employment of the proposed method we are capable of replacing five acceleration sensors with three current sensors, while the classification accuracy is also increased by more than 1%.

A fundamental assumption in machine learning is that training and test samples follow the same distribution. Therefore, for training a machine learning-based network traffic classifier, it is necessary to use samples obtained from the desired network. Collecting enough training data, however, can be challenging in many cases. Domain adaptation allows samples from other networks to be utilized. In order to satisfy the aforementioned assumption, domain adaptation reduces the distance between the distribution of the samples in the desired network and that of the available samples in other networks. However, it is important to note that the applications in two different networks can differ considerably. Taking this into account, in this paper, we present a new domain adaptation method for classifying network traffic. Thus, we use the labeled samples from a network and adapt them to the few labeled samples from the desired network; In other words, we adapt shared applications while preserving the information about non-shared applications. In order to demonstrate the efficacy of our method, we construct five different cross-network datasets using the Brazil dataset. These results indicate the effectiveness of adapting samples between different domains using the proposed method.

Domain adaptation (DA) transfers knowledge between domains by adapting them. The most well-known DA scenario in the literature is adapting two domains of source and target using the available labeled source samples to construct a model generalizable to the target domain. Although the primary purpose of DA is to compensate for the target domain’s labeled data shortage, the concept of adaptation can be utilized to solve other problems. One issue that may occur during adaptation is the problem of class misalignment, which would result in a negative transfer. Therefore, preventing negative transfer should be considered while designing DA methods. In addition, the sample availability in domains is another matter that should also be taken into account. Considering the two mentioned matters, this thesis aims to develop DA techniques to solve primary predictive maintenance problems. This thesis considers a spectrum of cases with different amounts of available target data. One endpoint is the case in which we have access to enough labeled target samples for all classes. In this case, we use the concept of DA for 1) Analyzing two different physical properties, i.e., vibration and current, to measure their robustness for fault identification and 2) Developing a denoising method to construct a robust model for a noisy test environment. Next, we consider the case where we have access to unlabeled and a few labeled target samples. Using the few labeled samples available, we aim to prevent negative transfer while adapting source and target domains. To achieve this, we construct a unified features representation using a few-shot and an adaptation learning technique. In the subsequent considered setting, we assume we only have access to very few labeled target samples, which are insufficient to train a domain-specific model. Furthermore, for the first time in the literature, we solve the DA for regression in a setting in which it adapts multiple domains with any arbitrary shift. Som

Noise will be unavoidably present in the data collected from physical environments, regardless of how sophisticated the measurement equipment is. Furthermore, collecting enough faulty data is a challenge since operating industrial machines in faulty modes not only has severe consequences to the machine health, but also may affect collateral machinery critically, from health state point of view. In this paper, we propose a method of denoising with limited data for the purpose of fault identification. In addition, our method is capable of removing multiple levels of noise simultaneously. For this purpose, inspired by unsupervised contrastive learning, we first augment the data with multiple levels of noise. Later, we construct a new feature representation using Contrastive Loss. The last step is building a classifier on top of the learned representation; this classifier can detect various faults in noisy environments. The experiments on the SOUTHEAST UNIVERSITY (SEU) dataset of bearings confirm that our method can simultaneously remove multiple noise levels.

Selecting the physical property capable of representing the health state of a machine is an important step in designing fault detection systems. In addition, variation of the loading condition is a challenge in deploying an industrial predictive maintenance solution. The robustness of the physical properties to variations in loading conditions is, therefore, an important consideration. In this paper, we focus specifically on squirrel cage induction motors and analyze the capabilities of three-phase current and five vibration signals acquired from different locations of the motor for the detection of Broken Rotor Bar generated in different loads. In particular, we examine the mentioned signals in relation to the performance of classifiers trained with them. Regarding the classifiers, we employ deep conventional classifiers and also propose a hybrid classifier that utilizes contrastive loss in order to mitigate the effect of different variations. The analysis shows that vibration signals are more robust under varying load conditions. Furthermore, the proposed hybrid classifier outperforms conventional classifiers and is able to achieve an accuracy of 90.96% when using current signals and 97.69% when using vibration signals.

In most industries, the working conditions of equipment vary significantly from one site to another, from one time of a year to another, and so on. This variation poses a severe challenge for data-driven fault identification methods: it introduces a change in the data distribution. This contradicts the underlying assumption of most machine learning methods, namely that training and test samples follow the same distribution. Domain Adaptation (DA) methods aim to address this problem by minimizing the distribution distance between training (source) and test (target) samples. However, in the area of predictive maintenance, this idea is complicated by the fact that different classes – fault categories – also vary across domains. Most of the state-of-the-art DA methods assume that the data in the target domain is complete, i.e., that we have access to examples from all the possible classes or faulty categories during adaptation. In reality, this is often very difficult to guarantee. Therefore, there is a need for a domain adaptation method that is able to align the source and target domains even in cases of having access to an incomplete set of test data. This paper presents our work in progress as we propose an approach for such a setting based on maintaining the geometry information of source samples during the adaptation. This way, the model can capture the relationships between different fault categories and preserve them in the constructed domain-invariant feature space, even in situations where some classes are entirely missing. This paper examines this idea using artificial data sets to demonstrate the effectiveness of geometry-preserving transformation. We have also started investigations on real-world predictive maintenance datasets, such as CWRU.

Utilizing machine learning methods to detect intrusion into computer networks is a trending topic in information security research. The limitation of labeled samples is one of the challenges in this area. This challenge makes it difficult to build accurate learning models for intrusion detection. Transfer learning is one of the methods to counter such a challenge in machine learning topics. On the other hand, the emergence of new technologies and applications might bring new vulnerabilities to computer networks. Therefore, the learning process cannot occur all at once. Incremental learning is a practical standpoint to confront this challenge. This research presents a new framework for intrusion detection systems called ITL-IDS that can potentially start learning in a network without prior knowledge. It begins with an incremental clustering algorithm to detect clusters’ numbers and shape without prior assumptions about the attacks. The outcomes are candidates to transfer knowledge between other instances of ITL-IDS. In each iteration, transfer learning provides target environments with incremental knowledge. Our evaluation shows that this method can combine incremental and transfer learning to identify new attacks. © 2022 Published by Elsevier B.V.

Domain Adaptation (DA) aims to transfer knowledge from a source to a target domain by aligning their respective data distributions. In the unsupervised setting, however, this may cause the source and target samples of different classes to align to each other, consequently leading to negative transfer. Semi-Supervised Domain Adaptation (SSDA) tries to solve such class misalignment problem by exploiting a few sample labels in the target domain. This paper proposes a new SSDA method called Adversarial Contrastive Semi-Supervised Domain Adaptation (ACSSDA) which combines two objectives, optimized for the case where very few target sample labels are available, to learn a shared feature representation for both source and target domains. ACSSDA uses a domain classifier to ensure that the resulting feature space is domain agnostic. Simultaneously, Contrastive loss aims to pull together samples of the same class and push apart samples of different classes. This is shown to reduce class misalignment and negative transfer even with as little as a single labeled sample per class. We demonstrate the effectiveness of ACSSDA with experiments on several benchmark data sets. The results show the superiority of our method over state-of-the-art approaches.

Som manuskript i avhandling / As manuscript in thesis

Federated Learning (FL) allows several clients to construct a common global machine-learning model without having to share their data. FL, however, faces the challenge of statistical heterogeneity between the client's data, which degrades performance and slows down the convergence toward the global model. In this paper, we provide theoretical proof that minimizing heterogeneity between clients facilitates the convergence of a global model for every single client. This becomes particularly important under empirical concept shifts among clients, rather than merely considering imbalanced classes, which have been studied until now. Therefore, we propose a method for knowledge transfer between clients where the server trains client-specific generators. Each generator generates samples for the corresponding client to remove the conflict with other clients' models. Experiments conducted on synthetic and real data, along with a theoretical study, support the effectiveness of our method in constructing a well-generalizable global model by reducing the conflict between local models., As manuscript in thesis