517 results
Search Results
2. Optimizing feature selection in intrusion detection systems: Pareto dominance set approaches with mutual information and linear correlation.
- Author
-
Barbosa, Guilherme Nunes Nasseh, Andreoni, Martin, and Mattos, Diogo Menezes Ferrazani
- Subjects
FEATURE selection ,INTRUSION detection systems (Computer security) ,MACHINE learning ,SOCIAL dominance ,PEARSON correlation (Statistics) ,FILTER paper - Abstract
In the realm of network intrusion detection using machine learning, feature selection aims for computational efficiency, enhanced performance, and model interpretability, preventing overfitting and optimizing data visualization. This paper proposes a filtering method for feature selection, which optimizes information quantity and linear correlation between resultant features. The method identifies Pareto dominant pairs of informative and correlated features, constructs a graph, and selects key features based on betweenness centrality in its connected components. The proposal yields a more concise and informative dataset representation. Experimental results, using three diverse datasets, demonstrate that the proposal achieves more than 95% accuracy in classifying network attacks with just 14% of the total number features in original datasets. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF
3. Using AI to detect panic buying and improve products distribution amid pandemic.
- Author
-
Adulyasak, Yossiri, Benomar, Omar, Chaouachi, Ahmed, Cohen, Maxime C., and Khern-am-nuai, Warut
- Subjects
CONSUMER behavior ,COVID-19 pandemic ,PRODUCT improvement ,ARTIFICIAL intelligence ,COVID-19 - Abstract
The COVID-19 pandemic has triggered panic-buying behavior around the globe. As a result, many essential supplies were consistently out-of-stock at common point-of-sale locations. Even though most retailers were aware of this problem, they were caught off guard and are still lacking the technical capabilities to address this issue. The primary objective of this paper is to develop a framework that can systematically alleviate this issue by leveraging AI models and techniques. We exploit both internal and external data sources and show that using external data enhances the predictability and interpretability of our model. Our data-driven framework can help retailers detect demand anomalies as they occur, allowing them to react strategically. We collaborate with a large retailer and apply our models to three categories of products using a dataset with more than 15 million observations. We first show that our proposed anomaly detection model can successfully detect anomalies related to panic buying. We then present a prescriptive analytics simulation tool that can help retailers improve essential product distribution in uncertain times. Using data from the March 2020 panic-buying wave, we show that our prescriptive tool can help retailers increase access to essential products by 56.74%. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF
4. Signature-based Adaptive Cloud Resource Usage Prediction Using Machine Learning and Anomaly Detection.
- Author
-
Sus, Wiktor and Nawrocki, Piotr
- Abstract
One of the challenges in managing cloud computing clusters is assigning resources based on the customers’ needs. For this mechanism to work efficiently, it is imperative that there are sufficient resources reserved to maintain continuous operation, but not too much to avoid overhead costs. Additionally, to avoid the overhead of acquisition time, it is important to reserve resources sufficiently in advance. This paper presents a novel reliable general-purpose mechanism for prediction-based resource usage reservation. The proposed solution should be capable of operating for long periods of time without drift-related problems, and dynamically adapt to changes in system usage. To achieve this, a novel signature-based ensemble prediction method is presented, which utilizes multiple distinct prediction algorithms suited for various use-cases, as well as an anomaly detection mechanism used to improve prediction accuracy. This ensures that the mechanism can operate efficiently in different real-life scenarios. Thanks to a novel signature-based selection algorithm, it is possible to use the best available prediction algorithm for each use-case, even over long periods of time, which would typically lead to drifts. The proposed approach has been evaluated using real-life historical data from various production servers, which include traces from more than 1,500 machines collected over more than a year. Experimental results have demonstrated an increase in prediction accuracy of up to 21.4 percent over the neural network approach. The evaluation of the proposed approach highlights the importance of choosing the appropriate prediction method, especially in diverse scenarios where the load changes frequently. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF
5. Anomalous process detection for Internet of Things based on K-Core.
- Author
-
Yue Chang, Teng Hu, Fang Lou, Tao Zeng, Mingyong Yin, Siqi Yang, Shaowei Wang, and Sheng Chen
- Subjects
INTERNET of things ,INTRUSION detection systems (Computer security) ,COMPUTER security ,ARTIFICIAL intelligence ,DISTRIBUTED computing ,OPTIMIZATION algorithms - Abstract
In recent years, Internet of Things security incidents occur frequently, which is often accompanied by malicious events. Therefore, anomaly detection is an important part of Internet of Things security defense. In this paper, we create a process whitelist based on the K-Core decomposition method for detecting anomalous processes in IoT devices. The method first constructs an IoT process network according to the relationships between processes and IoT devices. Subsequently, it creates a whitelist and detect anomalous processes. Our work innovatively transforms process data into a network framework, employing K-Core analysis to identify core processes that signify high popularity. Then, a threshold-based filtering mechanism is applied to formulate the process whitelist. Experimental results show that the unsupervised method proposed in this paper can accurately detect anomalous processes on real-world datasets. Therefore, we believe our algorithm can be widely applied to anomaly process detection, ultimately enhancing the overall security of the IoT. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF
6. Electricity Behavior Modeling and Anomaly Detection Services Based on a Deep Variational Autoencoder Network.
- Author
-
Lin, Rongheng, Chen, Shuo, He, Zheyu, Wu, Budan, Zou, Hua, Zhao, Xin, and Li, Qiushuang
- Abstract
Understanding electrical load profiles and detecting anomaly behaviors are important to the smart grid system. However, current load identification and anomaly analysis are based on static analysis, and less consideration is given to anomaly findings under load change conditions. This paper proposes a deep variational autoencoder network (DVAE) for load profiles, along with anomaly analysis services, and introduces auto-time series data updating strategies based on sliding window adjustment. DVAE can help reconstruct the load curve and measure the difference between the original and the newer curve, whose measurement indicators include reconstruction probability and Pearson similarity. Meanwhile, the design of the sliding window strategy updates the data and DVAE model in a time-series manner. Experiments were carried out based on datasets from the U.S. Department of Energy and from Southeast China. The results showed that the proposed services could result in a 5% improvement in the AUC value, which helps to identify the anomaly behavior. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF
7. A survey on social network's anomalous behavior detection.
- Author
-
Xing, Ling, Li, Shiyu, Zhang, Qi, Wu, Honghai, Ma, Huahong, and Zhang, Xiaohui
- Subjects
SOCIAL networks ,ANOMALY detection (Computer security) ,SOCIAL media ,RESEARCH personnel - Abstract
The onset of Web 3.0 has catalyzed the rapid advancement of social networking, transforming platforms into essential elements deeply embedded within the fabric of daily life. Researchers have proposed several methods for detecting anomalous behaviors in various scenarios. This article provides a comprehensive review of current research and the latest developments in anomalous behavior detection within social networks. We present a hierarchical three-layer categorization scheme based on the distinct characteristics of base-level detection technologies and various datasets. First, anomaly detection based on user behavioral characteristics can intuitively reflect deviations in individual behavior. However, it may overlook the overall network structure's impact. Second, detecting anomalies within a network's topological structure highlights structural significance, but may overlook the subtle nuances of individual behavior. Finally, the coordinated fusion method, which blends individual behavioral characteristics and the network's topological structure, addresses the multifaceted nature of anomalies, yielding a more thorough and accurate anomaly detection strategy. This paper provides an overview and assesses the performance of three anomaly detection methods. Furthermore, we explore the challenges associated with social network anomaly detection and the potential pathways for further research. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF
8. Beyond Supervised: The Rise of Self-Supervised Learning in Autonomous Systems.
- Author
-
Taherdoost, Hamed
- Subjects
SUPERVISED learning ,IMAGE analysis ,DIAGNOSTIC imaging ,FEATURE extraction ,INSTRUCTIONAL systems ,SCALABILITY - Abstract
Supervised learning has been the cornerstone of many successful medical imaging applications. However, its reliance on large labeled datasets poses significant challenges, especially in the medical domain, where data annotation is time-consuming and expensive. In response, self-supervised learning (SSL) has emerged as a promising alternative, leveraging unlabeled data to learn meaningful representations without explicit supervision. This paper provides a detailed overview of supervised learning and its limitations in medical imaging, underscoring the need for more efficient and scalable approaches. The study emphasizes the importance of the area under the curve (AUC) as a key evaluation metric in assessing SSL performance. The AUC offers a comprehensive measure of model performance across different operating points, which is crucial in medical applications, where false positives and negatives have significant consequences. Evaluating SSL methods based on the AUC allows for robust comparisons and ensures that models generalize well to real-world scenarios. This paper reviews recent advances in SSL for medical imaging, demonstrating their potential to revolutionize the field by mitigating challenges associated with supervised learning. Key results show that SSL techniques, by leveraging unlabeled data and optimizing performance metrics like the AUC, can significantly improve the diagnostic accuracy, scalability, and efficiency in medical image analysis. The findings highlight SSL's capability to reduce the dependency on labeled datasets and present a path forward for more scalable and effective medical imaging solutions. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF
9. Leak Event Diagnosis for Power Plants: Generative Anomaly Detection Using Prototypical Networks.
- Author
-
Jeong, Jaehyeok, Yeo, Doyeob, Roh, Seungseo, Jo, Yujin, and Kim, Minsuk
- Subjects
ANOMALY detection (Computer security) ,ARTIFICIAL intelligence ,LEAK detection ,ACQUISITION of data ,NOZZLES - Abstract
Anomaly detection systems based on artificial intelligence (AI) have demonstrated high performance and efficiency in a wide range of applications such as power plants and smart factories. However, due to the inherent reliance of AI systems on the quality of training data, they still demonstrate poor performance in certain environments. Especially in hazardous facilities with constrained data collection, deploying these systems remains a challenge. In this paper, we propose Generative Anomaly Detection using Prototypical Networks (GAD-PN) designed to detect anomalies using only a limited number of normal samples. GAD-PN is a structure that integrates CycleGAN with Prototypical Networks (PNs), learning from metadata similar to the target environment. This approach enables the collection of data that are difficult to gather in real-world environments by using simulation or demonstration models, thus providing opportunities to learn a variety of environmental parameters under ideal and normal conditions. During the inference phase, PNs can classify normal and leak samples using only a small number of normal data from the target environment by prototypes that represent normal and abnormal features. We also complement the challenge of collecting anomaly data by generating anomaly data from normal data using CycleGAN trained on anomaly features. It can also be adapted to various environments that have similar anomalous scenarios, regardless of differences in environmental parameters. To validate the proposed structure, data were collected specifically targeting pipe leakage scenarios, which are significant problems in environments such as power plants. In addition, acoustic ultrasound signals were collected from the pipe nozzles in three different environments. As a result, the proposed model achieved a leak detection accuracy of over 90% in all environments, even with only a small number of normal data. This performance shows an average improvement of approximately 30% compared with traditional unsupervised learning models trained with a limited dataset. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF
10. Understanding Regulatory Changes: Deep Learning in Sustainable Finance and Banking.
- Author
-
Anghel, Bogdan Ionut and Lupu, Radu
- Subjects
CONVOLUTIONAL neural networks ,SUSTAINABLE investing ,DEEP learning ,BANKING industry ,ANOMALY detection (Computer security) - Abstract
This paper examines the regulatory impact on the European Banking Sector using advanced deep learning techniques to analyze the relationship between Sustainable Finance guidelines and the SX7P Index from January 2012 to December 2023. Utilizing Long Short-Term Memory Auto-encoder (LSTM-AE), Variational Autoencoder (VAE), and Convolutional Neural Network (CNN) for anomaly detection, the study compares anomalies and investigates their correlation with European Banking Authority (EBA) events and Sustainable Finance guidelines from January 2020 to December 2023. Through the analysis of 43 pertinent EBA documents, the research identifies patterns and variations in anomalies, assessing their association with regulatory changes. The results reveal significant anomalies aligning with regulatory events, indicating a potential causal relationship. Notably, the VAE methodology shows the strongest correlation between EBA Sustainable Finance events and anomalies. This research advances the understanding of deep learning applications in financial markets and offers valuable insights for policymakers and financial institutions regarding regulatory shifts in Sustainable Finance. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF
11. A review of intrusion detection system and security threat in internet of things enabled environment.
- Author
-
Nisha, Gill, Nasib Singh, and Gulia, Preeti
- Subjects
INTERNET of things ,INTERNET security ,SECURITY systems ,INTRUSION detection systems (Computer security) ,TELECOMMUNICATION systems ,SENSOR networks - Abstract
Thousands of devices communicate globally to share data and information without any human intervention. A network of physical objects with numerous sensors and other network hardware to exchange data with servers and additional devices that are linked is referred to as the "internet of things (IoT)". The actions hurting the communication system are known as intrusions. Security features such as (integrity, and confidentiality) within IoT networks are compromised when any kind of intrusion occurs. To identify multiple infiltration types in an environment where IoT is enabled, an intrusion detection system (IDS) is required. In environments where IoT is enabled, security vulnerabilities are now more prevalent than ever. In this study, the IoT architecture is reviewed, and potential security risks at each tier are investigated. It is also hoped that this research will stimulate thought about the expanding risks posed by unprotected IoT devices. The paper also intends to provide an in-depth analysis of intrusion detection systems for identifying and classifying security threats in an IoT-enabled environment. Furthermore, this study investigates a variety of efficient machine learning-based methods for detecting cyberattacks on IoT devices. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF
12. Artificial intelligence powered internet of vehicles: securing connected vehicles in 6G.
- Author
-
Raja, Depa Ramachandraiah Kumar, Abas, Zuraida Abal, Akula, Chandra Sekhar, Kumar, Yellapalli Dileep, Kumar, Goshtu Hemanth, and Eswari, Venappagari
- Subjects
ARTIFICIAL intelligence ,DEEP learning ,INTERNET ,SIMPLE machines ,5G networks ,MACHINE learning - Abstract
The rapid advancements in automotive technology and the emergence of next-generation networks such as 5G and 6G are laying the foundation for the internet of vehicles (IoV), a revolutionary concept to transform transportation systems. The convergence of artificial intelligence (AI) and connected vehicles IoV is driving a paradigm shift in the transportation sector, especially in the dynamic framework of 5G and future 6G networks. This survey paper provides a thorough survey of the evolving AI-based IoV security landscape. We explore key areas of 5G/6G networks, focusing on the complex interplay of machine learning (ML) and deep learning (DL) in enhancing vehicle-to-everything (V2X) security and connected vehicles. Addressing the unique challenges of 6G, this paper outlines future directions for improving security and highlights open research issues. This comprehensive survey, which aims to provide information and guidance to both researchers and practitioners, contributes to a detailed understanding of the security issues associated with connected vehicles in the emerging 6G era. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF
13. Real-Time Monitoring of Data Pipelines: Exploring and Experimentally Proving that the Continuous Monitoring in Data Pipelines Reduces Cost and Elevates Quality.
- Author
-
Narayanan, Shammy, S., Maheswari, and Zephan, Prisha
- Subjects
DATA integrity ,DATA quality ,PIPELINE inspection ,ELECTRONIC data processing ,PIPELINE failures ,COST ,ELECTRONIC commerce - Abstract
Data pipelines are crucial for processing and transforming data in various domains, including finance, healthcare, and ecommerce. Ensuring the reliability and accuracy of data pipelines is of utmost importance to maintain data integrity and make informed business decisions. In this paper, we explore the significance of continuous monitoring in data pipelines and its contribution to data observability. This work discusses the challenges associated with monitoring data pipelines in realtime, propose a framework for real-time monitoring, and highlight its benefits in enhancing data observability. The findings of this work emphasize the need for organizations to adopt continuous monitoring practices to ensure data quality, detect anomalies, and improve overall system performance. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF
14. RAMFAE: a novel unsupervised visual anomaly detection method based on autoencoder.
- Author
-
Sun, Zhongju, Wang, Jian, and Li, Yakun
- Abstract
Traditional methods of visual anomaly detection based on reconstruction often use normal data to train autoencoder. Then the metric distance detection method is used to estimate whether the samples of detection belong to the exception class. However, this method has some problems that the autoencoder produces blurry images to cause false detection of normal pixel points. The model may still be able to fully reconstruct the undiscovered defects due to the large capacity of autoencoder, even if it is trained only on normal samples. Then, the metric distance detection method would ignore local key information. To solve this problem, this paper comes up with the random anomaly multi-scale feature focused autoencoder (RAMFAE), an innovative unsupervised visual anomaly detection technique, which incorporates three novel concepts. First, a multi-scale feature focused extraction (MFFE) network structure is designed and added between the encoder and decoder, which effectively solves the problem of reconstructing image blur and effectively improves the sensitivity of the model to normal regions. Second, this article employs Delete Paste, a novel data augmentation strategy for generating two different types of random anomalies, which pastes the cut part into a random location, while the pixels in the original position are filled with 0. In spite of the input anomalous images, the strategy makes the model be able to produce normal images to avoid the phenomenon of anomaly reconstruction, and then enables defect localization based on the error between the measured image and the reconstructed image. Third, the study adopts the image quality assessment with combining gradient magnitude similarity deviation (GMSD) and structural similarity (SSIM) to solve the problem that local key information and texture detail information are not easy to be paid attention to by the model, and alleviate the training pressure caused by Delete Paste enhancement. We perform an extensive evaluation on the challenging MVTec AD data set and compare it with the advanced visual anomaly detection methods in recent years as well. The AUC final result of RAMFAE in this text reaches 94.5, which is 3.6, 2.5 and 0.8 higher than the advanced IGD, FCDD and RIAD detection methods. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF
15. Insights into Modern Intrusion Detection Strategies for Internet of Things Ecosystems.
- Author
-
Isong, Bassey, Kgote, Otshepeng, and Abu-Mahfouz, Adnan
- Subjects
INTERNET of things ,CYBERTERRORISM ,ECOSYSTEMS ,BLOCKCHAINS ,MACHINE learning ,COMPUTER network security - Abstract
The swift explosion of Internet of Things (IoT) devices has brought about a new era of interconnectivity and ease of use while simultaneously presenting significant security concerns. Intrusion Detection Systems (IDS) play a critical role in the protection of IoT ecosystems against a wide range of cyber threats. Despite research advancements, challenges persist in improving IDS detection accuracy, reducing false positives (FPs), and identifying new types of attacks. This paper presents a comprehensive analysis of recent developments in IoT, shedding light on detection methodologies, threat types, performance metrics, datasets, challenges, and future directions. We systematically analyze the existing literature from 2016 to 2023, focusing on both machine learning (ML) and non-ML IDS strategies involving signature, anomaly, specification, and hybrid models to counteract IoT-specific threats. The findings include the deployment models from edge to cloud computing and evaluating IDS performance based on measures such as accuracy, FP rates, and computational costs, utilizing various IoT benchmark datasets. The study also explores methods to enhance IDS accuracy and efficiency, including feature engineering, optimization, and cutting-edge solutions such as cryptographic and blockchain technologies. Equally, it identifies key challenges such as the resource-constrained nature of IoT devices, scalability, and privacy issues and proposes future research directions to enhance IoT-based IDS and overall ecosystem security. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF
16. Anomaly‐background separation and particle swarm optimization based band selection for hyperspectral anomaly detection.
- Author
-
Shang, Xiaodi, Duan, Yiqi, Wang, Xiaopeng, Fu, Baijia, and Sun, Xudong
- Subjects
PARTICLE swarm optimization ,INTRUSION detection systems (Computer security) ,DISTRIBUTION (Probability theory) ,SURFACE analysis - Abstract
As one of the dimensionality reduction techniques of hyperspectral image (HSI), band selection (BS) does not change the spectral characteristics and physical meaning of HSIs, which is beneficial to the identification and analysis of surface objects. Recently, many BS methods for target detection have achieved promising results by making full use of the priori spectral features of the target to be detected. Conversely, anomaly detection separates the anomaly based solely on the statistical distribution difference between anomaly and background without any prior information. Therefore, the development of BS for anomaly detection has lagged far behind that of BS for target detection. To this end, this paper proposes a novel BS algorithm dedicated to anomaly detection tasks, named anomaly‐background separation and particle swarm optimization (PSO)‐based BS. Specifically, an anomaly‐background separation framework (ABSF) is established to predetermine a priori knowledge of anomaly distribution. Then, three band prioritization criteria are constructed with the anomaly‐background constraints generated by ABSF. Finally, PSO is used to find the optimal subset of bands in the solution space. The experiments on two real datasets demonstrate that the proposed method yields better detection results and greater stability compared to other BS methods discussed in this paper. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF
17. GAN-Based Anomaly Detection Tailored for Classifiers.
- Author
-
Králik, Ľubomír, Kontšek, Martin, Škvarek, Ondrej, and Klimo, Martin
- Subjects
GENERATIVE adversarial networks ,ARTIFICIAL neural networks ,DATABASES - Abstract
Pattern recognition systems always misclassify anomalies, which can be dangerous for uninformed users. Therefore, anomalies must be filtered out from each classification. The main challenge for the anomaly filter design is the huge number of possible anomaly samples compared with the number of samples in the training set. Tailoring the filter for the given classifier is just the first step in this reduction. Paper tests the hypothesis that the filter trained in avoiding "near" anomalies will also refuse the "far" anomalies, and the anomaly detector is then just a classifier distinguishing between "far real" and "near anomaly" samples. As a "far real" samples generator was used, a Generative Adversarial Network (GAN) fake generator that transforms normally distributed random seeds into fakes similar to the training samples. The paper proves the assumption that seeds unused in fake training will generate anomalies. These seeds are distinguished according to their Chebyshev norms. While the fakes have seeds within the hypersphere with a given radius, the near anomalies have seeds within the sphere near cover. Experiments with various anomaly test sets have shown that GAN-based anomaly detectors create a reliable anti-anomaly shield using the abovementioned assumptions. The proposed anomaly detector is tailored to the given classifier, but its limitation is due to the need for the availability of the database on which the classifier was trained. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF
18. An Anomaly Detection Method for UAV Based on Wavelet Decomposition and Stacked Denoising Autoencoder.
- Author
-
Zhou, Shenghan, He, Zhao, Chen, Xu, and Chang, Wenbing
- Subjects
INTRUSION detection systems (Computer security) ,DECOMPOSITION method ,FEATURE extraction ,DEEP learning ,DRONE aircraft ,DATA extraction - Abstract
The paper proposes an anomaly detection method for UAVs based on wavelet decomposition and stacked denoising autoencoder. This method takes the negative impact of noisy data and the feature extraction capabilities of deep learning models into account. It aims to improve the accuracy of the proposed anomaly detection method with wavelet decomposition and stacked denoising autoencoder methods. Anomaly detection based on UAV flight data is an important method of UAV condition monitoring and potential abnormal state mining, which is an important means to reduce the risk of UAV flight accidents. However, the diversity of UAV mission scenarios leads to a complex and harsh environment, so the acquired data are affected by noise, which brings challenges to accurate anomaly detection based on UAV data. Firstly, we use wavelet decomposition to denoise the original data; then, we used the stacked denoising autoencoder to achieve feature extraction. Finally, the softmax classifier is used to realize the anomaly detection of UAV. The experimental results demonstrate that the proposed method still has good performance in the case of noisy data. Specifically, the Accuracy reaches 97.53%, the Precision is 97.50%, the Recall is 91.81%, and the F1-score is 94.57%. Furthermore, the proposed method outperforms the four comparison models with more outstanding performance. Therefore, it has significant potential in reducing UAV flight accidents and enhancing operational safety. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF
19. Anomaly Detection in Blockchain Networks Using Unsupervised Learning: A Survey.
- Author
-
Cholevas, Christos, Angeli, Eftychia, Sereti, Zacharoula, Mavrikos, Emmanouil, and Tsekouras, George E.
- Subjects
DATA structures ,MACHINE learning ,PRIVATE networks ,BLOCKCHAINS ,ALGORITHMS - Abstract
In decentralized systems, the quest for heightened security and integrity within blockchain networks becomes an issue. This survey investigates anomaly detection techniques in blockchain ecosystems through the lens of unsupervised learning, delving into the intricacies and going through the complex tapestry of abnormal behaviors by examining avant-garde algorithms to discern deviations from normal patterns. By seamlessly blending technological acumen with a discerning gaze, this survey offers a perspective on the symbiotic relationship between unsupervised learning and anomaly detection by reviewing this problem with a categorization of algorithms that are applied to a variety of problems in this field. We propose that the use of unsupervised algorithms in blockchain anomaly detection should be viewed not only as an implementation procedure but also as an integration procedure, where the merits of these algorithms can effectively be combined in ways determined by the problem at hand. In that sense, the main contribution of this paper is a thorough study of the interplay between various unsupervised learning algorithms and how this can be used in facing malicious activities and behaviors within public and private blockchain networks. The result is the definition of three categories, the characteristics of which are recognized in terms of the way the respective integration takes place. When implementing unsupervised learning, the structure of the data plays a pivotal role. Therefore, this paper also provides an in-depth presentation of the data structures commonly used in unsupervised learning-based blockchain anomaly detection. The above analysis is encircled by a presentation of the typical anomalies that have occurred so far along with a description of the general machine learning frameworks developed to deal with them. Finally, the paper spotlights challenges and directions that can serve as a comprehensive compendium for future research efforts. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF
20. Unsupervised Anomaly Detection of Industrial Images Based on Dual Generator Reconstruction Networks.
- Author
-
Cong Gu, Siyv Ren, and Qiqiang Duan
- Subjects
ANOMALY detection (Computer security) ,DEEP learning - Abstract
At present, deep learning techniques are increasingly utilized in computer vision and anomaly detection. To address the limitations of inadequate reconstruction capability and subpar performance in reconstruction-based anomaly detection, this study enhances the existing algorithm and introduces an unsupervised anomaly detection of industrial images algorithm based on dual generator reconstruction networks-DGRNet. The network consists of two generators and a discriminator, introducing a widely recognized denoising diffusion probabilistic model (DDPM) as one of the generators, an autoencoder (AE) as the other generator, and a decoder as the discriminator. The model is tested on the MVTec AD dataset, and in the case of no additional training data, the anomaly detection AUC result of DGRNet exceeds the baseline method based on reconstruction by 19.6 percentage points. The experimental results show that DGRNet can improve the detection performance in the anomaly detection algorithm based on unsupervised and reconstructed networks. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF
21. Using Ensemble Learning for Anomaly Detection in Cyber–Physical Systems.
- Author
-
Jeffrey, Nicholas, Tan, Qing, and Villar, José R.
- Subjects
CYBER physical systems ,INTRUSION detection systems (Computer security) ,INFORMATION technology ,MACHINE learning ,TECHNOLOGY convergence ,SYSTEMS availability ,DEEP learning - Abstract
The swift embrace of Industry 4.0 paradigms has led to the growing convergence of Information Technology (IT) networks and Operational Technology (OT) networks. Traditionally isolated on air-gapped and fully trusted networks, OT networks are now becoming more interconnected with IT networks due to the advancement and applications of IoT. This expanded attack surface has led to vulnerabilities in Cyber–Physical Systems (CPSs), resulting in increasingly frequent compromises with substantial economic and life safety repercussions. The existing methods for the anomaly detection of security threats typically use simple threshold-based strategies or apply Machine Learning (ML) algorithms to historical data for the prediction of future anomalies. However, due to the high levels of heterogeneity across different CPS environments, minimizing the opportunities for transfer learning, and the scarcity of real-world data for training, the existing ML-based anomaly detection techniques suffer from a poor predictive performance. This paper introduces a hybrid anomaly detection approach designed to identify threats to CPSs by combining the signature-based anomaly detection typically utilized in IT networks, the threshold-based anomaly detection typically utilized in OT networks, and behavioural-based anomaly detection using Ensemble Learning (EL), which leverages the strengths of multiple ML algorithms against the same dataset to increase the accuracy. Multiple public research datasets were used to validate the proposed approach, with the hybrid methodology employing a divide-and-conquer strategy to offload the detection of certain cyber threats to computationally inexpensive signature-based and threshold-based methods using domain knowledge to minimize the size of the behavioural-based data needed for ML model training, thus achieving a higher accuracy over a reduced timeframe. The experimental results showed accuracy improvements of 4–7% over those of the conventional ML classifiers in performing anomaly detection across multiple datasets, which is particularly important to the operators of CPS environments due to the high financial and life safety costs associated with interruptions to system availability. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF
22. SSMSPC: self-supervised multivariate statistical in-process control in discrete manufacturing processes
- Author
-
Biegel, Tobias, Helm, Patrick, Jourdan, Nicolas, and Metternich, Joachim
- Published
- 2024
- Full Text
- View/download PDF
23. Enhancing anomaly detectors with LatentOut
- Author
-
Angiulli, Fabrizio, Fassetti, Fabio, and Ferragina, Luca
- Published
- 2024
- Full Text
- View/download PDF
24. Low-cost and high-performance abnormal trajectory detection based on the GRU model with deep spatiotemporal sequence analysis in cloud computing.
- Author
-
Tang, Guohao, Zhao, Huaying, and Yu, Baohua
- Subjects
SEQUENCE analysis ,RANDOM forest algorithms ,RECURRENT neural networks ,ANOMALY detection (Computer security) ,CLOUD computing ,PROCESS capability ,SEMANTICS - Abstract
Trajectory anomalies serve as early indicators of potential issues and frequently provide valuable insights into event occurrence. Existing methods for detecting abnormal trajectories primarily focus on comparing the spatial characteristics of the trajectories. However, they fail to capture the temporal dimension's pattern and evolution within the trajectory data, thereby inadequately identifying the behavioral inertia of the target group. A few detection methods that incorporate spatiotemporal features have also failed to adequately analyze the spatiotemporal sequence evolution information; consequently, detection methods that ignore temporal and spatial correlations are too one-sided. Recurrent neural networks (RNNs), especially gate recurrent unit (GRU) that design reset and update gate control units, process nonlinear sequence processing capabilities, enabling effective extraction and analysis of both temporal and spatial characteristics. However, the basic GRU network model has limited expressive power and may not be able to adequately capture complex sequence patterns and semantic information. To address the above issues, an abnormal trajectory detection method based on the improved GRU model is proposed in cloud computing in this paper. To enhance the anomaly detection ability and training efficiency of relevant models, strictly control the input of irrelevant features and improve the model fitting effect, an improved model combining the random forest algorithm and fully connected layer network is designed. The method deconstructs spatiotemporal semantics through reset and update gated units, while effectively capturing feature evolution information and target behavioral inertia by leveraging the integration of features and nonlinear mapping capabilities of the fully connected layer network. The experimental results based on the GeoLife GPS trajectory dataset indicate that the proposed approach improves both generalization ability by 1% and reduces training cost by 31.68%. This success do provides a practical solution for the task of anomaly trajectory detection. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF
25. 基于 SSA−LSTM 的风速异常波动检测方法.
- Author
-
邓立军, 袁金波, 刘剑, and 尚文天
- Abstract
Copyright of Coal Science & Technology (0253-2336) is the property of Coal Science & Technology and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
- Published
- 2024
- Full Text
- View/download PDF
26. Intelligent systems for sitting posture monitoring and anomaly detection: an overview.
- Author
-
Vermander, Patrick, Mancisidor, Aitziber, Cabanes, Itziar, and Perez, Nerea
- Subjects
ARTIFICIAL intelligence ,MEDICAL personnel ,POSTURE ,ELECTRIC wheelchairs ,FUNCTIONAL status ,RESEARCH personnel ,YOGA postures ,INTRUSION detection systems (Computer security) - Abstract
The number of people who need to use wheelchair for proper mobility is increasing. The integration of technology into these devices enables the simultaneous and objective assessment of posture, while also facilitating the concurrent monitoring of the functional status of wheelchair users. In this way, both the health personnel and the user can be provided with relevant information for the recovery process. This information can be used to carry out an early adaptation of the rehabilitation of patients, thus allowing to prevent further musculoskeletal problems, as well as risk situations such as ulcers or falls. Thus, a higher quality of life is promoted in affected individuals. As a result, this paper presents an orderly and organized analysis of the existing postural diagnosis systems for detecting sitting anomalies in the literature. This analysis can be divided into two parts that compose such postural diagnosis: on the one hand, the monitoring devices necessary for the collection of postural data and, on the other hand, the techniques used for anomaly detection. These anomaly detection techniques will be explained under two different approaches: the traditional generalized approach followed to date by most works, where anomalies are treated as incorrect postures, and a new individualized approach treating anomalies as changes with respect to the normal sitting pattern. In this way, the advantages, limitations and opportunities of the different techniques are analyzed. The main contribution of this overview paper is to synthesize and organize information, identify trends, and provide a comprehensive understanding of sitting posture diagnosis systems, offering researchers an accessible resource for navigating the current state of knowledge of this particular field. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF
27. Efficient Defect Detection of Rotating Goods under the Background of Intelligent Retail.
- Author
-
Hu, Zhengming, Zeng, Xuepeng, Xie, Kai, Wen, Chang, He, Jianbiao, and Zhang, Wei
- Subjects
VENDING machines ,PRODUCT returns ,CONSUMERS - Abstract
Dynamic visual vending machines are rapidly growing in popularity, offering convenience and speed to customers. However, there is a prevalent issue with consumers damaging goods and then returning them to the machine, severely affecting business interests. This paper addresses the issue from the standpoint of defect detection. Although existing industrial defect detection algorithms, such as PatchCore, perform well, they face challenges, including handling goods in various orientations, detection speeds that do not meet real-time monitoring requirements, and complex backgrounds that hinder detection accuracy. These challenges hinder their application in dynamic vending environments. It is crucial to note that efficient visual features play a vital role in memory banks, yet current memory repositories for industrial inspection algorithms do not adequately address the problem of location-specific feature redundancy. To tackle these issues, this paper introduces a novel defect detection algorithm for goods using adaptive subsampling and partitioned memory banks. Firstly, Grad-CAM is utilized to extract deep features, which, in combination with shallow features, mitigate the impact of complex backgrounds on detection accuracy. Next, graph convolutional networks extract rotationally invariant features. The adaptive subsampling partitioned memory bank is then employed to store features of non-defective goods, which reduces memory consumption and enhances training speed. Experimental results on the MVTec AD dataset demonstrate that the proposed algorithm achieves a marked improvement in detection speed while maintaining accuracy that is comparable to state-of-the-art models. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF
28. From Anomaly Detection to Defect Classification.
- Author
-
Klarák, Jaromír, Andok, Robert, Malík, Peter, Kuric, Ivan, Ritomský, Mário, Klačková, Ivana, and Tsai, Hung-Yin
- Subjects
INTRUSION detection systems (Computer security) ,SYSTEMS design ,CLASSIFICATION ,PROOF of concept ,DEEP learning ,INSPECTION & review - Abstract
This paper proposes a new approach to defect detection system design focused on exact damaged areas demonstrated through visual data containing gear wheel images. The main advantage of the system is the capability to detect a wide range of patterns of defects occurring in datasets. The methodology is built on three processes that combine different approaches from unsupervised and supervised methods. The first step is a search for anomalies, which is performed by defining the correct areas on the controlled object by using the autoencoder approach. As a result, the differences between the original and autoencoder-generated images are obtained. These are divided into clusters using the clustering method (DBSCAN). Based on the clusters, the regions of interest are subsequently defined and classified using the pre-trained Xception network classifier. The main result is a system capable of focusing on exact defect areas using the sequence of unsupervised learning (autoencoder)–unsupervised learning (clustering)–supervised learning (classification) methods (U2S-CNN). The outcome with tested samples was 177 detected regions and 205 occurring damaged areas. There were 108 regions detected correctly, and 69 regions were labeled incorrectly. This paper describes a proof of concept for defect detection by highlighting exact defect areas. It can be thus an alternative to using detectors such as YOLO methods, reconstructors, autoencoders, transformers, etc. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF
29. Dynamic Data Abstraction-Based Anomaly Detection for Industrial Control Systems.
- Author
-
Cho, Jake and Gong, Seonghyeon
- Subjects
INTRUSION detection systems (Computer security) ,ANOMALY detection (Computer security) ,INDUSTRIAL controls manufacturing ,INFORMATION technology ,MACHINE learning ,NOISE control - Abstract
Industrial control systems (ICS) are critical networks directly linked to the value of core national and societal assets, yet they are increasingly becoming primary targets for numerous cyberattacks today. The ICS network, a fusion of operational technology (OT) and information technology (IT) networks, possesses a broad attack vector, and attacks targeting ICS often take the form of advanced persistent threats (APTs) exploiting zero-day vulnerabilities. However, most existing ICS security techniques have been adaptations of security technologies for IT networks, and security measures tailored to the characteristics of ICS data are currently insufficient. To mitigate cyber threats to ICS networks, this paper proposes an anomaly detection technique based on dynamic data abstraction. The proposed method abstracts ICS data collected in real time using a dynamic data abstraction technique based on noise reduction. The abstracted data are then used to optimize both the update rate and the detection accuracy of the anomaly detection model through model adaptation and incremental learning processes. The proposed approach updates the model by quickly reflecting data on new attack patterns and their distributions, effectively shortening the dwell time in response to APTs utilizing zero-day vulnerabilities. We demonstrate the attack response performance and detection accuracy of the proposed dynamic data abstraction-based anomaly detection technique through experiments using the SWaT dataset generated from a testbed of an actual ICS process. The experiments show that the proposed model achieves high accuracy with a small number of abstracted data while rapidly learning new attack pattern data in real-time without compromising accuracy. The proposed technique can effectively respond to cyberattacks targeting ICS by quickly learning and reflecting trends in attack patterns that exploit zero-day vulnerabilities. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF
30. Multivariate time series anomaly detection via dynamic graph attention network and Informer.
- Author
-
Huang, Xiangheng, Chen, Ningjiang, Deng, Ziyue, and Huang, Suqun
- Subjects
ANOMALY detection (Computer security) ,TIME series analysis ,FALSE alarms ,COMPUTER software quality control ,TIMESTAMPS - Abstract
In the industrial Internet, industrial software plays a central role in enhancing the level of intelligent manufacturing. It enables the promotion of digital collaborative services. Effective anomaly detection of multivariate time series can ensure the quality of industrial software. Extensive research has been conducted on time series anomaly detection to identify abnormal data. However, detecting anomalies in multivariate time series, which consist of high-dimensional, high-noise, and random data, poses significant challenges. The states of different timestamps within a time series sample can influence the overall correlation of sensor features. Unfortunately, existing methods often overlook this impact, making it difficult to capture subtle variations in the delayed response of attacked sensors.Consequently, there are false alarms and abnormal omissions. To address these limitations, this paper proposes an anomaly detection method called DGINet. DGINet leverages a dynamic graph attention network and Informer to capture and integrate feature correlation across different time states. By combining GRU and Informer, DGINet effectively captures continuous correlations in long time series. Moreover, DGINet simultaneously optimizes the reconstruction and forecasting modules, enhancing its overall performance. Experimental results on four benchmark datasets demonstrate that DGINet outperforms state-of-the-art methods by achieving up to a 2 % improvement in accuracy. Further analysis reveals that DGINet excels in accurately detecting anomalies in long time series and locating candidate abnormal attack points. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF
31. Industrial Image Anomaly Detection via Self-Supervised Learning with Feature Enhancement Assistance.
- Author
-
Wu, Bin and Wang, Xiaoqi
- Abstract
Industrial anomaly detection is constrained by the scarcity of anomaly samples, limiting the applicability of supervised learning methods. Many studies have focused on anomaly detection by generating anomaly images and adopting self-supervised learning approaches. Leveraging pre-trained networks on ImageNet has been explored to assist in this training process. However, achieving accurate anomaly detection remains time-consuming due to the network's depth and parameter count not being reduced. In this paper, we propose a self-supervised learning method based on Feature Enhancement Patch Distribution Modeling (FEPDM), which generates simulated anomalies. Unlike direct training on the original feature extraction network, our approach utilizes a pre-trained network to extract multi-scale features. By aggregating these multi-scale features, we are able to train at the feature level, thereby adapting more efficiently to various network structures and reducing domain bias with respect to natural image classification. Additionally, it significantly reduces the number of parameters in the training process. Introducing this approach not only enhances the model's generalization ability but also significantly improves the efficiency of anomaly detection. The method was evaluated on MVTec AD and BTAD datasets, and (image-level, pixel-level) AUROC scores of (95.7%, 96.2%), (93.4%, 97.6%) were obtained, respectively. The experimental results have convincingly demonstrated the efficacy of our method in tackling the scarcity of abnormal samples in industrial scenarios, while simultaneously highlighting its broad generalizability. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF
32. Concept of Early Prediction and Identification of Truck Vehicle Failures Supported by In-Vehicle Telematics Platform Based on Abnormality Detection Algorithm.
- Author
-
Semenov, Iouri, Świderski, Andrzej, Borucka, Anna, and Guzanek, Patrycja
- Abstract
As automotive technology advances in the realm of digitization, vehicles are becoming smarter and, at the same time, more vulnerable to various threats. This paper focuses on techniques for detecting faults to mitigate the risk of freight transportation. Our observations show that vehicle uptime varies significantly even under similar operating conditions. This variation stems from differences in the wear and tear of moving and stationary parts, the characteristics of transported loads, driving styles, the quality of maintenance, etc. These factors are particularly crucial for abnormal vehicles designed to carry AILs (Abnormal Indivisible Loads). Such vehicles are especially prone to surprising threats, requiring efficient techniques for monitoring separate vehicle components and providing drivers with vital information about their operational status. The presented article proposes an original concept of an integrated three-level monitoring system based on the AOP (All-in-One Platform) principle, using the DBSCAN (Density-Based Spatial Clustering of Applications with Noise) algorithm, which is a tool oriented to distinguish points from three categories: basic, boundary, and external. This is a solution not yet found in the literature. It is based on assessments of LOFs (Local Outlier Factors) and to detect anomalies in the measured values of operational parameters. The purpose of our study was to determine whether providing truck drivers with current information from an active threat warning system could help reduce unplanned downtimes. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF
33. Non-Intrusive System for Honeybee Recognition Based on Audio Signals and Maximum Likelihood Classification by Autoencoder.
- Author
-
Libal, Urszula and Biernacki, Pawel
- Subjects
SIGNAL frequency estimation ,MULTIPLE Signal Classification ,ARTIFICIAL intelligence ,SIGNAL detection ,SIGNAL processing - Abstract
Artificial intelligence and Internet of Things are playing an increasingly important role in monitoring beehives. In this paper, we propose a method for automatic recognition of honeybee type by analyzing the sound generated by worker bees and drone bees during their flight close to an entrance to a beehive. We conducted a wide comparative study to determine the most effective preprocessing of audio signals for the detection problem. We compared the results for several different methods for signal representation in the frequency domain, including mel-frequency cepstral coefficients (MFCCs), gammatone cepstral coefficients (GTCCs), the multiple signal classification method (MUSIC) and parametric estimation of power spectral density (PSD) by the Burg algorithm. The coefficients serve as inputs for an autoencoder neural network to discriminate drone bees from worker bees. The classification is based on the reconstruction error of the signal representations produced by the autoencoder. We propose a novel approach to class separation by the autoencoder neural network with various thresholds between decision areas, including the maximum likelihood threshold for the reconstruction error. By classifying real-life signals, we demonstrated that it is possible to differentiate drone bees and worker bees based solely on audio signals. The attained level of detection accuracy enables the creation of an efficient automatic system for beekeepers. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF
34. Detecting GPS Interference Using Automatic Dependent Surveillance-Broadcast Data.
- Author
-
Ramchandra, Akshay Ram, Skurdal, Anton, Ranganathan, Prakash, and Semke, William
- Abstract
This paper investigates the detection of Global Positioning System (GPS) interference during the Dallas Fort Worth (DFW) event from 17 to 19 October 2022, utilizing various machine learning (ML) models. The study examines the effectiveness of several ML models, including neural networks (NN), tree-based models, regression-based models, Bayesian classifiers, distance-based models, and stochastic classifiers, in identifying GPS interference. A simulated training signature was created with 180,000 data points, of which 25,792 were modeled as positive samples indicating GPS interference. Preliminary results reveal that the Multi-Layer Perceptron (MLP) model outperformed others, achieving a 99.8% True Positive Rate (TPR). Additionally, permutation feature importance was utilized to understand how model feature prioritization impacts the detection outcomes. Given the increasing frequency of GPS interference, these findings underscore the critical importance of ML techniques in detecting GPS interference patterns in Automatic Dependent Surveillance-Broadcast (ADS-B) data. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF
35. Efficient Dual-Attention-Based Knowledge Distillation Network for Unsupervised Wafer Map Anomaly Detection.
- Author
-
Hasan, Mohammad Mehedi, Yu, Naigong, and Khan Mirani, Imran
- Subjects
ANOMALY detection (Computer security) ,SEMICONDUCTOR manufacturing ,SEMICONDUCTOR devices ,IMAGE segmentation ,DEEP learning - Abstract
Detecting wafer map anomalies is crucial for preventing yield loss in semiconductor fabrication, although intricate patterns and resource-intensive labeled data prerequisites hinder precise deep-learning segmentation. This paper presents an innovative, unsupervised method for detecting pixel-level anomalies in wafer maps. It utilizes an efficient dual attention module with a knowledge distillation network to learn defect distributions without anomalies. Knowledge transfer is achieved by distilling information from a pre-trained teacher into a student network with similar architecture, except an efficient dual attention module is incorporated atop the teacher network’s feature pyramid hierarchies, which enhances feature representation and segmentation across pyramid hierarchies that selectively emphasize relevant and discard irrelevant features by capturing contextual associations in positional and channel dimensions. Furthermore, it enables student networks to acquire an improved knowledge of hierarchical features to identify anomalies across different scales accurately. The dissimilarity in feature pyramids acts as a discriminatory function, predicting the likelihood of an abnormality, resulting in highly accurate pixel-level anomaly detection. Consequently, our proposed method excelled on the WM-811K and MixedWM38 datasets, achieving AUROC, AUPR, AUPRO, and F1-Scores of (99.65%, 99.35%), (97.31%, 92.13%), (90.76%, 84.66%) respectively, alongside an inference speed of 3.204 FPS, showcasing its high precision and efficiency. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF
36. Anomaly Detection With Transformer for a Railway Vehicle Air Compressor.
- Author
-
Jin, Min-Je and Kang, Chul-Goo
- Abstract
Recently, research on condition-based maintenance (CBM) using artificial intelligence has attracted attention to reduce the maintenance costs of railway vehicles. The air compressor of a railway vehicle that adopts an air brake system is a target for CBM, and in order to reduce maintenance costs and ensure driving stability, technology to detect anomalies in the air compressor is necessary. The long short-term memory (LSTM) autoencoder is used to process sequence data. However, LSTM has limitations in that it cannot perform parallel processing due to its recurrent neural network characteristics, and it is difficult to learn the dependency between data as the sequence data lengthens. In this paper, we propose a novel transformer architecture as an anomaly detection model to learn dependency between air compressor data of a railway vehicle and we demonstrate superior data reconstruction and generalization ability of proposed transformer. We conduct simulation tests based on actual railway vehicle air compressor data and confirm improved anomaly detection performance. We propose an anomaly score definition method using mean squared error to perform reconstruction error-based anomaly detection. With the successful anomaly detection results in the air compressor of a railway vehicle, we demonstrate the effectiveness of a proposed anomaly detection algorithm that applies the moving average of anomaly scores and defines anomaly criteria using three-sigma. We test the proposed anomaly detection method using sensor data received from actual urban railway air compressors and prove its usefulness. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF
37. Online Adaptive Kalman Filtering for Real-Time Anomaly Detection in Wireless Sensor Networks.
- Author
-
Ahmad, Rami and Alkhammash, Eman H.
- Subjects
ANOMALY detection (Computer security) ,WIRELESS sensor networks ,KALMAN filtering ,ADAPTIVE filters ,SMART cities - Abstract
Wireless sensor networks (WSNs) are essential for a wide range of applications, including environmental monitoring and smart city developments, thanks to their ability to collect and transmit diverse physical and environmental data. The nature of WSNs, coupled with the variability and noise sensitivity of cost-effective sensors, presents significant challenges in achieving accurate data analysis and anomaly detection. To address these issues, this paper presents a new framework, called Online Adaptive Kalman Filtering (OAKF), specifically designed for real-time anomaly detection within WSNs. This framework stands out by dynamically adjusting the filtering parameters and anomaly detection threshold in response to live data, ensuring accurate and reliable anomaly identification amidst sensor noise and environmental changes. By highlighting computational efficiency and scalability, the OAKF framework is optimized for use in resource-constrained sensor nodes. Validation on different WSN dataset sizes confirmed its effectiveness, showing 95.4% accuracy in reducing false positives and negatives as well as achieving a processing time of 0.008 s per sample. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF
38. JointNet: Multitask Learning Framework for Denoising and Detecting Anomalies in Hyperspectral Remote Sensing.
- Author
-
Shao, Yingzhao, Li, Shuhan, Yang, Pengfei, Cheng, Fei, Ding, Yueli, and Sun, Jianguo
- Subjects
IMAGE denoising ,REMOTE sensing ,DATA integrity ,NOISE ,INFORMATION sharing - Abstract
One of the significant challenges with traditional single-task learning-based anomaly detection using noisy hyperspectral images (HSIs) is the loss of anomaly targets during denoising, especially when the noise and anomaly targets are similar. This issue significantly affects the detection accuracy. To address this problem, this paper proposes a multitask learning (MTL)-based method for detecting anomalies in noisy HSIs. Firstly, a preliminary detection approach based on the JointNet model, which decomposes the noisy HSI into a pure background and a noise–anomaly target mixing component, is introduced. This approach integrates the minimum noise fraction rotation (MNF) algorithm into an autoencoder (AE), effectively isolating the noise while retaining critical features for anomaly detection. Building upon this, the JointNet model is further optimized to ensure that the noise information is shared between the denoising and anomaly detection subtasks, preserving the integrity of the training data during the anomaly detection process and resolving the issue of losing anomaly targets during denoising. A novel loss function is designed to enable the joint learning of both subtasks under the multitask learning model. In addition, a noise score evaluation metric is introduced to calculate the probability of a pixel being an anomaly target, allowing for a clear distinction between noise and anomaly targets, thus providing the final anomaly detection results. The effectiveness of the proposed model and method is validated via testing on the HYDICE and San Diego datasets. The denoising metric results of the PSNR, SSIM, and SAM are 41.79, 0.91, and 4.350 and 42.83, 0.93, and 3.558 on the HYDICE and San Diego datasets, respectively. The anomaly detection ACU is 0.943 and 0.959, respectively. The proposed method outperforms the other algorithms, demonstrating that the reconstructed images using this method exhibited lower noise levels and more complete image information, and the JointNet model outperforms the mainstream HSI anomaly detection algorithms in both the quantitative evaluation and visual effect, showcasing its improved detection capabilities. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF
39. A Novel Voltage-Abnormal Cell Detection Method for Lithium-Ion Battery Mass Production Based on Data-Driven Model with Multi-Source Time Series Data.
- Author
-
Wang, Xiang, He, Jianjun, Huang, Fuxin, Liu, Zhenjie, Deng, Aibin, and Long, Rihui
- Subjects
TIME series analysis ,MASS production ,PROBLEM solving ,DATA modeling ,SHIPMENT of goods - Abstract
Before leaving the factory, lithium-ion battery (LIB) cells are screened to exclude voltage-abnormal cells, which can increase the fault rate, troubleshooting difficulty, and degrade pack performance. However, the time interval to obtain the detection results through the existing voltage-abnormal cell method is too long, which can seriously affect production efficiency and delay shipment, especially in the mass production of LIBs when facing a large number of time-critical orders. In this paper, we propose a data-driven voltage-abnormal cell detection method, using a fast model with simple architecture, which can detect voltage-abnormal cells based on the multi-source time series data of the LIB without a time interval. Firstly, our method transforms the different source data of a cell into a multi-source time series data representation and utilizes a recurrent-based data embedding to model the relation within it. Then, a simplified MobileNet is used to extract hidden feature from the embedded data. Finally, we detect the voltage-abnormal cells according to the hidden feature with a cell classification head. The experiment results show that the accuracy and average running time of our model on the voltage-abnormal cell detection task is 95.42% and 0.0509 ms per sample, which is a considerable improvement over existing methods. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF
40. U-TFF: A U-Net-Based Anomaly Detection Framework for Robotic Manipulator Energy Consumption Auditing Using Fast Fourier Transform.
- Author
-
Song, Ge, Hong, Seong Hyeon, Kyzer, Tristan, and Wang, Yi
- Subjects
FAST Fourier transforms ,ENERGY auditing ,MANIPULATORS (Machinery) ,ENERGY consumption ,MANUFACTURING processes - Abstract
Robotic manipulators play a key role in modern industrial manufacturing processes. Monitoring their operational health is of paramount importance. In this paper, a novel anomaly detection framework named U-TFF is introduced for energy consumption auditing of robotic manipulators. It comprises a cascade of Time–Frequency Fusion (TFF) blocks to extract both time and frequency domain features from time series data. The block applies the Fast Fourier Transform to convert the input to the frequency domain, followed by two separate dense layers to process the resulting real and imaginary components, respectively. The frequency and time features are then combined to reconstruct the input. A U-shaped architecture is implemented to link corresponding TFF blocks of the encoder and decoder at the same level through skip connections. The semi-supervised model is trained using data exclusively from normal operations. Significant errors were generated during testing for anomalies with data distributions deviating from the training samples. Consequently, a threshold based on the magnitude of reconstruction errors was implemented to identify anomalies. Experimental validation was conducted using a custom dataset, including physical attacks as abnormal cases. The proposed framework achieved an accuracy and recall of approximately 0.93 and 0.83, respectively. A comparison with other benchmark models further verified its superior performance. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF
41. A semi-supervised anomaly detection strategy for drunk driving detection: a feasibility study.
- Author
-
Harrou, Fouzi, Kini, K. Ramakrishna, Madakyaru, Muddu, and Ying Sun
- Subjects
DRUNK driving ,ROAD safety measures ,INDEPENDENT component analysis ,DIGITAL cameras ,ROAD users ,GAS detectors - Abstract
Drunk driving poses a significant threat to road safety, necessitating effective detection methods to enhance preventive measures and ensure the well-being of road users. Recognizing the critical importance of identifying drunk driving incidents for public safety, this paper introduces an effective semi-supervised anomaly detection strategy. The proposed strategy integrates three key elements: Independent Component Analysis (ICA), Kantorovitch distance (KD), and double Exponentially Weighted Moving Average (DEWMA). ICA is used to handle non-gaussian and multivariate data, while KD is used to measure the dissimilarity between normal and abnormal events based on ICA features. The DEWMA is applied to KD charting statistics to detect changes in data and uses a nonparametric threshold to improve sensitivity. The primary advantage of this approach is its ability to perform anomaly detection without requiring labeled data. The study also used XGBoost for the later calculation of the SHAP (SHapley Additive exPlanations) values to identify the most important variables for detecting drunk driving behavior. The approach was evaluated using publicly available data from gas and temperature sensors, as well as digital cameras. The results showed that the proposed approach achieved an F1-score of 98% in detecting the driver's drunk status, outperforming conventional PCA-based and ICA-based methods. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF
42. ASOD: an adaptive stream outlier detection method using online strategy.
- Author
-
Hu, Zhichao, Yu, Xiangzhan, Liu, Likun, Zhang, Yu, and Yu, Haining
- Subjects
OUTLIER detection ,INFORMATION technology ,ANOMALY detection (Computer security) ,GAUSSIAN mixture models ,K-nearest neighbor classification ,ONLINE education - Abstract
In the current era of information technology, blockchain is widely used in various fields, and the monitoring of the security and status of the blockchain system is of great concern. Online anomaly detection for the real-time stream data plays vital role in monitoring strategy to find abnormal events and status of blockchain system. However, as the high requirements of real-time and online scenario, online anomaly detection faces many problems such as limited training data, distribution drift, and limited update frequency. In this paper, we propose an adaptive stream outlier detection method (ASOD) to overcome the limitations. It first designs a K-nearest neighbor Gaussian mixture model (KNN-GMM) and utilizes online learning strategy. So, it is suitable for online scenarios and does not rely on large training data. The K-nearest neighbor optimization limits the influence of new data locally rather than globally, thus improving the stability. Then, ASOD applies the mechanism of dynamic maintenance of Gaussian components and the strategy of dynamic context control to achieve self-adaptation to the distribution drift. And finally, ASOD adopts a dimensionless distance metric based on Mahalanobis distance and proposes an automatic threshold method to accomplish anomaly detection. In addition, the KNN-GMM provides the life cycle and the anomaly index for continuous tracking and analysis, which facilities the cause analysis and further interpretation and traceability. From the experimental results, it can be seen that ASOD achieves near-optimal F1 and recall on the NAB dataset with an improvement of 6% and 20.3% over the average, compared to baselines with sufficient training data. ASOD has the lowest F1 variance among the five best methods, indicating that it is effective and stable for online anomaly detection on stream data. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF
43. Drone Flight Detection at an Entrance to a Beehive Based on Audio Signals.
- Author
-
LIBAL, Urszula and BIERNACKI, Pawel
- Abstract
Spotting a significant number of drones flying near the entrance of a beehive during late Spring could indicate the occurrence of swarming mood, as the the surge in drone presence is related to an overcrowded hive. Swarming refers to a natural reproductive process witnessed in honey bees, wherein half of the bee colony departs from their hive alongside the aging queen. In the paper, we propose an early swarming detection mechanism that relies on the behavior of the drones. The proposed method is based on audio signals registered in a close proximity to the beehive entrance. A comparative study was performed to find the most effective preprocessing method for the audio signals for the detection problem. We have compared the results for three different power spectrum density coefficients estimation methods, which are used as an input of an autoencoder neural network to discriminate drones from worker bees. Through simulations employing real-life signals, it has been demonstrated that drone detection based solely on audio signals is indeed feasible. The attained level of detection accuracy enables the creation of an efficient alarm system for beekeepers. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF
44. Detection of Demagnetization Faults in Electric Motors by Analyzing Inverter Based Current Data Using Machine Learning Techniques.
- Author
-
Walch, Daniel, Blechinger, Christoph, Schellenberger, Martin, Hofmann, Maximilian, Eckardt, Bernd, and Lorentz, Vincent R.H.
- Subjects
ELECTRIC faults ,FAST Fourier transforms ,SUPPORT vector machines ,FOURIER transforms ,POWER electronics - Abstract
Demagnetization of the rotor magnets is a significant failure mode that can occur in permanent magnet synchronous machines (PMSMs). Early detection of demagnetization faults can help change system parameters to reduce power output or ensure safety. In this paper, the effects of demagnetization faults were analyzed both in simulation and experiments using the example of drone motors. An approach was investigated to detect even minor demagnetization faults that does not require any additional sensing effort. Machine learning (ML) techniques are used to analyze the phase current data directly received from the inverter to enable anomaly detection. For this purpose, the phase current is transformed by the Fast Fourier Transform (FFT), the spectral data is then reduced in dimensionality, followed by an anomaly detection algorithm using a one-class support vector machine (OC-SVM). To ensure simplified initialization of the ML model without the need for training sets of damaged drives, only data from magnetically undamaged motors was used to train the models for anomaly detection. Different selections of considered harmonics and different metrics were investigated using the experimental data, achieving a precision of up to 99%, a specificity of up to 98%, and an accuracy of up to 90%. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF
45. Lean Demonstration of On-Board Thermal Anomaly Detection Using Machine Learning.
- Author
-
Thoemel, Jan, Kanavouras, Konstantinos, Sachidanand, Maanasa, Hein, Andreas, Ortiz del Castillo, Miguel, Pauly, Leo, Rathinam, Arunkumar, and Aouada, Djamila
- Subjects
MOORE'S law ,INFRARED cameras ,COMPUTER performance ,MACHINE learning ,ORBITS (Astronomy) - Abstract
Moore's law states that the performance of computers doubles about every two years. This has dramatic consequences for any modern high development and for satellites. The long development cycles cause these expensive assets to be obsolete before the start of their operations. The advancement also presents challenges to their design, particularly from a thermal perspective, as more heat is dissipated and circuits are more fragile. These challenges mandate that faster spacecraft development methods are found and thermal management technologies are developed. We elaborate on existing development methodologies and present our own lean method. We explore the development of a thermal anomaly-detection payload, extending from conception to in-orbit commissioning, to stimulate discussions on space hardware development approaches. The payload consists of four miniaturized infrared cameras, heating sources in view of the cameras simulating an anomaly, an on-board processor, and peripherals for electrical and communication interfaces. The paper outlines our methodology and its application, showcasing the success of our efforts with the first-light activation of our cameras in orbit. We show our lean method, featuring reference technical and management models, from which we derive further development tools; such details are normally not available in the scientific-engineering literature. Additionally, we address the shortcomings identified during our development, such as the failure of an on-board component and propose improvements for future developments. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF
46. Malware Detection for Internet of Things Using One-Class Classification.
- Author
-
Shi, Tongxin, McCann, Roy A., Huang, Ying, Wang, Wei, and Kong, Jun
- Subjects
INTERNET of things ,ANOMALY detection (Computer security) ,CLASSIFICATION - Abstract
The increasing usage of interconnected devices within the Internet of Things (IoT) and Industrial IoT (IIoT) has significantly enhanced efficiency and utility in both personal and industrial settings but also heightened cybersecurity vulnerabilities, particularly through IoT malware. This paper explores the use of one-class classification, a method of unsupervised learning, which is especially suitable for unlabeled data, dynamic environments, and malware detection, which is a form of anomaly detection. We introduce the TF-IDF method for transforming nominal features into numerical formats that avoid information loss and manage dimensionality effectively, which is crucial for enhancing pattern recognition when combined with n-grams. Furthermore, we compare the performance of multi-class vs. one-class classification models, including Isolation Forest and deep autoencoder, that are trained with both benign and malicious NetFlow samples vs. trained exclusively on benign NetFlow samples. We achieve 100% recall with precision rates above 80% and 90% across various test datasets using one-class classification. These models show the adaptability of unsupervised learning, especially one-class classification, to the evolving malware threats in the IoT domain, offering insights into enhancing IoT security frameworks and suggesting directions for future research in this critical area. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF
47. Anomaly Detection Based on GCNs and DBSCAN in a Large-Scale Graph.
- Author
-
Retiti Diop Emane, Christopher, Song, Sangho, Lee, Hyeonbyeong, Choi, Dojin, Lim, Jongtae, Bok, Kyoungsoo, and Yoo, Jaesoo
- Subjects
ANOMALY detection (Computer security) ,INTRUSION detection systems (Computer security) ,REPRESENTATIONS of graphs ,DATA structures ,DEEP learning ,DATA integrity ,FUZZY algorithms - Abstract
Anomaly detection is critical across domains, from cybersecurity to fraud prevention. Graphs, adept at modeling intricate relationships, offer a flexible framework for capturing complex data structures. This paper proposes a novel anomaly detection approach, combining Graph Convolutional Networks (GCNs) and Density-Based Spatial Clustering of Applications with Noise (DBSCAN). GCNs, a specialized deep learning model for graph data, extracts meaningful node and edge representations by incorporating graph topology and attribute information. This facilitates learning expressive node embeddings capturing local and global structural patterns. For anomaly detection, DBSCAN, a density-based clustering algorithm effective in identifying clusters of varying densities amidst noise, is employed. By defining a minimum distance threshold and a minimum number of points within that distance, DBSCAN proficiently distinguishes normal graph elements from anomalies. Our approach involves training a GCN model on a labeled graph dataset, generating appropriately labeled node embeddings. These embeddings serve as input to DBSCAN, identifying clusters and isolating anomalies as noise points. The evaluation on benchmark datasets highlights the superior performance of our approach in anomaly detection compared to traditional methods. The fusion of GCNs and DBSCAN demonstrates a significant potential for accurate and efficient anomaly detection in graphs. This research contributes to advancing graph-based anomaly detection, with promising applications in domains where safeguarding data integrity and security is paramount. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF
48. Using heterogeneous computing and edge computing to accelerate anomaly detection in remotely sensed multispectral images
- Author
-
López-Fandiño, Javier, B. Heras, Dora, and Argüello, Francisco
- Published
- 2024
- Full Text
- View/download PDF
49. Navigating the metric maze: a taxonomy of evaluation metrics for anomaly detection in time series
- Author
-
Sørbø, Sondre and Ruocco, Massimiliano
- Published
- 2024
- Full Text
- View/download PDF
50. Development of Automatic Visual Anomaly Detection System for Data Centers.
- Author
-
Enkhbaatar, Misheel and Yamazaki, Tatsuya
- Subjects
SERVER farms (Computer network management) ,LIGHT emitting diodes ,INSPECTION & review ,SYSTEMS design ,ELECTRONIC equipment ,VIDEO recording ,DIGITAL cameras - Abstract
In this paper, we present a practical automated visual monitoring system designed to enhance the efficiency of visual inspection in data centers. Visual inspection is a manual process of detecting failures in electronic devices based on light-emitting diode (LED) lighting. The objective of data center monitoring is to implement real-time failure detection to prevent any service disruptions or loss of user data. To improve the reliability of data centers, we propose a monitoring system that automatically detects anomalies in electronic devices. The system integrates a digital camera and a novel algorithm that is tailored to distinguish normal LED lighting patterns from abnormal patterns. Experimental data were collected in an actual data center room and the system was evaluated with experiments involving LED region segmentation and anomaly detection. For the LED segmentation task, we propose a K-means-based method that outperformed a previous method based on background subtraction by 8%. For anomaly detection, recorded videos covering continuous monitoring of approximately 17 h were used. The proposed method successfully detected all five true anomalies in the video data. The results of another experiment for anomaly detection demonstrate that prolonged video recording for collecting patterns of LED lighting can positively contribute to a better understanding of normal patterns and can effectively be used to ensure the detection of device anomalies. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF
Discovery Service for Jio Institute Digital Library
For full access to our library's resources, please sign in.