Search

Unsupervised anomaly detection tackles the problem of finding anomalies inside datasets without the labels availability; since data tagging is typically hard or expensive to obtain, such approaches have seen huge applicability in recent years. In this context, Isolation Forest is a popular algorithm able to define an anomaly score by means of an ensemble of peculiar trees called isolation trees. These are built using a random partitioning procedure that is extremely fast and cheap to train. However, we find that the standard algorithm might be improved in terms of memory requirements, latency and performances; this is of particular importance in low resources scenarios and in TinyML implementations on ultra-constrained microprocessors. Moreover, Anomaly Detection approaches currently do not take advantage of weak supervisions: being typically consumed in Decision Support Systems, feedback from the users, even if rare, can be a valuable source of information that is currently unexplored. Beside showing iForest training limitations, we propose here TiWS-iForest, an approach that, by leveraging weak supervision is able to reduce Isolation Forest complexity and to enhance detection performances. We showed the effectiveness of TiWS-iForest on real word datasets and we share the code in a public repository to enhance reproducibility.

Within the context of Industry 4.0, quality assessment pro- cedures using data-driven techniques are becoming more critical due to the generation of massive amounts of production data. In this paper, we address the detection of abnormal screw tightening processes, which is a relevant industrial task. Since labeling is costly, requiring a manual effort, we focus on unsupervised approaches. In particular, we assume a low-dimensional input screw fastening approach that is based only on angle-torque pairs. Using such pairs, we explore three main unsuper- vised Machine Learning (ML) algorithms: Local Outlier Factor (LOF), Isolation Forest (iForest) and a deep learning Autoencoder (AE). For benchmarking purposes, we also explore a supervised Random Forest (RF) algorithm. Several computational experiments were held by us- ing recent industrial data with 2.8 million angle-torque pair records and a realistic and robust rolling window evaluation. Overall, high quality anomaly discrimination results were achieved by the iForest (99%) and AE (95% and 96%) unsupervised methods, which compared well against the supervised RF (99% and 91%). When compared with iForest, the AE requires less computation effort and provides faster anomaly detection response times., This work is supported by: European Structural and Investment Funds in the FEDER component, through the Operational Competitiveness and Internation- alization Programme (COMPETE 2020) [Project n 39479; Funding Reference: POCI-01-0247-FEDER-39479].

Данная работу посвящена разработке алгоритма выявления нетипичного поведения пользователя, выполняющего терминальные команды на удаленном сервере. Задачи, которые решались в Ñ Ð¾Ð´Ðµ разработки: 1. Моделирование поведения системного администратора и атакующей стороны. 2. Настройка audit сервиса для получения логов и Ð¸Ñ Ð¿Ð¾ÑÐ»ÐµÐ´ÑƒÑŽÑ‰ÐµÐ³Ð¾ преоб­ разования в данные. 3. Применение и анализ Ñ€Ð°Ð·Ð»Ð¸Ñ‡Ð½Ñ‹Ñ Ð¼ÐµÑ‚Ð¾Ð´Ð¾Ð² машинного обучения для задачи поиска аномалий. В работе были предложены модели для имитации поведения системного администратора и атакующей стороны, предложена конфигурация audit сервиса, разработана математическая модель Ð´Ð°Ð½Ð½Ñ‹Ñ Ð¸ рассмотрены алгоритмы изоляцион­ного леса, одноклассого метода Ð¾Ð¿Ð¾Ñ€Ð½Ñ‹Ñ Ð²ÐµÐºÑ‚Ð¾Ñ€Ð¾Ð² и Ð½ÐµÐ¹Ñ€Ð¾Ð½Ð½Ñ‹Ñ ÑÐµÑ‚ÐµÐ¹ для поиска нетипичного поведения пользователя. В результате был произведен сравнительный анализ Ñ€Ð°ÑÑÐ¼Ð¾Ñ‚Ñ€ÐµÐ½Ð½Ñ‹Ñ Ð°Ð»Ð³Ð¾Â­Ñ€Ð¸Ñ‚Ð¼Ð¾Ð², произведена оценка возможности применения методов к данной задаче., This program is designed to develop algorithms for detection anomaly user’s behavior, that execute terminal commands on the remote server. Tasks that were solved during development: 1. Modeling the behavior of the system administrator and the attacker. 2. Configuring the audit service to receive logs and then convert them into data. 3. Application and analysis of various machine learning methods for the problem of finding anomalies. The mathematical model of the data is developed. In this work, models were proposed to simulate the behavior of the system administrator and the attacker. The audit service configuration is proposed. The algorithms of the isolation forest, the one-class svm and neural networks for searching for anomaly user behavior are considered. As a result, comparative analysis of the considered algorithms was performed and the possibility of applying the methods to this problem is evaluated

This paper proposes an indoor positioning method based on iBeacon technology that combines anomaly detection and a weighted Levenberg&ndash, Marquadt (LM) algorithm. The proposed solution uses the isolation forest algorithm for anomaly detection on the collected Received Signal Strength Indicator (RSSI) data from different iBeacon base stations, and calculates the anomaly rate of each signal source while eliminating abnormal signals. Then, a weight matrix is set by using each anomaly ratio and the RSSI value after eliminating the abnormal signal. Finally, the constructed weight matrix and the weighted LM algorithm are combined to solve the positioning coordinates. An Android smartphone was used to verify the positioning method proposed in this paper in an indoor scene. This experimental scenario revealed an average positioning error of 1.540 m and a root mean square error (RMSE) of 1.748 m. A large majority (85.71%) of the positioning point errors were less than 3 m. Furthermore, the RMSE of the method proposed in this paper was, respectively, 38.69%, 36.60%, and 29.52% lower than the RMSE of three other methods used for comparison. The experimental results show that the iBeacon-based indoor positioning method proposed in this paper can improve the precision of indoor positioning and has strong practicability.

In fiscal year 2020, the U.S. Army spent nearly $77 billion on contracts. Auditors employ various techniques, including anomaly detection, to select contracts that merit scrutiny. But in a resource-constrained environment, auditors can review only a limited number of contracts. Using data obtained from USAspending.gov, we consider how anomaly detection combined with dimensionality reduction can be used to recommend contracts for investigation. We analyze over 20,000 fixed-price Army contracts between fiscal years 2017 to 2020, using more than one hundred combinations of dimensionality reduction and anomaly detection techniques, and formations of artificial anomalies. A consistent finding is that dimensionality reduction using principal components or autoencoders is not demonstrably beneficial. This finding may be due to the discrete nature of the USAspending.gov data and may not apply to other data sets. The best performance is obtained using isolation forests for anomaly detection without dimensionality reduction. Outstanding Thesis Major, United States Army Approved for public release. distribution is unlimited

Visual perception-based methods are a promising means of capturing the surface damage state of wire ropes and hence provide a potential way to monitor the condition of wire ropes. Previous methods mainly concentrated on the handcrafted feature-based flaw representation, and a classifier was constructed to realize fault recognition. However, appearances of outdoor wire ropes are seriously affected by noises like lubricating oil, dust, and light. In addition, in real applications, it is difficult to prepare a sufficient amount of flaw data to train a fault classifier. In the context of these issues, this study proposes a new flaw detection method based on the convolutional denoising autoencoder (CDAE) and Isolation Forest (iForest). CDAE is first trained by using an image reconstruction loss. Then, it is finetuned to minimize a cost function that penalizes the iForest-based flaw score difference between normal data and flaw data. Real hauling rope images of mine cableways were used to test the effectiveness and advantages of the newly developed method. Comparisons of various methods showed the CDAE-iForest method performed better in discriminative feature learning and flaw isolation with a small amount of flaw training data.

Anomaly detection for wind turbine condition monitoring is an active area of research within the wind energy operations and maintenance (O & M) community. In this paper three models were compared for multi-megawatt operational wind turbine SCADA data. The models used for comparison were One-Class Support Vector Machine (OCSVM), Isolation Forest (IF), and Elliptical Envelope (EE). Each of these were compared for the same fault, and tested under various different data configurations. IF and EE have not previously been used for fault detection for wind turbines, and OCSVM has not been used for SCADA data. This paper presents a novel method of condition monitoring that only requires two months of data per turbine. These months were separated by a year, the first being healthy and the second unhealthy. The number of anomalies is compared, with a greater number in the unhealthy month being considered correct. It was found that for accuracy IF and OCSVM had similar performances in both training regimes presented. OCSVM performed better for generic training, and IF performed better for specific training. Overall, IF and OCSVM had an average accuracy of 82% for all configurations considered, compared to 77% for EE.

In this PhD-thesis several new and existing data science application are described that are particularly focused on applications for tax administrations. The thesis contains a chapter on the managerial side of analytics with a balanced overview of the pros and cons of applying analytics within taxpayer supervision. Another topic is (tax) fraud detection with unsupervised anomaly detection techniques. Here a new type of outliers is described (singular outliers) and an algorithm is provided for finding them. Attention is also paid to improving risk selection models. It is noted that most current algorithms cannot treat interactions of categorical variables with many levels very well. An extension of logistic regression is provided that uses Factorization Machines, which resulted in a ten percent improvement in precision. A fourth topic is statistical testing on similar treatment of similar cases. A contribution is made by providing an algorithm to statistically test on similar treatment based on process logs. The thesis contains further a benchmark study of different anomaly detection algorithms. Finally HR Analytics, Reinforcement Learning and applications of fuzzy sets are shortly described.

Susidomėjimas blokų grandinės technologija auga nuo šios idėjos atsiradimo 2008 metais. Tai palyginti nauja technologija, kuri gyvuoja dar tik apie 12 metų, tačiau sulaukia nemažai žiniasklaidos ir mokslininkų dėmesio. Pagrindinis domėjimosi objektas yra bitkoino kriptografinė valiuta, kuriai blokų grandinė pirmiausia ir buvo sukurta. Kriptografinių valiutų populiarumas pritraukia ir įvairių sukčių, kurie vykdo nekorektiškas veiklas ir stengiasi gauti finansinės naudos. Kol kas 2019 metai pagal sukčiavimo apimtis buvo patys didžiausi ir per šiuos metus padaryta žala yra vertinama 4,3 milijardais dolerių. Bitkoinui tenka didžiausia šių nuostolių suma, tačiau antra pagal kapitalizacijos dydį eterio kriptografinė valiuta taip pat sulaukia sukčių dėmesio. Sukčiavimo aptikimas yra pirmas veiksmas norint sumažinti riziką ir apsisaugoti nuo galimų vagysčių ir apgavysčių. Šio darbo tikslas – pritaikant didžiųjų duomenų analitikos metodus sukurti mašininio mokymosi modelį, kuris įgalintų apdoroti didelius duomenų kiekius ir sėkmingai aptikti sukčiavimus bitkoino ir eterio blokų grandinėje. Visi bitkoino ir eterio sandoriai yra viešai prieinami. Panaudojant šiuos didžiuosius duomenis buvo išskirti požymiai (atliktų pavedimų skaičius, vidutinė atliktų pavedimų vertė ir pan.), kurie naudojami kuriant modelius. Sukčiavimo aptikimui buvo sukurti išskirčių nustatymo modeliai paremti k-vidurkių ir izoliavimo miško metodais. Dėl turimo didelio duomenų kiekio buvo kuriami atitinkamų modelių ansambliai. Sukurti mašininio mokymosi modeliai leido identifikuoti kriptografinių valiutų adresus (bankinės sąskaitos atitikmuo kriptografinių valiutų blokų grandinėje), kurie yra susiję su sukčiavimo atvejais. Bitkoino kriptografinės valiutos blokų grandinėje vienas k-vidurkių modelis, k-vidurkių modelių ansamblis, izoliavimo miško modelių ansamblis aptiko panašų apgaulių kiekį (29–30). Eterio kriptografinės valiutos blokų grandinėje sukčiavimą geriausiai sekėsi aptikti k-vidurkių modelių ansambliui, kuris iš viso aptiko 65 apgaules. Rezultatams patikrinti buvo naudojami trys skirtingi apgaulių duomenų rinkiniai. Iš „BitcoinTalk“ duomenų rinkinio pavyko identifikuoti 15 iš 16 bitkoino adresų, susijusių su apgaulėmis. Tai yra labai geras rezultatas, nes panašiuose tyrimuose, iš šio duomenų rinkinio, daugiausiai pavykdavo aptikti tik 5 apgaulės atvejus. Iš Ponzi schemų duomenų rinkinio eterio blokų grandinėje pavyko identifikuoti 64 apgaules iš 102. Iš „CryptoScamDB“ duomenų rinkinio bitkoino blokų grandinėje dėl ten patenkančių mažesnio mąsto apgaulių identifikuoti pavyko 14 apgaulių iš 140. Taip pat atlikti tyrimai parodė, kad mašininio mokymosi modeliai sukurti naudojant bitkoino sandorių duomenis, gali būti sėkmingai panaudoti aptinkant sukčiavimo atvejus eterio blokų grandinėje. Tačiau, modeliai, kurti naudojant bitkoino sandorių duomenis, aptinka mažiau apgaulių eterio blokų grandinėje, nei modeliai sukurti naudojant eterio sandorių duomenis., Interest in blockchain technology has been growing since 2008 when this concept was created. It is a relatively new technology that has been around for only 12 years but has received much attention in the media and from academics. The main object of the media’s focus is the bitcoin cryptocurrency, for which blockchain technology was first developed. The popularity of cryptocurrencies also attracts various scammers who engage in improper activities and seek financial gain. So far, the most significant damage has been done in 2019 and is estimated at $ 4.3 billion. As bitcoin is the most popular cryptocurrency, it bears the most considerable amount of damage caused by these thefts and frauds. Ethereum, the second-largest cryptocurrency by capitalization, is also receiving attention from scammers. Fraud detection is the first step in reducing risk and preventing potential theft and fraud. This study aims to develop a machine learning model using big data analytics methods that would be able to process large amounts of data and successfully identify fraud within the bitcoin and ethereum blockchain. All bitcoin and ethereum transactions are publicly available. Using these big data, the features (number of transactions received, an average value of the received transaction, etc.) that were used to develop the models were extracted. The k-means and the isolation forest methods were applied to create fraud detection model. Due to the big amount of data available, ensembles of these methods were developed. The developed machine learning models identified addresses that are associated with cases of fraud and scam. Looking at the overall results, one k-means model, an ensemble of k-means models, and an ensemble of isolation forest models found almost the same number of frauds in the bitcoin blockchain (29–30). In the ethereum blockchain, frauds were best detected by using an ensemble of k-means models, which caught a total of 65 scams. Three different data sets of fraud were used to verify the results. The developed models in the BitcoinTalk dataset identified 15 of the 16 bitcoin addresses associated with frauds. This is very good result, as a maximum 5 cases of fraud were detected in similar studies before. In the Ponzi schemes dataset were identified 64 scams in the ethereum blockchain out of 102. The developed models in the CryptoScamDB dataset identified 14 scams in the bitcoin blockchain of 140 because this dataset included smaller scams. Studies by other authors using Ponzi schemes and CryptoScamDB dataset use different methods (e.g. classification methods are used, results are calculated differently) and therefore the results are not comparable. This study has also shown that machine learning models developed using bitcoin transaction data can be successfully used to detect fraud in the ethereum blockchain. However, models developed using bitcoin transaction data detect fewer cases of scam in the ethereum blockchain than models developed using ethereum transaction data.

Micro resistance spot welding (MRSW) is an important technology widely used in electronics manufacturing for micro component joining. For the joining of micro enameled wire, quality control is heavily dependent on manual inspection till now. In this paper, a quality monitoring approach based on isolation forest (iForest) is proposed to identify abnormal welds and normal welds. Electrode voltage and welding current of over 110,000 spot welds were collected from a production line. The dynamic resistance and heat input were calculated for all welds and used for feature extraction. A class imbalance problem existed in the collected dataset because abnormal welds were far fewer than normal welds. The anomaly detection model based on iForest was established for the imbalanced data classification after comparison with other methods such as one-class (support vector machine) SVM and local outlier factor. Test results show that the similarity of dynamic resistance profile and heat input compared with the previous ten welds are valid features for detecting a part of the abnormal welds. The iForest model is effective for distinguishing incomplete fusion welds from normal welds with high efficiency. It can assist in the on-line quality monitoring of enameled wire welding process in production.

The spatial distribution of soil moisture (SM) was estimated by a multiple quantile regression (MQR) model with Terra Moderate Resolution Imaging Spectroradiometer (MODIS) and filtered SM data from 2013 to 2015 in South Korea. For input data, observed precipitation and SM data were collected from the Korea Meteorological Administration and various institutions monitoring SM. To improve the work of a previous study, prior to the estimation of SM, outlier detection using the isolation forest (IF) algorithm was applied to the observed SM data. The original observed SM data resulted in IF_SM data following outlier detection. This study obtained an average data removal rate of 20.1% at 58 stations. For various reasons, such as instrumentation, environment, and random errors, the original observed SM data contained approximately 20% uncertain data. After outlier detection, this study performed a regression analysis by estimating land surface temperature quantiles. The soil characteristics were considered through reclassification into four soil types (clay, loam, silt, and sand), and the five-day antecedent precipitation was considered in order to estimate the regression coefficient of the MQR model. For all soil types, the coefficient of determination (R2) and root mean square error (RMSE) values ranged from 0.25 to 0.77 and 1.86% to 12.21%, respectively. The MQR results showed a much better performance than that of the multiple linear regression (MLR) results, which yielded R2 and RMSE values of 0.20 to 0.66 and 1.08% to 7.23%, respectively. As a further illustration of improvement, the box plots of the MQR SM were closer to those of the observed SM than those of the MLR SM. This result indicates that the cumulative distribution functions (CDF) of MQR SM matched the CDF of the observed SM. Thus, the MQR algorithm with outlier detection can overcome the limitations of the MLR algorithm by reducing both the bias and variance.

This paper proposes a generic approach for crop anomaly detection at the parcel-level based on unsupervised point anomaly detection techniques. The input data is derived from synthetic aperture radar (SAR) and optical images acquired using Sentinel-1 and Sentinel-2 satellites. The proposed strategy consists of four sequential steps: acquisition and preprocessing of optical and SAR images, extraction of optical and SAR indicators, computation of zonal statistics at the parcel-level and point anomaly detection. This paper analyzes different factors that can affect the results of anomaly detection such as the considered features and the anomaly detection algorithm used. The proposed procedure is validated on two crop types in Beauce (France), namely, rapeseed and wheat crops. Two different parcel delineation databases are considered to validate the robustness of the strategy to changes in parcel boundaries.

In the last decade, a new class of cyber-threats has emerged. This new cybersecurity adversary is known with the name of "Advanced Persistent Threat" (APT) and is referred to different organizations that in the last years have been "in the center of the eye" due to multiple dangerous and effective attacks targeting financial and politic, news headlines, embassies, critical infrastructures, TV programs, etc. In order to early identify APT related malware, a semi-automatic approach for malware samples analysis is needed. In our previous work we introduced a "malware triage" step for a semi-automatic malware analysis architecture. This step has the duty to analyze as fast as possible new incoming samples and to immediately dispatch the ones that deserve a deeper analysis, among all the malware delivered per day in the cyber-space, the ones that really worth to be further examined by analysts. Our paper focuses on malware developed by APTs, and we build our knowledge base, used in the triage, on known APTs obtained from publicly available reports. In order to have the triage as fast as possible, we only rely on static malware features, that can be extracted with negligible delay, and use machine learning techniques for the identification. In this work we move from multiclass classification to a group of oneclass classifier, which simplify the training and allows higher modularity. The results of the proposed framework highlight high performances, reaching a precision of 100% and an accuracy over 95%

The analysis of big data is a fundamental challenge for the current and future stream of data coming from many different sources. Geospatial data is one of the sources currently less investigated. A typical example of always increasing data set is that produced by the distribution data of invasive species on the concerned territories. The dataset of Drosophila suzuki invasion sites in Europe up to 2011 was used to test a possible method to pinpoint its outliers (anomalies). Our aim was to find a method of analysis that would be able to treat large amount of data in order to produce easily readable outputs to summarize and predict the status and, possibly, the future development of a biological invasion. To do that, we aimed to identify the so called anomalies of the dataset, identified with a Python script based on the machine learning algorithm “Isolation Forest”. We used also the K-Means clustering method to partition the dataset. In our test, based on a real dataset, the Silhouette method yielded a number of clusters of 10 as the best result. The clusters were drawn on the map with a Voronoi tessellation, showing that 8 clusters were centered on industrial harbours, while the last two were in the hinterland. This fact led us to guess that: (1) the main entrance mechanisms in Europe may be the wares import fluxes through ports, occurring apparently several times; (2) the spreading into the inland may be due to road transportation of wares; (3) the outliers (anomalies) found with the isolation forest method would identify individuals or populations that tend to detach from their original cluster and hence represent indications about the lines of further spreading of the invasion. This type of analysis aims hence to identify the future direction of an invasion, rather than the center of origin as in the case of geographic profiling. Isolation Forest provides therefore complimentary results with respect to PGP. The recent records of the invasive species, mainly localized close to the outliers position, are an indication that the isolation forest method can be considered predictive and proved to be a useful method to treat large datasets of geospatial data.

Выпускная квалификационная работа посвящена разработке системы для выявления аномалий при магнитометрической диагностике. Данная система предназначена для выявления аномалий в трубопроводе. В Ñ Ð¾Ð´Ðµ работы рассмотрены существующие решения, основанные на Ð°Ð»Ð³Ð¾Ñ€Ð¸Ñ‚Ð¼Ð°Ñ Ð¼Ð°ÑˆÐ¸Ð½Ð½Ð¾Ð³Ð¾ обучения, проведен анализ ÑÑƒÑ‰ÐµÑÑ‚Ð²ÑƒÑŽÑ‰Ð¸Ñ ÑÐ¿Ð¾ÑÐ¾Ð±Ð¾Ð² распознавания аномалий с помощью машинного обучения. Результатом работы является система, обученная распознавать аномалии на ÑƒÑ‡Ð°ÑÑ‚ÐºÐ°Ñ Ñ‚Ñ€ÑƒÐ±Ð¾Ð¿Ñ€Ð¾Ð²Ð¾Ð´Ð¾Ð²., The final qualification work is devoted to the development of a system for detecting anomalies in magnetometric diagnostics. This system is designed to detect defects in the pipeline. In the course of the work, existing solutions based on machine learning algorithms were considered, an analysis was made of existing methods for recognizing anomalies using machine learning. The result of the work is a system trained to recognize defects in pipeline sections.

Метою дипломної роботи є визначення різниці між пацієнтами з нормальними сигналами електрокардиограми. Для цього необхідно створити норму для групи молодих людей та визначити їх «відстань» до створеної норми. На основі отриманих результатів складені висновки про різницю між клінічно здоровими та схожими пацієнтами. В першому розділі розглянуті теоретичні відомості про історію ЕКГ, методику отримання сигналу ЕКГ, способи отримання усередненого комплексу та можливе їх використання. В другому розділі описується теорія машинного навчання, запропонований метод вирішення поставленого перед роботою завдання. Відповідно запропонованого способу розглянуті доступні алгоритми машинного навчання та обраний найбільш підходящий (Isolation Forest). В третьому розділі описані використовувані вхідні дані. Вони отримані з бази даних курсантів військового училища. Вікова категорія – 17 - 19 років, всі пацієнти вважаються клінічно здоровими. На стадії формування завдання дипломної роботи було прийнято рішення використовувати усереднений кардіокомплекс. В практичній частині створений код для вичитування і форматування вхідних даних з файлів. Використовуючи обраний метод побудована область норми та визначені аномалії відносно неї. Отримані результати порівняні з аналізом програми Cardiol. Всі пацієнти, судячи з результатів програми, мають деякі відхилення в комплексі показників, що відповідають за стан міокарду в різній мірі. Зроблено висновок про поняття норми для різни вікових категорій пацієнтів та запропоновані методи покращення результатів експерименту. The purpose of this thesis is determination the difference between patients with normal ECG signals. Necessary to create a “normal cloud” for a group of young people and determine their “distance” to the established “cloud”. Based on the results obtained, it is possible to draw conclusions about difference between clinically healthy and similar patients. Partially answer the question: “How healthy is the patient?”. In the first section, there are theoretical information about the history of the ECG, method of obtaining an ECG signal, methods of obtaining an averaged ECG complex and their possible using. In the second section there are describes the theory of machine learning, proposed method of increasing main problem of labor. According to the proposed method, the available machine learning algorithms are considered and the most suitable (Isolation Forest) was selected. It is advantages are: 1. The possibility to work with multidimensional input data; 2. Low consumption computer resource; 3. The possibility of automatic formation of a “normal” sample. The third section describes the inputs used. They are obtained from the database of cadets of military school. Age category – 17-19 years, all of patients are clinically healthy. At the stage of forming the thesis task, it was decided to use an averaged cardio complex. In the practical part, code is created for reading and formatting input data from files. The “norm cloud” was constructed and anomalies are defined. The obtained results was comparable with the analysis of the program, that names Cardiol. All patients have some deviations in the set of parameters that are responsible for the state of the myocardium. Interesting, that 36% of them do not belong to a given age category. They are 67% of similar records in the entire database. Based on this fact conclusion – concept of norm for different age categories of patients is differ. The following steps have been proposed to improve the outcomes: 1. Refinement and increase the database; 2. To find the most weighty parameters for used algorithm; 3. More correctly find the parameters.

International audience; This paper studies the detection of anomalous crop development at the parcel-level based on an unsupervised outlier detection technique. The experimental validation is conducted on rapeseed and wheat parcels located in Beauce (France). The proposed methodology consists of four sequential steps: (1) preprocessing of synthetic aperture radar (SAR) and multispectral images acquired using Sentinel-1 and Sentinel-2 satellites, (2) extraction of SAR and multispectral pixel-level features, (3) computation of parcel-level features using zonal statistics and (4) outlier detection. The different types of anomalies that can affect the studied crops are analyzed and described. The different factors that can influence the outlier detection results are investigated with a particular attention devoted to the synergy between Sentinel-1 and Sentinel-2 data. Overall, the best performance is obtained when using jointly a selection of Sentinel-1 and Sentinel-2 features with the isolation forest algorithm. The selected features are co-polarized (VV) and cross-polarized (VH) backscattering coefficients for Sentinel-1 and five Vegetation Indexes for Sentinel-2 (among us, the Normalized Difference Vegetation Index and two variants of the Normalized Difference Water). When using these features with an outlier ratio of 10%, the percentage of detected true positives (i.e., crop anomalies) is equal to 94.1% for rapeseed parcels and 95.5% for wheat parcels.

Made available in DSpace on 2022-04-29T08:36:56Z (GMT). No. of bitstreams: 0 Previous issue date: 2022-02-15 Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES) Estimating the quality of GNSS vectors is decisive in planning GNSS networks and in several land surveying activities. These vectors are indirectly present in many civil infrastructures usually in the first stages of constructions and thus of high importance. In this research we assembled over 1000 baselines producing an extensive database with over 170,000 processed vectors. We propose a novel identification of outlying GNSS vectors based on the Isolation Forest (IF) algorithm based on the vectors deviations. And a new procedure to build linear models based on metaheuristics and a penalty function. The linear regressions presented models with a coefficient of determination R2 up to 0.996. The observation time span variable remained in all equations at least twice, showing its importance for the outcome quality of a vector. Overall, the three-dimensional deviation of vectors processed with broadcast ephemeris is 2.4 times higher than for precise ephemerides. Graduate Program in Applied Computing Unisinos University, Av. Unisinos, 950 Department of Civil Construction Federal Institute of Santa Catarina Graduate Program in Geodetic Sciences Federal University of Paraná Institute of Geography Federal University of Uberlandia Graduate Program in Remote Sensing Federal University of Rio Grande do Sul São Paulo State University UNESP, Presidente Prudente São Paulo State University UNESP, Presidente Prudente CAPES: 001

As a new kind of heating technology, microwave heating could replace traditional heating methods, because it has the advantages of high efficiency, no secondary pollution, and rapid heating. But the microwave heating process, which involves complex coupling between time-varying electromagnetic field and thermal field, is extremely complicated. At this point, the heated medium may produce local overheating. Worse, it may cause unexpected safety accidents, such as burning and even explosion. However, the temperature variation during the period of microwave heating could barely be obtained. In order to solve the problem of local overheating, this paper proposes a deep learning algorithm based on multi-dimensional data to construct an anomaly detection model for detecting local overheating. The algorithm consists of convolutional neural networks (CNNs) and unsupervised learning method named isolation forest algorithm (IFA). First, CNNs is utilized to extract features of the data collected from a WXD15S microwave heating system. Then, IFA detects the local overheating. Compared with the algorithm with common model, experiment results show that the proposed algorithm owns better measurement performance and higher precision.

The development of efficient and interpretable anomaly detection systems is fundamental to keeping production costs low, and is an active area of research in semiconductor manufacturing, particularly in the context of using Optical Emission Spectroscopy (OES) data. The high dimension and correlated nature of OES data can limit the performance achievable with anomaly detection systems. In this paper we present a dimensionality reducing variable selection and isolation forest based anomaly detection and diagnosis methodology that addresses these issues. In particular, it takes account of isolated variables that can be overlooked when using conventional approaches such as PCA, and provides greater interpretability than afforded by PCA. The proposed methodology is illustrated with the aid of simulated and industrial plasma etch case studies.

Background With the development and promotion of wearable devices and their mobile health (mHealth) apps, physiological signals have become a research hotspot. However, noise is complex in signals obtained from daily lives, making it difficult to analyze the signals automatically and resulting in a high false alarm rate. At present, screening out the high-quality segments of the signals from huge-volume data with few labels remains a problem. Signal quality assessment (SQA) is essential and is able to advance the valuable information mining of signals. Objective The aims of this study were to design an SQA algorithm based on the unsupervised isolation forest model to classify the signal quality into 3 grades: good, acceptable, and unacceptable; validate the algorithm on labeled data sets; and apply the algorithm on real-world data to evaluate its efficacy. Methods Data used in this study were collected by a wearable device (SensEcho) from healthy individuals and patients. The observation windows for electrocardiogram (ECG) and respiratory signals were 10 and 30 seconds, respectively. In the experimental procedure, the unlabeled training set was used to train the models. The validation and test sets were labeled according to preset criteria and used to evaluate the classification performance quantitatively. The validation set consisted of 3460 and 2086 windows of ECG and respiratory signals, respectively, whereas the test set was made up of 4686 and 3341 windows of signals, respectively. The algorithm was also compared with self-organizing maps (SOMs) and 4 classic supervised models (logistic regression, random forest, support vector machine, and extreme gradient boosting). One case validation was illustrated to show the application effect. The algorithm was then applied to 1144 cases of ECG signals collected from patients and the detected arrhythmia false alarms were calculated. Results The quantitative results showed that the ECG SQA model achieved 94.97% and 95.58% accuracy on the validation and test sets, respectively, whereas the respiratory SQA model achieved 81.06% and 86.20% accuracy on the validation and test sets, respectively. The algorithm was superior to SOM and achieved moderate performance when compared with the supervised models. The example case showed that the algorithm was able to correctly classify the signal quality even when there were complex pathological changes in the signals. The algorithm application results indicated that some specific types of arrhythmia false alarms such as tachycardia, atrial premature beat, and ventricular premature beat could be significantly reduced with the help of the algorithm. Conclusions This study verified the feasibility of applying the anomaly detection unsupervised model to SQA. The application scenarios include reducing the false alarm rate of the device and selecting signal segments that can be used for further research.

International audience; For the purpose of monitoring the behavior of complex infrastructures (e.g. aircrafts, transport or energy networks), high-rate sensors are deployed to capture multivariate data, generally unlabeled, in quasi continuous-time to detect quickly the occurrence of anomalies that may jeopardize the smooth operation of the system of interest. The statistical analysis of such massive data of functional nature raises many challenging methodological questions. The primary goal of this paper is to extend the popular Isolation Forest (IF) approach to Anomaly Detection, originally dedicated to finite dimensional observations, to functional data. The major difficulty lies in the wide variety of topological structures that may equip a space of functions and the great variety of patterns that may characterize abnormal curves. We address the issue of (randomly) splitting the functional space in a flexible manner in order to isolate progressively any trajectory from the others, a key ingredient to the efficiency of the algorithm. Beyond a detailed description of the algorithm, computational complexity and stability issues are investigated at length. From the scoring function measuring the degree of abnormality of an observation provided by the proposed variant of the IF algorithm, a Functional Statistical Depth function is defined and discussed as well as a multivariate functional extension. Numerical experiments provide strong empirical evidence of the accuracy of the extension proposed.

In the past recent years, Machine Learning methodologies have been applied in countless application areas. In particular, they play a key role in enabling Industry 4.0. However, one of the main obstacles to the diffusion of Machine Learning-based applications is related to the lack of interpretability of most of these methods. In this work, we propose an approach for defining a ‘feature importance’ in Anomaly Detection problems. Anomaly Detection is an important Machine Learning task that has an enormous applicability in industrial scenarios. Indeed, it is extremely relevant for the purpose of quality monitoring. Moreover, it is often the first step towards the design of a Machine Learning-based smart monitoring solution because Anomaly Detection can be implemented without the need of labelled data. The proposed feature importance evaluation approach is designed for Isolation Forest, one of the most commonly used algorithm for Anomaly Detection. The efficacy of the proposed method is tested on synthetic and real industrial datasets.

International audience; E-coupons (electronic coupons) have been a mainstay of online marketing to attract consumers and promote them to repeat purchase, distributing right e-coupons to right consumers is of critical importance. In big data era, analyzing consumers preferences for e-coupons by their online behavior and the impact of data imbalance caused by low active consumers are rarely studied. Thus, we propose a two-stage hybrid model. Firstly, consumer segmentation is implemented to analyze behavioral characteristics for each segment and distinguish low active consumers, then models are constructed for different consumer segments. The proposed model is applied to a real online consumption data. Consumers are aggregated into four segments: potential e-coupons user, low discount sensitive user, high discount sensitive user (including discount preference and fixed preference). The first one is defined as low active consumer segment and others are high active consumer segments. Isolation forest model and logistic regression model are respectively constructed for them. Result shows that data imbalance is effectively relieved, prediction performance is also significantly better than the traditional approaches. Finally, e-coupons’ usage characteristics for each consumer segment are summarized, according to that, companies can increase sales and improve consumer satisfaction as well.

Peak-load forecasting prevents energy waste and helps with environmental issues by establishing plans for the use of renewable energy. For that reason, the subject is still actively studied. Most of these studies are focused on improving predictive performance by using varying feature information, but most small industrial facilities cannot provide such information because of a lack of infrastructure. Therefore, we introduce a series of studies to implement a generalized prediction model that is applicable to these small industrial facilities. On the basis of the pattern of load information of most industrial facilities, new features were selected, and a generalized model was developed through the aggregation of ensemble models. In addition, a new method is proposed to improve prediction performance by providing additional compensation to the prediction results by reflecting the fewest opinions among the prediction results of each model. Actual data from two small industrial facilities were applied to our process, and the results proved the effectiveness of our proposed method.

usc Refereed/Peer-reviewed In this paper, we propose a model for detecting anomalies in elderly home care. Two scenarios are investigated in detecting anomalies: 1) the elderly person's vital signs and their surrounding environment; 2) the mobility patterns of the elderly. We evaluated our proposed model by employing the isolation forest which detects anomalies using an isolation approach on a random forest of decision trees. We compare isolation forest on unlabeled data with statistical methods on labelled data. Subsequently, to show the reliability of the isolation concept, we compare it with a distance measure concept. The experiment shows that isolation forest has higher detection accuracy and lower error prediction for two attributes in the first scenario: skin temperature and heart rate, whereas, in the second scenario, multi-covariance determinant has a slightly better accuracy compared to isolation forest (3.9% difference in accuracy) and has a small number of prediction errors compared to isolation forest.