Search

With the development of industrial automation, the accurate prediction of the health state of equipment becomes particularly important. This study aims to address the problem of application of small sample unbalanced data in device life prediction. A joint optimization model was constructed by combining the modified SMOTE algorithm and the modified KNN algorithm. To solve the sample imbalance problem, the modified KNN algorithm is used to improve the accuracy of classification. Through the simulation analysis of the hydraulic pump status data of Caterpillar Corporation and the vibration data of the water guide bearing of Lingjintan Hydropower Station, the proposed improved algorithm can accurately analyze the running status of the equipment and predict the future healthy development trend. Experimental results show that the joint optimization model has higher accuracy and reliability in the processing of small sample unbalanced data compared with traditional algorithms.

Fiber manufacturing is a complex and dynamic process where quality control poses unique challenges, requiring timely detection and accurate classification of faults. Automating the quality control process using machine-learning techniques has the potential to reduce costs and increase efficiency. However, training machine-learning models to classify faults requires labeling large quantities of time-series data generated from the sensors, which can be costly and time-consuming. In this work, we developed and evaluated Active Learning (AL) techniques to enhance the performance of fault classification models while reducing the number of instances that require labeling. AL is based on focusing the labeling effort on a small number of selected, informative samples, thereby reducing the cost and time requirements for creating a labeled dataset. Moreover, in manufacturing, the frequencies of different types of faults vary widely, resulting in a class imbalance problem. The selection of instances per class to be labeled is beyond the control of traditional AL techniques, which may lead to significant bias in the classification performance and the selection of instances to be labeled for AL strategies, particularly in a dynamic manufacturing environment. To address this problem, we also develop and incorporate a class-balancing instance selection algorithm that tries to select more instances from the classes with fewer labeled examples. The AL techniques implemented here reduce the amount of labeled data necessary for accurate fault classification by a factor of 5 compared to conventional supervised machine-learning techniques. Additionally, our class-balancing instance selection algorithm effectively addresses the class imbalance problem in our dataset. Overall, our results indicate that our AL pipeline constitutes a promising solution for efficient and accurate fault classification using time-series data from industrial manufacturing.

Detection of rare events is a major problem when dealing with unbalanced data. In the application of machine learning tools, data is split into training and test samples and preprocessing is applied to the training set, with the aim of obtaining a more balanced sample. In this paper we discuss preprocessing methods applied to heterogenous data clustered with respect to expected anomaly types. We propose a method for deciding on oversampling and under-sampling from each cluster, based on the variability of the items in each cluster, using Principal Component Analysis. The method is applied to the problem of detecting anomalies in a time series invoices, with an average rate of complaints of orders 10-4.

Detection of rare events is a major problem when dealing with unbalanced data. In the application of machine learning tools, data is split into training and test samples and preprocessing is applied to the training set, with the aim of obtaining a more balanced sample. In this paper we discuss preprocessing methods applied to heterogenous data clustered with respect to expected anomaly types. We propose a method for deciding on oversampling and under-sampling from each cluster, based on the variability of the items in each cluster, using Principal Component Analysis. The method is applied to the problem of detecting anomalies in a time series invoices, with an average rate of complaints of orders 10-4.

Detection of rare events is a major problem when dealing with unbalanced data. In the application of machine learning tools, data is split into training and test samples and preprocessing is applied to the training set, with the aim of obtaining a more balanced sample. In this paper we discuss preprocessing methods applied to heterogenous data clustered with respect to expected anomaly types. We propose a method for deciding on oversampling and under-sampling from each cluster, based on the variability of the items in each cluster, using Principal Component Analysis. The method is applied to the problem of detecting anomalies in a time series invoices, with an average rate of complaints of orders 10-4.

In today’s digital era, establishing an online presence and maintaining a well-structured website is vitalfor companies to remain competitive in their respective markets. A crucial aspect of online success liesin strategically selecting the right words to optimize customer engagement and search engine visibility.However, this process is often time-consuming, involving extensive analysis of a company’s website aswell as its competitors’. This thesis focuses on developing an efficient binary classification approachto identify key words and phrases extracted from multiple company websites. The data set used forthis solution consists of approximately 92,000 scraped samples, primarily comprising non-key samples.Various features were extracted, and a word embedding model was employed to assess each sample’srelevance to its specific industry and topic. The logistic regression, decision tree and random forestalgorithms were all explored and implemented as different solutions to the classification problem. Theresults indicated that the logistic regression model excelled in retaining keywords but was less effectivein eliminating non-keywords. Conversely, the tree-based methods demonstrated superior classificationof keywords, albeit at the cost of misclassifying a few keywords. Overall, the random forest approachoutperformed the others, achieving a result of 76 percent in recall and 20 percent in precision whenpredicting key samples. In summary, this thesis presents a solution for classifying words and phrasesfrom company websites into key and non-key categories, and the developed methodology could offervaluable insights for companies seeking to enhance their website optimization strategies.

In today’s digital era, establishing an online presence and maintaining a well-structured website is vitalfor companies to remain competitive in their respective markets. A crucial aspect of online success liesin strategically selecting the right words to optimize customer engagement and search engine visibility.However, this process is often time-consuming, involving extensive analysis of a company’s website aswell as its competitors’. This thesis focuses on developing an efficient binary classification approachto identify key words and phrases extracted from multiple company websites. The data set used forthis solution consists of approximately 92,000 scraped samples, primarily comprising non-key samples.Various features were extracted, and a word embedding model was employed to assess each sample’srelevance to its specific industry and topic. The logistic regression, decision tree and random forestalgorithms were all explored and implemented as different solutions to the classification problem. Theresults indicated that the logistic regression model excelled in retaining keywords but was less effectivein eliminating non-keywords. Conversely, the tree-based methods demonstrated superior classificationof keywords, albeit at the cost of misclassifying a few keywords. Overall, the random forest approachoutperformed the others, achieving a result of 76 percent in recall and 20 percent in precision whenpredicting key samples. In summary, this thesis presents a solution for classifying words and phrasesfrom company websites into key and non-key categories, and the developed methodology could offervaluable insights for companies seeking to enhance their website optimization strategies.

Contamination of maize with aflatoxins and fumonisins is one of the major food safety concerns worldwide. Knowing the contamination in advance can help to reduce food safety risks and related health issues and economic losses. The current study aimed to develop forecasting models for the contamination of maize grown in Serbia with aflatoxins and fumonisins. An integrated modeling approach was used, linking mechanistic modeling with artificial intelligence, in particular Bayesian network (BN) modeling. Two of such combined models, i.e., the prediction model for aflatoxins (PREMA) and for fumonisins (PREFUM) in maize, were developed. Data used for developing PREMA were from 867 maize samples, collected in Serbia during the period from 2012 to 2018, of which 190 were also used for developing PREFUM. Both datasets were split randomly in a model training set and a model validation set. With corresponding geographical and meteorological data, the so-called risk indices for total aflatoxins and total fumonisins were calculated using existing mechanistic models. Subsequently, these risk indices were used as input variables for developing the BN models, together with the longitudes and latitudes of the sites at which the samples were collected and related weather data. PREMA and PREFUM were internally and externally validated, resulting in a prediction accuracy of PREMA of, respectively, 83 and 70%, and of PREFUM of 76% and 80%. The capability of PREMA and PREFUM for predicting aflatoxins and fumonisins contamination using data from the early maize growth stages only was explored as well, and promising results were obtained. The integrated approach combining two different modeling techniques, as developed in the current study, was able to overcome the obstacles of unbalanced data and deficiency of the datasets, which are often seen in historical observational data from the food safety domain. The models provide predictions for mycotoxin contamination at the field level; this

[Abstract]: Advances in the information technologies have greatly contributed to the advent of larger datasets. These datasets often come from distributed sites, but even so, their large size usually means they cannot be handled in a centralized manner. A possible solution to this problem is to distribute the data over several processors and combine the different results. We propose a methodology to distribute feature selection processes based on selecting relevant and discarding irrelevant features. This preprocessing step is essential for current high-dimensional sets, since it allows the input dimension to be reduced. We pay particular attention to the problem of data imbalance, which occurs because the original dataset is unbalanced or because the dataset becomes unbalanced after data partitioning. Most works approach unbalanced scenarios by oversampling, while our proposal tests both over- and undersampling strategies. Experimental results demonstrate that our distributed approach to classification obtains comparable accuracy results to a centralized approach, while reducing computational time and efficiently dealing with data imbalance.

Deep Learning (DL) has become popular due to its ease of use and accuracy, with Transfer Learning (TL) effectively reducing the number of images needed to solve environmental problems. However, this approach has some limitations which we set out to explore: Our goal is to detect the presence of an invasive blueberry species in aerial images of wetlands. This is a key problem in ecosystem protection which is also challenging in terms of DL due to the severe imbalance present in the data. Results for the ResNet50 network show a high classification accuracy while largely ignoring the blueberry class, rendering these results of limited practical interest to detect that specific class. Moreover, by using loss function weighting and data augmentation results more akin to our practical application, our goals can be obtained. Our experiments regarding TL show that ImageNet weights do not produce satisfactory results when only the final layer of the network is trained. Furthermore, only minor gains are obtained compared with random weights when the whole network is retrained. Finally, in a study of state-of-the-art DL architectures best results were obtained by the ResNeXt architecture with 93.75 True Positive Rate and 98.11 accuracy for the Blueberry class with ResNet50, Densenet, and wideResNet obtaining close results. © 2020 by the authors. Licensee MDPI, Basel, Switzerland.

Deep Learning (DL) has become popular due to its ease of use and accuracy, with Transfer Learning (TL) effectively reducing the number of images needed to solve environmental problems. However, this approach has some limitations which we set out to explore: Our goal is to detect the presence of an invasive blueberry species in aerial images of wetlands. This is a key problem in ecosystem protection which is also challenging in terms of DL due to the severe imbalance present in the data. Results for the ResNet50 network show a high classification accuracy while largely ignoring the blueberry class, rendering these results of limited practical interest to detect that specific class. Moreover, by using loss function weighting and data augmentation results more akin to our practical application, our goals can be obtained. Our experiments regarding TL show that ImageNet weights do not produce satisfactory results when only the final layer of the network is trained. Furthermore, only minor gains are obtained compared with random weights when the whole network is retrained. Finally, in a study of state-of-the-art DL architectures best results were obtained by the ResNeXt architecture with 93.75 True Positive Rate and 98.11 accuracy for the Blueberry class with ResNet50, Densenet, and wideResNet obtaining close results. © 2020 by the authors. Licensee MDPI, Basel, Switzerland.

Outstanding claims liability is usually one of the largest liabilities on the balance sheet of a general insurer. Therefore, it is critical for insurers to accurately estimate their outstanding claims. Furthermore, a general insurer typically operates in multiple business lines whose risks are not perfectly dependent. This results in "diversification benefits", the consideration of which is crucial due to their effects on the aggregate reserves and capital. It is then essential to consider the dependence across business lines in the estimation of outstanding claims. The goal of this thesis is to develop new approaches to assess outstanding claims for portfolios of dependent lines. We explore the common shock technique for model developments, a very popular dependence modelling technique with distinctive strengths, such as explicit dependence structure, ease of interpretation, and parsimonious construction of correlation matrices. We also aim to enhance the practicality of our approaches by incorporating realistic and desirable model features. Motivated by the richness of the Tweedie distribution family which covers Poisson distributions, gamma distributions and many more, we introduce a common shock Tweedie framework with dependence across business lines. Desirable properties of this framework are studied, including its marginal flexibility, tractable moments, and ability to handle masses at 0. To overcome the complex distributional structure of the Tweedie framework, we formulate a Bayesian approach for model estimation and perform a real data illustration. Remarks on practical features of the framework are drawn. Loss reserving data possesses an unbalanced nature, that is, claims from different positions within and between loss triangles can vary widely as more claims typically develop in early development periods. We account for this feature explicitly in common shock models with a parsimonious common shock adjustment. Theoretical and real data illustrations are

We present a flexible deep convolutional neural network method for the analysis of arbitrary sized graph structures representing molecules. This method, which makes use of the Lipinski RDKit module, an open-source cheminformatics software, enables the incorporation of any global molecular (such as molecular charge and molecular weight) and local (such as atom hybridization and bond orders) information. In this paper, we show that this method significantly outperforms another recently proposed method based on deep convolutional neural networks on several datasets that are studied. Several best practices for training deep convolutional neural networks on chemical datasets are also highlighted within the article, such as how to select the information to be included in the model, how to prevent overfitting and how unbalanced classes in the data can be handled.

We present a flexible deep convolutional neural network method for the analysis of arbitrary sized graph structures representing molecules. This method, which makes use of the Lipinski RDKit module, an open-source cheminformatics software, enables the incorporation of any global molecular (such as molecular charge and molecular weight) and local (such as atom hybridization and bond orders) information. In this paper, we show that this method significantly outperforms another recently proposed method based on deep convolutional neural networks on several datasets that are studied. Several best practices for training deep convolutional neural networks on chemical datasets are also highlighted within the article, such as how to select the information to be included in the model, how to prevent overfitting and how unbalanced classes in the data can be handled.

A social network can be defined as a group of entities linked between each other by means of some relational tie. Several studies indicate the existence of an influence of social networks on job performance. This thesis is intended to assess such idea. More specifically, the objective is to examine the relationship between the position of the entities in a social network with their performances through time. To that purpose, an analysis with real data about a set of soldiers from an European country army is conducted by using the software R. Among others, it contains measures of the soldiers performance and the social networks that can be defined on them. In order to assess such influence, a multilevel model is considered as a proper tool to fit the data, which is characterized by its longitudinal and its hierarchical structure. Besides, the data is unbalanced. Thus, this study will cover the concept of social network and the theory around this matter, as well as the multilevel model theory and its estimation.

Real-time quality prediction in continuous casting process is of great significance to the increase of production and the improvement of casting billet quality. The process parameters have a great influence on the quality of the billet in the continuous casting process, and the quantity distribution of the superior and inferior products in the casting billet is extremely unbalanced. Therefore, this paper proposes an intelligent prediction method for casting billet quality based on multi-process parameters. Based on the analysis of the relationship between multiple process parameters and casting billet quality, a casting quality prediction model based on weighted random forest (WRF) algorithm was established. This algorithm solves the sample imbalance problem by weighting the decision tree results effectively, and can correctly identify negative samples. Based on real-time casting billet data in the production process, results of the case prove the effectiveness of the proposed method. © 2018 IEEE.

We present a flexible deep convolutional neural network method for the analysis of arbitrary sized graph structures representing molecules. This method, which makes use of the Lipinski RDKit module, an open-source cheminformatics software, enables the incorporation of any global molecular (such as molecular charge and molecular weight) and local (such as atom hybridization and bond orders) information. In this paper, we show that this method significantly outperforms another recently proposed method based on deep convolutional neural networks on several datasets that are studied. Several best practices for training deep convolutional neural networks on chemical datasets are also highlighted within the article, such as how to select the information to be included in the model, how to prevent overfitting and how unbalanced classes in the data can be handled.

We present a flexible deep convolutional neural network method for the analysis of arbitrary sized graph structures representing molecules. This method, which makes use of the Lipinski RDKit module, an open-source cheminformatics software, enables the incorporation of any global molecular (such as molecular charge and molecular weight) and local (such as atom hybridization and bond orders) information. In this paper, we show that this method significantly outperforms another recently proposed method based on deep convolutional neural networks on several datasets that are studied. Several best practices for training deep convolutional neural networks on chemical datasets are also highlighted within the article, such as how to select the information to be included in the model, how to prevent overfitting and how unbalanced classes in the data can be handled.

We present a flexible deep convolutional neural network method for the analysis of arbitrary sized graph structures representing molecules. This method, which makes use of the Lipinski RDKit module, an open-source cheminformatics software, enables the incorporation of any global molecular (such as molecular charge and molecular weight) and local (such as atom hybridization and bond orders) information. In this paper, we show that this method significantly outperforms another recently proposed method based on deep convolutional neural networks on several datasets that are studied. Several best practices for training deep convolutional neural networks on chemical datasets are also highlighted within the article, such as how to select the information to be included in the model, how to prevent overfitting and how unbalanced classes in the data can be handled.

Credit card fraud is a field with perpetrators performing illegal actions that may affect other individuals or companies negatively. For instance, a criminalcan steal credit card information from an account holder and then conduct fraudulent transactions. The activities are a potential contributory factor to how illegal organizations such as terrorists and drug traffickers support themselves financially. Within the machine learning area, there are several methods that possess the ability to detect credit card fraud transactions; supervised learning and unsupervised learning algorithms. This essay investigates the supervised approach, where two algorithms (Hellinger Distance Decision Tree (HDDT) and Random Forest) are evaluated on a real life dataset of 284,807 transactions. Under those circumstances, the main purpose is to develop a “well-functioning” model with a reasonable capacity to categorize transactions as fraudulent or legit. As the data is heavily unbalanced, reducing the false-positive rate is also an important part when conducting research in the chosen area. In conclusion, evaluated algorithms present a fairly similar outcome, where both models have the capability to distinguish the classes from each other. However, the Random Forest approach has a better performance than HDDT in all measures of interest.

Changing market dynamics make that mining companies need to increase their equipment utilization and use their equipment more effectively. The growing availability of data from mining equipment and mining operations enables the use of data analytics to achieve that goal. This thesis explores the potential of data analytics and machine learning to increase equipment utilization through the means of a case study on an underground South African diamond mine. To identify and quantify the potential of data analytics for operational improvement a benchmarking analysis of the operation was done using operational equipment effectiveness (OEE) as a measure. The outcome of this indicated that there was significant value to be unlocked if unplanned breakdowns were mitigated and unscheduled maintenance could be moved into scheduled maintenance. This led to believe that there was a solid case for breakdown prediction based on equipment sensor data using advanced analytical techniques. Data availability and quality made that the proof-of-model was initiated for one 10 ton loader with the target components for breakdown prediction being the engine, transmission and brakes. After extensive data cleaning and preparation various machine learning algorithms adopted from various non-mining disciplines were used for training predictive models. The best result was yielded using support vector machines (SVM) showing a very high accuracy (>80%) with little misclassifications. Random forests and anomaly detection were also tried but the unbalanced and noisy nature of the data made for the performance of these models not leading to satisfactory levels. The results of this modelling exercise combined with the OEE analysis give enough confidence to accept the hypothesis that there is good potential for application of advanced analytics to increase the operational effectiveness of underground mining equipment.

Changing market dynamics make that mining companies need to increase their equipment utilization and use their equipment more effectively. The growing availability of data from mining equipment and mining operations enables the use of data analytics to achieve that goal. This thesis explores the potential of data analytics and machine learning to increase equipment utilization through the means of a case study on an underground South African diamond mine. To identify and quantify the potential of data analytics for operational improvement a benchmarking analysis of the operation was done using operational equipment effectiveness (OEE) as a measure. The outcome of this indicated that there was significant value to be unlocked if unplanned breakdowns were mitigated and unscheduled maintenance could be moved into scheduled maintenance. This led to believe that there was a solid case for breakdown prediction based on equipment sensor data using advanced analytical techniques. Data availability and quality made that the proof-of-model was initiated for one 10 ton loader with the target components for breakdown prediction being the engine, transmission and brakes. After extensive data cleaning and preparation various machine learning algorithms adopted from various non-mining disciplines were used for training predictive models. The best result was yielded using support vector machines (SVM) showing a very high accuracy (>80%) with little misclassifications. Random forests and anomaly detection were also tried but the unbalanced and noisy nature of the data made for the performance of these models not leading to satisfactory levels. The results of this modelling exercise combined with the OEE analysis give enough confidence to accept the hypothesis that there is good potential for application of advanced analytics to increase the operational effectiveness of underground mining equipment., Applied Earth Sciences

Credit card fraud is a field with perpetrators performing illegal actions that may affect other individuals or companies negatively. For instance, a criminalcan steal credit card information from an account holder and then conduct fraudulent transactions. The activities are a potential contributory factor to how illegal organizations such as terrorists and drug traffickers support themselves financially. Within the machine learning area, there are several methods that possess the ability to detect credit card fraud transactions; supervised learning and unsupervised learning algorithms. This essay investigates the supervised approach, where two algorithms (Hellinger Distance Decision Tree (HDDT) and Random Forest) are evaluated on a real life dataset of 284,807 transactions. Under those circumstances, the main purpose is to develop a “well-functioning” model with a reasonable capacity to categorize transactions as fraudulent or legit. As the data is heavily unbalanced, reducing the false-positive rate is also an important part when conducting research in the chosen area. In conclusion, evaluated algorithms present a fairly similar outcome, where both models have the capability to distinguish the classes from each other. However, the Random Forest approach has a better performance than HDDT in all measures of interest.

Billions of dollars of loss are caused every year due to fraudulent credit card transactions. The design of efficient fraud detection algorithms is key for reducing these losses, and more algorithms rely on advanced machine learning techniques to assist fraud investigators. The design of fraud detection algorithms is however particularly challenging due to non-stationary distribution of the data, highly imbalanced classes distributions and continuous streams of transactions. At the same time public data are scarcely available for confidentiality issues, leaving unanswered many questions about which is the best strategy to deal with them. In this paper we provide some answers from the practitioner's perspective by focusing on three crucial issues: unbalancedness, non-stationarity and assessment. The analysis is made possible by a real credit card dataset provided by our industrial partner. © 2014 Elsevier Ltd. All rights reserved., SCOPUS: re.j, info:eu-repo/semantics/published

Billions of dollars of loss are caused every year due to fraudulent credit card transactions. The design of efficient fraud detection algorithms is key for reducing these losses, and more algorithms rely on advanced machine learning techniques to assist fraud investigators. The design of fraud detection algorithms is however particularly challenging due to non-stationary distribution of the data, highly imbalanced classes distributions and continuous streams of transactions. At the same time public data are scarcely available for confidentiality issues, leaving unanswered many questions about which is the best strategy to deal with them. In this paper we provide some answers from the practitioner's perspective by focusing on three crucial issues: unbalancedness, non-stationarity and assessment. The analysis is made possible by a real credit card dataset provided by our industrial partner. © 2014 Elsevier Ltd. All rights reserved., SCOPUS: re.j, info:eu-repo/semantics/published

Variance component estimation (VCE) is used to update the stochastic model in least-squares adjustments, but the uncertainty associated with the VCE-derived weights is rarely considered. Unbalanced data is where there is an unequal number of observations in each heterogeneous dataset comprising the variance component groups. As a case study using highly unbalanced data, we redefine a continent-wide vertical datum from a combined least-squares adjustment using iterative VCE and its uncertainties to update weights for each data set. These are: (1) a continent-wide levelling network, (2) a model of the ocean’s mean dynamic topography and mean sea level observations, and (3) GPS-derived ellipsoidal heights minus a gravimetric quasigeoid model. VCE uncertainty differs for each observation group in the highly unbalanced data, being dependent on the number of observations in each group. It also changes within each group after each VCE iteration, depending on the magnitude of change for each observation group’s variances. It is recommended that VCE uncertainty is computed for VCE updates to the weight matrix for unbalanced data so that the quality of the updates for each group can be properly assessed. This is particularly important if some groups contain relatively small numbers of observations. VCE uncertainty can also be used as a threshold for ceasing iterations, as it is shown—for this data set at least—that it is not necessary to continue time-consuming iterations to fully converge to unity.

Unbalanced data classification has been a long standing issue in the field of Medical Vision Science and Pattern Recognition. In our research, we introduced the methods of Support Vector Machines (SVM) with Active Learning (AL) to improve prediction of unbalanced classes of Scleroderma Lung Disease, Lung Fibrosis (LF) and Honeycomb (HC). Four different SVM with AL approaches are proposed: 1) random sampling to select the initial pool to begin the AL algorithm; 2) doubling the training instances of Honeycomb to reduce the imbalance ratio before the AL algorithm; 3) a balanced pool with equal number Honeycomb instances and Lung Fibrosis instances; 4) a balanced pool of Honeycomb and Lung Fibrosis and implements balanced sampling throughout the AL algorithm. Grid pixel data of Lung Fibrosis and Honeycomb was extracted from computed tomography (CT) images of 71 patients from 13 clinical centers around the United States to produce a training set of 348 HC and 3009 LF instances and a test set of 291 HC and 2665 LF. From our research, SVM with AL using balanced sampling compared to random sampling increased the sensitivity of HC by 0.56 (0.175 vs. 0.735) and 0.47 (0.23 vs. 0.70) for the original and de-noised dataset respectively. These results show that by implementing SVM with Active Learning paired with balanced sampling, we can improve the prediction performances of unbalanced data classification.

State-of-the-art classification algorithms suffer when the data is skewed towards one class. This led to the development of a number of techniques to cope with unbalanced data. However, as confirmed by our experimental comparison, no technique appears to work consistently better in all conditions. We propose to use a racing method to select adaptively the most appropriate strategy for a given unbalanced task. The results show that racing is able to adapt the choice of the strategy to the specific nature of the unbalanced problem and to select rapidly the most appropriate strategy without compromising the accuracy. © 2013 Springer-Verlag., SCOPUS: cp.k, info:eu-repo/semantics/published

Unbalanced data classification has been a long standing issue in the field of Medical Vision Science and Pattern Recognition. In our research, we introduced the methods of Support Vector Machines (SVM) with Active Learning (AL) to improve prediction of unbalanced classes of Scleroderma Lung Disease, Lung Fibrosis (LF) and Honeycomb (HC). Four different SVM with AL approaches are proposed: 1) random sampling to select the initial pool to begin the AL algorithm; 2) doubling the training instances of Honeycomb to reduce the imbalance ratio before the AL algorithm; 3) a balanced pool with equal number Honeycomb instances and Lung Fibrosis instances; 4) a balanced pool of Honeycomb and Lung Fibrosis and implements balanced sampling throughout the AL algorithm. Grid pixel data of Lung Fibrosis and Honeycomb was extracted from computed tomography (CT) images of 71 patients from 13 clinical centers around the United States to produce a training set of 348 HC and 3009 LF instances and a test set of 291 HC and 2665 LF. From our research, SVM with AL using balanced sampling compared to random sampling increased the sensitivity of HC by 0.56 (0.175 vs. 0.735) and 0.47 (0.23 vs. 0.70) for the original and de-noised dataset respectively. These results show that by implementing SVM with Active Learning paired with balanced sampling, we can improve the prediction performances of unbalanced data classification.

We apply the generalized cluster bootstrap to both Gaussian quasi-likelihood and robust estimates in the context of highly unbalanced clustered data. We compare it with the transformation bootstrap where the data are generated by the random effect and transformation models and all the random variables have different distributions. We also develop a fast approach (proposed by Salibian-Barrera et al. (2008)) and show that it produces some encouraging results. We show that the generalized bootstrap performs better than the transformation bootstrap for highly unbalanced clustered data. We apply the generalized cluster bootstrap to a sample of income data for Australian workers.

State-of-the-art classification algorithms suffer when the data is skewed towards one class. This led to the development of a number of techniques to cope with unbalanced data. However, as confirmed by our experimental comparison, no technique appears to work consistently better in all conditions. We propose to use a racing method to select adaptively the most appropriate strategy for a given unbalanced task. The results show that racing is able to adapt the choice of the strategy to the specific nature of the unbalanced problem and to select rapidly the most appropriate strategy without compromising the accuracy. © 2013 Springer-Verlag., SCOPUS: cp.k, info:eu-repo/semantics/published

In mixed models, the use of robust estimates raises several interesting inferential challenges. One of these challenges arises from the realization that the effect of contamination is to increase the variability in the data, but robust estimates of variance components are usually smaller than their nonrobust counterparts. The robust estimates reflect the variability of the bulk of the data, which is not the same as the variability in the data-generating process. This means that the naive implementation of bootstrap procedures might not work. In this article we consider several bootstrap procedures, including random effect, transformation, and weighted bootstraps. We give conditions for the asymptotic validity of the bootstraps and assess their performance via a small simulation study. Both the transformation and generalized cluster bootstrap perform well and are asymptotically valid under reasonable conditions.

O objetivo deste trabalho foi avaliar a conveniência de definir o número de componentes multiplicativos dos modelos de efeitos principais aditivos com interação multiplicativa (AMMI) em experimentos de interações genótipo x ambiente de algodão com dados imputados ou desbalanceados. Um estudo de simulação foi realizado com base em uma matriz de dados reais de produtividade de algodão em caroço, obtidos em ensaios de interação genótipo x ambiente, conduzidos com 15 cultivares em 27 locais no Brasil. A simulação foi feita com retiradas aleatórias de 10, 20 e 30% dos dados. O número ótimo de componentes multiplicativos para o modelo AMMI foi determinado usando o teste de Cornelius e o teste de razão de verossimilhança sobre as matrizes completadas por imputação. Para testar as hipóteses, quando a análise é feita a partir de médias e não são disponibilizadas as repetições, foi proposta uma correção com base nas observações ausentes no teste de Cornelius. Para a imputação de dados, foram considerados métodos usando submodelos robustos, mínimos quadrados alternados e imputação múltipla. Na análise de experimentos desbalanceados, é recomendável escolher o número de componentes multiplicativos do modelo AMMI somente a partir da informação observada e fazer a estimação clássica dos parâmetros com base nas matrizes completadas por imputação. The objective of this work was to evaluate the convenience of defining the number of multiplicative components of additive main effect and multiplicative interaction models (AMMI) in genotype x environment interaction experiments in cotton with imputed or unbalanced data. A simulation study was carried out based on a matrix of real seed-cotton productivity data obtained in trials with genotype x environment interaction carried out with 15 genotypes at 27 locations in Brazil. The simulation was made with random withdrawals of 10, 20 and 30% of the data. The optimal number of multiplicative components for the AMMI model was determined usin

The authors consider general estimators for the mean and variance parameters in the random effect model and in the transformation model for data with multiple levels of variation. They show that these estimators have different distributions under the two

Bee pollinators are currently recorded with many different sampling methods. However, the relative performances of these methods have not been systematically evaluated and compared. In response to the strong need to record ongoing shifts in pollinator diversity and abundance, global and regional pollinator initiatives must adopt standardized sampling protocols when developing large-scale and long-term monitoring schemes.We systematically evaluated the performance of six sampling methods (observation plots, pan traps, standardized and variable transect walks, trap nests with reed internodes or paper tubes) that are commonly used across a wide range of geographical regions in Europe and in two habitat types (agricultural and seminatural). We focused on bees since they represent the most important pollinator group worldwide. Several characteristics of the methods were considered in order to evaluate their performance in assessing bee diversity: sample coverage, observed species richness, species richness estimators, collector biases (identified by subunit-based rarefaction curves), species composition of the samples, and the indication of overall bee species richness (estimated from combined total samples).The most efficient method in all geographical regions, in both the agricultural and seminatural habitats, was the pan trap method. It had the highest sample coverage, collected the highest number of species, showed negligible collector bias, detected similar species as the transect methods, and was the best indicator of overall bee species richness. The transect methods were also relatively efficient, but they had a significant collector bias. The observation plots showed poor performance. As trap nests are restricted to cavity-nesting bee species, they had a naturally low sample coverage. However, both trap nest types detected additional species that were not recorded by any of the other methods.For large-scale and long-term monitoring schemes with surveyors with di

Tato práce se zaměřuje na problematiku segmentace nevyvážených dat pomocí uměléinteligence. V práci jsou prozkoumány známé metody pro vypořádání se s nevyváženýmidaty, z nichž jsou vybrány vhodné metody, a ty jsou aplikovány na reálný problém, vekterém je cílem segmentovat nevyvážená data s poměrem tříd větším než 6000:1., This thesis focuses on problematics of segmentation of unbalanced datasets by the useof artificial inteligence. Numerous existing methods for dealing with unbalanced datasetsare examined, and some of them are then applied to real problem that consist of seg-mentation of dataset with class ratio of more than 6000:1.

Tato práce se zaměřuje na problematiku segmentace nevyvážených dat pomocí uměléinteligence. V práci jsou prozkoumány známé metody pro vypořádání se s nevyváženýmidaty, z nichž jsou vybrány vhodné metody, a ty jsou aplikovány na reálný problém, vekterém je cílem segmentovat nevyvážená data s poměrem tříd větším než 6000:1., This thesis focuses on problematics of segmentation of unbalanced datasets by the useof artificial inteligence. Numerous existing methods for dealing with unbalanced datasetsare examined, and some of them are then applied to real problem that consist of seg-mentation of dataset with class ratio of more than 6000:1.

Tato práce se zaměřuje na problematiku segmentace nevyvážených dat pomocí uměléinteligence. V práci jsou prozkoumány známé metody pro vypořádání se s nevyváženýmidaty, z nichž jsou vybrány vhodné metody, a ty jsou aplikovány na reálný problém, vekterém je cílem segmentovat nevyvážená data s poměrem tříd větším než 6000:1., This thesis focuses on problematics of segmentation of unbalanced datasets by the useof artificial inteligence. Numerous existing methods for dealing with unbalanced datasetsare examined, and some of them are then applied to real problem that consist of seg-mentation of dataset with class ratio of more than 6000:1.

Tato práce se zaměřuje na problematiku segmentace nevyvážených dat pomocí uměléinteligence. V práci jsou prozkoumány známé metody pro vypořádání se s nevyváženýmidaty, z nichž jsou vybrány vhodné metody, a ty jsou aplikovány na reálný problém, vekterém je cílem segmentovat nevyvážená data s poměrem tříd větším než 6000:1., This thesis focuses on problematics of segmentation of unbalanced datasets by the useof artificial inteligence. Numerous existing methods for dealing with unbalanced datasetsare examined, and some of them are then applied to real problem that consist of seg-mentation of dataset with class ratio of more than 6000:1.

Tato práce se zaměřuje na problematiku segmentace nevyvážených dat pomocí uměléinteligence. V práci jsou prozkoumány známé metody pro vypořádání se s nevyváženýmidaty, z nichž jsou vybrány vhodné metody, a ty jsou aplikovány na reálný problém, vekterém je cílem segmentovat nevyvážená data s poměrem tříd větším než 6000:1., This thesis focuses on problematics of segmentation of unbalanced datasets by the useof artificial inteligence. Numerous existing methods for dealing with unbalanced datasetsare examined, and some of them are then applied to real problem that consist of seg-mentation of dataset with class ratio of more than 6000:1.

Tato práce se zaměřuje na problematiku segmentace nevyvážených dat pomocí uměléinteligence. V práci jsou prozkoumány známé metody pro vypořádání se s nevyváženýmidaty, z nichž jsou vybrány vhodné metody, a ty jsou aplikovány na reálný problém, vekterém je cílem segmentovat nevyvážená data s poměrem tříd větším než 6000:1., This thesis focuses on problematics of segmentation of unbalanced datasets by the useof artificial inteligence. Numerous existing methods for dealing with unbalanced datasetsare examined, and some of them are then applied to real problem that consist of seg-mentation of dataset with class ratio of more than 6000:1.