Search

The paper presents research on a specific approach to the issue of computed tomography with an incomplete data set. The case of incomplete information is quite common, for example when examining objects of large size or difficult to access. Algorithms devoted to this type of problems can be used to detect anomalies in coal seams that pose a threat to the life of miners. The most dangerous example of such an anomaly may be a compressed gas tank, which expands rapidly during exploitation, at the same time ejecting rock fragments, which are a real threat to the working crew. The approach presented in the paper is an improvement of the previous idea, in which the detected objects were represented by sequences of points. These points represent rectangles, which were characterized by sequences of their parameters. This time, instead of sequences in the representation, there are sets of objects, which allow for the elimination of duplicates. As a result, the reconstruction is faster. The algorithm presented in the paper solves the inverse problem of finding the minimum of the objective function. Heuristic algorithms are suitable for solving this type of tasks. The following heuristic algorithms are described, tested and compared: Aquila Optimizer (AQ), Firefly Algorithm (FA), Whale Optimization Algorithm (WOA), Butterfly Optimization Algorithm (BOA) and Dynamic Butterfly Optimization Algorithm (DBOA). The research showed that the best algorithm for this type of problem turned out to be DBOA. [ABSTRACT FROM AUTHOR]

Abstract Duplicate record is a common problem within data sets especially in huge volume databases. The accuracy of duplicate detection determines the efficiency of duplicate removal process. However, duplicate detection has become more challenging due to the presence of missing values within the records where during the clustering and matching process, missing values can cause records deemed similar to be inserted into the wrong group, hence, leading to undetected duplicates. In this paper, duplicate detection improvement was proposed despite the presence of missing values within a data set through Duplicate Detection within the Incomplete Data set (DDID) method. The missing values were hypothetically added to the key attributes of three data sets under study, using an arbitrary pattern to simulate both complete and incomplete data sets. The results were analyzed, then, the performance of duplicate detection was evaluated by using the Hot Deck method to compensate for the missing values in the key attributes. It was hypothesized that by using Hot Deck, duplicate detection performance would be improved. Furthermore, the DDID performance was compared to an early duplicate detection method namely DuDe, in terms of its accuracy and speed. The findings yielded that even though the data sets were incomplete, DDID was able to offer a better accuracy and faster duplicate detection as compared to DuDe. The results of this study offer insights into constraints of duplicate detection within incomplete data sets.

The paper presents research on a specific approach to the issue of computed tomography with an incomplete data set. The case of incomplete information is quite common, for example when examining objects of large size or difficult to access. Algorithms devoted to this type of problems can be used to detect anomalies in coal seams that pose a threat to the life of miners. The most dangerous example of such an anomaly may be a compressed gas tank, which expands rapidly during exploitation, at the same time ejecting rock fragments, which are a real threat to the working crew. The approach presented in the paper is an improvement of the previous idea, in which the detected objects were represented by sequences of points. These points represent rectangles, which were characterized by sequences of their parameters. This time, instead of sequences in the representation, there are sets of objects, which allow for the elimination of duplicates. As a result, the reconstruction is faster. The algorithm presented in the paper solves the inverse problem of finding the minimum of the objective function. Heuristic algorithms are suitable for solving this type of tasks. The following heuristic algorithms are described, tested and compared: Aquila Optimizer (AQ), Firefly Algorithm (FA), Whale Optimization Algorithm (WOA), Butterfly Optimization Algorithm (BOA) and Dynamic Butterfly Optimization Algorithm (DBOA). The research showed that the best algorithm for this type of problem turned out to be DBOA.

Duplicate record is a common problem within data sets especially in huge volume databases. The accuracy of duplicate detection determines the efficiency of duplicate removal process. However, duplicate detection has become more challenging due to the presence of missing values within the records where during the clustering and matching process, missing values can cause records deemed similar to be inserted into the wrong group, hence, leading to undetected duplicates. In this paper, duplicate detection improvement was proposed despite the presence of missing values within a data set through Duplicate Detection within the Incomplete Data set (DDID) method. The missing values were hypothetically added to the key attributes of three data sets under study, using an arbitrary pattern to simulate both complete and incomplete data sets. The results were analyzed, then, the performance of duplicate detection was evaluated by using the Hot Deck method to compensate for the missing values in the key attributes. It was hypothesized that by using Hot Deck, duplicate detection performance would be improved. Furthermore, the DDID performance was compared to an early duplicate detection method namely DuDe, in terms of its accuracy and speed. The findings yielded that even though the data sets were incomplete, DDID was able to offer a better accuracy and faster duplicate detection as compared to DuDe. The results of this study offer insights into constraints of duplicate detection within incomplete data sets. [ABSTRACT FROM AUTHOR]

Phase identification is a process to determine which of the three phases a particular house is connected to. The state-of-the-art identification methods usually exploit smart metering data. However, the data sets are not always available and the major challenge is hence to identify phases with incomplete data set. This paper proposes a novel spectral and saliency analysis identification method to overcome this hurdle. Spectral analysis is first performed to extract the high-frequency features from the incomplete data. Saliency analysis is then adopted to extract salient features from the variations of high-frequency loads in the time domain. Correlation analysis between customer features and the phase features is used to determine customers’ phase connectivity. The method is executed iteratively until all customers with smart meters have been allocated to a specific phase or no salient features can be found. It is validated against real data from over 6000 smart meters in Ireland and achieves an accuracy of over 93% with only 10% smart meter penetration ratio in a 100-household network. [ABSTRACT FROM AUTHOR]

Finite mixture models is a pattern recognition technique that is used for fitting complex data distributions. Parameters of this mixture models are usually determined via the Expectation Maximization (EM) algorithm. A modified version of the EM algorithm is proposed earlier to handle data sets with missing values. This algorithm is affected by the occurrence of outliers in the data, the overlap among classes in the data space and the bias in generating the data from its classes. In addition, it only works well when the missing value rate is low. In this paper, a new algorithm is proposed to overcome these problems. A comparison study shows the superiority of the new algorithm over the modified EM algorithm and other algorithms commonly used in the literature.

A procedure for evaluating candidate auxiliary variable correlations with response variables in incomplete data sets is outlined. The method provides point and interval estimates of the outcome-residual correlations with potentially useful auxiliaries, and of the bivariate correlations of outcome(s) with the latter variables. Auxiliary variables found in this way can enhance considerably the plausibility of the popular missing at random (MAR) assumption if included in ensuing maximum likelihood analyses, or can alternatively be incorporated in imputation models for subsequent multiple imputation analyses. The approach can be particularly helpful in empirical settings where violations of the MAR assumption are suspected, as is the case in many longitudinal studies, and is illustrated with data from cognitive aging research. [ABSTRACT FROM AUTHOR]

Partially missing data sets are a prevailing problem in clustering analysis. We propose a hybrid algorithm combining fuzzy clustering with particle swarm optimization (PSO) for incomplete data clustering, and missing attributes are represented as intervals. Furthermore, we develop a neighbor interval reconstruction (NIR) method based on pre-classification results that estimates the nearest-neighbor interval of missing attribute using the nearest-neighbor rule, which avoids endpoints of intervals determined by different species information, thereby improving the accuracy of missing attribute intervals and enhancing the robustness of missing attribute imputation. Then, the PSO and fuzzy c-means hybrid algorithm are used for clustering the interval-valued data set, and the global optimization ability of the PSO can improve the accuracy of clustering results compared with gradient-based optimization methods. The experimental results for several UCI data sets show the superiority of the proposed NIR hybrid algorithm. [ABSTRACT FROM AUTHOR]

Visualization of large‐scale data inherently requires dimensionality reduction to 1D, 2D, or 3D space. Autoassociative neural networks with a bottleneck layer are commonly used as a nonlinear dimensionality reduction technique. However, many real‐world problems suffer from incomplete data sets, i.e. some values can be missing. Common methods dealing with missing data include the deletion of all cases with missing values from the data set or replacement with mean or “normal” values for specific variables. Such methods are appropriate when just a few values are missing. But in the case when a substantial portion of data is missing, these methods can significantly bias the results of modeling. To overcome this difficulty, we propose a modified learning procedure for the autoassociative neural network that directly takes the missing values into account. The outputs of the trained network may be used for substitution of the missing values in the original data set. Daugiamatės erdvės duomenų grafinis vaizdavimas, kai trūksta reikšmių Santrauka Vaizduojant daugiamatę informaciją, paprastai reikia ją transformuoti į vienmatę, dvimatę arba trimatę erdvę. Nelinijinei daugiamatės erdvės transformacijai paprastai naudojami autoasociatyvieji neuroniniai tinklai. Tačiau, dažnai sprendžiant realias problemas, dalis informacijos dingsta. Taikant tradicinius metodus, elgiamasi dvejopai: trūkstama informacija ignoruojama; trūkstamos reikšmės pakeičiamos vidutinėmis arba tam tikromis konkrečiam kintamajam būdingomis reikšmėmis. Šie metodai tinka tada, kai trūksta tik kelių reikšmių. Kai trūksta daugelio duomenų, minėtieji metodai gali labai iškraipyti modeliavimo rezultatus. Šiai problemai išspręsti autoriai pasiūlė procedūrą, kuri, naudojant autoasociatyvųjį neuroninį tinklą duomenų transformacijai, įvertina trūkstamas reikšmes. Skaičiavimo rezultatai, tobulesniu neuroniniu tinklu gali būti naudojami trūkstamoms reikšmėms pirminėje duomenų aibėje pakeisti. First Published Online: 21 Oct 2010 Reikšminiai žodžiai: duomenų vaizdavimas, daugiamatės erdvės transformacija, autoasociatyvusis neuroninis tinklas, tinklo tobulinimas, neišsamių duomenų imtis.

An actual demand-forecasting problem of the US apparel dealers is studied. Demand is highly fluctuating during the peak sale season and low prior to the peak season. The model is described by the continuous time stochastic process applying the Bayesian process. The standard gamma distribution is selected for the demand process and an inverse gamma distribution is chosen as the conjugate prior for the model. The choice is supported by the maximum likelihood estimate among a number of non-negative distribution models. The proposed Bayesian models predict the probability of the future demand expressed explicitly conditional on the observed demand prior to the peak season. The data set illustrates partial demand of a seasonal product procured by the US dealers from overseas. In recent years, hazard and operational risks due to weather disasters and equipment shutdowns were felt significantly. These caused supply chain disruption and unrecorded demand. The model is extended to contribute to forecast from an unrecorded data set due to supply disruption. Forecasts are compared with real data and a widely implemented adaptive Holt-Winters (H-W) seasonal forecasting model. Results show that the forecasts calculated by the proposed methods do better than those of the adaptive H-W model. [ABSTRACT FROM AUTHOR]

Abstract: Finite mixture models is a pattern recognition technique that is used for fitting complex data distributions. Parameters of this mixture models are usually determined via the Expectation Maximization (EM) algorithm. A modified version of the EM algorithm is proposed earlier to handle data sets with missing values. This algorithm is affected by the occurrence of outliers in the data, the overlap among classes in the data space and the bias in generating the data from its classes. In addition, it only works well when the missing value rate is low. In this paper, a new algorithm is proposed to overcome these problems. A comparison study shows the superiority of the new algorithm over the modified EM algorithm and other algorithms commonly used in the literature. [ABSTRACT FROM AUTHOR]

Visualization of large-scale data inherently requires dimensionality reduction to 1D, 2D, or 3D space. Autoassociative neural networks with a bottleneck layer are commonly used as a nonlinear dimensionality reduction technique. However, many real-world problems suffer from incomplete data sets, i.e. some values can be missing. Common methods dealing with missing data include the deletion of all cases with missing values from the data set or replacement with mean or "normal" values for specific variables. Such methods are appropriate when just a few values are missing. But in the case when a substantial portion of data is missing, these methods can significantly bias the results of modeling. To overcome this difficulty, we propose a modified learning procedure for the autoassociative neural network that directly takes the missing values into account. The outputs of the trained network may be used for substitution of the missing values in the original data set. [ABSTRACT FROM AUTHOR]

Machine learning can extract desired knowledge from existing training examples and ease the development bottleneck in building expert systems. Most learning approaches derive rules from complete data sets. If some attribute values are unknown in a data set, it is called incomplete. Learning from incomplete data sets is usually more difficult than learning from complete data sets. In the past, the rough-set theory was widely used in dealing with data classification problems. In this paper, we deal with the problem of producing a set of certain and possible rules from incomplete data sets based on rough sets. A new learning algorithm is proposed, which can simultaneously derive rules from incomplete data sets and estimate the missing values in the learning process. Unknown values are first assumed to be any possible values and are gradually refined according to the incomplete lower and upper approximations derived from the given training examples. The examples and the approximations then interact on each other to derive certain and possible rules and to estimate appropriate unknown values. The rules derived can then serve as knowledge concerning the incomplete data set. [Copyright &y& Elsevier]

This paper is concerned with the assessment of reliability for equipment which is highly specialized. Of particular interest is the analysis of field data early in the life of the fleet of such equipment. Factors affecting the data underlying assessments are outlined, and a basic statistical model for describing these data is introduced. Methods of assessment of equipment reliability, and the precision of such assessments, are discussed, examples of data from this basic model are analysed, and further examples illustrate the role of model parameters. The model may also be used to analyse incomplete data sets, and we consider the penalty incurred at different levels of incompleteness. The paper concludes with a discussion outlining further possible modifications and refinements to the basic model. [ABSTRACT FROM AUTHOR]

In this article we assess the potential for developing mimics in situations where detailed data on the ecophysiology and ecosystem function of the native ecosystem are not available. We examine two ecosystems in south western Australia, and draw together available information on these. We discuss woodlands dominated by Eucalyptus salmonophloia, which formerly covered much of the valley bottom land in the Western Australian wheatbelt, and kwongan or heath which occurs on sandplain soils at the top of the landscape. From both of these we derive possible mimics which utilise aspects of the native vegetation which wehypothesise contributed to ecosystem function and resilience, but which still allow the production of a harvestable agricultural product.We conclude that situations for which we have adequate ecological knowledge from which to develop sound mimics are likely to be rare. We hence have to be prepared to make best-bet estimates and use these todevelop experimental mimic systems. [ABSTRACT FROM AUTHOR]