Search

This paper presents an assessment of the quality of selected filtration methods for the postprocessing of multibeam echosounder data. In this regard, the methodology used in the quality assessment of these data is an important factor. One of the most important final products derived from bathymetric data is the digital bottom model (DBM). Therefore, quality assessment is often based on factors related to it. In this paper, we propose some quantitative and qualitative factors to perform these assessments, and we analyze a few selected filtration methods as examples. This research makes use of real data gathered in real environments, preprocessed with typical hydrographic flow. The methods presented in this paper may be used in empirical solutions, and the filtration analysis may be useful for hydrographers choosing a filtration method for DBM interpolation. The results showed that both data-oriented and surface-oriented methods can be used in data filtration and that various evaluation methods show different perspectives on data filtration quality assessment. [ABSTRACT FROM AUTHOR]

The present article describes a method for enhanced accuracy in machining free-form surfaces produced on CNC milling machines. In this method, surface patch machining programs are generated based on their nominal CAD model. After the pretreatment, coordinate control measurements are carried out. The obtained results of the measurements contain information on the values and distribution of observed machining deviations. These data, after appropriate processing, are used to build a corrected CAD model of the surface produced. This model, made using reverse engineering techniques, compensates for the observed machining deviations. After regeneration of machining programs, the object processing and control measurements are repeated. As a result of the conducted procedure, the accuracy of the manufacture of the surface object is increased. This article also proposes the introduction of a simple procedure for the filtration of measurement data. Its purpose is to minimise the effect of random phenomena on the final machining error correction. The final part of the article presents the effects of the proposed method of increasing the accuracy of manufacturing on 'raw' and filtered measurement data. In both cases, a significant improvement in the accuracy of the machining process was achieved, with better final results obtained from the filtered measurement data. The method proposed in the article has been verified for three-axis machining with a ball-end cutter. [ABSTRACT FROM AUTHOR]

This paper presents an assessment of the quality of selected filtration methods for the postprocessing of multibeam echosounder data. In this regard, the methodology used in the quality assessment of these data is an important factor. One of the most important final products derived from bathymetric data is the digital bottom model (DBM). Therefore, quality assessment is often based on factors related to it. In this paper, we propose some quantitative and qualitative factors to perform these assessments, and we analyze a few selected filtration methods as examples. This research makes use of real data gathered in real environments, preprocessed with typical hydrographic flow. The methods presented in this paper may be used in empirical solutions, and the filtration analysis may be useful for hydrographers choosing a filtration method for DBM interpolation. The results showed that both data-oriented and surface-oriented methods can be used in data filtration and that various evaluation methods show different perspectives on data filtration quality assessment.

The world health organization (WHO) terms dengue as a serious illness that impacts almost half of the world's population and carries no specific treatment. Early and accurate detection of spread in affected regions can save precious lives. Despite the severity of the disease, a few noticeable works can be found that involve sentiment analysis to mine accurate intuitions from the social media text streams. However, the massive data explosion in recent years has led to difficulties in terms of storing and processing large amounts of data, as reliable mechanisms to gather the data and suitable techniques to extract meaningful insights from the data are required. This research study proposes a sentiment analysis polarity approach for collecting data and extracting relevant information about dengue via Apache Hadoop. The method consists of two main parts: the first part collects data from social media using Apache Flume, while the second part focuses on querying and extracting relevant information via the hybrid filtration-polarity algorithm using Apache Hive. To overcome the noisy and unstructured nature of the data, the process of extracting information is characterized by pre and post-filtration phases. As a result, only with the integration of Flume and Hive with filtration and polarity analysis, can a reliable sentiment analysis technique be offered to collect and process large-scale data from the social network. We introduce how the Apache Hadoop ecosystem - Flume and Hive - can provide a sentiment analysis capability by storing and processing large amounts of data. An important finding of this paper is that developing efficient sentiment analysis applications for detecting diseases can be more reliable through the use of the Hadoop ecosystem components than through the use of normal machines. [ABSTRACT FROM AUTHOR]

Deep neural networks (DNNs) have achieved astonishing results on a variety of supervised learning tasks owing to a large scale of well-labeled training data. However, as recent researches have pointed out, the generalization performance of DNNs is likely to sharply deteriorate when training data contains label noise. In order to address this problem, a novel loss function is proposed to guide DNNs to pay more attention to clean samples via adaptively weighing the traditional cross-entropy loss. Under the guidance of this loss function, a cross-training strategy is designed by leveraging two synergic DNN models, each of which plays the roles of both updating its own parameters and generating curriculums for the other one. In addition, this paper further proposes an online data filtration mechanism and integrates it into the final cross-training framework, which simultaneously optimizes DNN models and filters out noisy samples. The proposed approach is evaluated through a great deal of experiments on several benchmark datasets with man-made or real-world label noise, and the results have demonstrated its robustness to different noise types and noise scales.

In this article, we focus on the usage of MQTT (Message Queuing Telemetry Transport) within Connected Vehicles (CVs). Indeed, in the original version of MQTT protocol, the broker is responsible “only” for sending received data to subscribers; abstracting then the underlying mechanism of data exchange. However, within CVs context, subscribers (i.e., the processing infrastructure) may be overloaded with irrelevant data, in particular when the requirement is real or near real-time processing. To overcome this issue, we propose MQTT-CV; a new variant of MQTT protocol, in which the broker is able to perform local processing in order to reduce the workload at the infrastructure; i.e., filtering data before sending them. In this article, we first validate formally the correctness of MQTT-CV protocol (i.e., the three components of the proposed protocol are correctly interacting), through the use of Promela language and its system verification tool; the model checker SPIN. Secondly, using real-world data provided by our car manufacturer partner, we have conducted real implementation and experiments. The obtained results show the effectiveness of our approach in term of data workload reduction at the processing infrastructure. The mean improvement, besides the fact that it is dependent of the target application, was in general about 10 times less in comparison to native MQTT protocol. [ABSTRACT FROM AUTHOR]

The paper reports the forecasting model for multiple time-domain photovoltaic power plants, developed in response to the necessity of bad weather days' accurate and robust power generation forecasting. We provide a brief description of the piloted short-term forecasting system and place under close scrutiny the main sources of photovoltaic power plants' generation forecasting errors. The effectiveness of the empirical approach versus unsupervised learning was investigated in application to source data filtration in order to improve the power generation forecasting accuracy for unstable weather conditions. The k-nearest neighbors' methodology was justified to be optimal for initial data filtration, based on the clusterization results, associated with peculiar weather and seasonal conditions. The photovoltaic power plants' forecasting accuracy improvement was further investigated for a one hour-ahead time-domain. It was proved that operational forecasting could be implemented based on the results of short-term day-ahead forecast mismatches predictions, which form the basis for multiple time-domain integrated forecasting tools. After a comparison of multiple time series forecasting approaches, operational forecasting was realized based on the second-order autoregression function and applied to short-term forecasting errors with the resulting accuracy of 87%. In the concluding part of the article the authors from the points of view of computational efficiency and scalability proposed the hardware system composition. [ABSTRACT FROM AUTHOR]

The Internet of Things (IoT) is a network formed by smart devices whose core contains embedded technology in order to collect sensory information and exchange it with every single device present within the network. One major challenge in IoT is data stabilization of the sensory data collected by smart devices before it is fed into the cloud. Data collected by sensing devices suffer with random glitches, variation in volume; inter packet delay and data packet loss due to inline power fluctuations. The cleaning and filtration of such noisy characteristics must be done before restoring this raw data to some global repository. This paper advises a method for clean monitoring of captured environmental parameters and proposes few algorithms that help in stabilizing the captured sensory data on the basis of data packet volume, inter packet delay, data type and random amplitude levels. Such an approach supports service level implementation required by an enterprise for deploying error free stabilized data collection without any ambiguity in order to compute an accurate and sustainable result required for effective environmental parameter analysis. [ABSTRACT FROM AUTHOR]

Several natural languages have undergone a great deal of processing, but the problem of limited textual linguistic resources remains. The manual creation of parallel corpora by humans is rather expensive and time consuming, while the language data required for statistical machine translation (SMT) do not exist in adequate quantities for their statistical information to be used to initiate the research process. On the other hand, applying known approaches to build parallel resources from multiple sources, such as comparable or quasi-comparable corpora, is very complicated and provides rather noisy output, which later needs to be further processed and requires in-domain adaptation. To optimize the performance of comparable corpora mining algorithms, it is essential to use a quality parallel corpus for training of a good data classifier. In this research, we have developed a methodology for generating an accurate parallel corpus (Czech-English, Polish-English) from monolingual resources by calculating the compatibility between the results of three machine translation systems. We have created translations of large, single-language resources by applying multiple translation systems and strictly measuring translation compatibility using rules based on the Levenshtein distance. The results produced by this approach were very favorable. The generated corpora successfully improved the quality of SMT systems and seem to be useful for many other natural language processing tasks. [ABSTRACT FROM AUTHOR]

Abstract Lifetime maximization is an important issue while designing wireless sensor network. One of the ways to increase the lifetime of WSN is to reduce the energy demand of the sensor nodes and cluster head nodes, which can be decreased by filtering out redundant data traffic. A Two-Layer Hierarchal Aggregation (TLHA) protocol has been proposed, which filter out redundant data traffic. Simple aggregation algorithm has been used to classify the sensed data based on their normalized standard deviation for efficient data filtration. The algorithm also performs k- Means based re-classification of data to improve the efficiency of the algorithm. Two-layered redundant data filtration technique has been proposed to classify the data. Aggregation protocol reduces data traffic in lower layer hierarchy (sensor nodes to cluster head node) as well as upper layer hierarchy (cluster head nodes to base station node). In former, Energy efficient Time Division Multiple Access (EA-TDMA) Medium Access Control (MAC) protocol is used while in latter Bit-Map-Assisted (BMA) MAC protocol is used for transmission of data. Simulation and experimental results show that TLHA saves an enormous amount of energy, which finally increases the lifetime of the sensor network. The performance of the proposed TLHA protocol has been compared with already existing protocols viz. Data Aggregation Window Function (DAWF) and Spatial-Temporal Correlation Algorithm (STCA). [ABSTRACT FROM AUTHOR]

The paper reports the forecasting model for multiple time-domain photovoltaic power plants, developed in response to the necessity of bad weather days’ accurate and robust power generation forecasting. We provide a brief description of the piloted short-term forecasting system and place under close scrutiny the main sources of photovoltaic power plants’ generation forecasting errors. The effectiveness of the empirical approach versus unsupervised learning was investigated in application to source data filtration in order to improve the power generation forecasting accuracy for unstable weather conditions. The k-nearest neighbors’ methodology was justified to be optimal for initial data filtration, based on the clusterization results, associated with peculiar weather and seasonal conditions. The photovoltaic power plants’ forecasting accuracy improvement was further investigated for a one hour-ahead time-domain. It was proved that operational forecasting could be implemented based on the results of short-term day-ahead forecast mismatches predictions, which form the basis for multiple time-domain integrated forecasting tools. After a comparison of multiple time series forecasting approaches, operational forecasting was realized based on the second-order autoregression function and applied to short-term forecasting errors with the resulting accuracy of 87%. In the concluding part of the article the authors from the points of view of computational efficiency and scalability proposed the hardware system composition.

Rotation of coronal holes is studied using data from SDO/AIA for 2014 and 2015. A new approach to the treatment of data is applied. Instead of calculated average angular velocities of each coronal hole centroid and then grouping them in latitudinal bins for calculating average rotation rates of corresponding latitudes, we compiled instant rotation rates of centroids and their corresponding heliographic coordinates in one matrix for further processing. Even unfiltered data showed clear differential nature of rotation of coronal holes. We studied possible reasons for distortion of data by the limb effects to eliminate some discrepancies at high latitudes caused by the high order of scattering of data in that region. A study of the longitudinal distribution of angular velocities revealed the optimal longitudinal interval for the best result. We examined different methods of data filtering and realized that filtration using targeting on the local medians of data with a constant threshold is a more acceptable approach that is not biased towards a predefined notion of an expected result. The results showed a differential pattern of rotation of coronal holes. [ABSTRACT FROM AUTHOR]

The paper is devoted to the problem of consistency-based preprocessing of MMPI data for classification tasks. MMPI (Minnesota Multiphasic Personality Inventory) is a standardized psychometric test of adult personality and psychopathology. The MMPI test delivers psychometric data in a form of the so-called profiles consisting of values of thirteen descriptive attributes (corresponding to scales). The preprocessing procedure covers the filtration of cases using consistency factors calculated according to the original rough set based approach. The proposed filtration enables us to simplify the size of classifiers without a significant loss of their classification ability. [ABSTRACT FROM AUTHOR]

The article deals with the processing of log dimension data collected by electronic measurement during reception at sawmills. The subject of the work concerns the fi ltration of data before their use for calculation. Filtration methods were designed based on simple mathematical and statistical methods, and compared and evaluated with the use of a designed comparative methodology. As a result, five filtration methods were selected that best suit the reception requirements. At the same time, the impact of filtration on the measurement results is evaluated in relation to the calculation method of wood volume. Furthermore, the calculation by sections is recommended, as it is less affected by filtration errors.

Metabolomics generates massive and complex data. Redundant different analytical species and the high degree of correlation in datasets is a constraint for the use of data mining/statistical methods and interpretation. In this context, we developed a new tool to detect analytical correlation into datasets without confounding them with biological correlations. Based on several parameters, such as a similarity measure, retention time, and mass information from known isotopes, adducts, or fragments, the algorithm principle is used to group features coming from the same analyte, and to propose one single representative per group. To illustrate the functionalities and added-value of this tool, it was applied to published datasets and compared to one of the most commonly used free packages proposing a grouping method for metabolomics data: ‘CAMERA’. This tool was developed to be included in Galaxy and is available in Workflow4Metabolomics.

The data taken at the electronic reception of logs are tasked with mistaken values and do not correspond to the real shape of a stem. The aim of the data filtration is to remove the incorrect data and replace them by the ones, closer to the real values.The goal of presented work was to analyse the effect of different filtration methods on original data and to recommend the methods, which results correspond to the real shape of the logs with their defects on a surface as close as possible.Methods of filtration are based on simple mathematical and statistical procedures, which are subsequently variously combined, because the simplest comparative methods did not fulfil the expectations.More than 50 methods were combined and analyzed. Approximately 15 of them were selected and tested. The methods were applied to data scanned on ca 150 logs randomly selected from thousands of files in the sawmill log yard. The filtration effects of different methods were visually assessed. The most applicable three methods were selected and they will be tested practically at sawmill in the next step of research. These three methods are described in following paper.Results refer, that all proposed methods correspond to necessities for log dimensions determination. It is not possible to define the best method generally. Different properties of these methods make them suitable for calculation of log dimensions if different type of use is requested. It means for mid diameter determination necessary for volume calculation by using Huber method; for section dia­me­ter determination for volume calculation by using section method; for top diameter determination necessary for sorting; for filtration of drive dogs of the conveyor passing through the measuring equipment.

Copyright of Wood Industry / Drvna Industrija is the property of Drvna Industrija and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Success/failure of adaptive control algorithms – especially those designed using the Linear Quadratic Gaussian criterion – depends on the quality of the process data used for model identification. One of the most harmful types of process data corruptions are outliers, i.e. ‘wrong data’ lying far away from the range of real data. The presence of outliers in the data negatively affects an estimation of the dynamics of the system. This effect is magnified when the outliers are grouped into blocks. In this paper, we propose an algorithm for outlier detection and removal. It is based on modelling the corrupted data by a two-component probabilistic mixture. The first component of the mixture models uncorrupted process data, while the second models outliers. When the outlier component is detected to be active, a prediction from the uncorrupted data component is computed and used as a reconstruction of the observed data. The resulting reconstruction filter is compared to standard methods on simulated and real data. The filter exhibits excellent properties, especially in the case of blocks of outliers.

A critical step for DNA array analysis is data filtration, which can reduce thousands of detected signals to limited sets of genes. Commonly accepted rules for such filtration are still absent. We present a rational approach, based on thresholding of intensities with cutoff levels that are estimated by receiver operating characteristic (ROC) analysis. The technique compares test results with known distributions of positive and negative signals. We apply the method to Atlas cDNA arrays, GeneFilters, and Affymetrix GeneChip. ROC analysis demonstrates similarities in the distribution of false and true positive data for these different systems. We illustrate the estimation of an optimal cutoff level for intensity-based filtration, providing the highest ratio of true to false signals. For GeneChip arrays, we derived filtration thresholds consistent with the reported data based on replicate hybridizations. Intensity-based filtration optimized with ROC combined with other types of filtration (for example, based on significances of differences and/or ratios), should improve DNA array analysis. ROC methodology is also demonstrated for comparison of the performance of different types of arrays, imagers, and analysis software. [Copyright &y& Elsevier]

Surface topography measurements are vital in industrial quality control. Linear roughness measurements are among the most preferred methods, being quick to perform and easy to interpret. The ISO 16610 standard series prescribes filters that can be used for most cases, but has limitations for restricted measurement lengths. This is because the standard filter type is a Gaussian filter, which like most instances of kernel convolution filters has no output near the edges of the profile, effectively shortening the length of the filtered output profile as compared to the input. In some cases, this leads to a lack of representative data after filtration. Especially in fields such as Additive Manufacturing (AM) this becomes a problem, due to the high "roughness to measurable data length"-ratio that characterizes complex AM parts. This paper describes a method that allows to overcome this limitation:•A method for circular padding of short measured tracks is described and validated.•A flexible profile data post-processing tool was developed in MATLAB to grant users more control over the data analysis. Results obtained from roughness profiles long enough for normal ISO procedures are shown to not change significantly when circularly padded. When only a shorter section of the data is available, where the standard protocol would not be able to compute a filtered profile and related parameters anymore, the circular padding method is shown to lead to results that are in good agreement with the ISO standard procedures., Competing Interests: The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (© 2020 The Authors. Published by Elsevier B.V.)