Search

The recent work Unsupervised Linear Discriminant Analysis (Un-LDA) completes its clustering process during the alternating optimization by converting equivalently the objective and finally using the K-means algorithm. However, the K-means algorithm has its inherent drawbacks. It is hard for the K-means algorithm to deal well with some complex clustering cases where there are too many real clusters or non-convex clusters. In this paper, a novel clustering optimization method is presented to accomplish the clustering process in Un-LDA and the resulting method can be named Un-LDA(CD). Specifically, instead of the K-means algorithm, an elaborately designed coordinate descent algorithm is adopted to obtain the clusters after the objective function goes through a series of simple but deft equivalent conversions. Extensive experiments have demonstrated that the coordinate descent clustering solution for Un-LDA can outperform the original K-means based solution on the tested data sets especially those complex data sets with a pretty large number of real clusters.

While we instantaneously recognize a face as attractive, it is much harder to explain what exactly defines personal attraction. This suggests that attraction depends on implicit processing of complex, culturally and individually defined features. Generative adversarial neural networks (GANs), which learn to mimic complex data distributions, can potentially model subjective preferences unconstrained by pre-defined model parameterization. Here, we present generative brain-computer interfaces (GBCI), coupling GANs with brain-computer interfaces. GBCI first presents a selection of images and captures personalized attractiveness reactions toward the images via electroencephalography. These reactions are then used to control a GAN model, finding a representation that matches the features constituting an attractive image for an individual. We conducted an experiment (N=30) to validate GBCI using a face-generating GAN and producing images that are hypothesized to be individually attractive. In double-blind evaluation of the GBCI-produced images against matched controls, we found GBCI yielded highly accurate results. Thus, the use of EEG responses to control a GAN presents a valid tool for interactive information-generation. Furthermore, the GBCI-derived images visually replicated known effects from social neuroscience, suggesting that the individually responsive, generative nature of GBCI provides a powerful, new tool in mapping individual differences and visualizing cognitive-affective processing.

The healthcare industry has become increasingly challenging, requiring retrieval of knowledge from large amounts of complex data to find the best treatments. Several works have suggested the use of Data Mining tools to overcome the challenges; however, none of them has suggested the best tool to do so. To fill this gap, this paper presents a survey of popular open-source data mining tools in which data mining tool selection criteria based on healthcare application requirements is proposed and the best ones using the proposed selection criteria are identified. The following popular open-source data mining tools are assessed: KNIME, R, RapidMiner, Scikit-learn, and Spark. The study shows that KNIME and RapidMiner provide the largest coverage of healthcare data mining requirements.

In the era of big data, faced with massive and complex information, many mathematical concepts used to make judgments and decisions have emerged. In order to better integrate the multi-level fuzzy information, on the basis of the original covering rough set, we generalize the couple approximate operators defined by L.W. Ma to the information system and propose a new binary model—dyad fuzzy β-covering rough set models. This model can analyze and solve practical problems from multiple angles, so as to make more accurate decisions. In addition, in order to make the model more convenient for large and complex data processing, we use matrix to represent the model and realize it by computer programming. Finally, we illustrate the value of this model by solving a practical problem.

Deep learning has gained broad interest in remote sensing image scene classification thanks to the effectiveness of deep neural networks in extracting the semantics from complex data. However, deep networks require large amounts of training samples to obtain good generalization capabilities and are sensitive to errors in the training labels. This is a problem in remote sensing since highly reliable labels can be obtained at high costs and in limited amount. However, many sources of less reliable labeled data are available, e.g., obsolete digital maps. In order to train deep networks with larger datasets, we propose both the combination of single or multiple weak sources of labeled data with a small but reliable dataset to generate multisource labeled datasets and a novel training strategy where the reliability of each source is taken into consideration. This is done by exploiting the transition matrices describing the statistics of the errors of each source. The transition matrices are embedded into the labels and used during the training process to weigh each label according to the related source. The proposed method acts as a weighting scheme at gradient level, where each instance contributes with different weights to the optimization of different classes. The effectiveness of the proposed method is validated by experiments on different datasets. The results proved the robustness and capability of leveraging on unreliable source of labels of the proposed method.

This study proposes, for the first time, an approach to remove thermal noise from the wave coherency matrix, C₂, estimated from single-look complex dual-polarization Interferometric Wide Swath mode Sentinel-1 synthetic aperture radar data. The approach is straightforward; it exploits the ThermalNoiseRemoval module, provided by the European Space Agency (ESA) in its Sentinel Application Platform (SNAP) software, to remove thermal noise from the channel intensities. Then, noise correction on the complex data is applied, in order to estimate the noise-free C₂ matrix. As a further novelty, the proposed approach can be implemented in SNAP, through the use of a processing graph that is here provided. The method is applied on a dense time series of Sentinel-1 data, collected on an agricultural area located near Seville, Spain. The impact of thermal noise on the estimation of the eigendecomposition parameters of C₂, i.e., entropy (H₂), average alpha angle (α₂), and anisotropy (A₂), is assessed for different land-cover types, namely river, rice, forest, and urban areas. Monte Carlo simulations are implemented to assess the performance of the proposed approach in estimating H₂, α₂, and A₂. Results show that the proposed noise removal method improves the estimation of these parameters for the considered land-cover classes.

Entity resolution (ER) leads a key role in data preprocessing. ER identifies records corresponding to the same real-world entity. Recent years have witnessed a growing trend of deep learning based ER (deep ER). However, previous deep ER works do not fully utilize schema semantics, since they either use hard schema matching or disregard schema matching. In this work, we flexibly exploit schema matching to enhance deep ER. We define and implement soft schema matching, where attributes are flexibly associated in probabilities. Attribute associations are generated by aggregating token connections in coarse deep ER. Then we incorporate soft schema matching into hierarchical attention networks for ER, which tremendously improves resolution quality, especially for complex data and corrupted data. Different attentions are utilized for particular sub-tasks in ER networks, such as self-attention for contextualization, inter-attention for alignment and intra-attention for weighting. Finally comprehensive experiments are run over common data, complex data and corrupted data. Evaluation results show that our approach surpasses previous works.

La Trobe University Faculty of Science, Technology and Engineering Murray Darling Freshwater Research CentreMDFRC item.Terminal restriction fragment length polymorphism (T-RFLP) is increasingly being used to examine microbial community structure and accordingly, a range of approaches have been used to analyze data sets. A number of published reports have included data and results that were statistically flawed or lacked rigorous statistical testing. A range of simple, yet powerful techniques are available to examine community data, however their use is seldom, if ever, discussed in microbial literature. We describe an approach that overcomes some of the problems associated with analyzing community datasets and offer an approach that makes data interpretation simple and effective. The Bray-Curtis coefficient is suggested as an ideal coefficient to be used for the construction of similarity matrices. Its strengths include its ability to deal with data sets containing multiple blocks of zeros in a meaningful manner. Non-metric multi-dimensional scaling is described as a powerful, yet easily interpreted method to examine community patterns based on T-RFLP data. Importantly, we describe the use of significance testing of data sets to allow quantitative assessment of similarity, removing subjectivity in comparing complex data sets. Finally, we introduce a quantitative measure of sample dispersion and suggest its usefulness in describing site heterogeneity.

Maritime transports play a critical role in international trade and commerce. Massive vessels sailing around the world continuously generate vessel trajectory data that contain rich spatial- temporal patterns of vessel navigations. Analyzing and understanding these patterns are valuable for maritime traffic surveillance and management. As essential techniques in complex data analy- sis and understanding, visualization and visual analysis have been widely used in vessel trajectory data analysis. This paper presents a literature review on the visualization and visual analysis of vessel trajectory data. First, we introduce commonly used vessel trajectory data sets and sum- marize main operations in vessel trajectory data preprocessing. Then, we provide a taxonomy of visualization and visual analysis of vessel trajectory data based on existing approaches and introduce representative works in details. Finally, we expound on the prospects of the remaining challenges and directions for future research.

The implementation of high-resolution mass spectrometry systems offers new possibilities for the analysis of complex art samples such as historical oil paintings. However, these multicomponent systems generate large and complex data sets that require advanced visualization tools to aid interpretation, especially when no chromatographic separation is performed. In the context of this research, it was crucial to propose a data analysis tool to identify the products generated during the synthesis, drying, and aging of historical pigments. This study reports for the first time a nontraditional mass defect analysis of oil paint samples containing a fugitive brominated-organic pigment, eosin or geranium lake, by using direct infusion electrospray ionization in combination with a high-resolution Orbitrap mass spectrometer. The use of nontraditional Kendrick mass defect plots is presented in this study as a processing and visualization tool to recognize brominated species based on their specific mass defect and isotope pattern. The results demonstrate that this approach could provide valuable molecular compositional information on the degradation pathways of this pigment. We anticipate that mass defect analysis will become highly relevant in future degradation studies of many more historical organic pigments.

Structural equation modeling (SEM) is being applied to ever more complex data types and questions, often requiring extensions such as regularization or novel fitting functions. To extend SEM, researchers currently need to completely reformulate SEM and its optimization algorithm–a challenging and time–consuming task. In this paper, we introduce the computation graph for SEM, and show that this approach can extend SEM without the need for bespoke software development. We show that both existing and novel SEM improvements follow naturally. To demonstrate, we introduce three SEM extensions: least absolute deviation estimation, Bayesian LASSO optimization, and sparse high–dimensional mediation analysis. We provide an implementation of SEM in PyTorch–popular software in the machine learning community–to accelerate development of structural equation models adequate for modern–day data and research questions.

Processing huge and complex data to obtain useful information is challenging, even though several big data processing frameworks have been proposed and further enhanced. One of the prominent big data processing frameworks is MapReduce. The main concept of MapReduce framework relies on distributed and parallel processing. However, MapReduce framework is facing serious performance degradations due to the slow execution of certain tasks type called stragglers. Failing to handle stragglers causes delay and affects the overall job execution time. Meanwhile, several straggler reduction techniques have been proposed to improve the MapReduce performance. This study provides a comprehensive and qualitative review of the different existing straggler mitigation solutions. In addition, a taxonomy of the available straggler mitigation solutions is presented. Critical research issues and future research directions are identified and discussed to guide researchers and scholars

Generative adversarial networks play an important role in image generation, but the successful generation of high-resolution images from complex data sets remains a challenging goal. In this paper, we propose the LGAN (Link Generative Adversarial Networks) model, which can effectively enhance the quality of the synthesized images. The LGAN model consists of two parts, G1 and G2. G1 is responsible for the unconditional generation part, which generates anime images with highly abstract features containing few coefficients but continuous image elements covering the overall image features. Moreover, G2 is responsible for the conditional generation part (image translation), consisting of mapping and Superresolution networks. The mapping network fills the output of G1 into the real-world image after semantic segmentation or edge detection processing; the Superresolution network super-resolves the actual picture after completing mapping to improve the image’s resolution. In the comparison test with WGAN, SAGAN, WGAN-GP and PG-GAN, this paper’s LGAN(SEG) leads 64.36 and 12.28, respectively, fully proving the model’s superiority.

An early step of evaluating a nanomaterial’s potential for biological applications is investigating its interactions with cells and biological fluids. These experiments generate complex data, which...

With the rapid development of information technology, data streams in various fields are showing the characteristics of rapid arrival, complex structure and timely processing. Complex types of data streams make the classification performance worse. However, ensemble classification has become one of the main methods of processing data streams. Ensemble classification performance is better than traditional single classifiers. This article introduces the ensemble classification algorithms of complex data streams for the first time. Then overview analyzes the advantages and disadvantages of these algorithms for steady-state, concept drift, imbalanced, multi-label and multi-instance data streams. At the same time, the application fields of data streams are also introduced which summarizes the ensemble algorithms processing text, graph and big data streams. Moreover, it comprehensively summarizes the verification technology, evaluation indicators and open source platforms of complex data streams mining algorithms. Finally, the challenges and future research directions of ensemble learning algorithms dealing with uncertain, multi-type, delayed, multi-type concept drift data streams are given.

Hierarchical multilabel classification (HMC) aims to classify the complex data such as text with multiple topics and image with multiple semantics, in which the multiple labels are organized in hierarchical structures such as trees and direct acyclic graphs (DAG). To reduce the computational complexity, HMC methods generally assume that the class labels of different branches in a hierarchical structure are conditional independent. However, these class-independent HMC methods neglect the correlation between labels and thereby the precision of classification is affected. To tackle the problem, in this paper we propose a hierarchical multilabel classification method with class label correlation (HMC-CLC) which exploits the label correlations of different branches to benefit the discrimination of HMC. Specifically, in the training stage, for each label in the hierarchy, we use feature incremental learning to encode the labels of different branches into the input space. Based on this, the label correlations of different branches are reflected by the weights of classification model in corresponding dimensions. Then in the test stage, considering that the different samples have different label distributions, we propose a greedy label selection method to dynamically decide the correlated labels of different branches for each label. Therefore, for the same label in the hierarchy, the correlated labels could be different in different samples. Experimental results on a number of real-world data sets show that the proposed method outperforms the state-of-the-art HMC methods.

Reversible data hiding in encrypted domain (RDH-ED) technology can embed data into cover media without exposing the original content to third parties. In addition, the recipient can recover the cover media losslessly after extracting the embedded data. Image-based RDH-ED has been widely studied, but RDH-ED based on 3D meshes has obtained few research results due to the complex data structure and irregular geometric structure of 3D meshes. With the widespread application of 3D meshes, the research on 3D meshes has attracted extensive research from researchers in recent years. In this paper, we propose a reversible data hiding for encrypted 3D meshes based on integer mapping and most significant bit (MSB) prediction. The content owner divides all vertices into “embedded” sets and “reference” sets and then maps floating-point coordinates to integers. After calculating the MSB prediction error of the “embedded” sets, the encryption technology is performed. Then, additional data can be embedded through the MSB replacement strategy. According to different permissions, legal recipients can obtain the original meshes, the additional data or both of them by using the proposed separable method. Higher embedding capacity is achieved by adopting MSB embedding strategy, and perfect recovery of the original meshes is achieved by using ring prediction scheme. The experimental results show that the proposed method has greater embedding capacity compared with the state-of-the-art method.

This paper describes a collaboration organized around exchanging data between two technological systems to support teachers’ instructional decision-making. The goals of the collaboration among researchers, technology developers, and practitioners were not only to support teachers’ instructional decision-making but also to document the challenges and opportunities associated with bringing together data from instruction- and assessment-focused technologies. The approach described in this paper illustrates the potential importance of anchoring data products that combine data between two systems in the needs of teachers as well as aligning the content that students learn and are assessed on between systems. The increasing presence of data standards has made sharing complex data increasingly more feasible. The example collaboration described in this paper demonstrates the role that non-technical activities can play in supporting the exchange and use of learner event data.

The aim of this work is to analyse the performance of the new proposed hybrid parameterisation model in handling problematic data. Three types of problematic data will be highlighted in this paper: i) big data set, ii) uncertain and inconsistent data set and iii) imbalanced data set. The proposed hybrid model is an integration of three main phases which consist of the data decomposition, parameter reduction and parameter selection phases. Three main methods, which are soft set and rough set theories, were implemented to reduce and to select the optimised parameter set, while a neural network was used to classify the optimised data set. This proposed model can process a data set that might contain uncertain, inconsistent and imbalanced data. Therefore, one additional phase, data decomposition, was introduced and executed after the pre-processing task was completed in order to manage the big data issue. Imbalanced data sets were used to evaluate the capability of the proposed hybrid model in handling problematic data. The experimental results demonstrate that the proposed hybrid model has the potential to be implemented with any type of data set in a classification task, especially with complex data sets.

Sparsity-driven inverse synthetic aperture radar (ISAR) imaging can obtain a high-resolution image of non-cooperative targets using compressive sensing (CS)-based methods. However, traditional CS-based imaging methods generally require extensive computational complexity, and some fixed parameters need to be manually tuned to be optimal for better imaging quality. Iterative-thresholding methods are typical CS-based algorithms with representative framework and efficient performance. This paper proposes a general structure of hierarchical channel-mixed framework (HCMF) for sparsity-driven ISAR imaging. Specifically, HCMF is constructed by unrolling the iterative-thresholding methods, and each phase of HCMF is mapped from one iteration in traditional iterative-thresholding methods, maintaining the hierarchical framework. HCMF also involves mixed channels that flow separately and together at different processes, to process complex data properly. Besides, the parameters in each phase are set to be learnable and self-adapted optimal, contributing to greater robustness. Without loss of generality, we take approximate message-passing (AMP) for demonstration, dubbed as HCM-AMP-Net (HCM-AMPN). HCM-AMPN has the almost same computational complexity as traditional AMP, but the high-parallelization of deep network accelerates computational efficiency dramatically. Experimental results based on simulated and measured data demonstrate that HCM-AMPN has higher efficiency and superior performance compared with some state-of-the-art methods. In our experiments, HCM-AMPN is more than 100 times faster than traditional AMP, and tens of times faster than other network-based methods.

With the expansion of data scale and the increase in data complexity, it is particularly important to accurately identify clusters and efficiently save clustering results. To address this, we propose a novel clustering algorithm, Shape clustering based on data field (STATE), which can quickly identify clusters of arbitrary shapes and greatly reduce the storage space of clustering results in any datasets without reducing the accuracy. STATE mainly focuses on finding the edges of clusters and directions of edges instead of clustering centers through the data field. The results of STATE are presented as the edges of clusters without data objects inside clusters and without noise. Extensive experiments show that STATE can recognize complex data distribution in noisy environments without discrimination and greatly save the storage space of clustering results. When it is applied in a real-world scene, facial feature extraction, STATE can recognize eyes, nose, mouth, eyebrows and facial contours automatically without calibrating key features or training. Using the extracted facial features, we achieve facial recognition with high accuracy.

Background This paper exploits recent developments in topological data analysis to present a pipeline for clustering based on Mapper, an algorithm that reduces complex data into a one-dimensional graph. Results We present a pipeline to identify and summarise clusters based on statistically significant topological features from a point cloud using Mapper. Conclusions Key strengths of this pipeline include the integration of prior knowledge to inform the clustering process and the selection of optimal clusters; the use of the bootstrap to restrict the search to robust topological features; the use of machine learning to inspect clusters; and the ability to incorporate mixed data types. Our pipeline can be downloaded under the GNU GPLv3 license at https://github.com/kcl-bhi/mapper-pipeline.

Pharmacometric models can enhance clinical decision making, with covariates exposing potential contributions to variability of subpopulation characteristics, for example, demographics or disease status. Intuitive visualization of models with multiple covariates is needed because sparsity of data in visualizations trellised by covariate values can raise concerns about the credibility of the underlying model. V2ACHER, introduced here, is a stepwise transformation of data that can be applied to a variety of static (non‐ordinary‐differential‐equation‐based) pharmacometric analyses. This work uses four examples of increasing complexity to show how the transformation elucidates the relationship between observations and model results and how it can also be used in visual predictive checks to confirm the quality of a model. V2ACHER facilitates consistent, intuitive, single‐plot visualization of a multicovariate model with a complex data set, thereby enabling easier model communication for modelers and for cross‐functional development teams and facilitating confident use in support of decisions.

A method of mining and analyzing remote sensing user needs is presented in this paper. With the increasing number and types of satellites in China, satellites are more and more widely used in national economic construction, which can be used in land and resources census, emergency natural disaster relief, urban construction management, earth environment monitoring and so on. Therefore, the number of remote sensing users expand quickly and the specific content of remote sensing observation needs and remote sensing image data needs is diverse. How to find out the user’s remote sensing resources and data use rules from the large number of remote sensing needs is a key problem. The difficulty lies in that the rules representing the relationship between demand parameters are recessive rather than explicit, which can only be found by complex data analysis, especially under the condition of large amount of data. Data mining technology can find potential as well as valuable rules and knowledge from massive data. It is an effective tool to analyze massive data and an effective technical means to solve the above problems. In this paper, the association rules mining algorithm is adopted to mine and analyze mass of remote sensing user demand data, and find the potential association rules between the demand source, regional scope, time, demand type and other parameters.

Recently, big data are widely noticed in many fields like machine learning, pattern recognition, medical, financial, and transportation fields. Data analysis is crucial to converting data into more specific information fed to the decision-making systems. With the diverse and complex types of datasets, knowledge discovery becomes more difficult. One solution is to use feature subset selection preprocessing that reduces this complexity, so the computation and analysis become convenient. Preprocessing produces a reliable and suitable source for any data-mining algorithm. The effective features’ selection can improve a model’s performance and help us understand the characteristics and underlying structure of complex data. This study introduces a novel hybrid feature selection cloud-based model for imbalanced data based on the k nearest neighbor algorithm. The proposed model showed good performance compared with the simple weighted nearest neighbor. The proposed model combines the firefly distance metric and the Euclidean distance used in the k nearest neighbor. The experimental results showed good insights in both time usage and feature weights compared with the weighted nearest neighbor. It also showed improvement in the classification accuracy by 12% compared with the weighted nearest neighbor algorithm. And using the cloud-distributed model reduced the processing time up to 30%, which is deliberated to be substantial compared with the recent state-of-the-art methods.

Cutting-edge machine learning techniques often require millions of labeled data objects to train a robust model. Because relying on humans to supply such a huge number of labels is rarely practical, automated methods for label generation are needed. Unfortunately, critical challenges in auto-labeling remain unsolved, including the following research questions: (1) which objects to ask humans to label, (2) how to automatically propagate labels to other objects, and (3) when to stop labeling. These three questions are not only each challenging in their own right, but they also correspond to tightly interdependent problems. Yet existing techniques provide at best isolated solutions to a subset of these challenges. In this work, we propose the first approach, called LANCET, that successfully addresses all three challenges in an integrated framework. LANCET is based on a theoretical foundation characterizing the properties that the labeled dataset must satisfy to train an effective prediction model, namely the Covariate-shift and the Continuity conditions. First, guided by the Covariate-shift condition, LANCET maps raw input data into a semantic feature space, where an unlabeled object is expected to share the same label with its near-by labeled neighbor. Next, guided by the Continuity condition, LANCET selects objects for labeling, aiming to ensure that unlabeled objects always have some sufficiently close labeled neighbors. These two strategies jointly maximize the accuracy of the automatically produced labels and the prediction accuracy of the machine learning models trained on these labels. Lastly, LANCET uses a distribution matching network to verify whether both the Covariate-shift and Continuity conditions hold, in which case it would be safe to terminate the labeling process. Our experiments on diverse public data sets demonstrate that LANCET consistently outperforms the state-of-the-art methods from Snuba to GOGGLES and other baselines by a large margin - up to 30 percentage points increase in accuracy.

We introduce the Generalized Discrimination Value (GDV) that measures, in a non-invasive manner, how well different data classes separate in each given layer of an artificial neural network. It turns out that, at the end of the training period, the GDV in each given layer L attains a highly reproducible value, irrespective of the initialization of the network’s connection weights. In the case of multi-layer perceptrons trained with error backpropagation, we find that classification of highly complex data sets requires a temporal reduction of class separability, marked by a characteristic ‘energy barrier’ in the initial part of the GDV(L) curve. Even more surprisingly, for a given data set, the GDV(L) is running through a fixed ‘master curve’, independently from the total number of network layers. Finally, due to its invariance with respect to dimensionality, the GDV may serve as a useful tool to compare the internal representational dynamics of artificial neural networks with different architectures for neural architecture search or network compression; or even with brain activity in order to decide between different candidate models of brain function.

Data-driven is the use of data to drive knowledge and decisions. This has the potential to produce better results but can also suboptimal based on a misinterpretation of data, faulty data, and/or missing data. It is therefore interesting to study the problem of uncertainties in the data and how to integrate them into the models in order to improve performances. Principal component analysis (PCA) is one of the most popular data-driven approaches that is not designed to handle the uncertainty of the sensor measurements represented by interval type data. The complete information PCA (CIPCA) is one of the pioneering static approaches in the processing of interval data. Because the process operating conditions and their parameters may vary over time, as well as the oldest data that are not representative of the current process operation, a continuous update of the CIPCA model is necessary. The objective of this paper is to design an adaptive version of the CIPCA methodology for handling complex data, i.e., time-varying and interval data. At each time instant, when a new uncertain measurement of interval type is available, the CIPCA model is recursively updated, where recursive formulas for updating weighted mean and covariance matrix of interval-valued data are proposed. Indeed, the proposed CIPCA approach allows a robust and recursive representation, which improves fault detection and isolation decisions. Based on the updated CIPCA model, the interval $$T^2$$ and SPE statistics are calculated and used to monitor the operating conditions of a process. A wind turbine benchmark is adopted to confirm the effectiveness of the proposed method. The results show that the proposed adaptive CIPCA method for fault detection and isolation in wind turbines is quite promising.

Accurately estimating the interfacial bond capacity of the near-surface mounted (NSM) carbon fiber-reinforced polymer (CFRP) to concrete joint is a fundamental task in the strengthening and retrofit of existing reinforced concrete (RC) structures. The machine learning (ML) approach may provide an alternative to the commonly used semi-empirical or semi-analytical methods. Therefore, in this work we have developed a predictive model based on an artificial neural network (ANN) approach, i.e. using a back propagation neural network (BPNN), to map the complex data pattern obtained from an NSM CFRP to concrete joint. It involves a set of nine material and geometric input parameters and one output value. Moreover, by employing the neural interpretation diagram (NID) technique, the BPNN model becomes interpretable, as the influence of each input variable on the model can be tracked and quantified based on the connection weights of the neural network. An extensive database including 163 pull-out testing samples, collected from the authors’ research group and from published results in the literature, is used to train and verify the ANN. Our results show that the prediction given by the BPNN model agrees well with the experimental data and yields a coefficient of determination of 0.957 on the whole database. After removing one non-significant feature, the BPNN becomes even more computationally efficient and accurate. in addition, compared with the existed semi-analytical model, the ANN-based approach demonstrates a more accurate estimation. Therefore, the proposed ML method may be a promising alternative for predicting the bond strength of NSM CFRP to concrete joint for structural engineers.

The subchannel of a research reactor used to generate high power density is designed to be narrow and rectangular and comprises plate-type fuels operating under downward flow conditions. Critical heat flux (CHF) is a crucial parameter for estimating the safety of a nuclear fuel; hence, this parameter should be accurately predicted. Here, machine learning is applied for the prediction of CHF in a narrow rectangular channel. Although machine learning can effectively analyze large amounts of complex data, its application to CHF, particularly for narrow rectangular channels, remains challenging because of the limited flow conditions available in existing experimental databases. To resolve this problem, we used four CHF correlations to generate pseudo-data for training an artificial neural network. We also propose a network architecture that includes pre-training and prediction stages to predict and analyze the CHF. The trained neural network predicted the CHF with an average error of 3.65% and a root-mean-square error of 17.17% for the test pseudo-data; the respective errors of 0.9% and 26.4% for the experimental data were not considered during training. Finally, machine learning was applied to quantitatively investigate the parametric effect on the CHF in narrow rectangular channels under downward flow conditions.

The material removal rate (MRR) plays a critical role in the chemical mechanical planarization (CMP) process in the semiconductor industry. Many physics-based and data-driven approaches have been proposed to-date to predict the MRR. Nevertheless, most of them neglect the underlying equipment structure containing essential interaction mechanisms among different components. To fill the gap, this paper proposes a novel hypergraph convolution network (HGCN) based approach for predicting MRR in the CMP process. The main contributions include: (1) a generic hypergraph model to represent the interrelationships of complex equipment; and (2) a temporal-based prediction approach to learn the complex data correlation and high-order representation based on the hypergraph. To validate the effectiveness of the proposed approach, a case study is conducted by comparing with other cutting-edge models, of which it outperforms in several metrics. It is envisioned that this research can also bring insightful knowledge to similar scenarios in the manufacturing process.

Large and complex data becomes a valuable resource in biomedical discovery, which is highly facilitated to increase the scientific resources for retrieving the helpful information. However, indexing and retrieving the patient information from the disparate source of big data is challenging in biomedical research. Indexing and retrieving the patient information from big data is performed using the MapReduce framework. In this research, the indexing and retrieval of information are performed using the proposed Jaya-Sine Cosine Algorithm (Jaya–SCA)-based MapReduce framework. Initially, the input big data is forwarded to the mapper randomly. The average of each mapper data is calculated, and these data are forwarded to the reducer, where the representative data are stored. For each user query, the input query is matched with the reducer, and thereby, it switches over to the mapper for retrieving the matched best result. The bilevel matching is performed while retrieving the data from the mapper based on the distance between the query. The similarity measure is computed based on the parametric-enabled similarity measure (PESM), cosine similarity and the proposed Jaya–SCA, which is the integration of the Jaya algorithm and the SCA. Moreover, the proposed Jaya–SCA algorithm attained the maximum value of F-measure, recall and precision of 0.5323, 0.4400 and 0.6867, respectively, using the StatLog Heart Disease dataset.

A machine-based intrusion sensing system has become an essential component in network safeguarding and information security due to the regular development of large volumes of data and the increasing interconnection among global Internet infrastructures. Previous shallow learning and deep learning techniques follow a similar model learning approach to intrusion detection. To explain the increasingly complex data delivery intrusion patterns, a single learning model methodology can face problems. Specifically, a single deep learning model cannot capture special patterns in intrusive attacks with a few experiments. We propose a hierarchical deep learning system based on big data to further boost the efficiency of IDS-based machine learning. It uses behavioral and content-functional functionality to capture both network traffic and content details. Each deep learning model in the proposed framework focuses on learning the particular data distribution in a single cluster. Compared to previous single learning models, this technique improves the classification rate of disruptive attacks. The model building time of the system is reduced considerably when several devices are implemented.

In recent years, with the development of machine learning and big data technology, user data has become an important element in the production process of enterprises. For today’s e-commerce platforms, the deep mining of user’s purchase behavior is helpful to understand user’s purchase preferences and accurately recommend products that meet user expectations, which can not only improve user satisfaction but also reduce platform marketing cost. To accurately identify the user value of online purchasing on an e-commerce platform, this paper uses an improved RFM model to extract user features and uses the K -means++ clustering algorithm to realize user classification. The indicators of the traditional RFM model characterize user features from three angles: recent purchase time ( R ), purchase frequency ( F ), and total consumption amount ( M ). The user group and scenarios studied in this paper are different from the previous literature: (1) the user group is relatively fixed, (2) the consumer goods are relatively single, and (3) the characteristics of repeated purchase are obvious. Therefore, based on the existing literature, this paper extracts the user characteristics studied and improves and models the traditional indicators. Based on the real purchasing data from September to December 2018, it calculates the indicators that improved RFM, empowers the weight to indicators, and finally classifies the value of users by using the K -means++ algorithm. The experimental results show that the user classification based on the improved RFM model is more accurate than the user classification based on the traditional RFM model, and the improved RFM model can identify the user value more accurately, which provides a strong support for the e-commerce platform to realize the accurate marketing strategy based on big data.

In today’s world, the internet and distributed computing make things so convenient that web services can be easily built and fetch from any platform. Web services are loosely coupled, interoperable, and heterogeneous hence they help to connect web applications of different languages. Interoperability is the major factor while transferring web services from one platform to another platform.WS-I basic profile 1.0/1.1 organization provides guidelines to achieve interoperability of web services at a basic level but still, issues arise at a complex level. In this paper, we have discussed and shown the result of two interoperability issues i.e. Date with Null value and Collection of Complex data types of web services using JAVA and.NET environment. In the first issue.NET treat NULL as a value type and JAVA treats NULL as reference type in date-time data type. So, whenever a JAVA client fetches a web service built in.NET it will show a parsable error. In the second issue, a web service data structure contains elements of any type. Whenever, a web service built with the ‘ArrayOfAnyType’ data structure, it can be easily mapped to a.NET client but not to JAVA client. Hence, a data type mismatch issue arises here. To resolve these inter-platform issues, we have used JADE-WSIG as middleware between web services and agent-technology.

With the progress of financial technology (FinTech), real-time information from FinTech is huge and complicated. For various fields of research, identifying intrinsic features of complex data is important, not limited to financial big data. Reviewing previous studies, there are no suitable methods to deal with complex financial data. General methods are traditionally developed from statistics and machine learning. They are usually in some shallow model forms, which cannot fully represent complex, compositional, and hierarchical financial data features. Due to above drawbacks, this study tries to address the problem by advanced deep learning (DL) methods. In DL more layers will increase the power for abstract data representation. Recently, DL has achieved state-of-the-art performance in a wide range of tasks including speech, image, and vision. DL is effective in learning increasingly more abstract representations in a layer-wise manner. That just meets the characteristic of financial data. This study applies newly developed DL networks, the deep canonically correlation analysis and deep canonically correlated autoencoders to perform FinTech data mining. To test the proposed model, this study employed financial statement data regarding many listed high technology companies in Taiwan stock markets. The computation of deep learning is leveraged by multiple graphics processing unit. Our systems and traditional methods are compared by the same data. Empirical results showed that our systems outperform traditional techniques from statistics and machine learning.

Previous computer protection analysis focuses on current data sets that do not have an update and need one-time releases. Serial data publishing on a complex data collection has only a little bit of literature, although it is not completely considered either. They cannot be used against various backgrounds or the usefulness of the publication of serial data is weak. A new generalization hypothesis is developed on the basis of a theoretical analysis, which effectively decreases the risk of re-publication of certain sensitive attributes. The results suggest that our higher anonymity and lower hiding rates were present in our algorithm. Design and Implementation of new proposed privacy preserving technique: In this phase proposed technique is implemented for demonstrating the entire scenario of data aggregation and their privacy preserving data mining. Comparative Production between the proposed technology and the traditional technology for the application of C.45: In this stage, the performance is evaluated and a comparative comparison with the standard algorithm for the proposed data mining security model is presented

Community detection and its variations is one of the typically employed approaches for analyzing graph data originating from various diverse fields. In this paper, we focus on a particular approach for community detection capitalizing on hyperbolic network embedding, which is aimed at analyzing large data graphs. In order to enable its scaling to arbitrary sized data sets and respective data graphs, we extend it by incorporating a graph database approach. This allows for handling a larger number of nodes and edges in the data graph. Also, we turn our focus on the discovery and visualization of communities in Resource Description Framework (RDF) data, namely over linked datasets from diverse areas, explaining how our approach can accommodate relevant analysis. We demonstrate the applicability of the new approach over both real-world and artificially generated datasets showing its feasibility in producing correct results, while being able to scale seamlessly in large datasets. The approach can be used for multi-lateral analysis of feature-rich graph data, originating from diverse sources, enabling the discovery of hidden correlations through the hyperbolic network embedding.

With the rapid development of transportation systems, traffic data have been largely produced in daily lives. Finding the insights of all these complex data is of great significance to vehicle dispatching and public safety. In this work, we propose a multitask deep learning model called Multitask Recurrent Graph Convolutional Network (MRGCN) for accurately predicting traffic flows in the city. Specifically, we design a multitask framework consisting of four components: a region-flow encoder for modeling region-flow dynamics, a transition-flow encoder for exploring transition-flow correlations, a context modeling component for contextualized fusion of two types of traffic flows and a task-specific decoder for predicting traffic flows. Particularly, we introduce Dual-attention Graph Convolutional Gated Recurrent Units (DGCGRU) to simultaneously capture spatial and temporal dependencies, which integrate graph convolution and recurrent model as a whole. Extensive experiments are carried out on two real-world datasets and the results demonstrate that our proposed method outperforms several existing approaches.

Data are vital to collections—from managing their holdings to quantifying their impact. Data visualization can be a powerful asset for collections because it translates complex data into something intuitive. In recent years, data visualization (and dashboards specifically) has gradually been applied in collections for various tasks and audiences. However, this frontier is still new, and there are significant barriers making data visualization inaccessible for most collections, including time and resources, programming experience, training materials, and consistent examples. In this case study, I directly address these barriers by designing a suite of collection management dashboards using the free and user-friendly software Tableau Public and tutorials that explain how to use and customize the them. These resources were finalized with the input from a representative pool of the target audience. Everything is freely available online at http://www.CollVizDashboards.com . As museums continue to evolve, data visualization ought to be accessible to all collections.

Probability distributions are the building blocks of statistical modeling and inference. It is therefore of the utmost importance to know which distribution to use in what circumstances, as wrong choices will inevitably entail a biased analysis. In this article, we focus on circumstances involving complex data and describe the most popular flexible models for these settings. We focus on the following complex data: multivariate skew and heavy-tailed data, circular data, toroidal data, and cylindrical data. We illustrate the strength of flexible models on the basis of concrete examples and discuss major applications and challenges.

Classification and regression trees, as well as their variants, are off-the-shelf methods in Machine Learning. In this paper, we review recent contributions within the Continuous Optimization and the Mixed-Integer Linear Optimization paradigms to develop novel formulations in this research area. We compare those in terms of the nature of the decision variables and the constraints required, as well as the optimization algorithms proposed. We illustrate how these powerful formulations enhance the flexibility of tree models, being better suited to incorporate desirable properties such as cost-sensitivity, explainability, and fairness, and to deal with complex data, such as functional data.

The process of preparing potentially large and complex data sets for further analysis or manual examination is often called data wrangling. In classical warehousing environments, the steps in such a process are carried out using Extract-Transform-Load platforms, with significant manual involvement in specifying, configuring or tuning many of them. In typical big data applications, we need to ensure that all wrangling steps, including web extraction, selection, integration and cleaning, benefit from automation wherever possible. Towards this goal, in the paper we: (i) introduce a notion of data context, which associates portions of a target schema with extensional data of types that are commonly available; (ii) define a scalable methodology to bootstrap an end-to-end data wrangling process based on data profiling; (iii) describe how data context is used to inform automation in several steps within wrangling, specifically, matching, value format transformation, data repair, and mapping generation and selection to optimise the accuracy, consistency and relevance of the result; and (iv) we evaluate the approach with real estate data and financial data, showing substantial improvements in the results of automated wrangling.