Search

In the process industry, it is essential to establish a data-driven soft sensor to predict the key variable that is difficult to online measure directly. The accuracy performance of data-driven soft sensors relies heavily on data. Unfortunately, it is hard to acquire sufficient and informative data from the samples with limited number, which is called as the small sample problem. For handling the small sample problem, it is a good solution to generating virtual samples according to the distribution of original data. This paper proposes an enhanced method of virtual sample generation utilizing manifold features to develop soft sensors using small data. First, T-Distribution Stochastic Neighbor Embedding (t-SNE) is utilized to extract the features of input data. The main idea of generating virtual samples is to use the interpolation algorithm to obtain virtual t-SNE input features and then the random forest algorithm is utilized to get the virtual outputs using virtual t-SNE input features. Finally, virtual samples using the proposed t-SNE based virtual sample generation (t-SNE-VSG) can be achieved. For the sake of confirming the effectiveness and feasibility of the presented t-SNE-VSG, a standard data set is first used. What is more, a small data set from an actual industrial process of Purified Terephthalic Acid is used to establish a soft sensor model. The results from simulations show that the accuracy performance of the soft sensor established with small data can be effectively improved by adding the virtual samples generated by t-SNE-VSG. In addition, t-SNE-VSG achieves superior accuracy to state-of-the-art virtual sample generation methods.

In this present paper, we propose and analyze a C 1 -conforming virtual element method to solve the so-called one-layer stationary quasi-geostrophic equations (QGE) with applications in the large scale wind-driven ocean circulation, formulated in terms of the stream-function. This problem corresponds to a nonlinear fourth order partial differential equation. The C 1 virtual space and the discrete scheme are built in a straightforward way due to the flexibility of the virtual approach. Under the assumption of small data, we prove well-posedness of the discrete problem by using a fixed-point strategy and under standard assumptions on the computational domain, we establish error estimates in H 2 -norm for the stream-function. Finally, we report four numerical experiments that illustrate the behavior of the proposed scheme and confirm our theoretical results on different families of polygonal meshes.

Despite the increased sensor-based data collection in Industry 4.0, the practical use of this data is still in its infancy. In contrast, academic literature provides several approaches to detect machine failures but, in most cases, relies on simulations and vast amounts of training data. Since it is often not practical to collect such amounts of data in an industrial context, we propose an approach to detect the current production mode and machine degradation states on a comparably small data set. Our approach integrates domain knowledge about manufacturing systems into a highly generalizable end-to-end workflow ranging from raw data processing, phase segmentation, data resampling, and feature extraction to machine tool anomaly detection. The workflow applies unsupervised clustering techniques to identify the current production mode and supervised classification models for detecting the present degradation. A resampling strategy and classical machine learning models enable the workflow to handle small data sets and distinguish between normal and abnormal machine tool behavior. To the best of our knowledge, there exists no such end-to-end workflow in the literature that uses the entire machine signal as input to identify anomalies for individual tools. Our evaluation with data from a real multi-purpose machine shows that the proposed workflow detects anomalies with an average F1-score of almost 93%.

An aggregation method is proposed for proliferated small-size data networks (DN) to make a full use of DNs spatial load regulation potentials for demand response (DR). First, each DN is modeled as a virtual power network (VPN), where the Internet data center (IDC) load in each VPN is modeled as a set of linear constraints. Second, the aggregated VPN (AVPN) is proposed to model multiple VPNs to emulate DR in wholesale markets, where each aggregated virtual power flow is much larger than that of each virtual power line in a VPN and independent of DN and VPN structures. The coupling of AVPN and power network is considered to develop a linear load model for the aggregated IDC (AIDC), which is compatible with the models in wholesale markets. Furthermore, AVPN DR resource supply curves are formulated to capture heterogeneous VPN regulation costs and guarantee limited welfare losses in AVPN DR compared with individual VPN DR. Last, an AVPN DR implementation mechanism is deduced to reveal the potentials of the VPN aggregation method in power system applications. Simulation results verify the efficiency of the proposed AVPN and the AIDC load models, and the AVPN DR resource supply curves.

A bike-sharing system is a shared mobility mechanism that provides an alternative transportation mode for short trips with almost no added travel speed loss. However, this model’s low usage ratio and high depreciation rate pose a risk to the sustainable development of the bike-sharing industry. Our study proposes a new integrated station location and rebalancing vehicle service design model. This model aims to maximize daily revenue under a given total investment for station locations and bike acquisition. To address demand ambiguity due to possible bias and loss of data, we present a two-stage robust optimization model with a demand-related uncertainty set. The first stage of our model determines the station locations, initial bike inventory, and service areas of rebalancing vehicles. In contrast to the literature, which either simplifies the rebalancing process as an inventory transshipment problem or formulates it as a complex dynamic bike rebalancing problem, we assign each rebalancing vehicle to a service area composed of several specified stations. An approximate maximum travel distance for each rebalancing vehicle is also designed and constrained to ensure that the rebalancing operation can be performed within each period. In the second stage, our model optimizes the daily fleet operation and maximizes the total revenue minus the rebalancing cost. To solve our model, we design a customized row generation approach. Our numerical studies demonstrate that our algorithm can efficiently obtain exact solutions in small instances. For a real-size problem, the nearly optimal solutions of our model also reveal a high-quality worst-case performance with a small loss in mean performance, particularly when the value of the budget ratio (that is, the average number of bikes per station) is at a medium level. Moreover, the distribution of service areas depends on the bike supply and demand level at each station. The optimal fleet rebalancing operation does not have to be confined to one geographical area. Furthermore, our robust model can achieve larger mean and worst-case revenues and a higher revenue stability than a stochastic model with a small data set.

Representation learning with small labeled data have emerged in many problems, since the success of deep neural networks often relies on the availability of a huge amount of labeled data that is expensive to collect. To address it, many efforts have been made on training sophisticated models with few labeled data in an unsupervised and semi-supervised fashion. In this paper, we will review the recent progresses on these two major categories of methods. A wide spectrum of models will be categorized in a big picture, where we will show how they interplay with each other to motivate explorations of new ideas. We will review the principles of learning the transformation equivariant, disentangled, self-supervised and semi-supervised representations, all of which underpin the foundation of recent progresses. Many implementations of unsupervised and semi-supervised generative models have been developed on the basis of these criteria, greatly expanding the territory of existing autoencoders, generative adversarial nets (GANs) and other deep networks by exploring the distribution of unlabeled data for more powerful representations. We will discuss emerging topics by revealing the intrinsic connections between unsupervised and semi-supervised learning, and propose in future directions to bridge the algorithmic and theoretical gap between transformation equivariance for unsupervised learning and supervised invariance for supervised learning, and unify unsupervised pretraining and supervised finetuning. We will also provide a broader outlook of future directions to unify transformation and instance equivariances for representation learning, connect unsupervised and semi-supervised augmentations, and explore the role of the self-supervised regularization for many learning problems., Comment: published in IEEE Transactions on Pattern Analysis and Machine Intelligence

Occurrences of tropical cyclones at a location are rare, and for many locations, only short periods of observations or hindcasts are available. Hence, estimation of return values (corresponding to a period considerably longer than that for which data are available) for cyclone-induced significant wave height (SWH) from small samples is challenging. The STM-E (space-time maximum and exposure) model was developed to provide reduced bias in estimates of return values compared to competitor approaches in such situations and realistic estimates of return value uncertainty. STM-E exploits data from a spatial neighbourhood satisfying certain conditions, rather than data from a single location, for return value estimation. This article provides critical assessment of the STM-E model for tropical cyclones in the Caribbean Sea near Guadeloupe for which a large database of synthetic cyclones is available, corresponding to more than 3000 years of observation. Results indicate that STM-E yields values for the 500-year return value of SWH and its variability, estimated from 200 years of cyclone data, consistent with direct empirical estimates obtained by sampling 500 years of data from the full synthetic cyclone database; similar results were found for estimation of the 100-year return value from samples corresponding to approximately 50 years of data. In general, STM-E also provides reduced bias and more realistic uncertainty estimates for return values relative to single-location analysis.

This article develops a general yet effective hyperspectral image (HSI) classification framework that is trained with small data. To this end, two identically structured but differently initialized classifiers, which are referred to as two base classifiers, are trained in an iterative manner. Each iteration consists of three steps, that is: 1) the two base classifiers that are trained separately on guide data; 2) unclassified data that are processed by the two trained base classifiers; and 3) the classification results with high confidence that are explored as new guide data. In the first iteration, the guide data comprising the original small training data are the same for the two base classifiers. From the second iteration, the guide data for the two base classifiers start becoming different. Specifically, in each iteration, the guide data for training one base classifier keep being augmented by high confidence classification results provided by the other base classifier. It is in such an iterative manner that the two classifiers continuously provide different new guide data for each other, and thus increasingly augment labeled data from the original small training set to a reasonably larger amount of samples in a HSI. We refer to such a training strategy as mutual guide. We develop a mutual guide implementation scheme by exploiting extreme learning machines (ELMs) as base classifiers. Extensive experiments on four public HSI datasets, i.e., Indian Pines (IP), Kennedy Space Center (KSC), University of Pavia (UP), and Salinas (SA), validate the classification effectiveness of our mutual guide framework with small training data.

Classification is one of the fundamental supervised learning tasks which learns classifiers from the given training data and related labels. The quality of labels is important in classification tasks. However, in many real-world scenarios, data annotation is often corrupted, especially when the annotation process is done by humans. Vaguely labeled data is one of the common problems caused by limited domain knowledge or partial data observation. In this paper, a novel method is proposed to classify vaguely labeled data based on evidential fusion. Vaguely labeled data are divided into several small data groups by the proposed valid label-set cover assignment algorithm. Evidence theory is applied to vaguely labeled data classification by regarding each base classifier on a small data group as one piece of evidence. This gives the chance of classifying unseen precise labeled data from related vague labels. Note that our approach is not restricted to any specific classifiers. It can be generalized to any off-the-shelf classification methods with probabilistic outputs. Finally, experiments are conducted on both synthetic data and real-world data with different base classifiers. Experimental results show that the proposed method achieves superior performance against compared methods.

Stroke has become a leading cause of death and long-term disability in the world, and there is no effective treatment.Deep learning-based approaches have the potential to outperform existing stroke risk prediction models, they rely on large well-labeled data. Due to the strict privacy protection policy in health-care systems, stroke data is usually distributed among different hospitals in small pieces. In addition, the positive and negative instances of such data are extremely imbalanced. Transfer learning solves small data issue by exploiting the knowledge of a correlated domain, especially when multiple source are available.In this work, we propose a novel Hybrid Deep Transfer Learning-based Stroke Risk Prediction (HDTL-SRP) scheme to exploit the knowledge structure from multiple correlated sources (i.e.,external stroke data, chronic diseases data, such as hypertension and diabetes). The proposed framework has been extensively tested in synthetic and real-world scenarios, and it outperforms the state-of-the-art stroke risk prediction models. It also shows the potential of real-world deployment among multiple hospitals aided with 5G/B5G infrastructures. National Key R&D Program of China; National Nature Science Foundation of China; Natural Science Foundation of Guangdong Province; Guangdong “Pearl River Talent Recruitment Program”; Technology Research Project of Shenzhen City; Public Technology Platform of Shenzhen City

This study models the energy consumption of electric vehicles (EVs) under uncertain and small data conditions by combining the machine learning method and the idea of controlled experiments. We propose a Machine Learning-Control Variable model, termed the MLCV model, to estimate the trip energy consumption of EVs. Different data augmentation methods, ensemble methods, sampling factors are adopted as the parameters of the proposed method. Through parameter search, the accuracy of the base learner can be further improved. Our method utilizes real driving behaviours that are generated by real drivers and collected in a complex urban environment, making the approach generalizable. The experimental results demonstrate that the proposed MLCV model is superior to existing machine learning models in terms of estimation accuracy.

We propose a few-shot learning method for spatial regression. Although Gaussian processes (GPs), or kriging, have been successfully used for spatial regression, they require many observations in the target task to achieve a high predictive performance. Our model is trained using spatial datasets on various attributes in various regions, and predicts values on unseen attributes in unseen regions given a few observed data. With our model, a task representation is inferred from given small data using a neural network. Then, spatial values are predicted by neural networks with a GP framework, in which task-specific properties are controlled by the task representations. The GP framework allows us to analytically obtain predictions that are adapted to small data. By using the adapted predictions in the objective function, we can train our model efficiently and effectively so that the test predictive performance improves when adapted to newly given small data. In our experiments, we demonstrate that the proposed method achieves better predictive performance than existing meta-learning methods using spatial datasets.

Medical image segmentation often suffers from the challenges of class imbalance, blurred target boundaries, and small data. How to establish a framework to automatically segment medical images with these problems is an important task. Although there have been some studies on the issue, there is still a large room for improving the efficiency and quality of medical service. This paper utilizes the powerful ability of deep learning to extract features, and develops a two-stage decoding network with boundary attention (TSD-BA), which can locate the regions of interest in the target locating stage and obtain more spatial structure features in the detail refinement stage. Specifically, a deep fusion model (DFM) is used to aggregate high-level semantic features for accurately capturing the position of targets. Subsequently, a boundary attention module (BAM) is applied to further excavate the boundary features. Moreover, data augmentation and transfer learning are employed to avoid overfitting caused by small datasets. Finally, a pixel position aware (PPA) loss is introduced to focus on hard pixels and mitigate the class imbalance issues. Numerous experimental results indicate that the proposed TSD-BA achieves the best performance compared with state-of-the-art approaches.

Predictive modeling of new biochemical systems with small data is a great challenge. To fill this gap, transfer learning, a subdomain of machine learning that serves to transfer knowledge from a generalized model to a more domain-specific model, provides a promising solution. While transfer learning has been used in natural language processing, image analysis, and chemical engineering fault detection, its application within biochemical engineering has not been systematically explored. In this study, we demonstrated the benefits of transfer learning when applied to predict dynamic behaviors of new biochemical processes. Two different case studies were presented to investigate the accuracy, reliability, and advantage of this innovative modeling approach. We thoroughly discussed the different transfer learning strategies and the effects of topology on transfer learning, comparing the performance of the transfer learning models against benchmark kinetic and data-driven models. Furthermore, strong connections between the underlying process mechanism and the transfer learning model's optimal structure were highlighted, suggesting the interpretability of transfer learning to enable more accurate prediction than a naive data-driven modeling approach. Therefore, this study shows a novel approach to effectively combining data from different resources for bioprocess simulation.

This paper focuses on multi-label learning from small number of labelled data. We demonstrate that the straightforward binary-relevance extension of the interpolated label propagation algorithm, the harmonic function, is a competitive learning method with respect to many widely-used evaluation measures. This is achieved mainly by a new transition matrix that better captures the underlying manifold structure. Furthermore, we show that when there exists label dependence, we can use the outputs of a competitive learning method as part of the input to the harmonic function to provide improved results over those of the original model. Finally, since we are using multiple metrics to thoroughly evaluate the performance of the algorithm, we propose to use the game-theory based method of Kalai and Smorodinsky to output a single compromise solution. This method can be applied to any learning model irrespective of the number of evaluation measures used.

The Big Data phenomenon has imposed maturity on companies regarding the exploration of their data, as a prerogative to obtain valuable insights into their clients and the power of analysis to guide decision-making processes. Therefore, a general approach that describes how to extract knowledge for the execution of the business strategy needs to be established. The purpose of this research paper is to introduce and evaluate the implementation of a process for the experimental development of Data Mining (DM), AI and Data Science applications aligned with the strategic planning. A case study with the proposed process was conducted in a federal educational institution. The results generated evidence showing that it is possible to integrate a strategic alignment approach, an experimental method, and a methodology for the development of DM applications. Data Mining (DM) and Data Science (DS) applications also present the risks of other Information Systems, and the adoption of strategy-driven and scientific method processes are critical success factors. Moreover, it was possible to conclude that the application of the scientific method was facilitated, besides being an important tool to ensure the quality, reproducibility and transparency of intelligent applications. In conclusion, the process needs to be mapped to foment and guide the strategic alignment.

Application of data-scientific approach to conventional sciences, such as chemo-informatics, bio-informatics, and materials informatics (MI), has attracted much interest toward data-driven research...

Predicting materials properties from composition or structure is of great interest to the materials science community. Deep learning has recently garnered considerable interest in materials predictive tasks with low model errors when dealing with large materials data. However, deep learning models suffer in the small data regime that is common in materials science. Here we leverage the transfer learning concept and the graph network deep learning framework and develop the AtomSets machine learning framework for consistent high model accuracy at both small and large materials data. The AtomSets models can work with both compositional and structural materials data. By combining with transfer learned features from graph networks, they can achieve state-of-the-art accuracy from using small compositional data (130,000). The AtomSets models show much lower errors than the state-of-the-art graph network models at small data limits and the classical machine learning models at large data limits. They also transfer better in the simulated materials discovery process where the targeted materials have property values out of the training data limits. The models require minimal domain knowledge inputs and are free from feature engineering. The presented AtomSets model framework opens new routes for machine learning-assisted materials design and discovery.

Data centers have been highlighted as a major energy consumer and there has been an increasing trend towards the consolidation of smaller data centers into larger facilities. Yet, small data centers exist for a variety of reasons and account for a significant portion of the total number of servers in the US. Frequent refreshes of IT hardware has emerged as a trend in hyper-scale data centers but little attention has been paid to how these savings can be achieved in small data centers. This work provides a comprehensive framework for the energy saving opportunities, while determining when a return on investment can be achieved to enable small data center operators to create credible business cases for hardware refreshes. Various data center deployment scenarios are used as case studies (based on real-life datasets) to validate the proposed concepts.

We would like to study a weakly coupled system of semi-linear classical damped wave equations with moduli of continuity in nonlinearities whose powers belong to the critical curve in the p − q plane. The main goal of this paper is to find out sharp conditions of these moduli of continuity which classify between global (in time) existence of small data solutions and finite time blow-up of (even) small data solutions.

We present an efficient coreset construction algorithm for large-scale Support Vector Machine (SVM) training in Big Data and streaming applications. A coreset is a small, representative subset of the original data points such that a model trained on the coreset is provably competitive with that trained on the original data set. Since the size of the coreset is generally much smaller than the original set, our preprocess-then-train scheme has potential to lead to significant speedups when training SVM models. We prove lower and upper bounds on the size of the coreset required to obtain small data summaries for the SVM problem. As a corollary, we show that our algorithm can be used to extend the applicability of any off-the-shelf SVM solver to streaming, distributed, and dynamic data settings. We evaluate the performance of our algorithm on real-world and synthetic data sets. Our experimental results reaffirm the favorable theoretical properties of our algorithm and demonstrate its practical effectiveness in accelerating SVM training.

In this paper several statistical learning algorithms are used to predict the maximal length of fatigue cracks based on a sample composed of 31 observations. The small-data regime is still a problem for many professionals, especially in the areas where failures occur rarely. The analyzed object is a high-pressure Nozzle of a heavy-duty gas turbine. Operating parameters of the engines are used for the regression analysis. The following algorithms are used in this work: multiple linear and polynomial regression, random forest, kernel-based methods, AdaBoost and extreme gradient boosting and artificial neural networks. A substantial part of the paper provides advice on the effective selection of features. The paper explains how to process the dataset in order to reduce uncertainty; thus, simplifying the analysis of the results. The proposed loss and cost functions are custom and promote solutions accurately predicting the longest cracks. The obtained results confirm that some of the algorithms can accurately predict maximal lengths of the fatigue cracks, even if the sample is small.

Today’s classification task is getting more and more complex. This inevitably renders unanticipated compromises on the quality of data labels. In this paper, we consider learning label noise robust classifiers with focus on the tasks with limited training examples relative to the number of data classes and data dimensionality. In such cases, the existing label noise models tend to inaccurately estimate the noise proportions leading to suboptimal performance. To alleviate the problem, we formulated a regularised label noise model capable of expressing preference on the noise parameters. In addition, we treated the regularisation from a Bayesian perspective so that the regularisation parameters can be inferred from the data through the noise model, thereby facilitating model selection in the presence of label noise. This results in a more data and computationally efficient Bayesian label noise model which could be incorporated into any probabilistic classifier, including those that are known to be data intensive such as deep neural networks. We demonstrated the generality of the proposed method through its integrations with logistic regression, multinomial logistic regression and convolutional neural networks. Extensive empirical evaluations demonstrate that the proposed regularised label noise model can significantly improve, in terms of both the quality of noise parameters estimation and the classification accuracy, upon the existing ones when data is scarce, and is no worse than the existing approaches in the abundance of training data.

In the farmland Internet of Things, to achieve precise control of production, it is necessary to obtain more data support, which requires the deployment of many sensors, and this will inevitably bring about high investment and high-cost problems. This paper mainly studies the optimization of sensor placement in the agricultural field. Through compressed sensing and algorithm optimization, the number of sensors used is reduced and the cost is reduced on the premise of ensuring the effect. At present, there are many mature sensor layout optimization methods, but these methods will have incomplete parameters due to experimental conditions and environmental factors. They are more suitable for structural health monitoring and lack research in agricultural applications. Considering that the sensor layout optimization can be converted into the characteristics of image compression selection and the compression effect of the compressed sensing theory is better, therefore, this paper proposes a sensor layout optimization method based on compressed sensing. Due to the structural characteristics of the existing measurement matrix in the compressed sensing theory, the specific position distribution of the optimized sensor layout cannot be obtained directly. This paper improves the existing sparse random measurement matrix to determine the number of sensors required for a given region and the function of the specific location of each sensor. The experimental results show that soil moisture can be measured with a small error of 0.91 by using 1/3 of the original sensor number. The result of data reconstruction using 1/6 of the original sensor is average, and the average error is up to 1.68, which is suitable for the environment with small data fluctuation.