Search

In practical mechanical fault detection and diagnosis, it is difficult and expensive to collect enough large-scale supervised data to train deep networks. Transfer learning can reuse the knowledge obtained from the source task to improve the performance of the target task, which performs well on small data and reduces the demand for high computation power. However, the detection performance is significantly reduced by the direct transfer due to the domain difference. Domain adaptation (DA) can transfer the distribution information from the source domain to the target domain and solve a series of problems caused by the distribution difference of data. In this survey, we review various current DA strategies combined with deep learning (DL) and analyze the principles, advantages, and disadvantages of each method. We also summarize the application of DA combined with DL in the field of fault diagnosis. This paper provides a summary of the research results and proposes future work based on analysis of the key technologies.

Many deep-learning methods have been developed for fault diagnosis. However, due to the difficulty of collecting and labeling machine fault data, the datasets in some practical applications are relatively much smaller than the other big data benchmarks. In addition, the fault data come from different machines. Therefore, on some occasions, fault diagnosis is a multidomain problem with small data, where satisfactory transfer performance is difficult to obtain and has been rarely explored from the few-shot learning viewpoint. Different from the existing deep transfer learning solutions, a novel transfer relation network (TRN), combining a few-shot learning mechanism and transfer learning, is developed in this study. Specifically, the fault diagnosis problem has been treated as a similarity metric-learning problem instead of solely feature weighted classification. A feature net and a relation net have been, respectively, constructed for feature extraction and relation computation. The Siamese structure has been borrowed to extract the features of the source and the target domain samples with shared weights. Multikernel maximum mean discrepancy (MK-MMD) is employed on several higher layers with different tradeoff parameters to enable an efficient domain feature transfer considering different feature properties. To implement efficient diagnosis based on small data, an episode-based few-shot training strategy is adopted to train TRN. Average pooling has been adopted to suppress the noise influence from the vibration sequence which turns out to be important for the success of time sequence-based fault diagnosis. Transfer experiments on four datasets have verified the superior performance of TRN. A significant improvement of classification accuracy has been made compared with the state-of-the-art methods on the adopted datasets.

In the process industry, it is essential to establish a data-driven soft sensor to predict the key variable that is difficult to online measure directly. The accuracy performance of data-driven soft sensors relies heavily on data. Unfortunately, it is hard to acquire sufficient and informative data from the samples with limited number, which is called as the small sample problem. For handling the small sample problem, it is a good solution to generating virtual samples according to the distribution of original data. This paper proposes an enhanced method of virtual sample generation utilizing manifold features to develop soft sensors using small data. First, T-Distribution Stochastic Neighbor Embedding (t-SNE) is utilized to extract the features of input data. The main idea of generating virtual samples is to use the interpolation algorithm to obtain virtual t-SNE input features and then the random forest algorithm is utilized to get the virtual outputs using virtual t-SNE input features. Finally, virtual samples using the proposed t-SNE based virtual sample generation (t-SNE-VSG) can be achieved. For the sake of confirming the effectiveness and feasibility of the presented t-SNE-VSG, a standard data set is first used. What is more, a small data set from an actual industrial process of Purified Terephthalic Acid is used to establish a soft sensor model. The results from simulations show that the accuracy performance of the soft sensor established with small data can be effectively improved by adding the virtual samples generated by t-SNE-VSG. In addition, t-SNE-VSG achieves superior accuracy to state-of-the-art virtual sample generation methods.

Despite the increased sensor-based data collection in Industry 4.0, the practical use of this data is still in its infancy. In contrast, academic literature provides several approaches to detect machine failures but, in most cases, relies on simulations and vast amounts of training data. Since it is often not practical to collect such amounts of data in an industrial context, we propose an approach to detect the current production mode and machine degradation states on a comparably small data set. Our approach integrates domain knowledge about manufacturing systems into a highly generalizable end-to-end workflow ranging from raw data processing, phase segmentation, data resampling, and feature extraction to machine tool anomaly detection. The workflow applies unsupervised clustering techniques to identify the current production mode and supervised classification models for detecting the present degradation. A resampling strategy and classical machine learning models enable the workflow to handle small data sets and distinguish between normal and abnormal machine tool behavior. To the best of our knowledge, there exists no such end-to-end workflow in the literature that uses the entire machine signal as input to identify anomalies for individual tools. Our evaluation with data from a real multi-purpose machine shows that the proposed workflow detects anomalies with an average F1-score of almost 93%.

An aggregation method is proposed for proliferated small-size data networks (DN) to make a full use of DNs spatial load regulation potentials for demand response (DR). First, each DN is modeled as a virtual power network (VPN), where the Internet data center (IDC) load in each VPN is modeled as a set of linear constraints. Second, the aggregated VPN (AVPN) is proposed to model multiple VPNs to emulate DR in wholesale markets, where each aggregated virtual power flow is much larger than that of each virtual power line in a VPN and independent of DN and VPN structures. The coupling of AVPN and power network is considered to develop a linear load model for the aggregated IDC (AIDC), which is compatible with the models in wholesale markets. Furthermore, AVPN DR resource supply curves are formulated to capture heterogeneous VPN regulation costs and guarantee limited welfare losses in AVPN DR compared with individual VPN DR. Last, an AVPN DR implementation mechanism is deduced to reveal the potentials of the VPN aggregation method in power system applications. Simulation results verify the efficiency of the proposed AVPN and the AIDC load models, and the AVPN DR resource supply curves.

A bike-sharing system is a shared mobility mechanism that provides an alternative transportation mode for short trips with almost no added travel speed loss. However, this model’s low usage ratio and high depreciation rate pose a risk to the sustainable development of the bike-sharing industry. Our study proposes a new integrated station location and rebalancing vehicle service design model. This model aims to maximize daily revenue under a given total investment for station locations and bike acquisition. To address demand ambiguity due to possible bias and loss of data, we present a two-stage robust optimization model with a demand-related uncertainty set. The first stage of our model determines the station locations, initial bike inventory, and service areas of rebalancing vehicles. In contrast to the literature, which either simplifies the rebalancing process as an inventory transshipment problem or formulates it as a complex dynamic bike rebalancing problem, we assign each rebalancing vehicle to a service area composed of several specified stations. An approximate maximum travel distance for each rebalancing vehicle is also designed and constrained to ensure that the rebalancing operation can be performed within each period. In the second stage, our model optimizes the daily fleet operation and maximizes the total revenue minus the rebalancing cost. To solve our model, we design a customized row generation approach. Our numerical studies demonstrate that our algorithm can efficiently obtain exact solutions in small instances. For a real-size problem, the nearly optimal solutions of our model also reveal a high-quality worst-case performance with a small loss in mean performance, particularly when the value of the budget ratio (that is, the average number of bikes per station) is at a medium level. Moreover, the distribution of service areas depends on the bike supply and demand level at each station. The optimal fleet rebalancing operation does not have to be confined to one geographical area. Furthermore, our robust model can achieve larger mean and worst-case revenues and a higher revenue stability than a stochastic model with a small data set.

Representation learning with small labeled data have emerged in many problems, since the success of deep neural networks often relies on the availability of a huge amount of labeled data that is expensive to collect. To address it, many efforts have been made on training sophisticated models with few labeled data in an unsupervised and semi-supervised fashion. In this paper, we will review the recent progresses on these two major categories of methods. A wide spectrum of models will be categorized in a big picture, where we will show how they interplay with each other to motivate explorations of new ideas. We will review the principles of learning the transformation equivariant, disentangled, self-supervised and semi-supervised representations, all of which underpin the foundation of recent progresses. Many implementations of unsupervised and semi-supervised generative models have been developed on the basis of these criteria, greatly expanding the territory of existing autoencoders, generative adversarial nets (GANs) and other deep networks by exploring the distribution of unlabeled data for more powerful representations. We will discuss emerging topics by revealing the intrinsic connections between unsupervised and semi-supervised learning, and propose in future directions to bridge the algorithmic and theoretical gap between transformation equivariance for unsupervised learning and supervised invariance for supervised learning, and unify unsupervised pretraining and supervised finetuning. We will also provide a broader outlook of future directions to unify transformation and instance equivariances for representation learning, connect unsupervised and semi-supervised augmentations, and explore the role of the self-supervised regularization for many learning problems., Comment: published in IEEE Transactions on Pattern Analysis and Machine Intelligence

This article develops a general yet effective hyperspectral image (HSI) classification framework that is trained with small data. To this end, two identically structured but differently initialized classifiers, which are referred to as two base classifiers, are trained in an iterative manner. Each iteration consists of three steps, that is: 1) the two base classifiers that are trained separately on guide data; 2) unclassified data that are processed by the two trained base classifiers; and 3) the classification results with high confidence that are explored as new guide data. In the first iteration, the guide data comprising the original small training data are the same for the two base classifiers. From the second iteration, the guide data for the two base classifiers start becoming different. Specifically, in each iteration, the guide data for training one base classifier keep being augmented by high confidence classification results provided by the other base classifier. It is in such an iterative manner that the two classifiers continuously provide different new guide data for each other, and thus increasingly augment labeled data from the original small training set to a reasonably larger amount of samples in a HSI. We refer to such a training strategy as mutual guide. We develop a mutual guide implementation scheme by exploiting extreme learning machines (ELMs) as base classifiers. Extensive experiments on four public HSI datasets, i.e., Indian Pines (IP), Kennedy Space Center (KSC), University of Pavia (UP), and Salinas (SA), validate the classification effectiveness of our mutual guide framework with small training data.

Classification is one of the fundamental supervised learning tasks which learns classifiers from the given training data and related labels. The quality of labels is important in classification tasks. However, in many real-world scenarios, data annotation is often corrupted, especially when the annotation process is done by humans. Vaguely labeled data is one of the common problems caused by limited domain knowledge or partial data observation. In this paper, a novel method is proposed to classify vaguely labeled data based on evidential fusion. Vaguely labeled data are divided into several small data groups by the proposed valid label-set cover assignment algorithm. Evidence theory is applied to vaguely labeled data classification by regarding each base classifier on a small data group as one piece of evidence. This gives the chance of classifying unseen precise labeled data from related vague labels. Note that our approach is not restricted to any specific classifiers. It can be generalized to any off-the-shelf classification methods with probabilistic outputs. Finally, experiments are conducted on both synthetic data and real-world data with different base classifiers. Experimental results show that the proposed method achieves superior performance against compared methods.

Stroke has become a leading cause of death and long-term disability in the world, and there is no effective treatment.Deep learning-based approaches have the potential to outperform existing stroke risk prediction models, they rely on large well-labeled data. Due to the strict privacy protection policy in health-care systems, stroke data is usually distributed among different hospitals in small pieces. In addition, the positive and negative instances of such data are extremely imbalanced. Transfer learning solves small data issue by exploiting the knowledge of a correlated domain, especially when multiple source are available.In this work, we propose a novel Hybrid Deep Transfer Learning-based Stroke Risk Prediction (HDTL-SRP) scheme to exploit the knowledge structure from multiple correlated sources (i.e.,external stroke data, chronic diseases data, such as hypertension and diabetes). The proposed framework has been extensively tested in synthetic and real-world scenarios, and it outperforms the state-of-the-art stroke risk prediction models. It also shows the potential of real-world deployment among multiple hospitals aided with 5G/B5G infrastructures. National Key R&D Program of China; National Nature Science Foundation of China; Natural Science Foundation of Guangdong Province; Guangdong “Pearl River Talent Recruitment Program”; Technology Research Project of Shenzhen City; Public Technology Platform of Shenzhen City

This study models the energy consumption of electric vehicles (EVs) under uncertain and small data conditions by combining the machine learning method and the idea of controlled experiments. We propose a Machine Learning-Control Variable model, termed the MLCV model, to estimate the trip energy consumption of EVs. Different data augmentation methods, ensemble methods, sampling factors are adopted as the parameters of the proposed method. Through parameter search, the accuracy of the base learner can be further improved. Our method utilizes real driving behaviours that are generated by real drivers and collected in a complex urban environment, making the approach generalizable. The experimental results demonstrate that the proposed MLCV model is superior to existing machine learning models in terms of estimation accuracy.

We propose a few-shot learning method for spatial regression. Although Gaussian processes (GPs), or kriging, have been successfully used for spatial regression, they require many observations in the target task to achieve a high predictive performance. Our model is trained using spatial datasets on various attributes in various regions, and predicts values on unseen attributes in unseen regions given a few observed data. With our model, a task representation is inferred from given small data using a neural network. Then, spatial values are predicted by neural networks with a GP framework, in which task-specific properties are controlled by the task representations. The GP framework allows us to analytically obtain predictions that are adapted to small data. By using the adapted predictions in the objective function, we can train our model efficiently and effectively so that the test predictive performance improves when adapted to newly given small data. In our experiments, we demonstrate that the proposed method achieves better predictive performance than existing meta-learning methods using spatial datasets.

Medical image segmentation often suffers from the challenges of class imbalance, blurred target boundaries, and small data. How to establish a framework to automatically segment medical images with these problems is an important task. Although there have been some studies on the issue, there is still a large room for improving the efficiency and quality of medical service. This paper utilizes the powerful ability of deep learning to extract features, and develops a two-stage decoding network with boundary attention (TSD-BA), which can locate the regions of interest in the target locating stage and obtain more spatial structure features in the detail refinement stage. Specifically, a deep fusion model (DFM) is used to aggregate high-level semantic features for accurately capturing the position of targets. Subsequently, a boundary attention module (BAM) is applied to further excavate the boundary features. Moreover, data augmentation and transfer learning are employed to avoid overfitting caused by small datasets. Finally, a pixel position aware (PPA) loss is introduced to focus on hard pixels and mitigate the class imbalance issues. Numerous experimental results indicate that the proposed TSD-BA achieves the best performance compared with state-of-the-art approaches.

Predictive modeling of new biochemical systems with small data is a great challenge. To fill this gap, transfer learning, a subdomain of machine learning that serves to transfer knowledge from a generalized model to a more domain-specific model, provides a promising solution. While transfer learning has been used in natural language processing, image analysis, and chemical engineering fault detection, its application within biochemical engineering has not been systematically explored. In this study, we demonstrated the benefits of transfer learning when applied to predict dynamic behaviors of new biochemical processes. Two different case studies were presented to investigate the accuracy, reliability, and advantage of this innovative modeling approach. We thoroughly discussed the different transfer learning strategies and the effects of topology on transfer learning, comparing the performance of the transfer learning models against benchmark kinetic and data-driven models. Furthermore, strong connections between the underlying process mechanism and the transfer learning model's optimal structure were highlighted, suggesting the interpretability of transfer learning to enable more accurate prediction than a naive data-driven modeling approach. Therefore, this study shows a novel approach to effectively combining data from different resources for bioprocess simulation.

Data centers have been highlighted as a major energy consumer and there has been an increasing trend towards the consolidation of smaller data centers into larger facilities. Yet, small data centers exist for a variety of reasons and account for a significant portion of the total number of servers in the US. Frequent refreshes of IT hardware has emerged as a trend in hyper-scale data centers but little attention has been paid to how these savings can be achieved in small data centers. This work provides a comprehensive framework for the energy saving opportunities, while determining when a return on investment can be achieved to enable small data center operators to create credible business cases for hardware refreshes. Various data center deployment scenarios are used as case studies (based on real-life datasets) to validate the proposed concepts.

We present an efficient coreset construction algorithm for large-scale Support Vector Machine (SVM) training in Big Data and streaming applications. A coreset is a small, representative subset of the original data points such that a model trained on the coreset is provably competitive with that trained on the original data set. Since the size of the coreset is generally much smaller than the original set, our preprocess-then-train scheme has potential to lead to significant speedups when training SVM models. We prove lower and upper bounds on the size of the coreset required to obtain small data summaries for the SVM problem. As a corollary, we show that our algorithm can be used to extend the applicability of any off-the-shelf SVM solver to streaming, distributed, and dynamic data settings. We evaluate the performance of our algorithm on real-world and synthetic data sets. Our experimental results reaffirm the favorable theoretical properties of our algorithm and demonstrate its practical effectiveness in accelerating SVM training.

In this paper several statistical learning algorithms are used to predict the maximal length of fatigue cracks based on a sample composed of 31 observations. The small-data regime is still a problem for many professionals, especially in the areas where failures occur rarely. The analyzed object is a high-pressure Nozzle of a heavy-duty gas turbine. Operating parameters of the engines are used for the regression analysis. The following algorithms are used in this work: multiple linear and polynomial regression, random forest, kernel-based methods, AdaBoost and extreme gradient boosting and artificial neural networks. A substantial part of the paper provides advice on the effective selection of features. The paper explains how to process the dataset in order to reduce uncertainty; thus, simplifying the analysis of the results. The proposed loss and cost functions are custom and promote solutions accurately predicting the longest cracks. The obtained results confirm that some of the algorithms can accurately predict maximal lengths of the fatigue cracks, even if the sample is small.

Today’s classification task is getting more and more complex. This inevitably renders unanticipated compromises on the quality of data labels. In this paper, we consider learning label noise robust classifiers with focus on the tasks with limited training examples relative to the number of data classes and data dimensionality. In such cases, the existing label noise models tend to inaccurately estimate the noise proportions leading to suboptimal performance. To alleviate the problem, we formulated a regularised label noise model capable of expressing preference on the noise parameters. In addition, we treated the regularisation from a Bayesian perspective so that the regularisation parameters can be inferred from the data through the noise model, thereby facilitating model selection in the presence of label noise. This results in a more data and computationally efficient Bayesian label noise model which could be incorporated into any probabilistic classifier, including those that are known to be data intensive such as deep neural networks. We demonstrated the generality of the proposed method through its integrations with logistic regression, multinomial logistic regression and convolutional neural networks. Extensive empirical evaluations demonstrate that the proposed regularised label noise model can significantly improve, in terms of both the quality of noise parameters estimation and the classification accuracy, upon the existing ones when data is scarce, and is no worse than the existing approaches in the abundance of training data.

In the farmland Internet of Things, to achieve precise control of production, it is necessary to obtain more data support, which requires the deployment of many sensors, and this will inevitably bring about high investment and high-cost problems. This paper mainly studies the optimization of sensor placement in the agricultural field. Through compressed sensing and algorithm optimization, the number of sensors used is reduced and the cost is reduced on the premise of ensuring the effect. At present, there are many mature sensor layout optimization methods, but these methods will have incomplete parameters due to experimental conditions and environmental factors. They are more suitable for structural health monitoring and lack research in agricultural applications. Considering that the sensor layout optimization can be converted into the characteristics of image compression selection and the compression effect of the compressed sensing theory is better, therefore, this paper proposes a sensor layout optimization method based on compressed sensing. Due to the structural characteristics of the existing measurement matrix in the compressed sensing theory, the specific position distribution of the optimized sensor layout cannot be obtained directly. This paper improves the existing sparse random measurement matrix to determine the number of sensors required for a given region and the function of the specific location of each sensor. The experimental results show that soil moisture can be measured with a small error of 0.91 by using 1/3 of the original sensor number. The result of data reconstruction using 1/6 of the original sensor is average, and the average error is up to 1.68, which is suitable for the environment with small data fluctuation.

Microexpression can manifest the real mood of humans, which has been widely concerned in clinical diagnosis and depression analysis. To solve the problem of missing discriminative spatiotemporal features in a small data set caused by the short duration and subtle movement changes of microexpression, we present a dual-stream spatiotemporal attention network (DSTAN) that integrates dual-stream spatiotemporal network and attention mechanism to capture the deformation features and spatiotemporal features of microexpression in the case of small samples. The Spatiotemporal networks in DSTAN are based on two lightweight networks, namely, the spatiotemporal appearance network (STAN) learning the appearance features from the microexpression sequences and the spatiotemporal motion network (STMN) learning the motion features from optical flow sequences. To focus on the discriminative motion areas of microexpression, we construct a novel attention mechanism for the spatial model of STAN and STMN, including a multiscale kernel spatial attention mechanism and global dual-pool channel attention mechanism. To obtain the importance of each frame in the microexpression sequence, we design a temporal attention mechanism for the temporal model of STAN and STMN to form spatiotemporal appearance network-attention (STAN-A) and spatiotemporal motion network-attention (STMN-A), which can adaptively perform dynamic feature refinement. Finally, the feature concatenate-SVM method is used to integrate STAN-A and STMN-A to a novel network, DSTAN. The extensive experiments on three small spontaneous microexpression data sets of SMIC, CASME, and CASME II demonstrate the proposed DSTAN can effectively cope with the recognition of microexpressions.

As a high-efficiency and high-quality welding process, hybrid laser-MAG welding (HLMW) has significant potential of application in welding thick plates. In the present study, based on thermal elastic–plastic theory, a three-dimensional finite element model is developed to predict the weld shape characteristics, residual stress and distortion in HLMW for a butt joint of 12-mm-thick high strength steel plate. Metal active gas arc welding (MAG) heat input and laser energy are modeled as one double-ellipsoid body heat source and one cone body heat source with enhanced peak density along the central axis, respectively. The comparison between calculated molten pool shapes and those obtained by the experiment shows a good agreement. Then weld shape characteristic, residual stresses and distortions are calculated in four different welding process parameters. The results show that the increase of laser power and current can effectively increase the weld penetration width when the welding speed is fixed. At the top surfaces of weldment, the peak stress of high laser beam power is more significant than that of low laser beam power. A high compressive transverse stress of low laser beam power can be found at the welding zone and the surrounding heat affected zone. However, at the bottom surfaces of weldment, the peak stress of low laser beam power is larger than that of high laser beam power. The peak stress of low laser beam power is much larger than that of high laser beam power. A high compressive transverse stress of low laser beam power can be found at the welding zone. The vertical deformation in low laser beam power and low arc current welding has the lowest value.

Image transfer based on deep learning methods can achieve good results in face illumination processing. However, data constraints and generalization ability restrict the further development of these methods in this field. In this paper, we propose small data assisting face image illumination normalization. For data constraints, we train the network model on a small number of image pairs. In terms of generalization ability, the proposed normalization network parameters are different for processing different face images. Small data learning can provide prior knowledge, and the reconstruction process can guide detail generation. Therefore, the small data learning network and the reconstruction network are complementary to each other in image generating mode when only a small quantity of data is available. We use this network mode to normalize the illumination and reconstruct the super-resolution of face images. After illumination normalization, super-resolution reconstruction can obtain more precise face information and further improve the face recognition rate. Experiments show that the proposed method has good normalization performance when only 500 face image pairs are used for training.

With the development of smart grid information physical systems, some of the data processing functions gradually approach the edge layer of end-users. To better realize the energy theft detection function at the edge, we proposed an energy theft detection method based on the power consumption information acquisition system of power enterprises. The method involves the following steps. In the centralized data center, K-means is used to decompose a large amount of data into small data and then input and train neural network parameters to realize feature extraction. We design a neural network named DWMCNN, which can extract features from the day, week, and month and can extract more accurate features. In the edge data center, the random forest (RF) algorithm is used to classify the extracted features. The experimental results show that the clustering method accords with the idea of edge computing-distributed processing and improves the operation speed and that the feature extractor has good convergence performance. In addition, compared with the methods based on various classifiers, this method has higher accuracy and lower computational complexity, which is suitable for the deployment of edge data centers.

Machine learning (ML) is quickly becoming a premier tool for modeling chemical processes and materials. ML-based force fields, trained on large data sets of high-quality electron structure calculations, are particularly attractive due their unique combination of computational efficiency and physical accuracy. This Perspective summarizes some recent advances in the development of neural network-based interatomic potentials. Designing high-quality training data sets is crucial to overall model accuracy. One strategy is active learning, in which new data are automatically collected for atomic configurations that produce large ML uncertainties. Another strategy is to use the highest levels of quantum theory possible. Transfer learning allows training to a data set of mixed fidelity. A model initially trained to a large data set of density functional theory calculations can be significantly improved by retraining to a relatively small data set of expensive coupled cluster theory calculations. These advances are exemplified by applications to molecules and materials.

Streaming temporal data contains time stamps and values, challenging to quantify relationships of time stamps and corresponding values. Moreover, the characteristics and relationships of streaming temporal data are not invariable. Thus, it is impossible to analyse all data by a trained model at the beginning of data streams. Practically, the trained model to analyse streaming temporal data should change according to the increasing volume of data. Association rule mining, on the other hand, can find potential relationships from given data. This paper proposes an association rule mining method for streaming temporal data to discover potential relationships from streaming temporal data. Our experiments verify our proposed method. A public data set is applied to compare the performance of the proposed method and its counterpart. A small data set is also applied for two case studies to further illustrate our proposed method mine association rules with streaming temporal data with time stamps and corresponding values.

On farm experimentation (OFE) has been a long-standing method for farmers to assess alternative management at scales relevant to their farming practices. Through the use of spatially distributed designs, whether simple strips or other ‘whole-of-block’ trials, OFE can provide information such as which treatment should be recommended at specific locations, and make important contributions to precision agriculture. However, when treatment response data sets become large, such as with tens of thousands of field observations that are readily collected using on-the-go sensors, existing geostatistical systems for analysing such experiments become computationally intensive, if not impossible. To enable farmers, or their consultants, to generate high-resolution treatment response and recommendation maps on their own computers within a reasonable time, we present a fast and adaptive local cokriging tool for non-colocated and non-stationary OFE data. It uses a spatially-varying neighbourhood radius. It has a graphical user interface accessible via QGIS, a free and open source software. The adaptive local cokriging is demonstrated on three OFE examples. It performs indistinguishably from global cokriging on a small data set, but for large data sets, for which global cokriging is impractical, it predicts significantly more accurately than spatial splines or sampling-based cokriging. It outperforms cokriging base on a fixed number of nearest neighbours when this fixed number is not carefully chosen.

Big Data is a relative term, and Small Data can be equally important. Not only the volume of data defines if data is 'Big', but three more Vs characterise the term: velocity (speed of data generation and processing), veracity (referring to data quality) and variety. Perhaps the most defining is methodological: data becomes really big when new methods are needed to process and analyse it. In contrast, this paper demonstrates how even a tiny dataset can contribute to our understanding of the past, in this case of the historical geography of two provinces in Ottoman Greece in the 17th century. Graph analysis is used on a dataset of just 16 data pairs, illustrating the point that a close-up view of data complements the look from farther away at bigger data volumes.

The aim of this thesis was to compare the performance of two relational database management systems PostgreSQL and MySQL. For the purpose of the study a relational database was designed and a piece of software was implemented to connect desktop application with the database system. This software shall also create entities and relations between them in desired empty schemes on servers for databases. There has also been implemented a desktop application in Java programming language, that allows browsing data stored in database and performing the tests of database performance. Tests addressed basic operations of adding, collecting, updating and deleting data. Thisthesis makes a hypothesis "PostgreSQL is more efficient for desktop application while loaded with small data, in that case 1000 of queries", that was confirmed by achieved results from tests performed.

The issue of sentiment classification in short-term and small-scale data scenarios is considered in this paper. It is a hot topic because the text sentiment classification task in the public opinion analysis scene has two characteristics: short time and small data scale. Existing work focused on improving the accuracy at the cost of data and training time, without considering scenarios where time and data are lacked. The most commonly used method to solve the problem of small data scale is to use multi-modal information such as pictures, sounds and videos, which will lead to unbearable training time. The shorter training time determines that the classification model is generally selected as a deep neural network with fewer layers, such as TextCNN, TextRNN, and so on. However, such models are limited by the structure and have a low classification accuracy. In order to solve both short-term and small-scale data problems, a common information user attribute on social media is added to the model as multimodal information, which includes twelve attributes such as user age, location, and posting time. This paper proposed a sentiment classification algorithm based on multi-modal social media text information. The algorithm makes use of parallel convolutional neural networks (CNN) and recurrent neural network (RNN) to process text information and user attributes respectively, and combines the feature vectors of the two models for classification, which is called User attributes Convolutional and Recurrent Neural Network (UCRNN). The addition of user attributes can improve accuracy, and the CNN network used to extract user attributes features has fewer parameters, which proves that the algorithm can achieve high accuracy under short-term and small-scale data. Experiments verify that the training time of this model is slightly less than TextRNN. The classification accuracy can reach 90.2%, which is the state-of-the-art in the field of short-term and small-scale data sentiment classification.

In recent years, machine-learning-based approaches have become of considerable interest to the efficient processing of short or limited data samples. Its so-called small data approach. This is due to the significant growth of new intellectual analysis tasks in various industries, which are characterized by limited historical data. These include Materials Science, Economics, Medicine, and so on. An effective processing of short datasets is especially acute in medicine. Insufficient number of vectors, significant gaps in the data collected during the supervision of patient’s treatment or rehabilitation, reduces the effectiveness or prevents effective intellectual analysis based on them. This paper presents a new approach to processing short medical data samples. The basis of the developed method is SVR with RBF kernel. The algorithmic implementation of the method in both operation modes is described. Experimental modeling on a real short data set (Trabecular bone data) is conducted. It contained only 35 observations. A comparison of the method with a number of existing machine learning methods is conducted. It is experimental established the highest accuracy of the method among those considered. The developed method has potential opportunities for wide application in various fields of medicine.

International audience; Global sensitivity analysis provides insight into how sources of uncertainty contribute to uncertainty in predictions of computational models. Global sensitivity indices, also called variance-based sensitivity indices and Sobol indices, are most often computed with Monte Carlo methods. However, when the computational model is computationally expensive and only a small number of samples can be generated, that is, in so-called small-data settings, usual Monte Carlo estimates may lack sufficient accuracy. As a means of improving accuracy in such small-data settings, we explore the use of probabilistic learning. The objective of the probabilistic learning is to learn from the available samples a probabilistic model that can be used to generate additional samples, from which Monte Carlo estimates of the global sensitivity indices are then deduced. We demonstrate the interest of such a probabilistic learning method by applying it in an illustration concerned with forecasting the contribution of the Antarctic ice sheet to sea-level rise.

This study's main objective is to investigate the face validity and reliability of the determinants of SMEs' performance in Pakistan. For this purpose, researchers adopted a simple random sampling approach to collect a small data sample from the Owners/managers of 41 individual manufacturing SMEs. The model suggested four factors (PDI, PCI, OLC, TO) as the determinants of SMEs performance. Therefore, the instruments' reliability and validity were keenly examined with experts from academia and industry. Moreover, collected data were processed and analysed by using SPSS. However, the results of the study established the face validity and reliability of the adapted instruments.

A small data approach is an important area of research in the medical field. Many tasks in this area require the development of such direction due to the high cost of obtaining dataset or the inability to obtain sufficient observations for the analysis. Intelligent analysis of small datasets, which are very common in this area, will improve the quality of diagnosis, prevention or treatment. In this paper, the authors developed a new method of handling small datasets. The method is based on the use of a GRNN. The main steps of algorithms for preparation and application of the developed input doubling method are described. The modelling of the method is carried out; the optimal parameters for its functioning are selected. The efficiency of the method is established using a short set of clinical data by comparing its accuracy with a number of existing methods. It is shown that the developed input doubling method based on GRNN demonstrates the highest accuracy based on RMSE and MAE among all the methods considered. In addition, it provides unambiguous solutions, which avoids the use of such an approach as the "multiple runs" to choose the best solution. The main disadvantages of the method are analyzed, among which its considerable time delays of functioning in comparison with the basic GRNN that should be noted.

Optimization applications often depend on a huge number of uncertain parameters. In many contexts, however, the amount of relevant data per parameter is small, and hence, we may only have imprecise estimates. We term this setting—in which the number of uncertainties is large but all estimates have low precision—the small-data, large-scale regime. We formalize a model for this new regime, focusing on optimization problems with uncertain linear objectives. We show that common data-driven methods, such as sample average approximation, data-driven robust optimization, and certain regularized policies, may perform poorly in this new setting. We then propose a novel framework for selecting a data-driven policy from a given policy class. As with the aforementioned data-driven methods, our new policy enjoys provably good performance in the large-sample regime. Unlike these methods, we show that in the small-data, large-scale regime, our data-driven policy performs comparably to an oracle best-in-class policy under some mild conditions. We strengthen this result for linear optimization problems and two natural policy classes, the first inspired by the empirical Bayes literature and the second by regularization techniques. For both classes, the suboptimality gap between our proposed policy and the oracle policy decays exponentially fast in the number of uncertain parameters even for a fixed amount of data. Thus, these policies retain the strong large-sample performance of traditional methods and additionally enjoy provably strong performance in the small-data, large-scale regime. Numerical experiments confirm the significant benefits of our methods. This paper was accepted by Yinyu Ye, optimization.

In the complex industrial processes, the process data have the characteristics of imbalance and are incomplete due to the difficult-to-measure key variables, leading to the performance degradation of soft sensors. In order to deal with this problem, a novel noise injection virtual sample generation approach based on target-relevant autoencoder (NITAE-VSG) is proposed in this article. Different from the conventional VSG methods, the presented NITAE-VSG model can generate useful input and output virtual samples directly. To increase the diversity of virtual samples, the Gaussian noise (G_noise) is added into the features extracted from the target-relevant autoencoder (TAE) hidden layer. The generated virtual samples are combined with the original small samples to enhance the soft sensor model performance. To verify the reliable feasibility and superior effectiveness of the presented NITAE-VSG, two actual industrial cases: an industrial process of purified terephthalic acid shortened as PTA and a production system of ethylene are carried out. The results of simulations demonstrate that the presented NITAE-VSG can generate superior virtual samples to some other virtual sample generation (VSG) methods and can effectively improve the soft sensor modeling accuracy.

Fire detection technologies remain a critical component of building automation. With the recent significant advances in computer vision, visual fire detection methods have been developed and integrated into building surveillance systems. Overfitting and accuracy challenges remain in fire detection when training datasets are limited. In this work, we tackle these challenges by developing a deep convolutional generative adversarial network (DCGAN) for highly accurate visual fire detection when training images are limited. Our model addresses three types of errors in visual fire detection with small training datasets: model overfitting, fire probability overestimation, and fire probability underestimation. The DCGAN includes a generator of fake fire images for self-supervised learning (SSL) and a discriminator for image classification. We designed computational experiments with high-quality datasets to test and validate our model against other supervised learning approaches. We also benchmarked the performance of the DCGAN against a best-in-class deep visual fire detection model. The results show that our model significantly outperforms other fire detection models on all performance metrics when trained with the same small dataset. The results demonstrate that the DCGAN effectively mitigates the three types of error when the training dataset is limited.

In a context marked by the proliferation of smartphones and multimedia applications, the processing and transmission of images have become a real problem. Image compression is the first approach to address this problem, it nevertheless suffers from its inability to adapt to the dynamics of limited environments, consisting mainly of mobile equipment and wireless networks. In this work, we propose a stochastic model to gradually estimate an image upon information on its pixels that are transmitted progressively. We consider this transmission as a dynamical process, where the sender pushes the data in decreasing significance order. In order to adapt to network conditions and performances, instead of truncating the pixels, we suggest a new method called Fast Reconstruction Method by Kalman Filtering (FRM-KF) consisting of recursive inference of the not yet received layers belonging to a sequence of bitplanes. After empirical analysis, we estimate parameters of our model which is a linear discrete Kalman Filter. We assume the initial law of information to be the uniform distribution on the set [0, 255] corresponding to the range of gray levels. The performances of FRM-KF method have been evaluated in terms of the ratios in the quality of data image/size sent and in the quality of image/time required for treatment. A high quality was reached faster with relatively small data (less than 10% of image data is needed to obtain up to the sixth-quality image). The time for treatment also decreases faster with number of received layers. However, we found that the time of image treatment might be large starting from a image resolution of 1024 * 1024. Hence, we recommend FRM-KF method for resolutions less or equal to 512 * 512. A statistical comparative analysis reveals that FRM-KF is competitive and suitable to be implemented on limited resource environments.

Open science and data are yet to make a real breakthrough and research policies will have a critical role in it. The history and general context around open data is hence firstly addressed, including how researchers perceive the existing incentives, leading to recommendations on how to foster data sharing. Subsequently, the focus is on catalysis, with a particular emphasis on benchmarking the data sharing practices against other fields and surveying the type of data currently being shared. The current infrastructure, including data repositories, and standards formats is maped. The striking differences among different disciplines are discussed, serving as a basis to propose specific actions to promote data sharing in catalysis. Short-term initiatives are needed to boost the amount of openly available data, particularly in heterogeneous catalysis, but a high degree of standardization in data formats will be needed to ensure optimal and automated data mining in the long run. Because of its unique, central role in understanding the catalytic action, kinetic catalytic data is of particular interest. As formats and mining tools are dependant on the type of data, kinetic catalytic data is firstly characterized. Guidelines for a standardized sharing format are proposed, taking into account the small, well-structured nature of this type of data. To maximize the extraction of information, the low volume of kinetic catalytic data will be compensated by incorporating fundamental knowledge into statistics-based tools. Whencoupled with knowledge generation tools, i. e. kinetic models, new insights at the active site and mechanism levels will be reached in an ever more automated and powerful way.

When discussing interpretable machine learning results, researchers need to compare them and check for reliability, especially for health-related data. The reason is the negative impact of wrong results on a person, such as in wrong prediction of cancer, incorrect assessment of the COVID-19 pandemic situation, or missing early screening of dyslexia. Often only small data exists for these complex interdisciplinary research projects. Hence, it is essential that this type of research understands different methodologies and mindsets such as the Design Science Methodology, Human-Centered Design or Data Science approaches to ensure interpretable and reliable results. Therefore, we present various recommendations and design considerations for experiments that help to avoid over-fitting and biased interpretation of results when having small imbalanced data related to health. We also present two very different use cases: early screening of dyslexia and event prediction in multiple sclerosis.

Prediction methods of glioblastoma tumours growth constitute a hard task due to the lack of medical data, which is mostly related to the patients' privacy, the cost of collecting a large medical data set, and the availability of related notations by experts.In this study, the authors propose a synthetic medical image generator (SMIG) with the purpose of generating synthetic data based on the generative adversarial network in order to provide anonymised data. In addition, to predict the glioblastoma multiform tumour growth the authors developed a tumour growth predictor based on end to end convolution neural network architecture that allows training on a public data set from the cancer imaging archive (TCIA), combined with the generated synthetic data. The authors also highlighted the impact of implicating a synthetic data generated using SMIG as a data augmentation tool. Despite small data size provided by TCIA data set, the obtained results demonstrate valuable tumour growth prediction accuracy.

Quality control is essential for modern business success. The traditional statistical process control (SPC), however, lacks efficacy in current high-variety low-volume industrial practices since th...

As the world is digitizing fast, the increase in Big and Small Data offers opportunities to enrich official statistics for reporting on Sustainable Development Goals (SDG). However, survey data coming from an increased number of organizations (Small Data) and Big Data offer challenges in terms of data heterogeneity. This paper describes a methodology for combining various data sources to create a more comprehensive dataset on SDG 6.1.1. (proportion of population using safely managed drinking water services). We enabled digital volunteers to trace buildings on satellite imagery and used the traces on OpenStreetMap to facilitate visual detection of water points on Unmanned Aerial Vehicle (UAV) imagery and estimate the number of people served per water point. Combining data on water points identified on our UAV imagery with data on water points from field surveys improves the overall quality in terms of removal of inconsistencies and enrichment of attribute information. Satellite imagery enables scaling more easily than UAV imagery but is too costly to acquire at sufficiently high resolution. For small areas, our workflow is cost-effective in creating an up-to-date and consistent water point dataset by combining UAV imagery, Volunteered Geographic Information, and field survey data.

In order to address the clustering problem of intelligent medical data, the data sets were not preprocessed using the traditional method, leading to a large amount of calculation, low efficiency, and large data cluster center offset distance. We proposed a balanced clustering algorithm for intelligent medical heterogeneous big data set using deep learning. Firstly, a deep neural network model based on incremental updating was constructed, and adaptive training and adjustment were made according to data scale, and the multi-layer feature learning of heterogeneous big data sets of intelligent medical care. Secondly, under-sampling preprocessing was carried out on the data set so that the data of the heterogeneous big data set was in a balanced state, and on this basis, clustering calculation of the heterogeneous big data was conducted. Then, the clustering center was set according to the kernel density estimation results, and the data cluster center was updated iteratively until convergence by combining the data features obtained from deep learning and euclidean distance calculation, so as to complete the balanced clustering of the heterogeneous big data set of intelligent medical treatment. The results show that the proposed algorithm has the advantages of small data cluster center offset distance, short clustering time, low energy consumption, high Macro-F1 value and NMI value, and the accuracy of clustering can be as high as 95%, the calculational cost is low, which has certain advantages. 2020 Elsevier Ltd. All rights reserved.

Understanding the chaos of air traffic flow is significant to the achievement of advanced air traffic management, and trajectory data are the basic material for studying the chaotic characteristics. However, at present, there are two main obstacles to this task, namely, large amounts of noise in the measured data and the tedium of existing data processing methods. This paper improves the incorrect trajectory processing method based on ADS-B trajectory data and proposes a method by which to quickly extract the traffic flow through a certain waypoint. Currently, the commonly used theoretical analysis tools for nonlinear complex systems include the classical nonlinear dynamics analysis method and the newly developed complex network-based analysis method. The latter is currently in an exploratory stage because it has just been introduced into the study of air traffic flow. From these two perspectives, the chaotic characteristics of air traffic flow are studied in the present work. From the perspective of nonlinear dynamics, the improved C-C method is used to calculate the reliability parameters, namely, the time delay τ and embedding dimension m, of phase-space reconstruction, and the maximum Lyapunov index is calculated by using the small data volume method to prove the existence of chaos in the system. From the perspective of complex networks, the construction of a visibility graph and horizontal visibility graph is used to prove the existence of chaos in the system, and the goodness-of-fit parameters of the degree distributions of two fitting methods under different time scales are evaluated, which provides support for the air traffic flow theory.

Large-scale software systems and cloud services continue to produce a large amount of log data. Such log data is usually preserved for a long time (e.g., for auditing purposes). General compressors, like the LZ77 compressor used in gzip, are usually used in practice to compress log data to reduce the cost of long-term storage. However, such general compressors do not consider the unique nature of log data. In this paper, we study the performance of general compressors on compressing log data relative to their performance on compressing natural language data. We used 12 widely used general compressors to compress nine log files that are collected based on surveying prior literature on text compression, log compression and log analysis. We observe that log data is more repetitive than natural language data, and that log data can be compressed and decompressed faster with higher compression ratios. Besides, the compressor with the highest compression ratio for natural language data is rarely the one for log data. Nevertheless, the compressors with the highest compression ratio for log data are rarely adopted in practice by current logging libraries and log management tools. We also observe that the peak compression and decompression speeds of general compressors on log data is often achieved with a small data size, while such size may not be used by log management tools. Finally, we observe that the optimal compression performance (measured by a combined compression performance score) of log data usually requires the compression level to be configured higher than the default level. Our findings call for careful consideration of choosing general compressors and their associated compression levels for log data in practice. In addition, our findings shed lights on the opportunities for future research on compressors that better suit the characteristics of log data.

Overfitting and treatment of small data are among the most challenging problems in machine learning (ML), when a relatively small data statistics size [Formula: see text] is not enough to provide a robust ML fit for a relatively large data feature dimension [Formula: see text]. Deploying a massively parallel ML analysis of generic classification problems for different [Formula: see text] and [Formula: see text], we demonstrate the existence of statistically significant linear overfitting barriers for common ML methods. The results reveal that for a robust classification of bioinformatics-motivated generic problems with the long short-term memory deep learning classifier (LSTM), one needs in the best case a statistics [Formula: see text] that is at least 13.8 times larger than the feature dimension [Formula: see text]. We show that this overfitting barrier can be breached at a 10[Formula: see text] fraction of the computational cost by means of the entropy-optimal scalable probabilistic approximations algorithm (eSPA), performing a joint solution of the entropy-optimal Bayesian network inference and feature space segmentation problems. Application of eSPA to experimental single cell RNA sequencing data exhibits a 30-fold classification performance boost when compared to standard bioinformatics tools and a 7-fold boost when compared to the deep learning LSTM classifier.

Automatic extracting of knowledge from massive data samples, i.e., big data analytics (BDA), has emerged as a vital task in almost all scientific research fields. The BDA problems are rather difficult to solve due to their large-scale, high-dimensional, and dynamic properties, while the problems with small data are usually hard to handle due to insufficient data samples and incomplete information. Such difficulties lead to the search-based data analytics problem, where a data analysis task is modeled as a complex, dynamic, and computationally expensive optimization problem and then solved by using an iterative algorithm. In this paper, we intend to present an extensive and in-depth discussion on the utilizing of evolutionary computation (EC) based optimization methods [including evolutionary algorithms (EAs) and swarm intelligence (SI)] for solving search-based data analysis problems. Then, as an example for illustration, we provide a comprehensive review of the applications of state-of-the-art EC methods for different types of data mining problems in bioinformatics. Here, the detailed analysis and discussion are conducted on three types of data samples, which include sequences data, network data, and image data. Finally, we survey the challenges faced by EC methods and the trend for future directions. Based on the applications of EC methods for search-based data analysis problems involving inexact and uncertain information, the insights of data analytics are able to understand better, and more efficient algorithms could be designed to solve real-world complex BDA problems.