Search

The article presents a new approach to modeling nonlinear dependencies called composite Bayesian networks. The main emphasis is on integrating machine learning models into Bayesian networks while maintaining their fundamental principles. The novelty of the approach is that it allows us to solve the problem of data inconsistency with traditional assumptions about dependencies. The presented method consists in selecting a variety of machine learning models at the stage of training composite Bayesian networks. This allows you to flexibly customize the nature of the dependencies in accordance with the requirements and dictated characteristics of the modeled object. The software implementation is made in the form of a specialized framework that describes all the necessary functionality. The results of experiments to evaluate the effectiveness of modeling dependencies between features are presented. Data for the experiments was taken from the bnlearn repository for benchmarks and from the UCI repository for real data. The performance of composite Bayesian networks was validated by comparing the likelihood and F1 score with classical Bayesian networks trained with the Hill-Climbing algorithm, demonstrating high accuracy in representing multivariate distributions. The improvement in benchmarks is insignificant since they contain linear dependencies that are well modeled by the classical algorithm. An average 30 % improvement in likelihood was obtained on real UCI datasets. The obtained data can be applied in areas that require modeling complex dependencies between features, for example, in machine learning, statistics, data analysis, as well as in specific subject areas.

Federated learning makes it possible to train a machine learning model on decentralized data. Bayesian networks are widely used probabilistic graphical models. While some research has been published on the federated learning of Bayesian networks, publications on Bayesian networks in a vertically partitioned data setting are limited, with important omissions, such as handling missing data. We propose a novel method called VertiBayes to train Bayesian networks (structure and parameters) on vertically partitioned data, which can handle missing values as well as an arbitrary number of parties. For structure learning we adapted the K2 algorithm with a privacy-preserving scalar product protocol. For parameter learning, we use a two-step approach: first, we learn an intermediate model using maximum likelihood, treating missing values as a special value, then we train a model on synthetic data generated by the intermediate model using the EM algorithm. The privacy guarantees of VertiBayes are equivalent to those provided by the privacy preserving scalar product protocol used. We experimentally show VertiBayes produces models comparable to those learnt using traditional algorithms. Finally, we propose two alternative approaches to estimate the performance of the model using vertically partitioned data and we show in experiments that these give accurate estimates. [ABSTRACT FROM AUTHOR]

Summary In this article, the parameter learning scheme for the multi‐input multi‐output (MIMO) Hammerstein nonlinear systems under measurement noises is studied, which is derived by exploiting the correlation analysis and data filtering technique. The coupled MIMO Hammerstein system presented involves a static nonlinear subsystem modeled by neural fuzzy model (NFM), and a dynamic linear subsystem established by autoregressive moving average with extra input (ARMAX) model. To learn the unknown parameter of the MIMO Hammerstein system, the combined signals are designed to realize that identification of the nonlinear subsystem is separated from that of linear subsystem. First, the correlation properties of separable signals in a nonlinear system are analyzed, then the parameters of the linear subsystem are estimated utilizing correlation analysis, which can deal with the issue of unmeasured intermediate variable in the Hammerstein system. Second, the data filtering technique is introduced to derive the data filtering‐based recursive least squares technique for learning the nonlinear subsystem parameter, which can reduce the impact of the moving average noise and improve the precision of parameter estimation. Finally, the effectiveness and feasibility of the proposed identification scheme is proved by numerical simulation and nonlinear pH process. [ABSTRACT FROM AUTHOR]

Abstract Federated learning makes it possible to train a machine learning model on decentralized data. Bayesian networks are widely used probabilistic graphical models. While some research has been published on the federated learning of Bayesian networks, publications on Bayesian networks in a vertically partitioned data setting are limited, with important omissions, such as handling missing data. We propose a novel method called VertiBayes to train Bayesian networks (structure and parameters) on vertically partitioned data, which can handle missing values as well as an arbitrary number of parties. For structure learning we adapted the K2 algorithm with a privacy-preserving scalar product protocol. For parameter learning, we use a two-step approach: first, we learn an intermediate model using maximum likelihood, treating missing values as a special value, then we train a model on synthetic data generated by the intermediate model using the EM algorithm. The privacy guarantees of VertiBayes are equivalent to those provided by the privacy preserving scalar product protocol used. We experimentally show VertiBayes produces models comparable to those learnt using traditional algorithms. Finally, we propose two alternative approaches to estimate the performance of the model using vertically partitioned data and we show in experiments that these give accurate estimates.

Abstract With the acceleration of global climate change and urbanization, disaster chains are always connected to artificial systems like critical infrastructure. The complexity and uncertainty of the disaster chain development process and the severity of the consequences have brought great challenges to emergency decision makers. The Bayesian network (BN) was applied in this study to reason about disaster chain scenarios to support the choice of appropriate response strategies. To capture the interacting relationships among different factors, a scenario representation model of disaster chains was developed, followed by the determination of the BN structure. In deriving the conditional probability tables of the BN model, we found that, due to the lack of data and the significant uncertainty of disaster chains, parameter learning methodologies based on data or expert knowledge alone are insufficient. By integrating both sample data and expert knowledge with the maximum entropy principle, we proposed a parameter estimation algorithm under expert prior knowledge (PEUK). Taking the rainstorm disaster chain as an example, we demonstrated the superiority of the PEUK-built BN model over the traditional maximum a posterior (MAP) algorithm and the direct expert opinion elicitation method. The results also demonstrate the potential of our BN scenario reasoning paradigm to assist real-world disaster decisions.

When dealing with engineering design problems, designers often encounter nonlinear and nonconvex features, multiple objectives, coupled decision making, and various levels of fidelity of sub-systems. To realize the design with limited computational resources, problems with the features above need to be linearized and then solved using solution algorithms for linear programming. The adaptive linear programming (ALP) algorithm is an extension of the Sequential Linear Programming algorithm where a nonlinear compromise decision support problem (cDSP) is iteratively linearized, and the resulting linear programming problem is solved with satisficing solutions returned. The reduced move coefficient (RMC) is used to define how far away from the boundary the next linearization is to be performed, and currently, it is determined based on a heuristic. The choice of RMC significantly affects the efficacy of the linearization process and, hence, the rapidity of finding the solution. In this paper, we propose a rule-based parameter-learning procedure to vary the RMC at each iteration, thereby significantly increasing the speed of determining the ultimate solution. To demonstrate the efficacy of the ALP algorithm with parameter learning (ALPPL), we use an industry-inspired problem, namely, the integrated design of a hot-rolling process chain for the production of a steel rod. Using the proposed ALPPL, we can incorporate domain expertise to identify the most relevant criteria to evaluate the performance of the linearization algorithm, quantify the criteria as evaluation indices, and tune the RMC to return the solutions that fall into the most desired range of each evaluation index. Compared with the old ALP algorithm using the golden section search to update the RMC, the ALPPL improves the algorithm by identifying the RMC values with better linearization performance without adding computational complexity. The insensitive region of the RMC is better explored using the ALPPL—the ALP only explores the insensitive region twice, whereas the ALPPL explores four times throughout the iterations. With ALPPL, we have a more comprehensive definition of linearization performance—given multiple design scenarios, using evaluation indices (EIs) including the statistics of deviations, the numbers of binding (active) constraints and bounds, the numbers of accumulated linear constraints, and the number of iterations. The desired range of evaluation indices (DEI) is also learned during the iterations. The RMC value that brings the most EIs into the DEI is returned as the best RMC, which ensures a balance between the accuracy of the linearization and the robustness of the solutions. For our test problem, the hot-rolling process chain, the ALP returns the best RMC in twelve iterations considering only the deviation as the linearization performance index, whereas the ALPPL returns the best RMC in fourteen iterations considering multiple EIs. The complexity of both the ALP and the ALPPL is O(n2). The parameter-learning steps can be customized to improve the parameter determination of other algorithms. [ABSTRACT FROM AUTHOR]

The connection between numerical methods for solving differential equations and machine learning has been revealed recently. Differential equations have been proposed as continuous analogues of deep neural networks, and then used in handling certain tasks, such as image recognition, where the training of a model includes learning the parameters of systems of ODEs from certain points along their trajectories. Treating this inverse problem of determining the parameters of a dynamical system that minimize the difference between data and trajectory by a gradient-based optimization method presents the solution of the adjoint equation as the continuous analogue of backpropagation that yields the appropriate gradients. The paper explores an abstract approach that can be used to construct a family of loss functions with the aim of fitting the solution of an initial value problem to a set of discrete or continuous measurements. It is shown, that an extension of the adjoint equation can be used to derive the gradient of the loss function as a continuous analogue of backpropagation in machine learning. Numerical evidence is presented that under reasonably controlled circumstances the gradients obtained this way can be used in a gradient descent to fit the solution of an initial value problem to a set of continuous noisy measurements, and a set of discrete noisy measurements that are recorded at uncertain times. [ABSTRACT FROM AUTHOR]

With the acceleration of global climate change and urbanization, disaster chains are always connected to artificial systems like critical infrastructure. The complexity and uncertainty of the disaster chain development process and the severity of the consequences have brought great challenges to emergency decision makers. The Bayesian network (BN) was applied in this study to reason about disaster chain scenarios to support the choice of appropriate response strategies. To capture the interacting relationships among different factors, a scenario representation model of disaster chains was developed, followed by the determination of the BN structure. In deriving the conditional probability tables of the BN model, we found that, due to the lack of data and the significant uncertainty of disaster chains, parameter learning methodologies based on data or expert knowledge alone are insufficient. By integrating both sample data and expert knowledge with the maximum entropy principle, we proposed a parameter estimation algorithm under expert prior knowledge (PEUK). Taking the rainstorm disaster chain as an example, we demonstrated the superiority of the PEUK-built BN model over the traditional maximum a posterior (MAP) algorithm and the direct expert opinion elicitation method. The results also demonstrate the potential of our BN scenario reasoning paradigm to assist real-world disaster decisions. [ABSTRACT FROM AUTHOR]

We explore the performance of mixed‐frequency predictive regressions for stock returns from the perspective of a Bayesian investor. We develop a constrained parameter learning approach for sequential estimation allowing for belief revisions. Empirically, we find that mixed‐frequency models improve predictability, not only because of the combination of predictors with different frequencies but also due to the preservation of high‐frequency features such as time‐varying volatility. Temporally aggregated models misspecify the evolution frequency of the volatility dynamics, resulting in poor volatility timing and worse portfolio performance than the mixed‐frequency specification. These results highlight the importance of preserving the potential mixed‐frequency nature of predictors and volatility in predictive regressions. [ABSTRACT FROM AUTHOR]

In this paper, a new and effective parameter learning method for delayed Boolean control networks (DBCNs) with missing observations is presented in this paper, which can handle strong noise and converge to the true parameter values in finite times. Two types of noise that follow different time-varying Bernoulli distributions are considered in this paper: measurement noise and system noise. The concept of virtual nodes is introduced by the paper to cope with random missing observations, and the algebraic representation of DBCNs is obtained using the STP method. A parameter learning algorithm based on the forward and backward probability and EM algorithm is then proposed by the paper, which can estimate the model parameters from time series data. The performance and robustness of the proposed method are demonstrated by the paper through a numerical example.

The parameter gamma, which is used in distance-based knowledge graph embedding to distinguish positive and negative samples, plays an important role in model performance. Usually, gamma is considered a hyperparameter and taken from a discrete set. However, as the boundary of positive and negative samples, gamma may have a continuous property and be complementary to other parameters. The high consistency between gamma and the learnable parameter with respect to their attributes means that choosing gamma as a hyperparameter is probably not a good choice. To better explore the characteristics of gamma, we propose a multi-iterated parameterized scheme to convert gamma from a hyperparameter to a normal parameter. Two concrete implementation methods, the macro parametrized method and the micro parameterized method, are provided to achieve this conversion. The conversion for gamma can reduce dependency on expert knowledge in the gamma setting and can allow successors to become involved in the recent framework quickly. Experimental results on TransE, RotatE and LineaRE show that the proposed scheme and methods can achieve approximate or better results when compared with those of the original method. A series of experimental analyses further explore the characteristics and illustrate the availability of the proposed parameterized scheme and methods. [ABSTRACT FROM AUTHOR]

To explore the mechanism and influencing factors of coal and gas outburst accidents in coal mines, the factors inducing coal and gas outburst accidents were selected from four aspects: human, machine, environment and management, and the risk of accidents was studied by D-S evidence theory and Bayesian network. D-S evidence theory fuses expert knowledge to get the network structure, and then optimizes and corrects the network structure through greedy search algorithm. The EM algorithm is used to learn the parameters, obtain the conditional probability and posterior probability among the related factors, find out the key factors leading to the accident and identify the sensitive factors, and get the maximum risk source chain and analyze it. The results show that the main cause of coal and gas outburst is environmental factors(87.4%), followed by management factors (84.7%), mechanical factors(80.3%) and human factors(82.7%). Poor coal structure, geological conditions, without outburst prevention measures, inadequate investigation of potential safety hazards, gas overrun, imperfect monitoring system and insufficient ventilation facilities are the important influencing factors that induce coal and gas outburst accidents in coal mines.

Under the condition of Bayesian network (BN) small data set, the qualitative maximum a posteriori (QMAP) estimation tends to violate expert constraints, which causes the QMAP estimation to deviate the true value. In order to overcome the shortcomings of this algorithm, an improved QMAP algorithm was proposed. Firstly, the QMAP estimation was learned. Next, the parameters that violated the inequality constraints were regulated by the isotonic regression method. Then, the regulated parameters were further adjusted by using a fine-tuning strategy, in order to make the obtained parameters satisfy the expert constraints. Finally, a comparison was made with the maximum likelihood estimation (MLE) algorithm and QMAP algorithm. The simulation results show that under the condition of small data set, the proposed algorithm meets all the constraints:(1) the KL (Kullback-Leibler) divergence is always lower than the other two algorithms; (2) the running time is approximately 0.1 s more than the other two algorithms, which has little influence; and (3) the inference results are close to the real value, with the deviation maintaining between ±0.05. The comprehensive performance of the improved QMAP algorithm is better than that of MLE and QMAP algorithm, and it has good practicability.

A novel parameter learning scheme using multi-signals is proposed for estimating parameters of the Hammerstein nonlinear model in this research. The Hammerstein nonlinear model consists of a static nonlinear block and a dynamic linear block, and the designed multi-signals that composed of separable signals and random signals are used for learning separately the nonlinear block parameters and the linear block parameters. Firstly, the linear block parameters are learned by virtue of correlation analysis algorithm based on input–output data of separable signals, which effectively deals with the unmeasurable variable of the Hammerstein model. In addition, auxiliary error-based probability density function strategy is employed to estimate the nonlinear block parameters with the help of random signals, which makes the model error present normal distribution, as well as improves model accuracy effectively. Studies with a numerical simulation and a typical industrial process demonstrate that the developed approach obtains high learning accuracy and small modeling error, which verifies the accuracy and efficiency of the developed parameter learning scheme. [ABSTRACT FROM AUTHOR]

Bayesian networks (BNs) are one of the most compelling theoretical models in uncertain knowledge representation and inference. However, many domains are encountering the dilemma of insufficient data. Learning BN parameters using raw data may lead to low learning accuracy. Therefore, this paper seeks to solve the problem via two novel data extension methods. First, a constraint-based nonparametric bootstrap (CNB) method is proposed, which extends the raw data and guides the parameter distribution of the extended data through a constraint-based sample scoring function. The experimental results on 12 BNs show that the extended data can improve the parameter learning accuracy and enhance the existing parameter learning approaches. The CNB is still valid for medium and large networks with relatively large data. When the original data are of inferior quality, the CNB is unattainable to extend it. Then, a constraint-based parametric bootstrap (CPB) method is proposed, creating a new parameter distribution by constraints and the original samples. The experimental results for the missing data demonstrate that the extended data perform better. The CPB is insensitive to the proportion of missing data and remains superior in relatively large data. [ABSTRACT FROM AUTHOR]

A novel parameter learning scheme using multi‐signal processing is developed that aims at estimating parameters of the Hammerstein nonlinear model with output disturbance in this paper. The Hammerstein nonlinear model consists of a static nonlinear block and a dynamic linear block, and the multi‐signals are devised to estimate separately the nonlinear block parameters and the linear block parameters; the parameter estimation procedure is greatly simplified. Firstly, in view of the input–output data of separable signals, the linear block parameters are computed through correlation analysis method, thereby the influence of output noise is effectively handled. In addition, model error probability density function technology is employed to estimate the nonlinear block parameters with the help of measurable input–output data of random signals, which not only controls the space state distribution of model error but also makes error distribution tends to normal distribution. The simulation results demonstrate that the developed approach obtains high learning accuracy and small modeling error, which verifies the effectiveness of the developed approach. [ABSTRACT FROM AUTHOR]

In this paper, we focus on learning optimal parameters for PDE-based image denoising and decomposition models. First, we learn the regularization parameter and the differential operator for gray-scale image denoising using the fractional Laplacian in combination with a bilevel optimization problem. In our setting the fractional Laplacian allows the use of Fourier transform, which enables the optimization of the denoising operator. We prove stable and explainable results as an advantage in comparison to machine learning approaches. The numerical experiments correlate with our theoretical model settings and show a reduction of computing time in contrast to the Rudin-Osher-Fatemi model. Second, we introduce a new regularized image decomposition model with the fractional Laplacian and the Riesz potential. We provide an explicit formula for the unique solution and the numerical experiments illustrate the efficiency. [ABSTRACT FROM AUTHOR]

When dealing with engineering design problems, designers often encounter nonlinear and nonconvex features, multiple objectives, coupled decision making, and various levels of fidelity of sub-systems. To realize the design with limited computational resources, problems with the features above need to be linearized and then solved using solution algorithms for linear programming. The adaptive linear programming (ALP) algorithm is an extension of the Sequential Linear Programming algorithm where a nonlinear compromise decision support problem (cDSP) is iteratively linearized, and the resulting linear programming problem is solved with satisficing solutions returned. The reduced move coefficient (RMC) is used to define how far away from the boundary the next linearization is to be performed, and currently, it is determined based on a heuristic. The choice of RMC significantly affects the efficacy of the linearization process and, hence, the rapidity of finding the solution. In this paper, we propose a rule-based parameter-learning procedure to vary the RMC at each iteration, thereby significantly increasing the speed of determining the ultimate solution. To demonstrate the efficacy of the ALP algorithm with parameter learning (ALPPL), we use an industry-inspired problem, namely, the integrated design of a hot-rolling process chain for the production of a steel rod. Using the proposed ALPPL, we can incorporate domain expertise to identify the most relevant criteria to evaluate the performance of the linearization algorithm, quantify the criteria as evaluation indices, and tune the RMC to return the solutions that fall into the most desired range of each evaluation index. Compared with the old ALP algorithm using the golden section search to update the RMC, the ALPPL improves the algorithm by identifying the RMC values with better linearization performance without adding computational complexity. The insensitive region of the RMC is better explored using the ALPPL—the ALP only explores the insensitive region twice, whereas the ALPPL explores four times throughout the iterations. With ALPPL, we have a more comprehensive definition of linearization performance—given multiple design scenarios, using evaluation indices (EIs) including the statistics of deviations, the numbers of binding (active) constraints and bounds, the numbers of accumulated linear constraints, and the number of iterations. The desired range of evaluation indices (DEI) is also learned during the iterations. The RMC value that brings the most EIs into the DEI is returned as the best RMC, which ensures a balance between the accuracy of the linearization and the robustness of the solutions. For our test problem, the hot-rolling process chain, the ALP returns the best RMC in twelve iterations considering only the deviation as the linearization performance index, whereas the ALPPL returns the best RMC in fourteen iterations considering multiple EIs. The complexity of both the ALP and the ALPPL is O(n2). The parameter-learning steps can be customized to improve the parameter determination of other algorithms.

Copyright of Systems Engineering & Electronics is the property of Journal of Systems Engineering & Electronics Editorial Department and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Parameter learning for high-dimensional, partially observed, and nonlinear stochastic processes is a methodological challenge. Spatiotemporal disease transmission systems provide examples of such processes giving rise to open inference problems. We propose the iterated block particle filter (IBPF) algorithm for learning high-dimensional parameters over graphical state space models with general state spaces, measures, transition densities and graph structure. Theoretical performance guarantees are obtained on beating the curse of dimensionality (COD), algorithm convergence, and likelihood maximization. Experiments on a highly nonlinear and non-Gaussian spatiotemporal model for measles transmission reveal that the iterated ensemble Kalman filter algorithm (Li et al., 2020) is ineffective and the iterated filtering algorithm (Ionides et al., 2015) suffers from the COD, while our IBPF algorithm beats COD consistently across various experiments with different metrics. [ABSTRACT FROM AUTHOR]

Under the condition of Bayesian network（BN） small data set, the qualitative maximum a posteriori（QMAP） estimation tends to violate expert constraints, which causes the QMAP estimation to deviate the true value. In order to overcome the shortcomings of this algorithm, an improved QMAP algorithm was proposed. Firstly, the QMAP estimation was learned. Next, the parameters that violated the inequality constraints were regulated by the isotonic regression method. Then, the regulated parameters were further adjusted by using a fine-tuning strategy, in order to make the obtained parameters satisfy the expert constraints. Finally, a comparison was made with the maximum likelihood estimation (MLE） algorithm and QMAP algorithm. The simulation results show that under the condition of small data set, the proposed algorithm meets all the constraints:（1） the KL（Kullback-Leibler） divergence is always lower than the other two algorithms;（2） the running time is approximately 0.1 s more than the other two algorithms, which has little influence; and（3） the inference results are close to the real value, with the deviation maintaining between ±0.05. The comprehensive performance of the improved QMAP algorithm is better than that of MLE and QMAP algorithm, and it has good practicability. [ABSTRACT FROM AUTHOR]

We introduce a novel algorithm for estimating optimal parameters of linearized assignment flows for image labeling. An exact formula is derived for the parameter gradient of any loss function that is constrained by the linear system of ODEs determining the linearized assignment flow. We show how to efficiently evaluate this formula using a Krylov subspace and a low-rank approximation. This enables us to perform parameter learning by Riemannian gradient descent in the parameter space, without the need to backpropagate errors or to solve an adjoint equation. Experiments demonstrate that our method performs as good as highly-tuned machine learning software using automatic differentiation. Unlike methods employing automatic differentiation, our approach yields a low-dimensional representation of internal parameters and their dynamics which helps to understand how assignment flows and more generally neural networks work and perform. [ABSTRACT FROM AUTHOR]

At the beginning of the change in bearing condition, the bearing fault features are weak, often drowning in the background noise. It makes difficult to extract weak fault features from the vibration signal. If the fault features cannot be effectively extracted, the diagnostic accuracy will be greatly affected. Under this background, a fault diagnosis framework including two stages of signal enhancement and intelligent fault recognition is proposed in this work. Firstly, use a genetic algorithm to obtain the optimal combination of parameters, upon which the original fault signals are decomposed into intrinsic modal function components. Then, they are transformed into the spectrum signals and inputted into the proposed Bayesian network using dynamic weighted transfer learning (DWTL). This paper uses the DWTL method to set the source domain weight factor and the balance coefficient based on data quantity in different source and target domains. The DWTL method proposed in this paper can improve the fault diagnosis accuracy and effectively avoid the negative transfer phenomenon. An example of rolling bearing fault diagnosis is conducted. The results show that the accuracy of fault diagnosis based on the proposed framework is about 10% higher than that of other fault diagnosis methods. Therefore, the validity and feasibility of the proposed fault diagnosis framework are proved. [ABSTRACT FROM AUTHOR]

To realize unmanned aerial vehicle (UAV) situation assessment, a Bayesian network (BN) for situation assessment is established. Aimed at the problem that the parameters of the BN are difficult to obtain, an improved whale optimization algorithm based on prior parameter intervals (IWOA-PPI) for parameter learning is proposed. Firstly, according to the dependencies between the situation and its related factors, the structure of the BN is established. Secondly, in order to fully mine the prior knowledge of parameters, the parameter constraints are transformed into parameter prior intervals using Monte Carlo sampling and interval transformation formulas. Thirdly, a variable encircling factor and a nonlinear convergence factor are proposed. The former and the latter enhance the local and global search capabilities of the whale optimization algorithm (WOA), respectively. Finally, a simulated annealing strategy incorporating Levy flight is introduced to enable the WOA to jump out of the local optimum. In the experiment for the standard BNs, five parameter-learning algorithms are applied, and the results prove that the IWOA-PPI is not only effective but also the most accurate. In the experiment for the situation BN, the situations of the assumed mission scenario are evaluated, and the results show that the situation assessment method proposed in this article is correct and feasible.

Bearings are broadly applied in various types of industrial systems. Fault diagnosis, as a promising way for reliability of modern industrial internet of thing applications, has attracted increasing attention from both academia and industry fields. Being ideal modeling and inference tool in uncertainty situations, Bayesian network (BN) is becoming increasingly popular in many systems. However, in practical uncertain and complicated engineering surroundings, it’s difficult or expensive to collect massive labeled fault data for the sake of fault diagnosis model learning. To address the issue of BN parameter learning under small data set conditions, this paper proposes a Varying Coefficient Transfer Learning (VCTL) algorithm based on aggregation and transfer learning, that considers both knowledge from the resource domain and the resource relevance contributions. The balancing weight function is designed to determine whether the learning task in the resource domain is activated. Relevance weight factors are proposed to measure the relevance of resource and target parameters quantitatively, by combing parameter information from resource domains with those obtained from the target domain, using maximum a posterior (MAP) or maximum likelihood estimation (MLE). Finally, the target parameters are aggregated with both the target initial parameters and the parameter knowledge from the resource domain. Based on VCTL, a bearing fault diagnosis approach is proposed and verified. The experimental results show that, under the condition of the small data set, learning accuracy of VCTL algorithm with varying coefficient aggregation is better than MLE algorithm, MAP algorithm or state-of-the-art parameter transfer method, local linear pooling transfer learning (LoLP) algorithm. Under the condition of sufficient data set, learning accuracy of VCTL algorithm approaches the classical MLE or MAP, and the correctness of the proposed algorithm is verified. Moreover, we illustrate the successful application to real-world bearing fault diagnosis case with VCTL, where we had access to expert-provided resource knowledge and real fault diagnosis data.

Small data sets make the statistical information in Bayesian network parameter learning inaccurate, which makes it difficult to get accurate Bayesian network parameters based on data. Qualitative maximum a posteriori estimation (QMAP) is the most accurate algorithm for Bayesian network parameter learning under the condition of small data sets. However, when the number of parameter constraints is large or the parameter feasible region is small, the rejection-acceptance sampling process in QMAP algorithm will become extremely time-consuming. In order to improve the learning efficiency of QMAP algorithm and not affect its learning accuracy as much as possible, a new analytical calculation method of the center point of constrained region is designed to replace the original rejection-acceptance sampling calculation method. Firstly, a new objective function is designed, and a constrained objective optimization problem for solving the boundary points of the constrained region is constructed. Secondly, a new optimization engine is used to solve the objective optimization problem, and the boundary points and center points of the constrained region are obtained. Finally, the existing QMAP algorithm is improved by the obtained center points. The simulation results show that the CMAP algorithm proposed in this paper has a slightly worse parameter learning accuracy than the QMAP algorithm, but its computational efficiency is 2-5 times higher than that of the QMAP algorithm.

In Bayesian network parameter learning, it is difficult to obtain accurate parameters when the data are insufficient, and overfitting easily occurs. However, underfitting is prone to happen when the learning results are blindly close to the constraints generated by expert knowledge. A soft-hard constraint parameter learning method is proposed to balance the overfitting and underfitting problems in parameter learning. In this paper, the constraints applied to the parameters are called hard constraints, while those used to the prior are called soft constraints. A partial maximum entropy prior quantization method is proposed for the soft constraint to obtain proper parameters. The equivalent sample threshold is proposed to limit the hyperparameters based on the actual data size for the hard constraint. To combine soft and hard constraints effectively, the soft constraint maximization and hard constraint minimization model is proposed. Experimental results show that this method can significantly improve the accuracy of parameter learning and actually balance overfitting and underfitting. [ABSTRACT FROM AUTHOR]

Collapse is one of the most dangerous aspects of drilling–blasting construction in highway tunnels. To accurately control tunnel-collapse risk, a multistate dynamic Bayesian network (DBN) evaluation method for highway tunnel collapse based on parameter learning was proposed. First, by analyzing the risk mechanism of tunnel construction, the initial BN model was established based on the causal relationship between risk factors and construction risk in hydrogeological conditions, construction technology, and construction management. Next, the construction process was discretized into finite time slices. In consideration of the fuzzy uncertainty of nodes, node polymorphism was introduced to construct a multistate DBN. Then, 50 typical tunnel-collapse cases were taken as sample data, and the conditional probability distribution of initial BN was derived using parameter learning based on the expectation-maximization (EM) algorithm. Using DBN reasoning and sensitivity analysis, the dynamic risk probability and the dominant factors of tunnel collapse were predicted. Finally, the DBN model was fed back with the measured cumulative values and velocity of the crown settlement, which updated the dynamic risk probability assessment results. In analyzing the collapse probability of Jinzhupa tunnel passing through the angular unconformity contact zone as an example, the results demonstrated that dynamic risk assessment results combined with monitoring data could better reflect the reality of construction contingencies, providing real-time risk management guidance. [ABSTRACT FROM AUTHOR]

The small‐scale microphysical processes governing the formation of precipitation particles cannot be resolved explicitly by cloud resolving and climate models. Instead, they are represented by microphysics schemes that are based on a combination of theoretical knowledge, statistical assumptions, and fitting to data ("tuning"). Historically, tuning was done in an ad hoc fashion, leading to parameter choices that are not explainable or repeatable. Recent work has treated it as an inverse problem that can be solved by Bayesian inference. The posterior distribution of the parameters given the data—the solution of Bayesian inference—is found through computationally expensive sampling methods, which require over O105 $\mathcal{O}\left({10}^{5}\right)$ evaluations of the forward model; this is prohibitive for many models. We present a proof of concept of Bayesian learning applied to a new bulk microphysics scheme named "Cloudy," using the recently developed Calibrate‐Emulate‐Sample (CES) algorithm. Cloudy models collision‐coalescence and collisional breakup of cloud droplets with an adjustable number of prognostic moments and with easily modifiable assumptions for the cloud droplet mass distribution and the collision kernel. The CES algorithm uses machine learning tools to accelerate Bayesian inference by reducing the number of forward evaluations needed to O102 $\mathcal{O}\left({10}^{2}\right)$. It also exhibits a smoothing effect when forward evaluations are polluted by noise. In a suite of perfect‐model experiments, we show that CES enables computationally efficient Bayesian inference of parameters in Cloudy from noisy observations of moments of the droplet mass distribution. In an additional imperfect‐model experiment, a collision kernel parameter is successfully learned from output generated by a Lagrangian particle‐based microphysics model. Plain Language Summary: Clouds contain gazillions of cloud droplets, which grow by colliding and sticking together with each other, and eventually they become big enough to fall out as rain. Keeping track of every one of these droplets in weather and climate models is impossible, so the formation of rain has to be represented by simplified models, so‐called "microphysics schemes." These schemes have become a bit like black boxes, with baked‐in statistical assumptions and some empirical parameters whose values are somewhat obscure and not explainable. We show that we can use a method called Bayesian inference to determine the values of these parameters in a way that is both mathematically sound and reasonably fast. The idea of Bayesian inference is to come up with a first guess about the possible values of the parameters, and then to systematically refine that guess using observed data. We apply this method to a new microphysics scheme that we developed and named "Cloudy." To be honest, the data we use for our experiments are not real observations from, say, satellites, but are generated by Cloudy itself. With real observations, the problem becomes hairier, so what we do here is only a proof of concept—but hey, its a start! Key Points: Historically, microphysics schemes were tuned to data in an ad hoc way, resulting in parameter values that are not repeatable or explainableBayesian inference puts uncertainty quantification and parameter learning on solid mathematical grounds, but is computationally expensiveWe present a proof of concept of computationally efficient Bayesian learning applied to a new bulk microphysics scheme called "Cloudy" [ABSTRACT FROM AUTHOR]

Learning the conditional probability table (CPT) parameters of Bayesian networks (BNs) is a key challenge in real-world decision support applications, especially when there are limited data available. The traditional approach to this challenge is introducing domain knowledge/expert judgments that are encoded as qualitative parameter constraints. In this paper, we focus on multiplicative synergistic constraints. The negative multiplicative synergy constraint and positive multiplicative synergy constraint in this paper are symmetric. In order to integrate multiplicative synergistic constraints into the learning process of Bayesian Network parameters, we propose four methods to deal with the multiplicative synergistic constraints based on the idea of classical isotonic regression algorithm. The four methods are simulated by using the lawn moist model and Asia network, and we compared them with the maximum likelihood estimation (MLE) algorithm. Simulation results show that the proposed methods are superior to the MLE algorithm in the accuracy of parameter learning, which can improve the results of the MLE algorithm to obtain more accurate estimators of the parameters. The proposed methods can reduce the dependence of parameter learning on expert experiences. Combining these constraint methods with Bayesian estimation can improve the accuracy of parameter learning under small sample conditions. [ABSTRACT FROM AUTHOR]

Aiming at the problem of low accuracy of Bayesian network (BN) parameter learning results under the condition of small datasets, the necessity of variable weight design of BN parameters under the condition of small datasets is analyzed, and a BN parameter learning algorithm based on variable weight fusion, named VWPL, is proposed. Firstly, the inequality constraints are determined according to the expert experience, the minimum sample data set threshold is calculated and the variable weight factor function that changes with the sample size is designed. Then, the initial parameter set is calculated according to the sample, and the parameter expansion is carried out by the Bootstrap method to obtain the candidate parameter set that meets the constraints, and the final BN parameters can be obtained by substituting them into the BN variable weight parameter calculation model. The experimental results show that when the amount of learning data is small, the learning accuracy of the VWPL algorithm is higher than that of the MLE algorithm, the QMAP algorithm and the fixed-weight learning algorithm. In addition, the VWPL algorithm is successfully applied to the bearing fault diagnosis experiment, which provides a method of BN parameter estimation for small datasets. [ABSTRACT FROM AUTHOR]

The extended belief rule-based system has been used in the field of decision making in recent years for its advantage of expressing various kinds of information under uncertainty, where the extended belief rule base (EBRB) is used to store various types of uncertain knowledge in the form of belief structures. However, as data such as expert knowledge and experimental data is used to directly generate the EBRB, there could be noisy and redundant rules that not only increase the computation cost but also reduce the accuracy. To this end, a novel framework for EBR reduction and training with the greedy strategy and parameter learning is proposed in this paper. Firstly, a greedy-based EBRB reduction method is proposed, where noisy and redundant rules are be searched and removed. Then, the EBRB training method using parameter learning is introduced, where the parameters of the EBRB are trained to increase its accuracy. Next, the framework for EBRB reduction and training is introduced, and the procedure of the proposed method is detailed. Finally, two case studies are conducted to demonstrate the effectiveness and efficiency of the proposed method, and the results show that the proposed method could reduce the size of the EBRB while increasing its accuracy. [ABSTRACT FROM AUTHOR]

In this article, the parameter learning problem is studied for stochastic Boolean networks (SBNs). Both the measure noise and the system noise are assumed to be white and modeled by sequences of Bernoulli distributed stochastic variables which are mutually independent. An algebraic representation of the SBNs is obtained by taking advantage of vector expression of logic variable and applying the semi‐tensor product technique. Consequently, the parameter learning problem is reformulated as an optimization problem that makes it possible to identify the system matrices of SBNs in an efficient computation way. Subsequently, properties of forward and backward probabilities are investigated, and the EM algorithm is utilized to learn the model parameters from time series data. Finally, a numerical experiment is presented to show the usefulness of the designed parameter learning algorithm. [ABSTRACT FROM AUTHOR]

In this article, we study closed-loop digital predistortion (DPD) systems and associated learning algorithms. Specifically, we propose various low-complexity approaches to estimate and manipulate the inverse of the input data covariance matrix (CM) and combine them with the so-called self-orthogonalized (SO) learning rule. The inherent simplicity of the SO algorithm, combined with the proposed solutions, allows for remarkably reduced complexity in the DPD system while maintaining similar linearization performance compared to other state-of-the-art methods. This is demonstrated with thorough over-the-air (OTA) mmW measurement results at 28 GHz, incorporating a state-of-the-art 64-element active antenna array, and very wide channel bandwidths up to 800 MHz. In addition, complexity analyses are carried out, which together with the measured linearization performance demonstrates favorable performance–complexity tradeoffs in linearizing mmW active array transmitters through the proposed solutions. The techniques can find application in systems where the power amplifier (PA) nonlinearities are time-varying and thus frequent or even constant updating of the DPD is required, good examples being mmW adaptive antenna arrays as well as terminal transmitters in 5G and beyond networks. [ABSTRACT FROM AUTHOR]

Modeling a complex system with a large number of variables at one time and collecting global data on a large scale is usually not practical. It is ideal to model the global system according to only local data and structures, and then synthesize them together. Multiple local structures can be easily synthesized as a global structure through fusing same nodes, while multiple sets of local data cannot be combined as the global data. Therefore, we need a method to obtain the joint probability distribution (JPD) of the global system by utilizing local data only. When the global structure synthesized does not include directed cyclic graph (DCG), it is easy to calculate the JPD with Bayesian network (BN). When DCGs are included, BN does not work. This paper presents the multi-valued Dynamic Uncertain Causality Graph (M-DUCG) methodology to calculate the global JPD in the case of DCGs with only local data. The idea is to use multiple sets of local data to learn the parameters of the synthesized M-DUCG structure with DCGs, and then use these parameters to calculate the global JPD.

This paper applies the variational methods to learn the parameters and the probability of evidence of directed graphic models (also known as Bayesian networks (BNs)) when data contains missing values. One class of variational methods, the Bethe/Kikuchi approximate algorithm, is combined with Expectation-Maximization (EM) to learn BN model parameters from data using an upper limit on cluster size. Our proposed method is novel since the Bethe/Kikuchi algorithms are typically only used to approximate marginal distributions. We review popular Bethe/Kikuchi approximate algorithms, including Belief Propagation (BP), Iterative Join Graph Propagation (IJGP), and Fundamental Cycle Base (FCB) algorithms for discrete BNs, and conduct experiments to test their learning performance using a wide range of discrete BNs in practice. Experiments show that by using IJGP or FCB algorithms, with EM, we can obtain more accurate results than the conventional approximate algorithms, such as Evidence Pre-propagation Importance Sampling and Variational Inference. When exact methods are intractable the IJGP/FCB with EM can be an effective alternative solution.

This paper proposes a new mutation operator current−to−better/1 to keep the population diversity. Under the control of current−to−better/1, a high-ranking individual with a larger fitness value tend to search its nearby area, while the low-ranking individual is stimulated to move towards high-ranking comity in a not fixed direction. A learning matrix is set to store successful amplification factors with their corresponding individual vectors, and then to update the amplification factors matrix for each generation. To test the effectiveness of current−to−better/1, it is embedded into JADE as a collaborator of original mutation operator DE/current−to−pbest/1. The efficiency of this new algorithm, namely, L-JADE, is proved by the performance comparisons with other already known DE-variant algorithms on 2-D, 3-D, 5-D, 10-D, and 20-D COCO testbed. Experimental results show that L-JADE performs better than most algorithms.

Influence nets (INs) are proposed on the basis of causal logic for the purpose of depicting causal relationship strengths in complex systems. However, it is difficult to accurately determine the causal strength (CAST) parameters on the basis of expert knowledge only, particularly for practical problems with multiple attributes and multiple states. In this paper, the original IN with binary states is first extended into a multi-state IN, endowing the IN with the ability to model and infer under multiple states. Based on this method, an optimization framework is proposed to optimize the CAST parameters of multi-state INs. Finally, two practical cases are studied to verify the feasibility and efficiency of the proposed multi-state IN with an optimization framework in regression and classification. [ABSTRACT FROM AUTHOR]

The sensor-based human activity recognition has been wildly applied in behavior tracking, health monitoring, indoor localization etc. Using activity continuity to assist activity recognition is an important research issue, in which the activity transition matrix which describes the activity transformation in real scenarios is the most important parameter. Aiming at the problem that the current classic transition matrix learning algorithm cannot fuse weights of sample classification results, a weighted transition matrix learning algorithm is proposed in this paper. First, the basic definitions of an improved Hidden Markov Model (HMM) which fuses weights of classification results are given. Then, the recursive formula of transition matrix learning is derived, and the learning algorithm W-Trans is put forward. Finally, the proposed algorithm is simulated with the public data sets. The evaluation results show that the proposed algorithm outperforms the classical Baum-Welch algorithm under evaluation metrics of both the cosine similarity and the euler distance. By applying W-Trans to current activity recognition post-process methods, the advantage of our method is verified.

Recent advances have illustrated substantial benefits from learning Bayesian networks (BNs). However, when the available data size is small, the BN parameter learning becomes a key challenge in many intelligent applications. By integrating both sample data and expert constraints, we propose a BN parameter learning algorithm with extension method-parameter extension under constraints (PEUC) by introducing related domain expert knowledge. Knowledge is transformed into inequality constraints which candidate parameter sets arise from the relative constraints space.The maximum entropy principle helps to estimate the parameter in statistical averaging model while candidate sets of BN parameters satisfy the constrained knowledge by bootstrapping techniques. Then BN parameters are estimated based on the real available sampled data set and extension streams of candidate parameters samples from the constraints space. The sample size is also taken into account according to the contribution to the final parameters. Experimental results of benchmark BN modeling problems demonstrate that PEUC algorithm tends to the classical MLE algorithm when the modeling data size is sufficient. Furthermore, when the available data size is small, the parameters of BN can be estimated by PEUC as well, and the learned accuracy is superior to MLE, MAP or QMAP algorithm. Finally, PEUC is also applied to a real bearing fault diagnosis case. The presented approach provides a new promising BN parameter learning way for more intelligent system modeling problems, particularly when the data sets are small.

The classical belief rule-based (BRB) systems are usually constructed by arranging and combining referential values of antecedent attributes or by setting special fixed values, which can lead to overly large size of BRB systems in complex problems. This paper combines the decision tree classification method to analyze the information of data and extract the rules. Based on this, a new rule representation method with referential interval is proposed and the rule base is constructed according to the support degree and belief degree of the data. In the newly proposed method, the introduction of decision tree ensures that the size of the rule base is reasonable. Moreover, the rule parameters trained by the differential evolution (DE) algorithm are optimized and adjusted to further improve the system performance. The experiments are conducted on several commonly used public classification datasets. And the proposed algorithm can achieve better accuracy results compared with classical classification methods and the existing classification methods of BRB systems on average. The experimental results validate the reasonableness and effectiveness of the BRB construction method proposed in this paper.

Directed graphical models aim to represent the probabilistic relationships between variables in a system. Learning a directed graphical model from data includes parameter learning and structure learning. Several methods have been developed for directed graphical models with scalar variables. However, the case in which the variables are infinite-dimensional has not been studied thoroughly. Nowadays, in many applications, the variables are infinite-dimensional signals that need to be treated as functional random variables. This article proposes a novel method to learn directed graphical models in the functional setting. When the structure of the graph is known, function-to-function linear regression is used to estimate the parameters of the graph. When the goal is to learn the structure, a penalized least square loss function with a group LASSO penalty, for variable selection, and an L2 penalty, to handle group selection of nodes, is defined. Cyclic coordinate accelerated proximal gradient descent algorithm is employed to minimize the loss function and learn the structure of the directed graph. Through simulations and a case study, the advantage of the proposed method is proven. [ABSTRACT FROM AUTHOR]

Bayesian optimization (BO) is a sample-efficient global optimization algorithm for black-box functions which are expensive to evaluate. Existing literature on model based optimization in conditional parameter spaces are usually built on trees. In this work, we generalize the additive assumption to tree-structured functions and propose an additive tree-structured covariance function, showing improved sample-efficiency, wider applicability and greater flexibility. Furthermore, by incorporating the structure information of parameter spaces and the additive assumption in the BO loop, we develop a parallel algorithm to optimize the acquisition function and this optimization can be performed in a low dimensional space. We demonstrate our method on an optimization benchmark function, on a neural network compression problem and on pruning pre-trained VGG16 and ResNet50 models. Experimental results show our approach significantly outperforms the current state of the art for conditional parameter optimization including SMAC, TPE and Jenatton et al. (2017). [ABSTRACT FROM AUTHOR]

High-accuracy parameter learning in Bayesian Networks (BNs) is a key challenge in real-time decision support applications, particularly when the available data are limited. Prior/Expert knowledge was introduced to eliminate the drawbacks of insufficient information; however, this method is subjective. In this study, we explored the use of monotonicity constraints to control the causal relationships between the nodes and their parents-nodes in BNs and proposed a new learning algorithm called global domain monotonicity based on maximum information entropy (GDM-MIE), which was designed for parameter learning in uncertain discrete BNs with nonlinear equality constraints when only finite data are available. In the proposed algorithm, a class of monotonicity is encoded as a constraint on information entropy and the parameter learning problem is transformed into constraints among the node parameters based on a known network. Furthermore, we considered the parameters as uncertain entropy information and discussed the monotonicity among parameters in the global spatial domain, proving the accuracy of the logical relationships of the model, system reliability, and time complexity. Finally, the proposed method was validated using standard BNs, and its performance was analyzed by comparing the proposed method with the existing learning algorithms. The results showed that the proposed method is more accurate and has better Kullback–Leibler divergence. To revalidate the rationality of the proposed method, the Alarm and Asia networks were employed as special cases. The GDM-MIE was found to achieve the intended goal of observing the estimated parameters by closely approximating the original real parameters with a small sample size, indicating that the proposed algorithm can served as an efficient and feasible method for learning Bayesian parameter. [ABSTRACT FROM AUTHOR]

Learning the conditional probability table (CPT) parameters of Bayesian networks (BNs) is a key challenge in real-world decision support applications, especially when there are limited data available. The traditional approach to this challenge is introducing domain knowledge/expert judgments that are encoded as qualitative parameter constraints. In this paper, we focus on multiplicative synergistic constraints. The negative multiplicative synergy constraint and positive multiplicative synergy constraint in this paper are symmetric. In order to integrate multiplicative synergistic constraints into the learning process of Bayesian Network parameters, we propose four methods to deal with the multiplicative synergistic constraints based on the idea of classical isotonic regression algorithm. The four methods are simulated by using the lawn moist model and Asia network, and we compared them with the maximum likelihood estimation (MLE) algorithm. Simulation results show that the proposed methods are superior to the MLE algorithm in the accuracy of parameter learning, which can improve the results of the MLE algorithm to obtain more accurate estimators of the parameters. The proposed methods can reduce the dependence of parameter learning on expert experiences. Combining these constraint methods with Bayesian estimation can improve the accuracy of parameter learning under small sample conditions.

Relational data is structured and, in the real world, ambiguous. Logic can handle relations and probability can handle uncertainty. A probabilistic logic approach to learning can handle both relational structure and uncertainty in the data. Probabilistic logic approach works well with relational data. Incorporating domain knowledge in probabilistic logic approach further enhances learning, improving accuracy. A number of statistical techniques carry out predictive analytics based on historical data alone. Soccer, however, is a team game and the outcome of a soccer game depends on how well the team together and the players play against the opponent team. Thus, data about soccer games are better represented in relational form. In the present work, we propose to learn from soccer match data to predict their outcomes. We learn a model for the prediction of soccer game outcomes, taking into account the history of the matches played by the teams. We frame the background knowledge as rules in the logic program to enhance the prediction. Compared to the traditional machine learning approaches to soccer game outcome prediction, probabilistic logic approach is found to result in significant improvement in prediction accuracy. [ABSTRACT FROM AUTHOR]

In order to solve the problem that there are many restrictions on the source domain and the target domain in the process of Bayesian network parameter transfer， a unified framework based on Bayesian network parameter transfer learning was proposed under the condition of considering multiple information forms of source domain and target domain. The method considers the role of source domain structure and data volume in the migration. On the basis of structural similarity， the influence of alternative source domain data volume on parameter migration was discussed. The balance coefficient related to the target domain data was introduced in the migration process. According to the balance coefficient， the target domain data was linked with the migration process to realize the automatic adjustment of the balance coefficient. The Asia network verifies the accuracy of the method in this paper. [ABSTRACT FROM AUTHOR]

This study investigates the performance of four machine learning (ML) algorithms to evaluate the earthquake-induced liquefaction potential of soil based on the cone penetration test field case history records using the Bayesian belief network (BBN) learning software Netica. The BBN structures that were developed by ML algorithms-K2, hill climbing (HC), tree augmented naive (TAN) Bayes, and Tabu search were adopted to perform parameter learning in Netica, thereby fixing the BBN models. The performance measure indexes, namely, overall accuracy (OA), precision, recall, F-measure, and area under the receiver operating characteristic curve, were used to evaluate the training and testing BBN models' performance and highlight the capability of the K2 and TAN Bayes models over the Tabu search and HC models. The sensitivity analysis results showed that the cone tip resistance and vertical effective stress are the most sensitive factors, whereas the mean grain size is the least sensitive factor in the prediction of seismic soil liquefaction potential. The results of this study can provide theoretical support for researchers in selecting appropriate ML algorithms and improving the predictive performance of seismic soil liquefaction potential models. [ABSTRACT FROM AUTHOR]

Model-based approaches for target tracking and smoothing estimate the infinite number of possible target trajectories using a finite set of models. This article proposes a data-driven approach that represents the possible target trajectories using a distribution over an infinite number of functions. Recursive Gaussian process, and derivative-based Gaussian process approaches for target tracking, and smoothing are developed, with online training, and parameter learning. The performance evaluation over two highly maneuvering scenarios, shows that the proposed approach provides 80 and 62% performance improvement in the position, and 49 and 22% in the velocity estimation, respectively, as compared to the best model-based filter. [ABSTRACT FROM AUTHOR]