Search

The article presents a new approach to modeling nonlinear dependencies called composite Bayesian networks. The main emphasis is on integrating machine learning models into Bayesian networks while maintaining their fundamental principles. The novelty of the approach is that it allows us to solve the problem of data inconsistency with traditional assumptions about dependencies. The presented method consists in selecting a variety of machine learning models at the stage of training composite Bayesian networks. This allows you to flexibly customize the nature of the dependencies in accordance with the requirements and dictated characteristics of the modeled object. The software implementation is made in the form of a specialized framework that describes all the necessary functionality. The results of experiments to evaluate the effectiveness of modeling dependencies between features are presented. Data for the experiments was taken from the bnlearn repository for benchmarks and from the UCI repository for real data. The performance of composite Bayesian networks was validated by comparing the likelihood and F1 score with classical Bayesian networks trained with the Hill-Climbing algorithm, demonstrating high accuracy in representing multivariate distributions. The improvement in benchmarks is insignificant since they contain linear dependencies that are well modeled by the classical algorithm. An average 30 % improvement in likelihood was obtained on real UCI datasets. The obtained data can be applied in areas that require modeling complex dependencies between features, for example, in machine learning, statistics, data analysis, as well as in specific subject areas.

Federated learning makes it possible to train a machine learning model on decentralized data. Bayesian networks are widely used probabilistic graphical models. While some research has been published on the federated learning of Bayesian networks, publications on Bayesian networks in a vertically partitioned data setting are limited, with important omissions, such as handling missing data. We propose a novel method called VertiBayes to train Bayesian networks (structure and parameters) on vertically partitioned data, which can handle missing values as well as an arbitrary number of parties. For structure learning we adapted the K2 algorithm with a privacy-preserving scalar product protocol. For parameter learning, we use a two-step approach: first, we learn an intermediate model using maximum likelihood, treating missing values as a special value, then we train a model on synthetic data generated by the intermediate model using the EM algorithm. The privacy guarantees of VertiBayes are equivalent to those provided by the privacy preserving scalar product protocol used. We experimentally show VertiBayes produces models comparable to those learnt using traditional algorithms. Finally, we propose two alternative approaches to estimate the performance of the model using vertically partitioned data and we show in experiments that these give accurate estimates. [ABSTRACT FROM AUTHOR]

Summary In this article, the parameter learning scheme for the multi‐input multi‐output (MIMO) Hammerstein nonlinear systems under measurement noises is studied, which is derived by exploiting the correlation analysis and data filtering technique. The coupled MIMO Hammerstein system presented involves a static nonlinear subsystem modeled by neural fuzzy model (NFM), and a dynamic linear subsystem established by autoregressive moving average with extra input (ARMAX) model. To learn the unknown parameter of the MIMO Hammerstein system, the combined signals are designed to realize that identification of the nonlinear subsystem is separated from that of linear subsystem. First, the correlation properties of separable signals in a nonlinear system are analyzed, then the parameters of the linear subsystem are estimated utilizing correlation analysis, which can deal with the issue of unmeasured intermediate variable in the Hammerstein system. Second, the data filtering technique is introduced to derive the data filtering‐based recursive least squares technique for learning the nonlinear subsystem parameter, which can reduce the impact of the moving average noise and improve the precision of parameter estimation. Finally, the effectiveness and feasibility of the proposed identification scheme is proved by numerical simulation and nonlinear pH process. [ABSTRACT FROM AUTHOR]

Abstract Federated learning makes it possible to train a machine learning model on decentralized data. Bayesian networks are widely used probabilistic graphical models. While some research has been published on the federated learning of Bayesian networks, publications on Bayesian networks in a vertically partitioned data setting are limited, with important omissions, such as handling missing data. We propose a novel method called VertiBayes to train Bayesian networks (structure and parameters) on vertically partitioned data, which can handle missing values as well as an arbitrary number of parties. For structure learning we adapted the K2 algorithm with a privacy-preserving scalar product protocol. For parameter learning, we use a two-step approach: first, we learn an intermediate model using maximum likelihood, treating missing values as a special value, then we train a model on synthetic data generated by the intermediate model using the EM algorithm. The privacy guarantees of VertiBayes are equivalent to those provided by the privacy preserving scalar product protocol used. We experimentally show VertiBayes produces models comparable to those learnt using traditional algorithms. Finally, we propose two alternative approaches to estimate the performance of the model using vertically partitioned data and we show in experiments that these give accurate estimates.

Abstract With the acceleration of global climate change and urbanization, disaster chains are always connected to artificial systems like critical infrastructure. The complexity and uncertainty of the disaster chain development process and the severity of the consequences have brought great challenges to emergency decision makers. The Bayesian network (BN) was applied in this study to reason about disaster chain scenarios to support the choice of appropriate response strategies. To capture the interacting relationships among different factors, a scenario representation model of disaster chains was developed, followed by the determination of the BN structure. In deriving the conditional probability tables of the BN model, we found that, due to the lack of data and the significant uncertainty of disaster chains, parameter learning methodologies based on data or expert knowledge alone are insufficient. By integrating both sample data and expert knowledge with the maximum entropy principle, we proposed a parameter estimation algorithm under expert prior knowledge (PEUK). Taking the rainstorm disaster chain as an example, we demonstrated the superiority of the PEUK-built BN model over the traditional maximum a posterior (MAP) algorithm and the direct expert opinion elicitation method. The results also demonstrate the potential of our BN scenario reasoning paradigm to assist real-world disaster decisions.

When dealing with engineering design problems, designers often encounter nonlinear and nonconvex features, multiple objectives, coupled decision making, and various levels of fidelity of sub-systems. To realize the design with limited computational resources, problems with the features above need to be linearized and then solved using solution algorithms for linear programming. The adaptive linear programming (ALP) algorithm is an extension of the Sequential Linear Programming algorithm where a nonlinear compromise decision support problem (cDSP) is iteratively linearized, and the resulting linear programming problem is solved with satisficing solutions returned. The reduced move coefficient (RMC) is used to define how far away from the boundary the next linearization is to be performed, and currently, it is determined based on a heuristic. The choice of RMC significantly affects the efficacy of the linearization process and, hence, the rapidity of finding the solution. In this paper, we propose a rule-based parameter-learning procedure to vary the RMC at each iteration, thereby significantly increasing the speed of determining the ultimate solution. To demonstrate the efficacy of the ALP algorithm with parameter learning (ALPPL), we use an industry-inspired problem, namely, the integrated design of a hot-rolling process chain for the production of a steel rod. Using the proposed ALPPL, we can incorporate domain expertise to identify the most relevant criteria to evaluate the performance of the linearization algorithm, quantify the criteria as evaluation indices, and tune the RMC to return the solutions that fall into the most desired range of each evaluation index. Compared with the old ALP algorithm using the golden section search to update the RMC, the ALPPL improves the algorithm by identifying the RMC values with better linearization performance without adding computational complexity. The insensitive region of the RMC is better explored using the ALPPL—the ALP only explores the insensitive region twice, whereas the ALPPL explores four times throughout the iterations. With ALPPL, we have a more comprehensive definition of linearization performance—given multiple design scenarios, using evaluation indices (EIs) including the statistics of deviations, the numbers of binding (active) constraints and bounds, the numbers of accumulated linear constraints, and the number of iterations. The desired range of evaluation indices (DEI) is also learned during the iterations. The RMC value that brings the most EIs into the DEI is returned as the best RMC, which ensures a balance between the accuracy of the linearization and the robustness of the solutions. For our test problem, the hot-rolling process chain, the ALP returns the best RMC in twelve iterations considering only the deviation as the linearization performance index, whereas the ALPPL returns the best RMC in fourteen iterations considering multiple EIs. The complexity of both the ALP and the ALPPL is O(n2). The parameter-learning steps can be customized to improve the parameter determination of other algorithms. [ABSTRACT FROM AUTHOR]

The connection between numerical methods for solving differential equations and machine learning has been revealed recently. Differential equations have been proposed as continuous analogues of deep neural networks, and then used in handling certain tasks, such as image recognition, where the training of a model includes learning the parameters of systems of ODEs from certain points along their trajectories. Treating this inverse problem of determining the parameters of a dynamical system that minimize the difference between data and trajectory by a gradient-based optimization method presents the solution of the adjoint equation as the continuous analogue of backpropagation that yields the appropriate gradients. The paper explores an abstract approach that can be used to construct a family of loss functions with the aim of fitting the solution of an initial value problem to a set of discrete or continuous measurements. It is shown, that an extension of the adjoint equation can be used to derive the gradient of the loss function as a continuous analogue of backpropagation in machine learning. Numerical evidence is presented that under reasonably controlled circumstances the gradients obtained this way can be used in a gradient descent to fit the solution of an initial value problem to a set of continuous noisy measurements, and a set of discrete noisy measurements that are recorded at uncertain times. [ABSTRACT FROM AUTHOR]

In this paper, a new and effective parameter learning method for delayed Boolean control networks (DBCNs) with missing observations is presented in this paper, which can handle strong noise and converge to the true parameter values in finite times. Two types of noise that follow different time-varying Bernoulli distributions are considered in this paper: measurement noise and system noise. The concept of virtual nodes is introduced by the paper to cope with random missing observations, and the algebraic representation of DBCNs is obtained using the STP method. A parameter learning algorithm based on the forward and backward probability and EM algorithm is then proposed by the paper, which can estimate the model parameters from time series data. The performance and robustness of the proposed method are demonstrated by the paper through a numerical example.

When dealing with engineering design problems, designers often encounter nonlinear and nonconvex features, multiple objectives, coupled decision making, and various levels of fidelity of sub-systems. To realize the design with limited computational resources, problems with the features above need to be linearized and then solved using solution algorithms for linear programming. The adaptive linear programming (ALP) algorithm is an extension of the Sequential Linear Programming algorithm where a nonlinear compromise decision support problem (cDSP) is iteratively linearized, and the resulting linear programming problem is solved with satisficing solutions returned. The reduced move coefficient (RMC) is used to define how far away from the boundary the next linearization is to be performed, and currently, it is determined based on a heuristic. The choice of RMC significantly affects the efficacy of the linearization process and, hence, the rapidity of finding the solution. In this paper, we propose a rule-based parameter-learning procedure to vary the RMC at each iteration, thereby significantly increasing the speed of determining the ultimate solution. To demonstrate the efficacy of the ALP algorithm with parameter learning (ALPPL), we use an industry-inspired problem, namely, the integrated design of a hot-rolling process chain for the production of a steel rod. Using the proposed ALPPL, we can incorporate domain expertise to identify the most relevant criteria to evaluate the performance of the linearization algorithm, quantify the criteria as evaluation indices, and tune the RMC to return the solutions that fall into the most desired range of each evaluation index. Compared with the old ALP algorithm using the golden section search to update the RMC, the ALPPL improves the algorithm by identifying the RMC values with better linearization performance without adding computational complexity. The insensitive region of the RMC is better explored using the ALPPL—the ALP only explores the insensitive region twice, whereas the ALPPL explores four times throughout the iterations. With ALPPL, we have a more comprehensive definition of linearization performance—given multiple design scenarios, using evaluation indices (EIs) including the statistics of deviations, the numbers of binding (active) constraints and bounds, the numbers of accumulated linear constraints, and the number of iterations. The desired range of evaluation indices (DEI) is also learned during the iterations. The RMC value that brings the most EIs into the DEI is returned as the best RMC, which ensures a balance between the accuracy of the linearization and the robustness of the solutions. For our test problem, the hot-rolling process chain, the ALP returns the best RMC in twelve iterations considering only the deviation as the linearization performance index, whereas the ALPPL returns the best RMC in fourteen iterations considering multiple EIs. The complexity of both the ALP and the ALPPL is O(n2). The parameter-learning steps can be customized to improve the parameter determination of other algorithms.

High-accuracy parameter learning in Bayesian Networks (BNs) is a key challenge in real-time decision support applications, particularly when the available data are limited. Prior/Expert knowledge was introduced to eliminate the drawbacks of insufficient information; however, this method is subjective. In this study, we explored the use of monotonicity constraints to control the causal relationships between the nodes and their parents-nodes in BNs and proposed a new learning algorithm called global domain monotonicity based on maximum information entropy (GDM-MIE), which was designed for parameter learning in uncertain discrete BNs with nonlinear equality constraints when only finite data are available. In the proposed algorithm, a class of monotonicity is encoded as a constraint on information entropy and the parameter learning problem is transformed into constraints among the node parameters based on a known network. Furthermore, we considered the parameters as uncertain entropy information and discussed the monotonicity among parameters in the global spatial domain, proving the accuracy of the logical relationships of the model, system reliability, and time complexity. Finally, the proposed method was validated using standard BNs, and its performance was analyzed by comparing the proposed method with the existing learning algorithms. The results showed that the proposed method is more accurate and has better Kullback–Leibler divergence. To revalidate the rationality of the proposed method, the Alarm and Asia networks were employed as special cases. The GDM-MIE was found to achieve the intended goal of observing the estimated parameters by closely approximating the original real parameters with a small sample size, indicating that the proposed algorithm can served as an efficient and feasible method for learning Bayesian parameter. [ABSTRACT FROM AUTHOR]

[Display omitted] Very often the performance of a Bayesian Network (BN) is affected when applied to a new target population. This is mainly because of differences in population characteristics. External validation of the model performance on different populations is a standard approach to test model's generalisability. However, a good predictive performance is not enough to show that the model represents the unique population characteristics and can be adopted in the new environment. In this paper, we present a methodology for updating and recalibrating developed BN models – both their structure and parameters - to better account for the characteristics of the target population. Attention has been given on incorporating expert knowledge and recalibrating latent variables, which are usually omitted from data-driven models. The method is successfully applied to a clinical case study about the prediction of trauma-induced coagulopathy, where a BN has already been developed for civilian trauma patients and now it is recalibrated on combat casualties. The methodology proposed in this study is important for developing credible models that can demonstrate a good predictive performance when applied to a target population. Another advantage of the proposed methodology is that it is not limited to data-driven techniques and shows how expert knowledge can also be used when updating and recalibrating the model. [ABSTRACT FROM AUTHOR]