Search

• Proposing physics-informed DCNN for HFI-SCB temperature field reconstruction. • Proposing Deep MC-QR method to quantify data uncertainty caused by noise. • Proposing HFI-SCB reliability analysis method based on interval multilevel BN. Temperature field reconstruction is essential for analyzing the reliability of a high-density functionally integrated satellite circuit board (HFI-SCB). As a representative deep learning model, the deep convolutional neural network (DCNN) is a powerful tool for reconstructing the HFI-SCB temperature field. However, DCNN needs a lot of labeled data to learn its parameters, which is contrary to the fact that actual satellite engineering can only acquire noisy unlabeled data. Thus, this paper proposes an unsupervised method, i.e., the physics-informed deep Monte Carlo quantile regression method, for reconstructing the HFI-SCB temperature field and quantifying the data uncertainty caused by sensor noise. The proposed method combines a DCNN with known physics knowledge to reconstruct an accurate HFI-SCB temperature field using only monitoring point temperatures. Besides, the proposed method can quantify the data uncertainty by the Monte Carlo quantile regression. Based on the reconstructed temperature field and the quantified data uncertainty, this paper builds an interval multilevel Bayesian Network to analyze the HFI-SCB reliability. Two case studies are used to validate the proposed method. [ABSTRACT FROM AUTHOR]

Modeling of large and complex industrial processes such as chemical processes has always been an ongoing research topic. Hybrid modeling approaches draw more and more attentions due to its ability to combine the benefits of both mechanism(first-principle) models and data-based models in the term of the balance between accuracy and cost. However, there is an insurmountable problem in the application of hybrid modeling, that is, when the accurate formulated mechanism information of the system is difficult to obtain, the hybrid modeling strategy is often difficult to obtain the desired results in existing hybrid modeling strategies. In order to solve the contradiction between the need for accurate formulated mechanism and the reality that obtaining them in a large actual process is difficult or even impossible, we utilized the directed graph and the method of Bayesian Neural Network(BNN)-Structure Learning to combine the mechanism information and data information into a hybrid information graph, which is called BNN-Data Augmented Graph(BNN-DAG) to form a hybrid model without accurate first-principles formulas. On the basis of the hybrid information graph, the system is first partitioned into small blocks according to the demand of the target variables, then an appropriate modeling method is utilized based on the requirements of the object. Then, the selected modeling method is implemented into blocks and contributes to a complete predictive model, so that the complex system can still be modeled with a certain accuracy without the mechanism information expressed by accurate formulas. The method in this manuscript is validated on a Tennessee Eastman(TE) benchmark process and a Hydrogenation Fractionation Unit(HFU) in a real plant. The prediction results are compared with the currently popular deep learning methods. It is proved that under the framework of the method in this manuscript, it is possible to obtain a system prediction model with the same or higher accuracy as the existing deep learning methods but with a relatively a low cost. Under the framework, the interpretability of the prediction model could be obtained clearly. Also, the improvement is beneficial to the application in the actual industrial process. • This paper proposes an enhanced Bayesian neural network structure learning strategy to address the hybrid modeling of complex systems without formalized mechanistic information. • In order to solve the problem of representing the non-formulated mechanism information of complex systems, directed graph networks are introduced to represent the mechanism information of the system. • Different from the fixed model structure in the previous hybrid modeling strategy, the model structure in the strategy proposed in this paper can be adjusted according to actual requirements, so as to obtain a reasonable balance of model complexity and accuracy. • A real chemical plant is used to verify the method proposed in this paper. The results show that the method can guarantee a certain accuracy and reduce the computational burden. [ABSTRACT FROM AUTHOR]

Bayesian Neural Networks provide a tool to estimate the uncertainty of a neural network by considering a distribution over weights and sampling different models for each input. In this paper, we propose a method for uncertainty estimation in neural networks which, instead of considering a distribution over weights, samples outputs of each layer from a corresponding Gaussian distribution, parametrized by the predictions of mean and variance sub-layers. In uncertainty quality estimation experiments, we show that the proposed method achieves better uncertainty quality than other single-bin Bayesian Model Averaging methods, such as Monte Carlo Dropout or Bayes By Backpropagation methods. [ABSTRACT FROM AUTHOR]

To enhance RUL prediction accuracy and uncertainty quantification, numerous methods have been developed, including model-based, data-driven, and hybrid approaches. However, model-based approaches struggle with complex relationships and uncertainties. Data-driven methods might overlook prior knowledge and struggle with limited data. Hybrid models for RUL prediction face two key challenges: inadequate use of physical information and difficulty in accurately quantifying uncertainty. Aiming at the problems of non-linearity, small sample sizes, and the absence of uncertainty quantification in RUL prediction, this paper introduces a hybrid method aimed at achieving RUL prediction and uncertainty quantification in few-slot scenarios. In this study, a hybrid approach that combines model-based approach and data-driven approach is proposed to achieve accurate RUL prediction. The uncertainty is measured based on a Bayesian framework. Specifically, the proposed method combines the degradation trend model using a double-exponential degradation model (DEDM), and the degradation fluctuations is predicted by a Gated Recurrent Unit (GRU) network to achieve point estimation of the RUL. Subsequently, an ensemble learning method is adopted to integrate the different modules using a Bayesian neural network (BNN) for uncertainty quantification. The applicability and effectiveness of the proposed method is investigated through case studies conducted on the three lithium battery datasets. The comparative analyses are also conducted with commonly used EMD-based approaches. Experimental results demonstrate that the proposed method has good performance in both data leakage and non-leakage scenarios and is more effective than individual methods to achieve accurate RUL prediction under small sample datasets. Finally, this study dedicates to integrate physical models and data-driven models to address the challenge of data drift in future research. [ABSTRACT FROM AUTHOR]

Deep neural networks exhibit extreme superiority in natural language processing and computer vision. With the fast development of 3D sensing technology, point cloud has been wildly used in robotics and autonomous driving. In recent years, several attempts have been made to utilize deep neural networks in the field of point cloud processing. However, most top performing network cannot provide the confidence of each prediction. In this paper, we implement Bayesian deep learning method in point cloud processing and propose ProbConv, a three-dimensional convolutional kernel with stochastic weights. Based on ProbConv, a Bayesian deep neural network named ProbNet is further designed to effectively accomplish classification and segmentation tasks on point cloud data. Since Bayesian neural network can naturally calculate the confidence of its prediction, ProbNet is able to provide the confidence of the segmentation result. The experimental results on ModelNet40, ShapeNet and S3DIS demonstrate that ProbNet improves accuracy in both object classification and semantic segmentation tasks. The confidence provided by the ProbNet is reliable to reflect the accuracy of its prediction. [Display omitted] • Bayesian deep learning and pyramid-divided respective field is implemented on ProbConv. • ProbNet is based on the proposed ProbConv operator to provide confidence of its prediction. • Experiment result shows that ProbNet improves accuracy while providing reliable confidence. [ABSTRACT FROM AUTHOR]

In this paper, three state-of-the-art deep learning uncertainty quantification (UQ) methods – Flipout probabilistic convolutional neural network (CNN), deep ensemble probabilistic CNN, and Bayesian probabilistic CNN – based on the Visual Geometry Group 13 architecture are proposed. They are compared with a traditional Bayesian inference approach for localizing delamination damage in composite. The law of conditional covariance is used to separate and quantify the predictive variance of the three networks into aleatoric and epistemic uncertainty. The network models' performance is enhanced through hyperparameter optimization using Hyperband and warm-up optimization algorithms. The performance of the three networks in measuring the uncertainty is assessed on an out-of-distribution (OOD) dataset and validated on an in-distribution (ID) dataset for localization of composite delamination damage. Results indicate high accuracy in predicting damage locations for all methods on the ID dataset. On the OOD dataset, the Flipout and deep ensemble network have better performance, stably measuring aleatoric uncertainty in both trained and untrained areas, while the Bayesian network's aleatoric uncertainty shows a discernible change across both areas. All three networks effectively measure epistemic uncertainty. Overall in both ID and OOD datasets, the Flipout network provides an optimal balance among training efficiency, UQ effectiveness and accuracy in predicting damage locations. [ABSTRACT FROM AUTHOR]

Accurate traffic congestion estimation and prediction are critical building blocks for smart trip planning and rerouting decisions in transportation systems. Over the decades, there have been many studies focusing on traffic congestion estimation and prediction with different statistical approaches (e.g., Markov chain) and machine learning models (e.g., clustering, Bayesian networks, and artificial neural networks). However, there is a lack of a unified framework to address the mechanisms of different models and integrate the advantages of different methods through combinations. This paper introduces the FD-Markov-LSTM model, a hybrid interpretable approach that combines the fundamental diagram (FD), Markov chain, and long short-term memory (LSTM). The aim is to estimate and predict traffic states by integrating statistical data in both congested and uncongested scenarios. The FD-Markov-LSTM model leverages the FD to identify hierarchical traffic states and utilizes the Markov process to capture the probabilistic transitions between these states. We employ the LSTM model to further capture the residual time series produced by the Markov chain model (assuming a memoryless property) to enhance the estimation and prediction performance. The proposed model's accuracy in estimating and predicting traffic flow is evaluated using empirical data from three case studies conducted in Beijing and Los Angeles. The results highlight a significant improvement in accuracy compared to classical benchmark models such as the Markov model, ARIMA model, k-Nearest Neighbor model, Random Forest model, and LSTM. Specifically, the FD-Markov-LSTM model achieves reductions of over 39% in mean absolute error, 35% in root mean squared error, and 7.4% in mean absolute percentage error. These results clearly demonstrate that the FD-Markov-LSTM model outperforms the benchmark models, enabling more precise predictions of traffic flow. [ABSTRACT FROM AUTHOR]

As Deep Neural Networks (DNN) make their way into safety-critical decision processes, it becomes imperative to have robust and reliable uncertainty estimates for their predictions for both in-distribution and out-of-distribution (OOD) examples. This is particularly important in real-life high-risk settings such as healthcare, where OOD examples (e.g., patients with previously unseen or rare labels, i.e., diagnoses) are frequent, and an incorrect clinical decision might put human life in danger, in addition to having severe ethical and financial costs. While evidential uncertainty estimates for deep learning have been studied for multi-class problems, research in multi-label settings remains untapped. In this paper, we propose a DEep Evidential Doctor (DEED), which is a novel deterministic approach to estimate multi-label targets along with uncertainty. We achieve this by placing evidential priors over the original likelihood functions and directly estimating the parameters of the evidential distribution using a novel loss function. Additionally, we build a redundancy layer (particularly for high uncertainty and OOD examples) to minimize the risk associated with erroneous decisions based on dubious predictions. We achieve this by learning the mapping between the evidential space and a continuous semantic label embedding space via a recurrent decoder. Thereby inferring, even in the case of OOD examples, reasonably close predictions to avoid catastrophic consequences. We demonstrate the effectiveness of DEED on a digit classification task based on a modified multi-label MNIST dataset, and further evaluate it on a diagnosis prediction task from a real-life electronic health record dataset. We highlight that in terms of prediction scores, our approach is on par with the existing state-of-the-art having a clear advantage of generating reliable, memory and time-efficient uncertainty estimates with minimal changes to any multi-label DNN classifier. [ABSTRACT FROM AUTHOR]

• Public ecological aesthetic perception basically follows a three-step process (seeing ecological elements—concept perception processing—aesthetic experience). • Ecological and aesthetic perceptions vary in terms of conceptual relevance and processing. • Natural landscape elements are more likely to contribute to positive ecosystem aesthetic perception. • Evaluation models developed using Bayesian networks combined with deep learning and eye-tracking have practical applications. Public perceptions of ecological aesthetics are a crucial component in evaluations of the quality of urban green spaces (UGSs) and in guiding urban planning, design and renewal practices. In this paper, we propose a novel methodological framework that combines deep learning and eye-tracking to investigate the relationships between landscape elements, features, visual focus and public ecosystem aesthetic perceptions of UGSs. Our Public Ecosystem Aesthetic Perceptions (PEAP) model emphasises interactions between ecosystem quality and landscape aesthetic quality. To validate the model, we conducted a study in Shanghai, China, where we selected 54 representative UGSs for photography, perception surveys, and image processing. The results demonstrated that visual focus acts as a mediator between landscape stimuli (landscape elements and features) and perceptions of ecosystem aesthetics. Notably, we found that natural elements elicited significantly stronger visual focus than artificial elements. Thus, our PEAP approach can enhance understanding of aesthetic qualities of landscapes and ecological qualities of urban ecosystems, enabling creation of more aesthetically pleasing and ecologically sustainable UGSs. [ABSTRACT FROM AUTHOR]

Bayesian learning has been recently considered as an effective means of accounting for uncertainty in trained deep network parameters. This is of crucial importance when dealing with small or sparse training datasets. On the other hand, shallow models that compute weighted sums of their inputs, after passing them through a bank of arbitrary randomized nonlinearities, have been recently shown to enjoy good test error bounds that depend on the number of nonlinearities. Inspired from these advances, in this paper we examine novel deep network architectures, where each layer comprises a bank of arbitrary nonlinearities, linearly combined using multiple alternative sets of weights. We effect model training by means of approximate inference based on a t -divergence measure; this generalizes the Kullback–Leibler divergence in the context of the t -exponential family of distributions. We adopt the t -exponential family since it can more flexibly accommodate real-world data, that entail outliers and distributions with fat tails, compared to conventional Gaussian model assumptions. We extensively evaluate our approach using several challenging benchmarks, and provide comparative results to related state-of-the-art techniques. [ABSTRACT FROM AUTHOR]

[Display omitted] Non-Intrusive Load Monitoring (NILM) can be used to detect, recognize, and classify switching events of individual electrical appliances from an aggregate power signal that is measured at the main line of the grid connection. A limitation of existing solutions is that deep learning models tend to overfit the data and do not express their uncertainty when making predictions. This paper shows that uncertainty information can be obtained in a natural way by making use of Bayesian Neural Networks. Having this information is very valuable, because it supplies relevant information about potential misclassifications of the model to an end-user. The source of these misclassifications can be attributed to ambiguous data, or the model requiring more examples to learn from. In this work, an increase in generalization performance is observed when making use of Stochastic Gradient Hamiltonian Monte Carlo over Stochastic Gradient descent, and the usefulness of uncertainty in a NILM context is discussed. [ABSTRACT FROM AUTHOR]

Analyzing and predicting user activity is important in the current digital era with a lot of use-cases and applications. In this paper, we present an approach that facilitates a modification of the attention mechanism in a Transformer model. This work enables to improve the predictive capacity of a forecasting model which is progressively fed by somewhat erratic and small data generated by the early stages of online activity. The key element of the work is to use a Bayesian Network (BN) as a tool for feature engineering that helps to modify the attention mechanism in the Transformer model in that scenario. The model predicts the next activity on a sequence of online activities that the user will engage in while interacting with a Learning Management System (LMS). Click-stream data refers to a detailed log of how participants navigate through an online platform during a working session. The main application of our work is to improve the Predictor module of a smart hybrid-classifier for an LMS. Several configurations and architectures for the RNN-powered predictor, are tested and assessed. The results of the improved predictive capacity of this work can be useful to users in an online learning environment where early assistance in quasi-real time is required. This research answers the questions of how click-stream data can assist in refining the tasks of the Attention mechanism to improve the quality of the prediction. Performance is measured by the accuracy , right-content and first-state accuracy scores for the incoming sequence and compared across alternative models. The method also provides systematic customization of the attention mechanism in Transformers that can be applied to a range of problems involving click-stream data. [ABSTRACT FROM AUTHOR]