Search

The extensive implementation of cloud computing has brought about a significant transformation in multiple industries, encompassing major corporations, individual consumers, and nascent technological advancements. Cloud computing services have been widely adopted by Intelligent Transportation Systems (ITS) in order to optimize communication, data storage, and processing capabilities. ITS infrastructure is very vulnerable to security concerns due to its sensitive nature, hence requiring the implementation of efficient Intrusion Detection Systems (IDS) to identify potential threats. This study presents a new method to improve the accuracy of IDS in identifying attacks in the ITS Cloud environment by using game theoretic and bayesian optimized bayesian neural network (GTBNN). The Game-theoretic Model effectively tackles the issue of non-cooperative behavior between attackers and defenders. This model is combined with a Bayesian Optimized Bayesian Neural Network (BNN) to achieve efficient optimization and testing. The performance of our framework is evaluated on three benchmark datasets, namely UNSW-NB15, CICIDS, and Bot-IoT. The experimental findings demonstrate significant enhancements in detection rates across all datasets, exhibiting respective increases of 9.66%, 3.75%, and 4.16% and significant decreases in False Positive Rates (FPR) of 0.01%, 0.026%, and 0.138% for the respective datasets. The presented approach utilizes game-theoretic ideas and Bayesian optimization techniques to provide a distinctive and influential solution for improving the accuracy and efficiency of IDS in protecting vital ITS infrastructure. [ABSTRACT FROM AUTHOR]

This research proposes a novel approach leveraging deep learning techniques to enhance scour severity prediction that aims to analyse various parameters’ effects on scour severity and scour patterns around piers under different bed conditions. The methodology involves integrating deep learning techniques, including Linear Discriminant Analysis-t-Distributed Stochastic Neighbour Embedding (LDA-t-SNE) and Bayesian Neural Networks (BNNs), to extract features, reduce dimensionality, and improve detection accuracy. This study conducted correlation analysis to understand the relationships between parameters such as drainage area, stream slope, pier characteristics, flow dynamics, and sediment properties and their influence on scour severity. Additionally, sensitivity analysis was performed to assess the impact of different pier shapes and bed conditions on scour severity. Our results demonstrate the effectiveness of the proposed approach, with metrics such as RMSE (%) values of 0.025 and MAE (%) values of 0.011 and 0.01, respectively, outperforming traditional scour detection methods, achieving an accuracy of 98% consistently affirming the superior accuracy and reliability of the model. This research provides valuable insights into proactive scour management and infrastructure resilience, offering practical solutions for safeguarding hydraulic structures against scour-induced risks in civil engineering applications. [ABSTRACT FROM AUTHOR]

Stochastic models with global parameters and latent variables are common, and for which variational inference (VI) is popular. However, existing methods are often either slow or inaccurate in high dimensions. We suggest a fast and accurate VI method for this case that employs a well-defined natural gradient variational optimization that targets the joint posterior of the global parameters and latent variables. It is a hybrid method, where at each step the global parameters are updated using the natural gradient and the latent variables are generated from their conditional posterior. A fast to compute expression for the Tikhonov damped Fisher information matrix is used, along with the re-parameterization trick, to provide a stable natural gradient. We apply the approach to deep mixed models, which are an emerging class of Bayesian neural networks with random output layer coefficients to allow for heterogeneity. A range of simulations show that using the natural gradient is substantially more efficient than using the ordinary gradient, and that the approach is faster and more accurate than two cutting-edge natural gradient VI methods. In a financial application we show that accounting for industry level heterogeneity using the deep mixed model improves the accuracy of asset pricing models. MATLAB code to implement the method and replicate the results can be found at [ABSTRACT FROM AUTHOR]

In this paper, we propose and experimentally assess an innovative framework for scaling posterior distributions over different-curation datasets, based on Bayesian-Neural-Networks (BNN). Another innovation of our proposed study consists in enhancing the accuracy of the Bayesian classifier via intelligent sampling algorithms. The proposed methodology is relevant in emerging applicative settings, such as provenance detection and analysis and cybercrime. Our contributions are complemented by a comprehensive experimental evaluation and analysis over both static and dynamic image datasets. Derived results confirm the successful application of our proposed methodology to emerging big data analytics settings. [ABSTRACT FROM AUTHOR]

This paper presents a learning- and scenario-based model predictive control (MPC) design approach for systems modelled in the linear parameter-varying (LPV) framework. Using input-output data collected from the system, a state-space LPV model with uncertainty quantification is first learned through the variational Bayesian inference Neural Network (BNN) approach. The learned probabilistic model is assumed to contain the true dynamics of the system with a high probability and is used to generate scenarios that ensure safety for a scenario-based MPC. Moreover, to guarantee stability and enhance the performance of the closed-loop system, a parameter-dependent terminal cost and controller, as well as a terminal robust positive invariant set are designed. Numerical examples will be used to demonstrate that the proposed control design approach can ensure safety and achieve desired control performance. [ABSTRACT FROM AUTHOR]

Childhood allergies are a problem that seems to be forgotten by the Artificial Intelligence community, even if they are affecting millions of children. In this paper we are interested in studying the prevalence of childhood allergies, some demographic stats, and to predict the severity of the most encountered allergy, asthma. For this we have used two publicly available datasets, one for Data Engineering and Exploratory Data Analysis, and the other for Bayesian Neural Networks. [ABSTRACT FROM AUTHOR]

This paper investigates uncertainty quantification (UQ) techniques in multi-class classification of chest X-ray images (COVID-19, Pneumonia, and Normal). We evaluate Bayesian Neural Networks (BNN) and the Deep Neural Network with UQ (DNN with UQ) techniques, including Monte Carlo dropout, Ensemble Bayesian Neural Network (EBNN), Ensemble Monte Carlo (EMC) dropout, across different evaluation metrics. Our analysis reveals that DNN with UQ, especially EBNN and EMC dropout, consistently outperform BNNs. For example, in Class 0 vs. All, EBNN achieved a UAcc of 92.6%, UAUC-ROC of 95.0%, and a Brier Score of 0.157, significantly surpassing BNN's performance. Similarly, EMC Dropout excelled in Class 1 vs. All with a UAcc of 83.5%, UAUC-ROC of 95.8%, and a Brier Score of 0.165. These advanced models demonstrated higher accuracy, better discriaminative capability, and more accurate probabilistic predictions. Our findings highlight the efficacy of DNN with UQ in enhancing model reliability and interpretability, making them highly suitable for critical healthcare applications like chest X-ray imageQ6 classification.

Abstract With the increased use of data-driven approaches and machine learning-based methods in material science, the importance of reliable uncertainty quantification (UQ) of the predicted variables for informed decision-making cannot be overstated. UQ in material property prediction poses unique challenges, including multi-scale and multi-physics nature of materials, intricate interactions between numerous factors, limited availability of large curated datasets, etc. In this work, we introduce a physics-informed Bayesian Neural Networks (BNNs) approach for UQ, which integrates knowledge from governing laws in materials to guide the models toward physically consistent predictions. To evaluate the approach, we present case studies for predicting the creep rupture life of steel alloys. Experimental validation with three datasets of creep tests demonstrates that this method produces point predictions and uncertainty estimations that are competitive or exceed the performance of conventional UQ methods such as Gaussian Process Regression. Additionally, we evaluate the suitability of employing UQ in an active learning scenario and report competitive performance. The most promising framework for creep life prediction is BNNs based on Markov Chain Monte Carlo approximation of the posterior distribution of network parameters, as it provided more reliable results in comparison to BNNs based on variational inference approximation or related NNs with probabilistic outputs.

Extensive regions in the permafrost zone are projected to become climatically unsuitable to sustain permafrost peatlands over the next century, suggesting transformations in these landscapes that can leave large amounts of permafrost carbon vulnerable to post‐thaw decomposition. We present 3 years of eddy covariance measurements of CH4 and CO2 fluxes from the degrading permafrost peatland Iškoras in Northern Norway, which we disaggregate into separate fluxes of palsa, pond, and fen areas using information provided by the dynamic flux footprint in a novel ensemble‐based Bayesian deep neural network framework. The 3‐year mean CO2‐equivalent flux is estimated to be 106 gCO2 m−2 yr−1 for palsas, 1,780 gCO2 m−2 yr−1 for ponds, and −31 gCO2 m−2 yr−1 for fens, indicating that possible palsa degradation to thermokarst ponds would strengthen the local greenhouse gas forcing by a factor of about 17, while transformation into fens would slightly reduce the current local greenhouse gas forcing. Plain Language Summary: Arctic and sub‐arctic regions on the southern border of the permafrost zone often feature peatlands with a patchy surface of peat mounds, thaw ponds, and surrounding fens. As the permafrost underneath peat mounds thaws, these areas transform and can change their emission or uptake of greenhouse gases like CO2 and methane. Assessing this gas exchange on the patchy surface is difficult because our measurement techniques cannot directly observe the variability in space and time. We collected 3 years of gas exchange measurements at a Norwegian permafrost peatland and developed a new method using a collection of uncertainty‐aware neural networks to predict the greenhouse gas exchange of different surface types. Our work suggests that large amounts of methane are emitted by ponds and fens, while the elevated peat mounds have almost no methane emissions. For CO2, we see that ponds are strong emitters, while fens take up substantial amounts as their vegetation absorbs this gas. We are still unsure when the peat mounds will collapse and if they turn into ponds or fens, but we can say that pond formation would give a 17 fold increase in greenhouse gas emissions, while fen formation would slightly reduce today's emissions of permafrost peatlands. Key Points: Eddy covariance fluxes are disaggregated for different surfaces using Bayesian neural networks to derive uncertainty‐aware carbon balancesWhile palsa areas have a near‐zero annual methane balance, the fens and ponds that form upon palsa degradation emit large amounts of methaneFens compensate for methane emissions with strong annual CO2 sinks, while ponds appear as strong, yet uncertain, CO2 emission hotspots [ABSTRACT FROM AUTHOR]

Offshore wind structures are exposed to a harsh marine environment and are subject to deterioration mechanisms throughout their operational lifetime. Even if the deterioration evolution of structural elements can be estimated through physics-based deterioration models, the uncertainties involved in the process hurdle the selection of lifecycle management decisions, e.g., lifetime extension. In this scenario, the collection of relevant information through an efficient monitoring system enables the reduction of uncertainties, ultimately driving more optimal lifecycle decisions. However, a full monitoring instrumentation implemented on all wind turbines in a farm may become unfeasible due to practical and economical constraints. Besides, certain load monitoring systems often become defective after a few years of marine environment exposure. Addressing the aforementioned concerns, a farm-wide virtual load monitoring scheme directed by a fleet-leader wind turbine offers an attractive solution. Fetched with data retrieved from a fully instrumented wind turbine, a model can be first trained and then deployed, yielding load predictions for non-fully monitored wind turbines, from which only standard data are available, e.g., supervisory control and data acquisition. During the deployment stage, the pretrained virtual monitoring model may, however, receive previously unseen monitoring data, leading to inaccurate load predictions. In this article, we propose a virtual load monitoring framework formulated via Bayesian neural networks (BNNs) and provide relevant implementation details needed for the construction, training, and deployment of BNN data-based virtual monitoring models. As opposed to their deterministic counterparts, BNNs intrinsically announce the uncertainties associated with generated load predictions and allow to detect inaccurate load estimations generated for non-fully monitored wind turbines. The proposed virtual load monitoring is thoroughly tested through an experimental campaign in an operational offshore wind farm and the results demonstrate the effectiveness of BNN models for "fleet-leader"-based farm-wide virtual monitoring. [ABSTRACT FROM AUTHOR]

Generative adversarial networks (GANs) and diffusion models (DMs) have revolutionized the creation of synthetically generated but realistic-looking images. Distinguishing such generated images from real camera captures is one of the key tasks in current multimedia forensics research. One particular challenge is the generalization to unseen generators or post-processing. This can be viewed as an issue of handling out-of-distribution inputs. Forensic detectors can be hardened by the extensive augmentation of the training data or specifically tailored networks. Nevertheless, such precautions only manage but do not remove the risk of prediction failures on inputs that look reasonable to an analyst but in fact are out of the training distribution of the network. With this work, we aim to close this gap with a Bayesian Neural Network (BNN) that provides an additional uncertainty measure to warn an analyst of difficult decisions. More specifically, the BNN learns the task at hand and also detects potential confusion between post-processing and image generator artifacts. Our experiments show that the BNN achieves on-par performance with the state-of-the-art detectors while producing more reliable predictions on out-of-distribution examples. [ABSTRACT FROM AUTHOR]

AbstractWe propose a new approach to Bayesian prediction that caters for models with a large number of parameters and is robust to model misspecification. Given a class of high-dimensional (but parametric) predictive models, this new approach constructs a posterior predictive using a variational approximation to a generalized posterior that is directly focused on predictive accuracy. The theoretical behavior of the new prediction approach is analyzed and a form of optimality demonstrated. Applications to both simulated and empirical data using high-dimensional Bayesian neural network and autoregressive mixture models demonstrate that the approach provides more accurate results than various alternatives, including misspecified likelihood-based predictions. Supplementary materials for this article are available online. [ABSTRACT FROM AUTHOR]

To train machine learning algorithms to predict emotional expressions in terms of arousal and valence, annotated datasets are needed. However, as different people perceive others’ emotional expressions differently, their annotations are subjective. To account for this, annotations are typically collected from multiple annotators and averaged to obtain ground-truth labels. However, when exclusively trained on this averaged ground-truth, the model is agnostic to the inherent subjectivity in emotional expressions. In this work, we therefore propose an end-to-end Bayesian neural network capable of being trained on a distribution of annotations to also capture the subjectivity-based label uncertainty. Instead of a Gaussian, we model the annotation distribution using Student's $t$ t -distribution, which also accounts for the number of annotations available. We derive the corresponding Kullback-Leibler divergence loss and use it to train an estimator for the annotation distribution, from which the mean and uncertainty can be inferred. We validate the proposed method using two in-the-wild datasets. We show that the proposed $t$ t -distribution based approach achieves state-of-the-art uncertainty modeling results in speech emotion recognition, and also consistent results in cross-corpora evaluations. Furthermore, analyses reveal that the advantage of a $t$ t -distribution over a Gaussian grows with increasing inter-annotator correlation and a decreasing number of annotations available. [ABSTRACT FROM AUTHOR]

Bayesian neural networks (BNNs) have recently regained a significant amount of attention in the deep learning community due to the development of scalable approximate Bayesian inference techniques. There are several advantages of using a Bayesian approach: parameter and prediction uncertainties become easily available, facilitating more rigorous statistical analysis. Furthermore, prior knowledge can be incorporated. However, the construction of scalable techniques that combine both structural and parameter uncertainty remains a challenge. In this paper, we apply the concept of model uncertainty as a framework for structural learning in BNNs and, hence, make inferences in the joint space of structures/models and parameters. Moreover, we suggest an adaptation of a scalable variational inference approach with reparametrization of marginal inclusion probabilities to incorporate the model space constraints. Experimental results on a range of benchmark datasets show that we obtain comparable accuracy results with the competing models, but based on methods that are much more sparse than ordinary BNNs. [ABSTRACT FROM AUTHOR]

While Bayesian neural networks (BNNs) have gained popularity for their theoretical guarantees and robustness, they have yet to see a convincing implementation at scale. This study investigates a variational inference-based neural architecture called Variational Density Propagation (VDP) that boasts noise robustness, self-compression and improved explanations over traditional (deterministic) neural networks. Due to the large computational burden associated with BNNs, however, these methods have yet to scale efficiently for real-world problems. In this study, we simplify the VDP architecture by reducing its time and space requirements and allowing for efficient scaling to ImageNet level problems. Additionally, we evaluate the inherent properties of the VDP method in order to validate the simplified method. Across all datasets and architectures, our method exhibits exceptional self-compression capabilities, retaining performance even with over 90% of its parameters pruned. The method also presents improved visual explanations via saliency maps, suggesting superior explanation quality compared to deterministic models. Lastly, we employ the VDP method to train a vision transformer on ImageNet-1k, something that was previously impossible due to the inherent computational constraints of the method. Our code has been made readily available at the link below.

The rapid growth of the global population, economy, and urbanization is significantly increasing energy consumption, necessitating the integration of renewable energy sources. This integration presents challenges that demand innovative solutions to maintain grid stability and efficiency. Smart grids offer enhanced reliability, efficiency, sustainability, and bi-directional communication. However, the reliance on advanced technologies in smart grids introduces vulnerabilities, particularly concerning adversarial attacks. This paper addresses two critical issues in smart grid fault prediction: the vulnerability of machine learning models to adversarial attacks and the operational challenges posed by false alarms. We propose a Bayesian Neural Network (BNN) framework for fault zone prediction that quantifies uncertainty in predictions, enhancing robustness and reducing false alarms. Our BNN model achieves up to 0.958 accuracy and 0.960 precision in fault zone prediction. To counter adversarial attacks, we developed an uncertainty-based detection scheme that leverages prediction uncertainty. This framework distinguishes between normal and adversarial data using predictive entropy and mutual information as metrics. It detects complex white-box adversarial attacks, which are challenging due to attackers’ detailed knowledge of the model, with a mean accuracy of 0.891 using predictive entropy and 0.981 using mutual information. The model’s performance, combined with minimal computational overhead, underscores its practicality and robustness for enhancing smart grid security.

Bayesian inference provides a methodology for parameter estimation and uncertainty quantification in machine learning and deep learning methods. Variational inference and Markov Chain Monte-Carlo (MCMC) sampling methods are used to implement Bayesian inference. In the past few decades, MCMC sampling methods have faced challenges in being adapted to larger models (such as deep learning models) and big data problems. Advanced proposal distributions that incorporate gradients, such as a Langevin proposal distribution, provide a means to address some of the limitations of MCMC sampling for Bayesian neural networks. Furthermore, MCMC methods have typically been constrained to statisticians, and hence not well-known among deep learning researchers. We present a tutorial for MCMC methods that covers simple Bayesian linear and logistic models, and Bayesian neural networks. The aim of this tutorial is to bridge the gap between theory and implementation via Python code, given a general sparsity of libraries and tutorials. This tutorial provides code in Python with data and instructions that enable their use and extension. We provide results for selected benchmark problems showing the strengths and weaknesses of implementing the respective Bayesian models via MCMC. We highlight the challenges in sampling multi-modal posterior distributions for the case of Bayesian neural networks and the need for further improvement of convergence diagnosis methods.

This paper proposes a Bayesian neural network method for predicting equipment operational trends based on a channel attention mechanism. Traditional time series prediction methods have limitations in handling complex data and nonlinear relationships. To enhance prediction accuracy and stability, the paper introduces a channel attention mechanism to capture crucial features and contextual information within the data. This mechanism automatically adjusts the weights of feature channels to focus on the influence of key features. By leveraging the advantages of Bayesian neural networks, the model undergoes multiple updates and adjustments while considering uncertainty factors, progressively improving the predictive outcomes. In experiments, the paper utilizes power transformer data from a Kaggle public dataset and a substantial amount of temporary facility equipment data from the Winter Olympics site, comparing the performance against other commonly used prediction methods. Results demonstrate the significant superiority of the Bayesian neural network method with channel attention mechanism in equipment trend prediction, outperforming traditional time series models and other commonly used methods.

Bayesian model reduction provides an efficient approach for comparing the performance of all nested sub-models of a model, without re-evaluating any of these sub-models. Until now, Bayesian model reduction has been applied mainly in the computational neuroscience community on simple models. In this paper, we formulate and apply Bayesian model reduction to perform principled pruning of Bayesian neural networks, based on variational free energy minimization. Direct application of Bayesian model reduction, however, gives rise to approximation errors. Therefore, a novel iterative pruning algorithm is presented to alleviate the problems arising with naive Bayesian model reduction, as supported experimentally on the publicly available UCI datasets for different inference algorithms. This novel parameter pruning scheme solves the shortcomings of current state-of-the-art pruning methods that are used by the signal processing community. The proposed approach has a clear stopping criterion and minimizes the same objective that is used during training. Next to these benefits, our experiments indicate better model performance in comparison to state-of-the-art pruning schemes.

For the first time, a simplified approach to constructing Bayesian neural networks is proposed, combining computational efficiency with the ability to analyze the learning process. The proposed approach is based on Bayesianization of a deterministic neural network by randomizing parameters only at the interface level, i.e., the formation of a Bayesian neural network based on a given network by replacing its parameters with probability distributions that have the parameters of the original model as the average value. Evaluations of the efficiency metrics of the neural network were obtained within the framework of the approach under consideration, and the Bayesian neural network constructed through variation inference were performed using topological data analysis methods. The Bayesianization procedure is implemented through graded variation of the randomization intensity. As an alternative, two neural networks with identical structure were used — deterministic and classical Bayesian networks. The input of the neural network was supplied with the original data of two datasets in versions without noise and with added Gaussian noise. The zero and first persistent homologies for the embeddings of the formed neural networks on each layer were calculated. To assess the quality of classification, the accuracy metric was used. It is shown that the barcodes for embeddings on each layer of the Bayesianized neural network in all four scenarios are between the corresponding barcodes of the deterministic and Bayesian neural networks for both zero and first persistent homologies. In this case, the deterministic neural network is the lower bound, and the Bayesian neural network is the upper bound. It is shown that the structure of data associations within a Bayesianized neural network is inherited from a deterministic model, but acquires the properties of a Bayesian one. It has been experimentally established that there is a relationship between the normalized persistent entropy calculated on neural network embeddings and the accuracy of the neural network. For predicting accuracy, the topology of embeddings on the middle layer of the neural network model turned out to be the most revealing. The proposed approach can be used to simplify the construction of a Bayesian neural network from an already trained deterministic neural network, which opens up the possibility of increasing the accuracy of an existing neural network without ensemble with additional classifiers. It becomes possible to proactively evaluate the effectiveness of the generated neural network on simplified data without running it on a real dataset, which reduces the resource intensity of its development.

Purpose: Deep neural networks need to be able to indicate error likelihood via reliable estimates of their predictive uncertainty when used in high-risk scenarios, such as medical decision support. This work contributes a systematic overview of state-of-the-art approaches for decomposing predictive uncertainty into aleatoric and epistemic components, and a comprehensive comparison for Bayesian neural networks (BNNs) between mutual information decomposition and the explicit modelling of both uncertainty types via an additional loss-attenuating neuron. Methods: Experiments are performed in the context of liver segmentation in CT scans. The quality of the uncertainty decomposition in the resulting uncertainty maps is qualitatively evaluated, and quantitative behaviour of decomposed uncertainties is systematically compared for different experiment settings with varying training set sizes, label noise, and distribution shifts. Results: Our results show the mutual information decomposition to robustly yield meaningful aleatoric and epistemic uncertainty estimates, while the activation of the loss-attenuating neuron appears noisier with non-trivial convergence properties. We found that the addition of a heteroscedastic neuron does not significantly improve segmentation performance or calibration, while slightly improving the quality of uncertainty estimates. Conclusions: Mutual information decomposition is simple to implement, has mathematically pleasing properties, and yields meaningful uncertainty estimates that behave as expected under controlled changes to our data set. The additional extension of BNNs with loss-attenuating neurons provides no improvement in terms of segmentation performance or calibration in our setting, but marginal benefits regarding the quality of decomposed uncertainties. [ABSTRACT FROM AUTHOR]

The increasing importance of uncertainty quantification in the regulatory evaluation of pharmaceutical products has triggered an explosion of Bayesian methods in recent years. In biomarker discovery and early clinical development, Bayesian methods have established a foothold in developing new drugs, in part due to the increasing availability of greater computational power, often complementing both traditional pharmaceutical statistics and classical statistical methods. In this paper, we present a selective survey of the recent efforts that have been made toward the development and application of effective statistical and computational models in early development statistics. The survey introduces four such case studies and methods that can be used for end-to-end biomarker discovery that includes pre-clinical and early clinical development, from unsupervised clustering to supervised machine learning to modern statistical inference, including but not limited to Bayesian clustering, Bayesian additive regression trees, Bayesian neural networks, and empirical Bayes procedures with the overarching goal of promoting their use and applications among pharmaceutical statisticians. Finally, we present some open issues in Bayesian early clinical methods to help guide the future advancement and wide adoption of Bayesian applications in early clinical pharmaceutical statistics. [ABSTRACT FROM AUTHOR]

Recent developments in information technology are due to the development of smart cities, which act as a crucial facilitator for the building of next-generation intelligent systems to improve security, reliability, and efficiency. Intelligent biosensors and biochips for molecular robotics are being developed with the help of integrated optoelectronic devices, which are essential. These tools combine optics and electronics to provide very sensitive and accurate detection, analysis, and manipulation of biological molecules and cells. This research proposed novel method in detection of lung cancer utilizing feature selection as well as classification utilizingdeep learning architectures with optical bio sensor. Here input MRI lung image has been collected based on the pre-historic medical data of the patients. This image is processed and segmented for noise removal as well as normalization. Features of this processed image is selected using Lasso regression and classify the selected features using Bayesian neural networks.The cells of this imagesisclassified for identification of nanoparticle Fe3O4 in the images in detecting tumor region. Patients diagnosed with COCO dataset, TEM dataset, S2NANOdataset were used to test developed feature selection, classification and prediction methods on 4various data sets for HDdetection. Testing results revealed that developed classification, as well as prediction methods, attained an accuracy of 98%, precision of 95%, specificity of 91%, F-1 score of 90%, Area under the ROC Curve (AUC) of 88% and mAP(Mean Average Precision)of 81% for CCF dataset; for HIC dataset proposed method obtained accuracy of 95%, precision of 90%, specificity of 88%, F-1 score of 86%, AUC of 88% and mAP of 71%; proposed method obtained accuracy of 96%, precision of 94%, specificity of 81%, F-1 score of 89%, AUC of 85% and mAP of 68% for LBMC dataset; for SUH dataset proposed method obtained accuracy of 97%, precision of 92%, specificity of 88%, F-1 score of 91%, AUC of 77% and mAP of 71%exceeding accuracies of previously published research. [ABSTRACT FROM AUTHOR]

Abstract Extensive regions in the permafrost zone are projected to become climatically unsuitable to sustain permafrost peatlands over the next century, suggesting transformations in these landscapes that can leave large amounts of permafrost carbon vulnerable to post‐thaw decomposition. We present 3 years of eddy covariance measurements of CH4 and CO2 fluxes from the degrading permafrost peatland Iškoras in Northern Norway, which we disaggregate into separate fluxes of palsa, pond, and fen areas using information provided by the dynamic flux footprint in a novel ensemble‐based Bayesian deep neural network framework. The 3‐year mean CO2‐equivalent flux is estimated to be 106 gCO2 m−2 yr−1 for palsas, 1,780 gCO2 m−2 yr−1 for ponds, and −31 gCO2 m−2 yr−1 for fens, indicating that possible palsa degradation to thermokarst ponds would strengthen the local greenhouse gas forcing by a factor of about 17, while transformation into fens would slightly reduce the current local greenhouse gas forcing.

The last decade witnessed a growing interest in Bayesian learning. Yet, the technicality of the topic and the multitude of ingredients involved therein, besides the complexity of turning theory into practical implementations, limit the use of the Bayesian learning paradigm, preventing its widespread adoption across different fields and applications. This self-contained survey engages and introduces readers to the principles and algorithms of Bayesian Learning for Neural Networks. It provides an introduction to the topic from an accessible, practical-algorithmic perspective. Upon providing a general introduction to Bayesian Neural Networks, we discuss and present both standard and recent approaches for Bayesian inference, with an emphasis on solutions relying on Variational Inference and the use of Natural gradients. We also discuss the use of manifold optimization as a state-of-the-art approach to Bayesian learning. We examine the characteristic properties of all the discussed methods, and provide pseudo-codes for their implementation, paying attention to practical aspects, such as the computation of the gradients. [ABSTRACT FROM AUTHOR]

In this work, minibatch MCMC sampling for feedforward neural networks is made more feasible. To this end, it is proposed to sample subgroups of parameters via a blocked Gibbs sampling scheme. By partitioning the parameter space, sampling is possible irrespective of layer width. It is also possible to alleviate vanishing acceptance rates for increasing depth by reducing the proposal variance in deeper layers. Increasing the length of a non-convergent chain increases the predictive accuracy in classification tasks, so avoiding vanishing acceptance rates and consequently enabling longer chain runs have practical benefits. Moreover, non-convergent chain realizations aid in the quantification of predictive uncertainty. An open problem is how to perform minibatch MCMC sampling for feedforward neural networks in the presence of augmented data. [ABSTRACT FROM AUTHOR]

The global economy and way of life will be impacted by the increase in heat that the Earth is experiencing daily. Storms, cyclones, droughts, floods, and fires are examples of natural disasters that can strike without warning and have devastating effects on living things. Not only will this have a negative impact on the commercial and industrial development of the global economy, but it could also result in fatalities. Overall, it would seriously affect the upkeep of the Earth's ecosystems. With the development of machine learning algorithms, it is essential for us to comprehend how to use the available climate expert systems and various systematic procedures that can predict critical climatic conditions in advance so that potential disasters can be anticipated, identified, and mitigated. This study analyses effective machine learning methods for forecasting the risk of adverse weather events, such as heavy rain, temperature rise, wind, and drought. A recent study found that using artificial intelligence in data processing can be highly successful in producing a potentially effective climate forecast. Natural climate-related occurrences occur with predictable regularity. However, several of them exhibit diverse behaviour within their intervals. Compared to other conventional ways, artificial intelligence outfitted with potent machine learning strategies has shown to be effective in anticipating catastrophic tragedies. [ABSTRACT FROM AUTHOR]

The prediction of financial markets is a challenging yet important task. In modern electronically driven markets, traditional time‐series econometric methods often appear incapable of capturing the true complexity of the multilevel interactions driving the price dynamics. While recent research has established the effectiveness of traditional machine learning (ML) models in financial applications, their intrinsic inability to deal with uncertainties, which is a great concern in econometrics research and real business applications, constitutes a major drawback. Bayesian methods naturally appear as a suitable remedy conveying the predictive ability of ML methods with the probabilistically oriented practice of econometric research. By adopting a state‐of‐the‐art second‐order optimization algorithm, we train a Bayesian bilinear neural network with temporal attention, suitable for the challenging time‐series task of predicting mid‐price movements in ultra‐high‐frequency limit‐order book markets. We thoroughly compare our Bayesian model with traditional ML alternatives by addressing the use of predictive distributions to analyze errors and uncertainties associated with the estimated parameters and model forecasts. Our results underline the feasibility of the Bayesian deep‐learning approach and its predictive and decisional advantages in complex econometric tasks, prompting future research in this direction. [ABSTRACT FROM AUTHOR]

Generative adversarial networks (GANs) and diffusion models (DMs) have revolutionized the creation of synthetically generated but realistic-looking images. Distinguishing such generated images from real camera captures is one of the key tasks in current multimedia forensics research. One particular challenge is the generalization to unseen generators or post-processing. This can be viewed as an issue of handling out-of-distribution inputs. Forensic detectors can be hardened by the extensive augmentation of the training data or specifically tailored networks. Nevertheless, such precautions only manage but do not remove the risk of prediction failures on inputs that look reasonable to an analyst but in fact are out of the training distribution of the network. With this work, we aim to close this gap with a Bayesian Neural Network (BNN) that provides an additional uncertainty measure to warn an analyst of difficult decisions. More specifically, the BNN learns the task at hand and also detects potential confusion between post-processing and image generator artifacts. Our experiments show that the BNN achieves on-par performance with the state-of-the-art detectors while producing more reliable predictions on out-of-distribution examples.

Bayesian neural networks (BNNs) have recently regained a significant amount of attention in the deep learning community due to the development of scalable approximate Bayesian inference techniques. There are several advantages of using a Bayesian approach: parameter and prediction uncertainties become easily available, facilitating more rigorous statistical analysis. Furthermore, prior knowledge can be incorporated. However, the construction of scalable techniques that combine both structural and parameter uncertainty remains a challenge. In this paper, we apply the concept of model uncertainty as a framework for structural learning in BNNs and, hence, make inferences in the joint space of structures/models and parameters. Moreover, we suggest an adaptation of a scalable variational inference approach with reparametrization of marginal inclusion probabilities to incorporate the model space constraints. Experimental results on a range of benchmark datasets show that we obtain comparable accuracy results with the competing models, but based on methods that are much more sparse than ordinary BNNs.

A Bayesian treatment of deep learning allows for the computation of uncertainties associated with the predictions of deep neural networks. We show how the concept of Errors-in-Variables can be used in Bayesian deep regression to also account for the uncertainty associated with the input of the employed neural network. The presented approach thereby exploits a relevant, but generally overlooked, source of uncertainty and yields a decomposition of the predictive uncertainty into an aleatoric and epistemic part that is more complete and, in many cases, more consistent from a statistical perspective. We discuss the approach along various simulated and real examples and observe that using an Errors-in-Variables model leads to an increase in the uncertainty while preserving the prediction performance of models without Errors-in-Variables. For examples with known regression function we observe that this ground truth is substantially better covered by the Errors-in-Variables model, indicating that the presented approach leads to a more reliable uncertainty estimation. [ABSTRACT FROM AUTHOR]

Few-shot models aim at making predictions using a minimal number of labeled examples from a given task. The main challenge in this area is the one-shot setting, where only one element represents each class. We propose the general framework for few-shot learning via kernel HyperNetworks—the fusion of kernels and hypernetwork paradigm. Firstly, we introduce the classical realization of this framework, dubbed HyperShot. Compared to reference approaches that apply a gradient-based adjustment of the parameters, our models aim to switch the classification module parameters depending on the task’s embedding. In practice, we utilize a hypernetwork, which takes the aggregated information from support data and returns the classifier’s parameters handcrafted for the considered problem. Moreover, we introduce the kernel-based representation of the support examples delivered to hypernetwork to create the parameters of the classification module. Consequently, we rely on relations between the support examples’ embeddings instead of the backbone models’ direct feature values. Thanks to this approach, our model can adapt to highly different tasks. While such a method obtains very good results, it is limited by typical problems such as poorly quantified uncertainty due to limited data size. We further show that incorporating Bayesian neural networks into our general framework, an approach we call BayesHyperShot, solves this issue. [ABSTRACT FROM AUTHOR]

Sequential Bayesian inference can be used for continual learning to prevent catastrophic forgetting of past tasks and provide an informative prior when learning new tasks. We revisit sequential Bayesian inference and assess whether using the previous task's posterior as a prior for a new task can prevent catastrophic forgetting in Bayesian neural networks. Our first contribution is to perform sequential Bayesian inference using Hamiltonian Monte Carlo. We propagate the posterior as a prior for new tasks by approximating the posterior via fitting a density estimator on Hamiltonian Monte Carlo samples. We find that this approach fails to prevent catastrophic forgetting, demonstrating the difficulty in performing sequential Bayesian inference in neural networks. From there, we study simple analytical examples of sequential Bayesian inference and CL and highlight the issue of model misspecification, which can lead to sub-optimal continual learning performance despite exact inference. Furthermore, we discuss how task data imbalances can cause forgetting. From these limitations, we argue that we need probabilistic models of the continual learning generative process rather than relying on sequential Bayesian inference over Bayesian neural network weights. Our final contribution is to propose a simple baseline called Prototypical Bayesian Continual Learning, which is competitive with the best performing Bayesian continual learning methods on class incremental continual learning computer vision benchmarks. [ABSTRACT FROM AUTHOR]

Accurate net load forecasting is a cost-effective technique, crucial for the planning, stability, reliability, and integration of variable solar photovoltaic (PV) systems in modern power systems. This work presents a direct short-term net load forecasting (STNLF) methodology for solar-integrated microgrids by leveraging machine learning (ML) principles. The proposed data-driven method comprises of an initial input feature engineering and filtering step, construction of forecasting model using Bayesian neural networks, and an optimization stage. The performance of the proposed model was validated on historical net load data obtained from a university campus solar-powered microgrid. The results demonstrated the effectiveness of the model for providing accurate and robust STNLF. Specifically, the optimally constructed model yielded a normalized root mean square error of 3.98% when benchmarked using a 1-year historical microgrid data. The $k$ -fold cross-validation method was then used and proved the stability of the forecasting model. Finally, the obtained ML-based forecasts demonstrated improvements of 17.77% when compared against forecasts of a baseline naïve persistence model. To this end, this work provides insights on how to construct high-performance STNLF models for solar-integrated microgrids. Such insights on the development of accurate STNLF architectures can have positive implications in actual microgrid decision-making by utilities/operators.

The expansion of explainable artificial intelligence as a field of research has generated numerous methods of visualizing and understanding the black box of a machine learning model. Attribution maps are commonly used to highlight the parts of the input image that influence the model to make a specific decision. At the same time, numerous recent research papers in the fields of machine learning and explainable artificial intelligence have demonstrated the essential role of robustness to natural noise and adversarial attacks in determining the features learned by a model. This paper focuses on evaluating methods of attribution mapping to find whether robust neural networks are more explainable, particularly within the application of classification for medical imaging. However, there is no consensus on how to evaluate attribution maps. To solve this, we propose a new explainability faithfulness metric, EvalAttAI, that addresses the limitations of prior metrics. We evaluate various attribution methods on multiple datasets and find that Bayesian deep neural networks using the Variational Density Propagation technique are consistently more explainable when used with the best performing attribution method, the Vanilla Gradient. Our results suggest that robust neural networks may not always be more explainable, despite producing more visually plausible attribution maps.

A probabilistic neural network has been implemented to predict the malignancy of breast cancer cells, based on a data set, the features of which are used for the formulation and training of a model for a binary classification problem. The focus is placed on considerations when building the model, in order to achieve not only accuracy but also a safe quantification of the expected uncertainty of the calculated network parameters and the medical prognosis. The source code is included to make the results reproducible, also in accordance with the latest trending in machine learning research, named Papers with Code. The various steps taken for the code development are introduced in detail but also the results are visually displayed and critically analyzed also in the sense of explainable artificial intelligence. In statistical-classification problems, the decision boundary is the region of the problem space in which the classification label of the classifier is ambiguous. Problem aspects and model parameters which influence the decision boundary are a special aspect of practical investigation considered in this work. Classification results issued by technically transparent machine learning software can inspire more confidence, as regards their trustworthiness which is very important, especially in the case of medical prognosis. Furthermore, transparency allows the user to adapt models and learning processes to the specific needs of a problem and has a boosting influence on the development of new methods in relevant machine learning fields (transfer learning).

• A Deep Operator Network (DeepONet) for data-driven transient response prediction. • The Bayesian DeepONet enables a reliable prediction via uncertainty quantification. • The Bayesian DeepONet enables learning even when data is scarce. • The Probabilistic DeepONet enables a reliable prediction at virtually no extra cost This paper proposes a novel data-driven method for the reliable prediction of the power grid's post-fault trajectories, i.e., the power grid's dynamic response after a disturbance or fault. The proposed method is based on the recently proposed concept of Deep Operator Networks (DeepONets). Unlike traditional neural networks that learn to approximate functions, DeepONets are designed to approximate nonlinear operators, i.e., mappings between infinite-dimensional spaces. Under this operator framework, we design a novel and efficient DeepONet that (i) takes as inputs the trajectories collected before and during the fault and (ii) outputs the predicted post-fault trajectories. In addition, we endow our method with the much-needed ability to balance efficiency with reliable/trustworthy predictions via uncertainty quantification. To this end, we propose and compare two novel methods that enable quantifying the predictive uncertainty. First, we propose a Bayesian DeepONet (B-DeepONet) that uses stochastic gradient Hamiltonian Monte-Carlo to sample from the posterior distribution of the DeepONet trainable parameters. Then, we design a Probabilistic DeepONet (Prob-DeepONet) that uses a probabilistic training strategy to enable quantifying uncertainty at virtually no extra computational cost. Finally, we validate the proposed methods' predictive power and uncertainty quantification capability using the New York-New England power grid model. [ABSTRACT FROM AUTHOR]

In the last few years, many works have tried to explain the predictions of deep learning models. Few methods, however, have been proposed to verify the accuracy or faithfulness of these explanations. Recently, influence functions, which is a method that approximates the effect that leave-one-out training has on the loss function, has been shown to be fragile. The proposed reason for their fragility remains unclear. Although previous work suggests the use of regularization to increase robustness, this does not hold in all cases. In this work, we seek to investigate the experiments performed in the prior work in an effort to understand the underlying mechanisms of influence function fragility. First, we verify influence functions using procedures from the literature under conditions where the convexity assumptions of influence functions are met. Then, we relax these assumptions and study the effects of non-convexity by using deeper models and more complex datasets. Here, we analyze the key metrics and procedures that are used to validate influence functions. Our results indicate that the validation procedures may cause the observed fragility. • Influence functions do not appear to be as fragile as previously thought. • Bayesian neural nets enhance influence function explanations. • Large hessian eigenvalues do not correlate with influence function performance. • The retraining from optimal procedure may lead to erroneous results. • Spearman correlation is not an effective metric to evaluate influence functions. [ABSTRACT FROM AUTHOR]

We propose a framework for the assessment of uncertainty quantification in deep regression. The framework is based on regression problems where the regression function is a linear combination of nonlinear functions. Basically, any level of complexity can be realized through the choice of the nonlinear functions and the dimensionality of their domain. Results of an uncertainty quantification for deep regression are compared against those obtained by a statistical reference method. The reference method utilizes knowledge about the underlying nonlinear functions and is based on Bayesian linear regression using a prior reference. The flexibility, together with the availability of a reference solution, makes the framework suitable for defining benchmark sets for uncertainty quantification. Reliability of uncertainty quantification is assessed in terms of coverage probabilities, and accuracy through the size of calculated uncertainties. We illustrate the proposed framework by applying it to current approaches for uncertainty quantification in deep regression. In addition, results for three real-world regression tasks are presented. [ABSTRACT FROM AUTHOR]