Search

The residual loss in Physics-Informed Neural Networks (PINNs) alters the simple recursive relation of layers in a feed-forward neural network by applying a differential operator, resulting in a loss landscape that is inherently different from those of common supervised problems. Therefore, relying on the existing theory leads to unjustified design choices and suboptimal performance. In this work, we analyze the residual loss by studying its characteristics at critical points to find the conditions that result in effective training of PINNs. Specifically, we first show that under certain conditions, the residual loss of PINNs can be globally minimized by a wide neural network. Furthermore, our analysis also reveals that an activation function with well-behaved high-order derivatives plays a crucial role in minimizing the residual loss. In particular, to solve a $k$-th order PDE, the $k$-th derivative of the activation function should be bijective. The established theory paves the way for designing and choosing effective activation functions for PINNs and explains why periodic activations have shown promising performance in certain cases. Finally, we verify our findings by conducting a set of experiments on several PDEs. Our code is publicly available at https://github.com/nimahsn/pinns_tf2., Comment: Accepted at IJCAI 2024. V2: Corrected typos

The increasing frequency and scale of wildfires carry significant ecological, socioeconomic, and environmental implications, prompting the need for a deeper grasp of wildfire characteristics. Essential meteorological factors like temperature, humidity, and precipitation wield a crucial impact on fire behavior and the estimation of burned areas. This study aims to unravel the interconnections between meteorological conditions and fire attributes within the Salmon-Challis National Forest located in east-central Idaho, USA. Through the utilization of remotely sensed data from the Fire Monitoring, Mapping, and Modeling system (Fire M3) alongside meteorological variables recorded between 2010 and 2020, an exploration is conducted into varied meteorological patterns associated with wildfire events. By integrating the computed burned area into the clustering process, valuable insights are gained into the specific influences of fire weather conditions on the extent of burned areas. The Salmon-Challis National Forest, encompassing more than 4.3 million acres and encompassing the largest wilderness area in the Continental United States, emerges as a pivotal research site for wildfire investigations. This work elucidates the data attributes employed for clustering and visualization, along with the algorithms employed. Additionally, the study presents research findings and delineates potential future applications, ultimately contributing to the advancement of fire management and mitigation strategies in regions prone to wildfires.

We have constructed a Bayesian neural network able of retrieving tropospheric temperature profiles from rotational Raman-scatter measurements of nitrogen and oxygen and applied it to measurements taken by the RAman Lidar for Meteorological Observations (RALMO) in Payerne, Switzerland. We give a detailed description of using a Bayesian method to retrieve temperature profiles including estimates of the uncertainty due to the network weights and the statistical uncertainty of the measurements. We trained our model using lidar measurements under different atmospheric conditions, and we tested our model using measurements not used for training the network. The computed temperature profiles extend over the altitude range of 0.7 km to 6 km. The mean bias estimate of our temperatures relative to the MeteoSwiss standard processing algorithm does not exceed 0.05 K at altitudes below 4.5 km, and does not exceed 0.08 K in an altitude range of 4.5 km to 6 km. This agreement shows that the neural network estimated temperature profiles are in excellent agreement with the standard algorithm. The method is robust and is able to estimate the temperature profiles with high accuracy for both clear and cloudy conditions. Moreover, the trained model can provide the statistical and model uncertainties of the estimated temperature profiles. Thus, the present study is a proof of concept that the trained NNs are able to generate temperature profiles along with a full-budget uncertainty. We present case studies showcasing the Bayesian neural network estimations for day and night measurements, as well as in clear and cloudy conditions. We have concluded that the proposed Bayesian neural network is an appropriate method for the statistical retrieval of temperature profiles.

Physics-informed neural network (PINN) algorithms have shown promising results in solving a wide range of problems involving partial differential equations (PDEs). However, they often fail to converge to desirable solutions when the target function contains high-frequency features, due to a phenomenon known as spectral bias. In the present work, we exploit neural tangent kernels (NTKs) to investigate the training dynamics of PINNs evolving under stochastic gradient descent with momentum (SGDM). This demonstrates SGDM significantly reduces the effect of spectral bias. We have also examined why training a model via the Adam optimizer can accelerate the convergence while reducing the spectral bias. Moreover, our numerical experiments have confirmed that wide-enough networks using SGDM still converge to desirable solutions, even in the presence of high-frequency features. In fact, we show that the width of a network plays a critical role in convergence.