Search

Uncertainty quantification during atmospheric chemistry modeling is computationally expensive as it typically requires a large number of simulations using complex models. As large-scale modeling is typically performed with simplified chemical mechanisms for computational tractability, we describe a probabilistic surrogate modeling method using principal components analysis (PCA) and Ensemble Sparse Identification of Nonlinear Dynamics (E-SINDy) to both automatically simplify a gas-phase chemistry mechanism and to quantify the uncertainty introduced when doing so. We demonstrate the application of this method on a small photochemical box model for ozone formation. With 100 ensemble members, the calibration $R$-squared value is 0.96 among the three latent species on average and 0.98 for ozone, demonstrating that predicted model uncertainty aligns well with actual model error. In addition to uncertainty quantification, this probabilistic method also improves accuracy as compared to an equivalent deterministic version, by $\sim$60% for the ensemble prediction mean or $\sim$50% for deterministic prediction by the best-performing single ensemble member. Overall, the ozone testing root mean square error (RMSE) is 15.1% of its root mean square (RMS) concentration. Although our probabilistic ensemble simulation ends up being slower than the reference model it emulates, we expect that use of a more complex reference model in future work will result in additional opportunities for acceleration. Versions of this approach applied to full-scale chemical mechanisms may result in improved uncertainty quantification in models of atmospheric composition, leading to enhanced atmospheric understanding and improved support for air quality control and regulation.

Sparse-view Computed Tomography (CT) image reconstruction is a promising approach to reduce radiation exposure, but it inevitably leads to image degradation. Although diffusion model-based approaches are computationally expensive and suffer from the training-sampling discrepancy, they provide a potential solution to the problem. This study introduces a novel Cascaded Diffusion with Discrepancy Mitigation (CDDM) framework, including the low-quality image generation in latent space and the high-quality image generation in pixel space which contains data consistency and discrepancy mitigation in a one-step reconstruction process. The cascaded framework minimizes computational costs by moving some inference steps from pixel space to latent space. The discrepancy mitigation technique addresses the training-sampling gap induced by data consistency, ensuring the data distribution is close to the original manifold. A specialized Alternating Direction Method of Multipliers (ADMM) is employed to process image gradients in separate directions, offering a more targeted approach to regularization. Experimental results across two datasets demonstrate CDDM's superior performance in high-quality image generation with clearer boundaries compared to existing methods, highlighting the framework's computational efficiency.

Modeling atmospheric chemistry is computationally expensive and limits the widespread use of atmospheric chemical transport models. This computational cost arises from solving high-dimensional systems of stiff differential equations. Previous work has demonstrated the promise of machine learning (ML) to accelerate air quality model simulations but has suffered from numerical instability during long-term simulations. This may be because previous ML-based efforts have relied on explicit Euler time integration -- which is known to be unstable for stiff systems -- and have used neural networks which are prone to overfitting. We hypothesize that the creation of parsimonious models combined with modern numerical integration techniques can overcome this limitation. Using a small-scale photochemical mechanism to explore the potential of these methods, we have created a machine-learned surrogate by (1) reducing dimensionality using singular value decomposition to create an interpretably-compressed low-dimensional latent space, and (2) using Sparse Identification of Nonlinear Dynamics (SINDy) to create a differential-equation-based representation of the underlying chemical dynamics in the compressed latent space with reduced numerical stiffness. The root mean square error of the ML model prediction for ozone concentration over nine days is 37.8% of the root mean concentration across all simulations in our testing dataset. The surrogate model is 11$\times$ faster with 12$\times$ fewer integration timesteps compared to the reference model and is numerically stable in all tested simulations. Overall, we find that SINDy can be used to create fast, stable, and accurate surrogates of a simple photochemical mechanism. In future work, we will explore the application of this method to more detailed mechanisms and their use in large-scale simulations.

Foundation models hold promise for transforming AI in healthcare by providing modular components that are easily adaptable to downstream healthcare tasks, making AI development more scalable and cost-effective. Structured EHR foundation models, trained on coded medical records from millions of patients, demonstrated benefits including increased performance with fewer training labels, and improved robustness to distribution shifts. However, questions remain on the feasibility of sharing these models across different hospitals and their performance for local task adaptation. This multi-center study examined the adaptability of a recently released structured EHR foundation model ($FM_{SM}$), trained on longitudinal medical record data from 2.57M Stanford Medicine patients. Experiments were conducted using EHR data at The Hospital for Sick Children and MIMIC-IV. We assessed both adaptability via continued pretraining on local data, and task adaptability compared to baselines of training models from scratch at each site, including a local foundation model. We evaluated the performance of these models on 8 clinical prediction tasks. In both datasets, adapting the off-the-shelf $FM_{SM}$ matched the performance of GBM models locally trained on all data while providing a 13% improvement in settings with few task-specific training labels. With continued pretraining on local data, label efficiency substantially improved, such that $FM_{SM}$ required fewer than 1% of training examples to match the fully trained GBM's performance. Continued pretraining was also 60 to 90% more sample-efficient than training local foundation models from scratch. Our findings show that adapting shared EHR foundation models across hospitals provides improved prediction performance at less cost, underscoring the utility of base foundation models as modular components to streamline the development of healthcare AI., Comment: 46 pages, 5 figures, 3 tables, 14 appendices

Localization of objects is vital for robot-object interaction. Light Detection and Ranging (LiDAR) application in robotics is an emerging and widely used object localization technique due to its accurate distance measurement, long-range, wide field of view, and robustness in different conditions. However, LiDAR is unable to identify the objects when they are obstructed by obstacles, resulting in inaccuracy and noise in localization. To address this issue, we present an approach incorporating LiDAR and Ultra-Wideband (UWB) ranging for object localization. The UWB is popular in sensor fusion localization algorithms due to its low weight and low power consumption. In addition, the UWB is able to return ranging measurements even when the object is not within line-of-sight. Our approach provides an efficient solution to combine an anonymous optical sensor (LiDAR) with an identity-based radio sensor (UWB) to improve the localization accuracy of the object. Our approach consists of three modules. The first module is an object-identification algorithm that compares successive scans from the LiDAR to detect a moving object in the environment and returns the position with the closest range to UWB ranging. The second module estimates the moving object's moving direction using the previous and current estimated position from our object-identification module. It removes the suspicious estimations through an outlier rejection criterion. Lastly, we fuse the LiDAR, UWB ranging, and odometry measurements in pose graph optimization (PGO) to recover the entire trajectory of the robot and object. Extensive experiments were performed to evaluate the performance of the proposed approach., Comment: This paper has been accepted by The 2023 IEEE International Conference on Robotics and Biomimetics (IEEE ROBIO 2023)

As models for nature language processing (NLP), computer vision (CV) and recommendation systems (RS) require surging computation, a large number of GPUs/TPUs are paralleled as a large batch (LB) to improve training throughput. However, training such LB tasks often meets large generalization gap and downgrades final precision, which limits enlarging the batch size. In this work, we develop the variance reduced gradient descent technique (VRGD) based on the gradient signal to noise ratio (GSNR) and apply it onto popular optimizers such as SGD/Adam/LARS/LAMB. We carry out a theoretical analysis of convergence rate to explain its fast training dynamics, and a generalization analysis to demonstrate its smaller generalization gap on LB training. Comprehensive experiments demonstrate that VRGD can accelerate training ($1\sim 2 \times$), narrow generalization gap and improve final accuracy. We push the batch size limit of BERT pretraining up to 128k/64k and DLRM to 512k without noticeable accuracy loss. We improve ImageNet Top-1 accuracy at 96k by $0.52pp$ than LARS. The generalization gap of BERT and ImageNet training is significantly reduce by over $65\%$., Comment: 25 pages, 5 figures