Search

Reduced-order models, also known as proxy model or surrogate model, are approximate models that are less computational expensive as opposed to fully descriptive models. With the integration of machine learning, these models have garnered increasing research interests recently. However, many existing reduced-order modeling methods, such as embed to control (E2C) and embed to control and observe (E2CO), fall short in long-term predictions due to the accumulation of prediction errors over time. This issue arises partly from the one-step prediction framework inherent in E2C and E2CO architectures. This paper introduces a deep learning-based surrogate model, referred as multi-step embed-to-control model, for the construction of proxy models with improved long-term prediction performance. Unlike E2C and E2CO, the proposed network considers multiple forward transitions in the latent space at a time using Koopman operator, allowing the model to incorporate a sequence of state snapshots during training phrases. Additionally, the loss function of this novel approach has been redesigned to accommodate these multiple transitions and to respect the underlying physical principles. To validate the efficacy of the proposed method, the developed framework was implemented within two-phase (oil and water) reservoir model under a waterflooding scheme. Comparative analysis demonstrate that the proposed model significantly outperforms the conventional E2C model in long-term simulation scenarios. Notably, there was a substantial reduction in temporal errors in the prediction of saturation profiles and a decent improvement in pressure forecasting accuracy., Comment: 12 pages, 8 figures

Injecting greenhouse gas into deep underground reservoirs for permanent storage can inadvertently lead to fault reactivation, caprock fracturing and greenhouse gas leakage when the injection-induced stress exceeds the critical threshold. Extraction of pre-existing fluids at various stages of injection process, referred as pressure management, can mitigate associated risks and lessen environmental impact. However, identifying optimal pressure management strategies typically requires thousands of full-order simulations due to the need for function evaluations, making the process computationally prohibitive. This paper introduces a novel surrogate model-based reinforcement learning method for devising optimal pressure management strategies for geological CO2 sequestration efficiently. Our approach comprises two steps. Firstly, a surrogate model is developed through the embed to control method, which employs an encoder-transition-decoder structure to learn latent dynamics. Leveraging this proxy model, reinforcement learning is utilized to find an optimal strategy that maximizes economic benefits while satisfying various control constraints. The reinforcement learning agent receives the latent state space representation and immediate reward tailored for CO2 sequestration and choose real-time controls which are subject to predefined engineering constraints in order to maximize the long-term cumulative rewards. To demonstrate its effectiveness, this framework is applied to a compositional simulation model where CO2 is injected into saline aquifer. The results reveal that our surrogate model-based reinforcement learning approach significantly optimizes CO2 sequestration strategies, leading to notable economic gains compared to baseline scenarios., Comment: 23 pages, 10 figures

A physics-informed convolutional neural network is proposed to simulate two phase flow in porous media with time-varying well controls. While most of PICNNs in existing literatures worked on parameter-to-state mapping, our proposed network parameterizes the solution with time-varying controls to establish a control-to-state regression. Firstly, finite volume scheme is adopted to discretize flow equations and formulate loss function that respects mass conservation laws. Neumann boundary conditions are seamlessly incorporated into the semi-discretized equations so no additional loss term is needed. The network architecture comprises two parallel U-Net structures, with network inputs being well controls and outputs being the system states. To capture the time-dependent relationship between inputs and outputs, the network is well designed to mimic discretized state space equations. We train the network progressively for every timestep, enabling it to simultaneously predict oil pressure and water saturation at each timestep. After training the network for one timestep, we leverage transfer learning techniques to expedite the training process for subsequent timestep. The proposed model is used to simulate oil-water porous flow scenarios with varying reservoir gridblocks and aspects including computation efficiency and accuracy are compared against corresponding numerical approaches. The results underscore the potential of PICNN in effectively simulating systems with numerous grid blocks, as computation time does not scale with model dimensionality. We assess the temporal error using 10 different testing controls with variation in magnitude and another 10 with higher alternation frequency with proposed control-to-state architecture. Our observations suggest the need for a more robust and reliable model when dealing with controls that exhibit significant variations in magnitude or frequency.

This paper presents a novel surrogate model for modeling subsurface fluid flow with well controls using a physics-informed convolutional recurrent neural network (PICRNN). The model uses a convolutional long-short term memory (ConvLSTM) to capture the spatiotemporal dependencies of the state evolution dynamics in the porous flow. The ConvLSTM is linked to the state space equations, enabling the incorporation of a discrete-time sequence of well control. The model requires initial state condition and a sequence of well controls as inputs, and predicts the state variables of the system, such as pressure, as output. By minimizing the residuals of reservoir flow state-space equations, the network is trained without the need for labeled data. The model is designed to serve as a surrogate model for predicting future reservoir states based on the initial reservoir state and input engineering controls. Boundary conditions are enforced into the state-space equations so no additional loss term is needed. Three numerical cases are studied, demonstrating the model's effectiveness in predicting reservoir dynamics based on future well/system controls. The proposed model provides a new approach for efficient and accurate prediction of subsurface fluid flow, with potential applications in optimal control design for reservoir engineering.

Subsurface earth models (referred to as geo-models) are crucial for characterizing complex subsurface systems. Multiple-point statistics are commonly used to generate geo-models. In this paper, a deep-learning-based generative method is developed as an alternative to the traditional Geomodel generation procedure. The generative method comprises two deep-learning models, namely the hierarchical vector-quantized variational autoencoder (VQ-VAE-2) and PixelSNAIL autoregressive model. Based on the principle of neural discrete representation learning, the VQ-VAE-2 learns to massively compress the Geomodels to extract the low-dimensional, discrete latent representation corresponding to each Geomodel. Following that, PixelSNAIL uses the deep autoregressive network to learn the prior distribution of the latent codes. For the purpose of Geomodel generation, PixelSNAIL samples from the newly learned prior distribution of latent codes, and then the decoder of the VQ-VAE-2 converts the newly sampled latent code to a newly constructed geo-model. PixelSNAIL can be used for unconditional or conditional geo-model generation. In an unconditional generation, the generative workflow generates an ensemble of geo-models without any constraint. On the other hand, in the conditional geo-model generation, the generative workflow generates an ensemble of geo-models similar to a user-defined source image, which ultimately facilitates the control and manipulation of the generated geo-models. To better construct the fluvial channels in the geo-models, the perceptual loss is implemented in the VQ-VAE-2 model instead of the traditional mean squared error loss. At a specific compression ratio, the quality of multi-attribute geo-model generation is better than that of single-attribute geo-model generation.