Search

For decades, corporations and governments have relied on scanned documents to record vast amounts of information. However, extracting this information is a slow and tedious process due to the overwhelming amount of documents. The rise of vision language models presents a way to efficiently and accurately extract the information out of these documents. The current automated workflow often requires a two-step approach involving the extraction of information using optical character recognition software, and subsequent usage of large language models for processing this information. Unfortunately, these methods encounter significant challenges when dealing with noisy scanned documents. The high information density of such documents often necessitates using computationally expensive language models to effectively reduce noise. In this study, we propose PatchFinder, an algorithm that builds upon Vision Language Models (VLMs) to address the information extraction task. First, we devise a confidence-based score, called Patch Confidence, based on the Maximum Softmax Probability of the VLMs' output to measure the model's confidence in its predictions. Then, PatchFinder utilizes that score to determine a suitable patch size, partition the input document into overlapping patches of that size, and generate confidence-based predictions for the target information. Our experimental results show that PatchFinder can leverage Phi-3v, a 4.2 billion parameter vision language model, to achieve an accuracy of 94% on our dataset of 190 noisy scanned documents, surpassing the performance of ChatGPT-4o by 18.5 percentage points., Comment: This paper has been accepted to IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) 2025

Understanding material failure is critical for designing stronger and lighter structures by identifying weaknesses that could be mitigated. Existing full-physics numerical simulation techniques involve trade-offs between speed, accuracy, and the ability to handle complex features like varying boundary conditions, grid types, resolution, and physical models. We present the first foundation model specifically designed for predicting material failure, leveraging large-scale datasets and a high parameter count (up to 3B) to significantly improve the accuracy of failure predictions. In addition, a large language model provides rich context embeddings, enabling our model to make predictions across a diverse range of conditions. Unlike traditional machine learning models, which are often tailored to specific systems or limited to narrow simulation conditions, our foundation model is designed to generalize across different materials and simulators. This flexibility enables the model to handle a range of material properties and conditions, providing accurate predictions without the need for retraining or adjustments for each specific case. Our model is capable of accommodating diverse input formats, such as images and varying simulation conditions, and producing a range of outputs, from simulation results to effective properties. It supports both Cartesian and unstructured grids, with design choices that allow for seamless updates and extensions as new data and requirements emerge. Our results show that increasing the scale of the model leads to significant performance gains (loss scales as $N^{-1.6}$, compared to language models which often scale as $N^{-0.5}$)., Comment: Accepted at NeurIPS 2024 "Foundation Models for Science: Progress, Opportunities, and Challenges" Workshop

This paper presents a novel benchmark where the large language model (LLM) must write code that computes integer sequences from the Online Encyclopedia of Integer Sequences (OEIS), a widely-used resource for mathematical sequences. The benchmark is designed to evaluate both the correctness of the generated code and its computational efficiency. Our benchmark reveals that the o1 series of models outperform other frontier models from OpenAI, Anthropic, Meta, and Google in accuracy and cheating rates across both easy and hard integer sequences. In order to ensure models do not exploit memorized sequence values, we introduce an automated cheating detection mechanism that flags the use of lookup tables and validated this automation against human cheating evaluations. This benchmark provides a meaningful challenge for current LLMs, offering insights into their mathematical reasoning and code writing capabilities, which can guide future research directions and model development in mathematical reasoning and code synthesis.

This work introduces an online greedy method for constructing quadratic manifolds from streaming data, designed to enable in-situ analysis of numerical simulation data on the Petabyte scale. Unlike traditional batch methods, which require all data to be available upfront and take multiple passes over the data, the proposed online greedy method incrementally updates quadratic manifolds in one pass as data points are received, eliminating the need for expensive disk input/output operations as well as storing and loading data points once they have been processed. A range of numerical examples demonstrate that the online greedy method learns accurate quadratic manifold embeddings while being capable of processing data that far exceed common disk input/output capabilities and volumes as well as main-memory sizes., Comment: 20 pages, 10 figures

The advent of large language models (LLMs) has significantly advanced the field of code translation, enabling automated translation between programming languages. However, these models often struggle with complex translation tasks due to inadequate contextual understanding. This paper introduces a novel approach that enhances code translation through Few-Shot Learning, augmented with retrieval-based techniques. By leveraging a repository of existing code translations, we dynamically retrieve the most relevant examples to guide the model in translating new code segments. Our method, based on Retrieval-Augmented Generation (RAG), substantially improves translation quality by providing contextual examples from which the model can learn in real-time. We selected RAG over traditional fine-tuning methods due to its ability to utilize existing codebases or a locally stored corpus of code, which allows for dynamic adaptation to diverse translation tasks without extensive retraining. Extensive experiments on diverse datasets with open LLM models such as Starcoder, Llama3-70B Instruct, CodeLlama-34B Instruct, Granite-34B Code Instruct, and Mixtral-8x22B, as well as commercial LLM models like GPT-3.5 Turbo and GPT-4o, demonstrate our approach's superiority over traditional zero-shot methods, especially in translating between Fortran and CPP. We also explored varying numbers of shots i.e. examples provided during inference, specifically 1, 2, and 3 shots and different embedding models for RAG, including Nomic-Embed, Starencoder, and CodeBERT, to assess the robustness and effectiveness of our approach., Comment: LLM for code translation

We establish a number of results concerning conditions for minimum energy dissipation and advective travel time in porous and fractured media. First, we establish a pair of converse results concerning fluid motion along a streamline between two points of fixed head: the minimal advective time is achieved under conditions of constant energy dissipation, and minimal energy dissipation is achieved under conditions of constant velocity along the streamline (implying homogeneous conductivity in the vicinity of the streamline). We also show directly by means of variational methods that minimum advection time along a streamline with a given average conductivity is achieved when the conductivity is constant. Finally, we turn our attention to minimum advection time and energy dissipation in parallel and sequential fracture systems governed by the cubic law: for which fracture cross-section and conductivity are intimately linked. We show that, as in porous domains, flow partitioning between different pathways always acts to minimize system energy dissipation. Finally, we consider minimum advection time as a function of aperture distribution in a sequence of fracture segments. We show that, for a fixed average aperture, a uniform-aperture system displays the shortest advection time. However, we also show that any sufficiently small small perturbations in aperture away from uniformity always act to reduce advection time.

To reduce environmental risks and impacts from orphaned wells (abandoned oil and gas wells), it is essential to first locate and then plug these wells. Although some historical documents are available, they are often unstructured, not cleaned, and outdated. Additionally, they vary widely by state and type. Manual reading and digitizing this information from historical documents are not feasible, given the high number of wells. Here, we propose a new computational approach for rapidly and cost-effectively locating these wells. Specifically, we leverage the advanced capabilities of large language models (LLMs) to extract vital information including well location and depth from historical records of orphaned wells. In this paper, we present an information extraction workflow based on open-source Llama 2 models and test them on a dataset of 160 well documents. Our results show that the developed workflow achieves excellent accuracy in extracting location and depth from clean, PDF-based reports, with a 100% accuracy rate. However, it struggles with unstructured image-based well records, where accuracy drops to 70%. The workflow provides significant benefits over manual human digitization, including reduced labor and increased automation. In general, more detailed prompting leads to improved information extraction, and those LLMs with more parameters typically perform better. We provided a detailed discussion of the current challenges and the corresponding opportunities/approaches to address them. Additionally, a vast amount of geoscientific information is locked up in old documents, and this work demonstrates that recent breakthroughs in LLMs enable us to unlock this information more broadly.

Quantum computing is an emerging topic in engineering that promises to enhance supercomputing using fundamental physics. In the near term, the best candidate algorithms for achieving this advantage are variational quantum algorithms (VQAs). We design and numerically evaluate a novel ansatz for VQAs, focusing in particular on the variational quantum eigensolver (VQE). As our ansatz is inspired by classical multigrid hierarchy methods, we call it "multigrid" ansatz. The multigrid ansatz creates a parameterized quantum circuit for a quantum problem on $n$ qubits by successively building and optimizing circuits for smaller qubit counts $j < n$, reusing optimized parameter values as initial solutions to next level hierarchy at $j+1$. We show through numerical simulation that the multigrid ansatz outperforms the standard hardware-efficient ansatz in terms of solution quality for the Laplacian eigensolver as well as for a large class of combinatorial optimization problems with specific examples for MaxCut and Maximum $k$-Satisfiability. Our studies establish the multi-grid ansatz as a viable candidate for many VQAs and in particular present a promising alternative to the QAOA approach for combinatorial optimization problems., Comment: 11 pages, 9 figures

We perform a set of flow and reactive transport simulations within three-dimensional fracture networks to learn the factors controlling mineral reactions. CO$_2$ mineralization requires CO$_2$-laden water, dissolution of a mineral that then leads to precipitation of a CO$_2$-bearing mineral. Our discrete fracture networks (DFN) are partially filled with quartz that gradually dissolves until it reaches a quasi-steady state. At the end of the simulation, we measure the quartz remaining in each fracture within the domain. We observe that a small backbone of fracture exists, where the quartz is fully dissolved which leads to increased flow and transport. However, depending on the DFN topology and the rate of dissolution, we observe a large variability of these changes, which indicates an interplay between the fracture network structure and the impact of geochemical dissolution. In this work, we developed a machine learning framework to extract the important features that support mineralization in the form of dissolution. In addition, we use structural and topological features of the fracture network to predict the remaining quartz volume in quasi-steady state conditions. As a first step to characterizing carbon mineralization, we study dissolution with this framework. We studied a variety of reaction and fracture parameters and their impact on the dissolution of quartz in fracture networks. We found that the dissolution reaction rate constant of quartz and the distance to the flowing backbone in the fracture network are the two most important features that control the amount of quartz left in the system. For the first time, we use a combination of a finite-volume reservoir model and graph-based approach to study reactive transport in a complex fracture network to determine the key features that control dissolution., Comment: 23 pages, 5 figures, 2 tables

Recreating complex, high-dimensional global fields from limited data points is a grand challenge across various scientific and industrial domains. Given the prohibitive costs of specialized sensors and the frequent inaccessibility of certain regions of the domain, achieving full field coverage is typically not feasible. Therefore, the development of algorithms that intelligently improve sensor placement is of significant value. In this study, we introduce a general approach that employs differentiable programming to exploit sensor placement within the training of a neural network model in order to improve field reconstruction. We evaluated our method using two distinct datasets; the results show that our approach improved test scores. Ultimately, our method of differentiable placement strategies has the potential to significantly increase data collection efficiency, enable more thorough area coverage, and reduce redundancy in sensor deployment.

We introduce JuliQAOA, a simulation package specifically built for the Quantum Alternating Operator Ansatz (QAOA). JuliQAOA does not require a circuit-level description of QAOA problems, or another package to simulate such circuits, instead relying on a more direct linear algebra implementation. This allows for increased QAOA-specific performance improvements, as well as improved flexibility and generality. JuliQAOA is the first QAOA package designed to aid in the study of both constrained and unconstrained combinatorial optimization problems, and can easily include novel cost functions, mixer Hamiltonians, and other variations. JuliQAOA also includes robust and extensible methods for learning optimal angles. Written in the Julia language, JuliQAOA outperforms existing QAOA software packages and scales well to HPC-level resources. JuliQAOA is available at https://github.com/lanl/JuliQAOA.jl.

Modeling effective transport properties of 3D porous media, such as permeability, at multiple scales is challenging as a result of the combined complexity of the pore structures and fluid physics - in particular, confinement effects which vary across the nanoscale to the microscale. While numerical simulation is possible, the computational cost is prohibitive for realistic domains, which are large and complex. Although machine learning models have been proposed to circumvent simulation, none so far has simultaneously accounted for heterogeneous 3D structures, fluid confinement effects, and multiple simulation resolutions. By utilizing numerous computer science techniques to improve the scalability of training, we have for the first time developed a general flow model that accounts for the pore-structure and corresponding physical phenomena at scales from Angstrom to the micrometer. Using synthetic computational domains for training, our machine learning model exhibits strong performance (R$^2$=0.9) when tested on extremely diverse real domains at multiple scales.

Limiting the injection rate to restrict the pressure below a threshold at a critical location can be an important goal of simulations that model the subsurface pressure between injection and extraction wells. The pressure is approximated by the solution of Darcy's partial differential equation (PDE) for a given permeability field. The subsurface permeability is modeled as a random field since it is known only up to statistical properties. This induces uncertainty in the computed pressure. Solving the PDE for an ensemble of random permeability simulations enables estimating a probability distribution for the pressure at the critical location. These simulations are computationally expensive, and practitioners often need rapid online guidance for real-time pressure management. An ensemble of numerical PDE solutions is used to construct a Gaussian process regression model that can quickly predict the pressure at the critical location as a function of the extraction rate and permeability realization. Our first novel contribution is to identify a sampling methodology for the random environment and matching kernel technology for which fitting the Gaussian process regression model scales as O(n log n) instead of the typical O(n^3) rate in the number of samples n used to fit the surrogate. The surrogate model allows almost instantaneous predictions for the pressure at the critical location as a function of the extraction rate and permeability realization. Our second contribution is a novel algorithm to calibrate the uncertainty in the surrogate model to the discrepancy between the true pressure solution of Darcy's equation and the numerical solution. Although our method is derived for building a surrogate for the solution of Darcy's equation with a random permeability field, the framework broadly applies to solutions of other PDE with random coefficients., Comment: 20 pages, 8 figures, 1 table

Data-driven modeling can suffer from a constant demand for data, leading to reduced accuracy and impractical for engineering applications due to the high cost and scarcity of information. To address this challenge, we propose a progressive reduced order modeling framework that minimizes data cravings and enhances data-driven modeling's practicality. Our approach selectively transfers knowledge from previously trained models through gates, similar to how humans selectively use valuable knowledge while ignoring unuseful information. By filtering relevant information from previous models, we can create a surrogate model with minimal turnaround time and a smaller training set that can still achieve high accuracy. We have tested our framework in several cases, including transport in porous media, gravity-driven flow, and finite deformation in hyperelastic materials. Our results illustrate that retaining information from previous models and utilizing a valuable portion of that knowledge can significantly improve the accuracy of the current model. We have demonstrated the importance of progressive knowledge transfer and its impact on model accuracy with reduced training samples. For instance, our framework with four parent models outperforms the no-parent counterpart trained on data nine times larger. Our research unlocks data-driven modeling's potential for practical engineering applications by mitigating the data scarcity issue. Our proposed framework is a significant step toward more efficient and cost-effective data-driven modeling, fostering advancements across various fields.

Quantum algorithms provide an exponential speedup for solving certain classes of linear systems, including those that model geologic fracture flow. However, this revolutionary gain in efficiency does not come without difficulty. Quantum algorithms require that problems satisfy not only algorithm-specific constraints, but also application-specific ones. Otherwise, the quantum advantage carefully attained through algorithmic ingenuity can be entirely negated. Previous work addressing quantum algorithms for geologic fracture flow has illustrated core algorithmic approaches while incrementally removing assumptions. This work addresses two further requirements for solving geologic fracture flow systems with quantum algorithms: efficient system state preparation and efficient information extraction. Our approach to addressing each is consistent with an overall exponential speed-up., Comment: 13 pages, 12 figures, LA-UR-23-31328

We model flow and transport in three-dimensional fracture networks with varying degrees of fracture-to-fracture aperture/permeability heterogeneity and network density to show how changes in these properties can cause the emergence of anomalous flow and transport behavior. If fracture-to-fracture aperture heterogeneity is increased in sparse networks, velocity fluctuations can inhibit high flow rates and solute transport can be delayed, even in cases where hydraulic aperture is monotonically increased. As the density of the networks is increased, more connected pathways allow for particles to bypass these effects. We discover transition behavior where with relatively few connected pathways in a network from inflow to outflow boundaries, the first arrival times of particles are not heavily affected by fracture-to-fracture aperture heterogeneity, but the scaling behavior of the tails is strongly influenced due to the particles being forced to sample some of the heterogeneity in the velocity field caused by aperture differences. These results reinforce the importance of considering multi-scale effects in fractured systems and can inform flow and transport processes in both natural and engineered fracture systems, especially the latter where high aperture fractures are often stimulated and connect to existing fracture networks with smaller apertures.

We comparatively study, through large-scale numerical simulation, the performance across a large set of Quantum Alternating Operator Ansatz (QAOA) implementations for finding approximate and optimum solutions to unconstrained combinatorial optimization problems. Our survey includes over 100 different mixing unitaries, and we combine each mixer with both the standard phase separator unitary representing the objective function and a thresholded version. Our numerical tests for randomly chosen instances of the unconstrained optimization problems Max 2-SAT and Max 3-SAT reveal that the traditional transverse-field mixer with the standard phase separator performs best for problem sizes of 8 through 14 variables, while the recently introduced Grover mixer with thresholding wins at problems of size 6. This result (i) corrects earlier work suggesting that the Grover mixer is a superior mixer based only on results from problems of size 6, thus illustrating the need to push numerical simulation to larger problem sizes to more accurately predict performance; and (ii) it suggests that more complicated mixers and phase separators may not improve QAOA performance., Comment: LA-UR-23-20702

Damage models for ductile materials typically need to be parameterized, often with the appropriate parameters changing for a given material depending on the loading conditions. This can make parameterizing these models computationally expensive, since an inverse problem must be solved for each loading condition. Using standard inverse modeling techniques typically requires hundreds or thousands of high-fidelity computer simulations to estimate the optimal parameters. Additionally, the time of a human expert is required to set up the inverse model. Machine learning has recently emerged as an alternative approach to inverse modeling in these settings, where the machine learning model is trained in an offline manner and new parameters can be quickly generated on the fly, after training is complete. This work utilizes such a workflow to enable the rapid parameterization of a ductile damage model called TEPLA with a machine learning inverse model. The machine learning model can efficiently estimate the model parameters much faster, as compared to previously employed methods, such as Bayesian calibration. The results demonstrate good accuracy on a synthetic test dataset and is validated against experimental data., Comment: 13 pages, 9 figures; v2 minor revision

Solving large systems of equations is a challenge for modeling natural phenomena, such as simulating subsurface flow. To avoid systems that are intractable on current computers, it is often necessary to neglect information at small scales, an approach known as coarse-graining. For many practical applications, such as flow in porous, homogenous materials, coarse-graining offers a sufficiently-accurate approximation of the solution. Unfortunately, fractured systems cannot be accurately coarse-grained, as critical network topology exists at the smallest scales, including topology that can push the network across a percolation threshold. Therefore, new techniques are necessary to accurately model important fracture systems. Quantum algorithms for solving linear systems offer a theoretically-exponential improvement over their classical counterparts, and in this work we introduce two quantum algorithms for fractured flow. The first algorithm, designed for future quantum computers which operate without error, has enormous potential, but we demonstrate that current hardware is too noisy for adequate performance. The second algorithm, designed to be noise resilient, already performs well for problems of small to medium size (order 10 to 1000 nodes), which we demonstrate experimentally and explain theoretically. We expect further improvements by leveraging quantum error mitigation and preconditioning., Comment: 20 pages, 12 figures

Inverse analysis has been utilized to understand unknown underground geological properties by matching the observational data with simulators. To overcome the underconstrained nature of inverse problems and achieve good performance, an approach is presented with embedded physics and a technique known as automatic differentiation. We use a physics-embedded generative model, which takes statistically simple parameters as input and outputs subsurface properties (e.g., permeability or P-wave velocity), that embeds physical knowledge of the subsurface properties into inverse analysis and improves its performance. We tested the application of this approach on four geologic problems: two heterogeneous hydraulic conductivity fields, a hydraulic fracture network, and a seismic inversion for P-wave velocity. This physics-embedded inverse analysis approach consistently characterizes these geological problems accurately. Furthermore, the excellent performance in matching the observational data demonstrates the reliability of the proposed method. Moreover, the application of automatic differentiation makes this an easy and fast approach to inverse analysis when dealing with complicated geological structures.

Avoiding over-pressurization in subsurface reservoirs is critical for applications like CO2 sequestration and wastewater injection. Managing the pressures by controlling injection/extraction are challenging because of complex heterogeneity in the subsurface. The heterogeneity typically requires high-fidelity physics-based models to make predictions on CO$_2$ fate. Furthermore, characterizing the heterogeneity accurately is fraught with parametric uncertainty. Accounting for both, heterogeneity and uncertainty, makes this a computationally-intensive problem challenging for current reservoir simulators. To tackle this, we use differentiable programming with a full-physics model and machine learning to determine the fluid extraction rates that prevent over-pressurization at critical reservoir locations. We use DPFEHM framework, which has trustworthy physics based on the standard two-point flux finite volume discretization and is also automatically differentiable like machine learning models. Our physics-informed machine learning framework uses convolutional neural networks to learn an appropriate extraction rate based on the permeability field. We also perform a hyperparameter search to improve the model's accuracy. Training and testing scenarios are executed to evaluate the feasibility of using physics-informed machine learning to manage reservoir pressures. We constructed and tested a sufficiently accurate simulator that is 400000 times faster than the underlying physics-based simulator, allowing for near real-time analysis and robust uncertainty quantification., Comment: 12 pages, 5 figures

Modeling hydrological fracture networks is a hallmark challenge in computational earth sciences. Accurately predicting critical features of fracture systems, e.g. percolation, can require solving large linear systems far beyond current or future high performance capabilities. Quantum computers can theoretically bypass the memory and speed constraints faced by classical approaches, however several technical issues must first be addressed. Chief amongst these difficulties is that such systems are often ill-conditioned, i.e. small changes in the system can produce large changes in the solution, which can slow down the performance of linear solving algorithms. We test several existing quantum techniques to improve the condition number, but find they are insufficient. We then introduce the inverse Laplacian preconditioner, which improves the scaling of the condition number of the system from $O(N)$ to $O(\sqrt{N})$ and admits a quantum implementation. These results are a critical first step in developing a quantum solver for fracture systems, both advancing the state of hydrological modeling and providing a novel real-world application for quantum linear systems algorithms.

Quantum algorithms for solving linear systems of equations have generated excitement because of the potential speed-ups involved and the importance of solving linear equations in many applications. However, applying these algorithms can be challenging. The Harrow-Hassidim-Lloyd algorithm and improvements thereof require complex subroutines suitable for fault-tolerant hardware such as Hamiltonian simulation, making it ill-suited to current hardware. Variational algorithms, on the other hand, involve expensive optimization loops, which can be prone to barren plateaus and local optima. We describe a quantum algorithm for solving linear systems of equations that avoids these problems. Our algorithm is based on the Woodbury identity, which analytically describes the inverse of a matrix that is a low-rank modification of another (easily-invertible) matrix. This approach only utilizes basic quantum subroutines like the Hadamard test or the swap test, so it is well-suited to current hardware. There is no optimization loop, so barren plateaus and local optima do not present a problem. The low-rank aspect of the identity enables us to efficiently transfer information to and from the quantum computer. This approach can produce accurate results on current hardware. As evidence of this, we estimate an inner product involving the solution of a system of more than 16 million equations with 2% error using IBM's Auckland quantum computer. To our knowledge, no system of equations this large has previously been solved to this level of accuracy on a quantum computer.

We propose a unified data-driven reduced order model (ROM) that bridges the performance gap between linear and nonlinear manifold approaches. Deep learning ROM (DL-ROM) using deep-convolutional autoencoders (DC-AE) has been shown to capture nonlinear solution manifolds but fails to perform adequately when linear subspace approaches such as proper orthogonal decomposition (POD) would be optimal. Besides, most DL-ROM models rely on convolutional layers, which might limit its application to only a structured mesh. The proposed framework in this study relies on the combination of an autoencoder (AE) and Barlow Twins (BT) self-supervised learning, where BT maximizes the information content of the embedding with the latent space through a joint embedding architecture. Through a series of benchmark problems of natural convection in porous media, BT-AE performs better than the previous DL-ROM framework by providing comparable results to POD-based approaches for problems where the solution lies within a linear subspace as well as DL-ROM autoencoder-based techniques where the solution lies on a nonlinear manifold; consequently, bridges the gap between linear and nonlinear reduced manifolds. Furthermore, this BT-AE framework can operate on unstructured meshes, which provides flexibility in its application to standard numerical solvers, on-site measurements, experimental data, or a combination of these sources., Comment: arXiv admin note: text overlap with arXiv:2107.11460

The "Workshop on Machine learning in heterogeneous porous materials" brought together international scientific communities of applied mathematics, porous media, and material sciences with experts in the areas of heterogeneous materials, machine learning (ML) and applied mathematics to identify how ML can advance materials research. Within the scope of ML and materials research, the goal of the workshop was to discuss the state-of-the-art in each community, promote crosstalk and accelerate multi-disciplinary collaborative research, and identify challenges and opportunities. As the end result, four topic areas were identified: ML in predicting materials properties, and discovery and design of novel materials, ML in porous and fractured media and time-dependent phenomena, Multi-scale modeling in heterogeneous porous materials via ML, and Discovery of materials constitutive laws and new governing equations. This workshop was part of the AmeriMech Symposium series sponsored by the National Academies of Sciences, Engineering and Medicine and the U.S. National Committee on Theoretical and Applied Mechanics., Comment: The workshop link is: https://amerimech.mech.utah.edu

Despite much recent work, the true promise and limitations of the Quantum Alternating Operator Ansatz (QAOA) are unclear. A critical question regarding QAOA is to what extent its performance scales with the input size of the problem instance, in particular the necessary growth in the number of QAOA rounds to reach a high approximation ratio. We present numerical evidence for an exponential speed-up of QAOA over Grover-style unstructured search in finding approximate solutions to constrained optimization problems. Our result provides a strong hint that QAOA is able to exploit the structure of an optimization problem and thus overcome the lower bound for unstructured search. To this end, we conduct a comprehensive numerical study on several Hamming-weight constrained optimization problems for which we include combinations of all standardly studied mixer and phase separator Hamiltonians (Ring mixer, Clique mixer, Objective Value phase separator) as well as quantum minimum-finding inspired Hamiltonians (Grover mixer, Threshold-based phase separator). We identify Clique-Objective-QAOA with an exponential speed-up over Grover-Threshold-QAOA and tie the latter's scaling to that of unstructured search, with all other QAOA combinations coming in at a distant third. Our result suggests that maximizing QAOA performance requires a judicious choice of mixer and phase separator, and should trigger further research into other QAOA variations.

Quantum annealers manufactured by D-Wave Systems, Inc., are computational devices capable of finding high-quality solutions of NP-hard problems. In this contribution, we explore the potential and effectiveness of such quantum annealers for computing Boolean tensor networks. Tensors offer a natural way to model high-dimensional data commonplace in many scientific fields, and representing a binary tensor as a Boolean tensor network is the task of expressing a tensor containing categorical (i.e., {0, 1}) values as a product of low dimensional binary tensors. A Boolean tensor network is computed by Boolean tensor decomposition, and it is usually not exact. The aim of such decomposition is to minimize the given distance measure between the high-dimensional input tensor and the product of lower-dimensional (usually three-dimensional) tensors and matrices representing the tensor network. In this paper, we introduce and analyze three general algorithms for Boolean tensor networks: Tucker, Tensor Train, and Hierarchical Tucker networks. The computation of a Boolean tensor network is reduced to a sequence of Boolean matrix factorizations, which we show can be expressed as a quadratic unconstrained binary optimization problem suitable for solving on a quantum annealer. By using a novel method we introduce called \textit{parallel quantum annealing}, we demonstrate that tensor with up to millions of elements can be decomposed efficiently using a DWave 2000Q quantum annealer., Comment: Updated with new figures and fixed typos. 18 pages

Modern NISQ devices are subject to a variety of biases and sources of noise that degrade the solution quality of computations carried out on these devices. A natural question that arises in the NISQ era, is how fairly do these devices sample ground state solutions. To this end, we run five fair sampling problems (each with at least three ground state solutions) that are based both on quantum annealing and on the Grover Mixer-QAOA algorithm for gate-based NISQ hardware. In particular, we use seven IBM~Q devices, the Aspen-9 Rigetti device, the IonQ device, and three D-Wave quantum annealers. For each of the fair sampling problems, we measure the ground state probability, the relative fairness of the frequency of each ground state solution with respect to the other ground state solutions, and the aggregate error as given by each hardware provider. Overall, our results show that NISQ devices do not achieve fair sampling yet. We also observe differences in the software stack with a particular focus on compilation techniques that illustrate what work will still need to be done to achieve a seamless integration of frontend (i.e. quantum circuit description) and backend compilation., Comment: 11 pages

We propose and study Th-QAOA (pronounced Threshold QAOA), a variation of the Quantum Alternating Operator Ansatz (QAOA) that replaces the standard phase separator operator, which encodes the objective function, with a threshold function that returns a value $1$ for solutions with an objective value above the threshold and a $0$ otherwise. We vary the threshold value to arrive at a quantum optimization algorithm. We focus on a combination with the Grover Mixer operator; the resulting GM-Th-QAOA can be viewed as a generalization of Grover's quantum search algorithm and its minimum/maximum finding cousin to approximate optimization. Our main findings include: (i) we provide intuitive arguments and show empirically that the optimum parameter values of GM-Th-QAOA (angles and threshold value) can be found with $O(\log(p) \times \log M)$ iterations of the classical outer loop, where $p$ is the number of QAOA rounds and $M$ is an upper bound on the solution value (often the number of vertices or edges in an input graph), thus eliminating the notorious outer-loop parameter finding issue of other QAOA algorithms; (ii) GM-Th-QAOA can be simulated classically with little effort up to 100 qubits through a set of tricks that cut down memory requirements; (iii) somewhat surprisingly, GM-Th-QAOA outperforms non-thresholded GM-QAOA in terms of approximation ratios achieved. This third result holds across a range of optimization problems (MaxCut, Max k-VertexCover, Max k-DensestSubgraph, MaxBisection) and various experimental design parameters, such as different input edge densities and constraint sizes., Comment: V2

This work is the first to employ and adapt the image-to-image translation concept based on conditional generative adversarial networks (cGAN) towards learning a forward and an inverse solution operator of partial differential equations (PDEs). Even though the proposed framework could be applied as a surrogate model for the solution of any PDEs, here we focus on steady-state solutions of coupled hydro-mechanical processes in heterogeneous porous media. Strongly heterogeneous material properties, which translate to the heterogeneity of coefficients of the PDEs and discontinuous features in the solutions, require specialized techniques for the forward and inverse solution of these problems. Additionally, parametrization of the spatially heterogeneous coefficients is excessively difficult by using standard reduced order modeling techniques. In this work, we overcome these challenges by employing the image-to-image translation concept to learn the forward and inverse solution operators and utilize a U-Net generator and a patch-based discriminator. Our results show that the proposed data-driven reduced order model has competitive predictive performance capabilities in accuracy and computational efficiency as well as training time requirements compared to state-of-the-art data-driven methods for both forward and inverse problems.

Quantum annealing is an emerging technology with the potential to solve some of the computational challenges that remain unresolved as we approach an era beyond Moore's Law. In this work, we investigate the capabilities of the quantum annealers of D-Wave Systems, Inc., for computing a certain type of Boolean tensor decomposition called Boolean Hierarchical Tucker Network (BHTN). Boolean tensor decomposition problems ask for finding a decomposition of a high-dimensional tensor with categorical, [true, false], values, as a product of smaller Boolean core tensors. As the BHTN decompositions are usually not exact, we aim to approximate an input high-dimensional tensor by a product of lower-dimensional tensors such that the difference between both is minimized in some norm. We show that BHTN can be calculated as a sequence of optimization problems suitable for the D-Wave 2000Q quantum annealer. Although current technology is still fairly restricted in the problems they can address, we show that a complex problem such as BHTN can be solved efficiently and accurately.

We study the status of fair sampling on Noisy Intermediate Scale Quantum (NISQ) devices, in particular the IBM Q family of backends. Using the recently introduced Grover Mixer-QAOA algorithm for discrete optimization, we generate fair sampling circuits to solve six problems of varying difficulty, each with several optimal solutions, which we then run on twenty backends across the IBM Q system. For a given circuit evaluated on a specific set of qubits, we evaluate: how frequently the qubits return an optimal solution to the problem, the fairness with which the qubits sample from all optimal solutions, and the reported hardware error rate of the qubits. To quantify fairness, we define a novel metric based on Pearson's $\chi^2$ test. We find that fairness is relatively high for circuits with small and large error rates, but drops for circuits with medium error rates. This indicates that structured errors dominate in this regime, while unstructured errors, which are random and thus inherently fair, dominate in noisier qubits and longer circuits. Our results show that fairness can be a powerful tool for understanding the intricate web of errors affecting current NISQ hardware.

Simulation of the crack network evolution on high strain rate impact experiments performed in brittle materials is very compute-intensive. The cost increases even more if multiple simulations are needed to account for the randomness in crack length, location, and orientation, which is inherently found in real-world materials. Constructing a machine learning emulator can make the process faster by orders of magnitude. There has been little work, however, on assessing the error associated with their predictions. Estimating these errors is imperative for meaningful overall uncertainty quantification. In this work, we extend the heteroscedastic uncertainty estimates to bound a multiple output machine learning emulator. We find that the response prediction is accurate within its predicted errors, but with a somewhat conservative estimate of uncertainty.

Homomorphic encryption has been an area of study in classical computing for decades. The fundamental goal of homomorphic encryption is to enable (untrusted) Oscar to perform a computation for Alice without Oscar knowing the input to the computation or the output from the computation. Alice encrypts the input before sending it to Oscar, and Oscar performs the computation directly on the encrypted data, producing an encrypted result. Oscar then sends the encrypted result of the computation back to Alice, who can decrypt it. We describe an approach to homomorphic encryption for quantum annealing based on spin reversal transformations and show that it comes with little or no performance penalty. This is in contrast to approaches to homomorphic encryption for classical computing, which incur a significant additional computational cost. This implies that the performance gap between quantum annealing and classical computing is reduced when both paradigms use homomorphic encryption. Further, homomorphic encryption is critical for quantum annealing because quantum annealers are native to the cloud -- a third party (such as untrusted Oscar) performs the computation. If sensitive information, such as health-related data subject to the Health Insurance Portability and Accountability Act, is to be processed with quantum annealers, such a technique could be useful., Comment: to be published at IEEE HPEC 2020

Sampling all ground states of a Hamiltonian with equal probability is a desired feature of a sampling algorithm, but recent studies indicate that common variants of transverse field quantum annealing sample the ground state subspace unfairly. In this note, we present perturbation theory arguments suggesting that this deficiency can be corrected by employing reverse annealing-inspired paths. We confirm that this conclusion holds in simulations of previously studied small instances with degeneracy, as well as larger instances on quantum annealing hardware., Comment: 5 pages, 5 figures

It was recently shown that quantum annealing can be used as an effective, fast subroutine in certain types of matrix factorization algorithms. The quantum annealing algorithm performed best for quick, approximate answers, but performance rapidly plateaued. In this paper, we utilize reverse annealing instead of forward annealing in the quantum annealing subroutine for nonnegative/binary matrix factorization problems. After an initial global search with forward annealing, reverse annealing performs a series of local searches that refine existing solutions. The combination of forward and reverse annealing significantly improves performance compared to forward annealing alone for all but the shortest run times., Comment: 9 pages, 5 figures

Failure in brittle materials under dynamic loading conditions is a result of the propagation and coalescence of microcracks. Simulating this mechanism at the continuum level is computationally expensive or, in some cases, intractable. The computational cost is due to the need for highly resolved computational meshes required to capture complex crack growth behavior, such as branching, turning, etc. Typically, continuum-scale models that account for brittle damage evolution homogenize the crack network in some way, which reduces the overall computational cost, but can also neglect key physics of the subgrid crack growth behavior, sacrificing accuracy for efficiency. We have developed an approach using machine learning that overcomes the current inability to represent micro-scale physics at the macro-scale. Our approach leverages damage and stress data from a high-fidelity model that explicitly resolves microcrack behavior to build an inexpensive machine learning emulator, which runs in seconds as opposed to the high-fidelity model, which takes hours. Once trained, the machine learning emulator is used to predict the evolution of crack length statistics. A continuum-scale constitutive model is then informed with these crack statistics, speeding up the workflow by four orders of magnitude. Both the machine learning model and the continuum-scale model are validated against a high-fidelity model and experimental data, respectively, showing excellent agreement. There are two key findings. The first is that we can reduce the dimensionality of the problem, establishing that the machine learning emulator only needs the length of the longest crack and one of the maximum stress components to capture the necessary physics. Another compelling finding is that the emulator can be trained in one experimental setting and transferred successfully to predict behavior in a different setting., Comment: Keywords: Computational Material Science, Machine Learning. 27 pages,13 figures, in review at COMMAT Elsevier journal

It was recently shown that certain subsurface hydrological inverse problems -- here framed as determining the composition of an aquifer from pressure readings -- can be solved on a quantum annealer. However, the quantum annealer performance suffered when solving problems where the aquifer was composed of materials with vastly different permeability, which is often encountered in practice. In this paper, we study why this regime is difficult and use several pre- and post-processing tools to address these issues. This study has three benefits: it improves quantum annealing performance for real-world problems in hydrology, it studies the scaling behavior for these problems (which were previously studied at a fixed size), and it elucidates a challenging class of problems that are amenable to quantum annealers.

Inverse problems often involve matching observational data using a physical model that takes a large number of parameters as input. These problems tend to be under-constrained and require regularization to impose additional structure on the solution in parameter space. A central difficulty in regularization is turning a complex conceptual model of this additional structure into a functional mathematical form to be used in the inverse analysis. In this work we propose a method of regularization involving a machine learning technique known as a variational autoencoder (VAE). The VAE is trained to map a low-dimensional set of latent variables with a simple structure to the high-dimensional parameter space that has a complex structure. We train a VAE on unconditioned realizations of the parameters for a hydrological inverse problem. These unconditioned realizations neither rely on the observational data used to perform the inverse analysis nor require any forward runs of the physical model, thus making the computational cost of generating the training data minimal. The central benefit of this approach is that regularization is then performed on the latent variables from the VAE, which can be regularized simply. A second benefit of this approach is that the VAE reduces the number of variables in the optimization problem, thus making gradient-based optimization more computationally efficient when adjoint methods are unavailable. After performing regularization and optimization on the latent variables, the VAE then decodes the problem back to the original parameter space. Our approach constitutes a novel framework for regularization and optimization, readily applicable to a wide range of inverse problems. We call the approach RegAE.

Sensitivity analysis plays an important role in searching for constitutive parameters (e.g. permeability) subsurface flow simulations. The mathematics behind is to solve a dynamic constrained optimization problem. Traditional methods like finite difference and forward sensitivity analysis require computational cost that increases linearly with the number of parameters times number of cost functions. Discrete adjoint sensitivity analysis (SA) is gaining popularity due to its computational efficiency. This algorithm requires a forward run followed by a backward run who involves integrating adjoint equation backward in time. This was done by doing one forward solve and store the snapshot by checkpointing. Using the checkpoint data, the adjoint equation is numerically integrated. The computational cost of this algorithm only depends on the number of cost functions and does not depend on the number of parameters. The algorithm is highly powerful when the parameter space is large, and in our case of heterogeneous permeability the number of parameters is proportional to the number of grid cells. The aim of this project is to implement the discrete sensitivity analysis method in parallel to solve realistic subsurface problems. To achieve this goal, we propose to implement the algorithm in parallel using data structures such as TSAdjoint and TAO. This paper dealt with large-scale subsurface flow inversion problem with discrete adjoint method. This method can effectively reduce the computational cost in sensitivity analysis.

As quantum computers become available to the general public, the need has arisen to train a cohort of quantum programmers, many of whom have been developing classical computer programs for most of their careers. While currently available quantum computers have less than 100 qubits, quantum computing hardware is widely expected to grow in terms of qubit count, quality, and connectivity. This review aims to explain the principles of quantum programming, which are quite different from classical programming, with straightforward algebra that makes understanding of the underlying fascinating quantum mechanical principles optional. We give an introduction to quantum computing algorithms and their implementation on real quantum hardware. We survey 20 different quantum algorithms, attempting to describe each in a succinct and self-contained fashion. We show how these algorithms can be implemented on IBM's quantum computer, and in each case, we discuss the results of the implementation with respect to differences between the simulator and the actual hardware runs. This article introduces computer scientists, physicists, and engineers to quantum algorithms and provides a blueprint for their implementations., Comment: ACM Transactions on Quantum Computing

D-Wave quantum annealers represent a novel computational architecture and have attracted significant interest, but have been used for few real-world computations. Machine learning has been identified as an area where quantum annealing may be useful. Here, we show that the D-Wave 2X can be effectively used as part of an unsupervised machine learning method. This method can be used to analyze large datasets. The D-Wave only limits the number of features that can be extracted from the dataset. We apply this method to learn the features from a set of facial images.