Search

We investigate the use of the Senseiver, a transformer neural network designed for sparse sensing applications, to estimate full-field surface height measurements of tsunami waves from sparse observations. The model is trained on a large ensemble of simulated data generated via a shallow water equations solver, which we show to be a faithful reproduction for the underlying dynamics by comparison to historical events. We train the model on a dataset consisting of 8 tsunami simulations whose epicenters correspond to historical USGS earthquake records, and where the model inputs are restricted to measurements obtained at actively deployed buoy locations. We test the Senseiver on a dataset consisting of 8 simulations not included in training, demonstrating its capability for extrapolation. The results show remarkable resolution of fine scale phase and amplitude features from the true field, provided that at least a few of the sensors have obtained a non-zero signal. Throughout, we discuss which forecasting techniques can be improved by this method, and suggest ways in which the flexibility of the architecture can be leveraged to incorporate arbitrary remote sensing data (eg. HF Radar and satellite measurements) as well as investigate optimal sensor placements.

Understanding material failure is critical for designing stronger and lighter structures by identifying weaknesses that could be mitigated. Existing full-physics numerical simulation techniques involve trade-offs between speed, accuracy, and the ability to handle complex features like varying boundary conditions, grid types, resolution, and physical models. We present the first foundation model specifically designed for predicting material failure, leveraging large-scale datasets and a high parameter count (up to 3B) to significantly improve the accuracy of failure predictions. In addition, a large language model provides rich context embeddings, enabling our model to make predictions across a diverse range of conditions. Unlike traditional machine learning models, which are often tailored to specific systems or limited to narrow simulation conditions, our foundation model is designed to generalize across different materials and simulators. This flexibility enables the model to handle a range of material properties and conditions, providing accurate predictions without the need for retraining or adjustments for each specific case. Our model is capable of accommodating diverse input formats, such as images and varying simulation conditions, and producing a range of outputs, from simulation results to effective properties. It supports both Cartesian and unstructured grids, with design choices that allow for seamless updates and extensions as new data and requirements emerge. Our results show that increasing the scale of the model leads to significant performance gains (loss scales as $N^{-1.6}$, compared to language models which often scale as $N^{-0.5}$)., Comment: Accepted at NeurIPS 2024 "Foundation Models for Science: Progress, Opportunities, and Challenges" Workshop

This study introduces a hybrid fluid simulation approach that integrates generative diffusion models with physics-based simulations, aiming at reducing the computational costs of flow simulations while still honoring all the physical properties of interest. These simulations enhance our understanding of applications such as assessing hydrogen and CO$_2$ storage efficiency in underground reservoirs. Nevertheless, they are computationally expensive and the presence of nonunique solutions can require multiple simulations within a single geometry. To overcome the computational cost hurdle, we propose a hybrid method that couples generative diffusion models and physics-based modeling. We introduce a system to condition the diffusion model with a geometry of interest, allowing to produce variable fluid saturations in the same geometry. While training the model, we simultaneously generate initial conditions and perform physics-based simulations using these conditions. This integrated approach enables us to receive real-time feedback on a single compute node equipped with both CPUs and GPUs. By efficiently managing these processes within one compute node, we can continuously evaluate performance and stop training when the desired criteria are met. To test our model, we generate realizations in a real Berea sandstone fracture which shows that our technique is up to 4.4 times faster than commonly used flow simulation initializations.

We present a lattice model for polymer solutions, explicitly incorporating interactions with a bath of solvent and cosolvent molecules. By exploiting the well-known analogy between polymer systems and the $O(n)$-vector spin model in the limit $n \to 0$, we derive an exact field-theoretic expression for the partition function of the system. The latter is then evaluated at the saddle point, providing a mean-field estimate of the free energy. The resulting expression, which conforms to the Flory-Huggins type, is then used to analyze the system's stability with respect to phase separation, complemented by a numerical approach based on convex hull evaluation. We demonstrate that this simple lattice model can effectively explain the behavior of a variety of seemingly unrelated polymer systems, which have been predominantly investigated in the past only through numerical simulations. This includes both, single-chain and multi-chain, solutions. Our findings emphasize the fundamental, mutually competing, roles of solvent and cosolvent in polymer systems., Comment: 12 pages, 6 figures (main); 9 pages, 5 figures (suppl info); Physical Review Materials, accepted for publication

Using Langevin dynamics complemented by Wang-Landau Monte Carlo simulations, we study the phase behavior of single and multiple semiflexible polymer chains in solution under poor-solvent conditions. In the case of a single chain, we obtain the full phase diagram in the temperature-bending rigidity (stiffness) plane and we provide connections with a classical mean field result on a lattice as well as with past results on the same model. At low bending rigidity and upon cooling, we find a second order coil-globule transition, followed by a subsequent first order globule-crystal transition at lower temperatures. The obtained crystals have the shape of a twisted rod whose length increases with the increase of the stiffness of the chain. Above a critical value of the stiffness, we also find a direct first order globule-crystal transition, with the crystal having the form of a twisted toroid. Close to the triple point, we find a region with isoenergetic structures with frequent switching from rods to toroids, with the toroid eventually becoming the only observed stable phase at higher stiffness. The model is then extended to many thermally equilibrated chains in a box and the analogous phase diagram is deduced where the chains are observed to first fold into a globule bundle at low stiffness upon cooling, and then rearrange into a nematic bundle via a nucleation process involving an isotropic-nematic transition. As in the single chain counterpart, above a critical stiffness the chains are observed to undergo a direct transition from a gas of isotropically distributed chains to a nematic bundle as the temperature decreases in agreement with recent suggestions from mean field theory. The consequences of these findings for self-assembly of biopolymers in solutions are discussed., Comment: 59 pages, supporting information; published in Macromolecules (2024)

Miniaturization of light-emitting diodes (LEDs) can enable high-resolution augmented and virtual reality displays and on-chip light sources for ultra-broadband chiplet communication. However, unlike silicon scaling in electronic integrated circuits, patterning of inorganic III-V semiconductors in LEDs considerably compromises device efficiencies at submicrometer scales. Here, we present the scalable fabrication of nanoscale organic LEDs (nano-OLEDs), with the highest array density (>84,000 pixels per inch) and the smallest pixel size (~100 nm) ever reported to date. Direct nanomolecular patterning of organic semiconductors is realized by self-aligned evaporation through nanoapertures fabricated on a free-standing silicon nitride film adhering to the substrate. The average external quantum efficiencies (EQEs) extracted from a nano-OLED device of more than 4 megapixels reach up to 10%. At the subwavelength scale, individual pixels act as electroluminescent meta-atoms forming metasurfaces that directly convert electricity into modulated light. The diffractive coupling between nano-pixels enables control over the far-field emission properties, including directionality and polarization. The results presented here lay the foundation for bright surface light sources of dimension smaller than the Abbe diffraction limit, offering new technological platforms for super-resolution imaging, spectroscopy, sensing, and hybrid integrated photonics.

The urgent need for decarbonization in the energy sector has led to an increased emphasis on the integration of renewable energy sources, such as wind and solar, into power grids. While these resources offer significant environmental benefits, they also introduce challenges related to intermittency and variability. Long-duration energy storage (LDES) technologies have emerged as a very promising solution to address these challenges by storing excess energy during periods of high generation and delivering it when demand is high or renewable resources are scarce for a sustained amount of time. This paper introduces a novel methodology for estimating the boundary technology cost of LDES systems for economic viability in decarbonized energy systems. Our methodology is applied to estimate the boundary costs in 2050 for the state of California to achieve full retirement of gas power plants. California's ambitious decarbonization goals and transition to a renewable energy-based power system present an ideal context for examining the role of LDES. The results also offer insights into the needed capacity expansion planning and the operational contribution of LDES in the California's energy landscape, taking into account the unique energy demand profiles and renewable resource availability of the region. Our findings are intended to provide complementary information to guide decision-makers, energy planners, and any other stakeholders in making informed choices about LDES investment in the context of a decarbonized energy future.

Recreating complex, high-dimensional global fields from limited data points is a grand challenge across various scientific and industrial domains. Given the prohibitive costs of specialized sensors and the frequent inaccessibility of certain regions of the domain, achieving full field coverage is typically not feasible. Therefore, the development of algorithms that intelligently improve sensor placement is of significant value. In this study, we introduce a general approach that employs differentiable programming to exploit sensor placement within the training of a neural network model in order to improve field reconstruction. We evaluated our method using two distinct datasets; the results show that our approach improved test scores. Ultimately, our method of differentiable placement strategies has the potential to significantly increase data collection efficiency, enable more thorough area coverage, and reduce redundancy in sensor deployment.

Pore-scale simulations accurately describe transport properties of fluids in the subsurface. These simulations enhance our understanding of applications such as assessing hydrogen storage efficiency and forecasting CO$_2$ sequestration processes in underground reservoirs. Nevertheless, they are computationally expensive due to their mesoscopic nature. In addition, their stationary solutions are not guaranteed to be unique, so multiple runs with different initial conditions must be performed to ensure sufficient sample coverage. These factors complicate the task of obtaining representative and reliable forecasts. To overcome the high computational cost hurdle, we propose a hybrid method that couples generative diffusion models and physics-based modeling. Upon training a generative model, we synthesize samples that serve as the initial conditions for physics-based simulations. We measure the relaxation time (to stationary solutions) of the simulations, which serves as a validation metric and early-stopping criterion. Our numerical experiments revealed that the hybrid method exhibits a speed-up of up to 8.2 times compared to commonly used initialization methods. This finding offers compelling initial support that the proposed diffusion model-based hybrid scheme has potentials to significantly decrease the time required for convergence of numerical simulations without compromising the physical robustness.

Modeling effective transport properties of 3D porous media, such as permeability, at multiple scales is challenging as a result of the combined complexity of the pore structures and fluid physics - in particular, confinement effects which vary across the nanoscale to the microscale. While numerical simulation is possible, the computational cost is prohibitive for realistic domains, which are large and complex. Although machine learning models have been proposed to circumvent simulation, none so far has simultaneously accounted for heterogeneous 3D structures, fluid confinement effects, and multiple simulation resolutions. By utilizing numerous computer science techniques to improve the scalability of training, we have for the first time developed a general flow model that accounts for the pore-structure and corresponding physical phenomena at scales from Angstrom to the micrometer. Using synthetic computational domains for training, our machine learning model exhibits strong performance (R$^2$=0.9) when tested on extremely diverse real domains at multiple scales.

We study a solution of interacting semiflexible polymers with curvature energy in poor-solvent conditions on the d-dimensional cubic lattice using mean-field theory and Monte Carlo computer simulations. Building upon past studies on a single chain, we construct a field-theory representation of the system and solve it within a mean-field approximation supported by Monte Carlo simulations in d=3. A gas-liquid transition is found in the temperature-density plane that is then interpreted in terms of real systems. Interestingly, we find this transition to be independent of the bending rigidity. Past classical Flory-Huggins and Flory mean-field results are shown to be particular cases of this more general framework. Perspectives in terms of guiding experimental results towards optimal conditions are also proposed., Comment: 19 pages, 10 figures, 3 appendices, accepted for publication

Segmentation is an essential step for remote sensing image processing. This study aims to advance the application of the Segment Anything Model (SAM), an innovative image segmentation model by Meta AI, in the field of remote sensing image analysis. SAM is known for its exceptional generalization capabilities and zero-shot learning, making it a promising approach to processing aerial and orbital images from diverse geographical contexts. Our exploration involved testing SAM across multi-scale datasets using various input prompts, such as bounding boxes, individual points, and text descriptors. To enhance the model's performance, we implemented a novel automated technique that combines a text-prompt-derived general example with one-shot training. This adjustment resulted in an improvement in accuracy, underscoring SAM's potential for deployment in remote sensing imagery and reducing the need for manual annotation. Despite the limitations encountered with lower spatial resolution images, SAM exhibits promising adaptability to remote sensing data analysis. We recommend future research to enhance the model's proficiency through integration with supplementary fine-tuning techniques and other networks. Furthermore, we provide the open-source code of our modifications on online repositories, encouraging further and broader adaptations of SAM to the remote sensing domain., Comment: 20 pages, 9 figures

Multi-task learning has proven to be effective in improving the performance of correlated tasks. Most of the existing methods use a backbone to extract initial features with independent branches for each task, and the exchange of information between the branches usually occurs through the concatenation or sum of the feature maps of the branches. However, this type of information exchange does not directly consider the local characteristics of the image nor the level of importance or correlation between the tasks. In this paper, we propose a semantic segmentation method, MTLSegFormer, which combines multi-task learning and attention mechanisms. After the backbone feature extraction, two feature maps are learned for each task. The first map is proposed to learn features related to its task, while the second map is obtained by applying learned visual attention to locally re-weigh the feature maps of the other tasks. In this way, weights are assigned to local regions of the image of other tasks that have greater importance for the specific task. Finally, the two maps are combined and used to solve a task. We tested the performance in two challenging problems with correlated tasks and observed a significant improvement in accuracy, mainly in tasks with high dependence on the others., Comment: Accepted 4th Agriculture-Vision Workshop - CVPRW

Recent advancements in Natural Language Processing (NLP), particularly in Large Language Models (LLMs), associated with deep learning-based computer vision techniques, have shown substantial potential for automating a variety of tasks. One notable model is Visual ChatGPT, which combines ChatGPT's LLM capabilities with visual computation to enable effective image analysis. The model's ability to process images based on textual inputs can revolutionize diverse fields. However, its application in the remote sensing domain remains unexplored. This is the first paper to examine the potential of Visual ChatGPT, a cutting-edge LLM founded on the GPT architecture, to tackle the aspects of image processing related to the remote sensing domain. Among its current capabilities, Visual ChatGPT can generate textual descriptions of images, perform canny edge and straight line detection, and conduct image segmentation. These offer valuable insights into image content and facilitate the interpretation and extraction of information. By exploring the applicability of these techniques within publicly available datasets of satellite images, we demonstrate the current model's limitations in dealing with remote sensing images, highlighting its challenges and future prospects. Although still in early development, we believe that the combination of LLMs and visual models holds a significant potential to transform remote sensing image processing, creating accessible and practical application opportunities in the field.

Deep learning-based networks are among the most prominent methods to learn linear patterns and extract this type of information from diverse imagery conditions. Here, we propose a deep learning approach based on graphs to detect plantation lines in UAV-based RGB imagery presenting a challenging scenario containing spaced plants. The first module of our method extracts a feature map throughout the backbone, which consists of the initial layers of the VGG16. This feature map is used as an input to the Knowledge Estimation Module (KEM), organized in three concatenated branches for detecting 1) the plant positions, 2) the plantation lines, and 3) for the displacement vectors between the plants. A graph modeling is applied considering each plant position on the image as vertices, and edges are formed between two vertices (i.e. plants). Finally, the edge is classified as pertaining to a certain plantation line based on three probabilities (higher than 0.5): i) in visual features obtained from the backbone; ii) a chance that the edge pixels belong to a line, from the KEM step; and iii) an alignment of the displacement vectors with the edge, also from KEM. Experiments were conducted in corn plantations with different growth stages and patterns with aerial RGB imagery. A total of 564 patches with 256 x 256 pixels were used and randomly divided into training, validation, and testing sets in a proportion of 60\%, 20\%, and 20\%, respectively. The proposed method was compared against state-of-the-art deep learning methods, and achieved superior performance with a significant margin, returning precision, recall, and F1-score of 98.7\%, 91.9\%, and 95.1\%, respectively. This approach is useful in extracting lines with spaced plantation patterns and could be implemented in scenarios where plantation gaps occur, generating lines with few-to-none interruptions., Comment: 19 pages, 11 figures, 4 tables

Semantic segmentation is a classic computer vision task with multiple applications, which includes medical and remote sensing image analysis. Despite recent advances with deep-based approaches, labeling samples (pixels) for training models is laborious and, in some cases, unfeasible. In this paper, we present two novel meta learning methods, named WeaSeL and ProtoSeg, for the few-shot semantic segmentation task with sparse annotations. We conducted extensive evaluation of the proposed methods in different applications (12 datasets) in medical imaging and agricultural remote sensing, which are very distinct fields of knowledge and usually subject to data scarcity. The results demonstrated the potential of our method, achieving suitable results for segmenting both coffee/orange crops and anatomical parts of the human body in comparison with full dense annotation.

Recently, methods based on Convolutional Neural Networks (CNN) achieved impressive success in semantic segmentation tasks. However, challenges such as the class imbalance and the uncertainty in the pixel-labeling process are not completely addressed. As such, we present a new approach that calculates a weight for each pixel considering its class and uncertainty during the labeling process. The pixel-wise weights are used during training to increase or decrease the importance of the pixels. Experimental results show that the proposed approach leads to significant improvements in three challenging segmentation tasks in comparison to baseline methods. It was also proved to be more invariant to noise. The approach presented here may be used within a wide range of semantic segmentation methods to improve their robustness., Comment: 15 pages, 9 figures, 3 tables

This paper presents a Convolutional Neural Network (CNN) approach for counting and locating objects in high-density imagery. To the best of our knowledge, this is the first object counting and locating method based on a feature map enhancement and a Multi-Stage Refinement of the confidence map. The proposed method was evaluated in two counting datasets: tree and car. For the tree dataset, our method returned a mean absolute error (MAE) of 2.05, a root-mean-squared error (RMSE) of 2.87 and a coefficient of determination (R$^2$) of 0.986. For the car dataset (CARPK and PUCPR+), our method was superior to state-of-the-art methods. In the these datasets, our approach achieved an MAE of 4.45 and 3.16, an RMSE of 6.18 and 4.39, and an R$^2$ of 0.975 and 0.999, respectively. The proposed method is suitable for dealing with high object-density, returning a state-of-the-art performance for counting and locating objects., Comment: 15 pages, 10 figures, 8 tables