Search

We report a mixture of expert strategy to create fine-tuned large language models using a deep layer-wise token-level approach based on low-rank adaptation (LoRA). Starting with a set of pre-trained LoRA adapters, our gating strategy uses the hidden states to dynamically mix adapted layers, allowing the resulting X-LoRA model to draw upon different capabilities and create never-before-used deep layer-wise combinations to solve tasks. The design is inspired by the biological principles of universality and diversity, where neural network building blocks are reused in different hierarchical manifestations. Hence, the X-LoRA model can be easily implemented for any existing large language model without a need for modifications of the underlying structure. We develop a tailored X-LoRA model that offers scientific capabilities, including forward/inverse analysis tasks and enhanced reasoning capability, focused on biomaterial analysis, protein mechanics, and design. The impact of this work includes access to readily expandable and adaptable models with strong domain knowledge and the capability to integrate across areas of knowledge. Featuring experts in biology, mathematics, reasoning, bio-inspired materials, mechanics and materials, chemistry, protein biophysics, mechanics, and quantum-mechanics based molecular properties, we conduct a series of physics-focused case studies. We examine knowledge recall, protein mechanics forward/inverse tasks, protein design, adversarial agentic modeling including ontological knowledge graph construction, and molecular design. The model is capable not only of making quantitative predictions of nanomechanical properties of proteins or quantum mechanical molecular properties but also reasoning over the results and correctly predicting likely mechanisms that explain distinct molecular behaviors.

Abstract The study of biological materials and bio‐inspired materials science is well established; however, surprisingly little knowledge is systematically translated to engineering solutions. To accelerate discovery and guide insights, an open‐source autoregressive transformer large language model (LLM), BioinspiredLLM, is reported. The model is finetuned with a corpus of over a thousand peer‐reviewed articles in the field of structural biological and bio‐inspired materials and can be prompted to recall information, assist with research tasks, and function as an engine for creativity. The model has proven that it is able to accurately recall information about biological materials and is further strengthened with enhanced reasoning ability, as well as with Retrieval‐Augmented Generation (RAG) to incorporate new data during generation that can also help to traceback sources, update the knowledge base, and connect knowledge domains. BioinspiredLLM also has shown to develop sound hypotheses regarding biological materials design and remarkably so for materials that have never been explicitly studied before. Lastly, the model shows impressive promise in collaborating with other generative artificial intelligence models in a workflow that can reshape the traditional materials design process. This collaborative generative artificial intelligence method can stimulate and enhance bio‐inspired materials design workflows. Biological materials are at a critical intersection of multiple scientific fields and models like BioinspiredLLM help to connect knowledge domains.

Abstract Architected materials can achieve enhanced properties compared to their plain counterparts. Specific architecting serves as a powerful design lever to achieve targeted behavior without changing the base material. Thus, the connection between architected structure and resultant properties remains an open field of great interest to many fields, from aerospace to civil to automotive applications. Here, we focus on properties related to mechanical compression, and design hierarchical honeycomb structures to meet specific values of stiffness and compressive stress. To do so, we employ a combination of techniques in a singular workflow, starting with molecular dynamics simulation of the forward design problem, augmenting with data-driven artificial intelligence models to address the inverse design problem, and verifying the behavior of de novo structures with experimentation of additively manufactured samples. We thereby demonstrate an approach for architected design that is generalizable to multiple material properties and agnostic to the identity of the base material.

Abstract Structural defects are abundant in solids, and vital to the macroscopic materials properties. However, a defect-property linkage typically requires significant efforts from experiments or simulations, and often contains limited information due to the breadth of nanoscopic design space. Here we report a graph neural network (GNN)-based approach to achieve direct translation between mesoscale crystalline structures and atom-level properties, emphasizing the effects of structural defects. Our end-to-end method offers great performance and generality in predicting both atomic stress and potential energy of multiple systems with different defects. Furthermore, the approach also precisely captures derivative properties which strictly observe physical laws and reproduces evolution of properties with varying boundary conditions. By incorporating a genetic algorithm, we then design de novo atomic structures with optimum global properties and target local patterns. The method would significantly enhance the efficiency of evaluating atomic behaviors given structural imperfections and accelerates the design process at the meso-level.

The dynamics of material failure is a critical phenomenon relevant to a range of scientific and engineering fields, from healthcare to structural materials. We propose a specially designed deep neural network, DyFraNet, which can predict dynamic fracture behaviors by identifying a complete history of fracture propagation—from the onset of cracking, as a crack grows through the material, modeled as a series of frames evolving over time and dependent on each other. Furthermore, the model can not only forecast future fracture processes but also backcast to elucidate past fracture histories. In this scenario, once provided with the outcome of a fracture event, the model will reveal past events that led to this state and can also predict future evolutions of the failure process. By comparing the predicted results with atomistic-level simulations and theory, we show that DyFraNet can capture dynamic fracture mechanics by accurately predicting how cracks develop over time, including measures such as the crack speed, as well as when cracks become unstable. We use Gradient-weighted Class Activation Mapping, Grad-CAM, to interpret how DyFraNet perceives the relationship between geometric conditions and fracture dynamics, and we find that DyFraNet pays special attention to the areas around crack tips that have a critical influence in the early stage of fracture propagation. In later stages, the model pays increased attention to the existing or newly formed damaged regions in the material. The proposed approach offers the potential to accelerate the exploration of dynamical processes in material design against failure and can be adapted for all kinds of dynamical problems.

Machine learning (ML) has emerged as an indispensable methodology to describe, discover, and predict complex physical phenomena that efficiently help us learn underlying functional rules, especially in cases when conventional modeling approaches cannot be applied. While conventional feedforward neural networks are typically limited to performing tasks related to static patterns in data, recursive models can both work iteratively based on a changing input and discover complex dynamical relationships in the data. Deep language models can model flexible modalities of data and are capable of learning rich dynamical behaviors as they operate on discrete or continuous symbols that define the states of a physical system, yielding great potential toward end-to-end predictions. Similar to how words form a sentence, materials can be considered as a self-assembly of physically interacted building blocks, where the emerging functions of materials are analogous to the meaning of sentences. While discovering the fundamental relationships between building blocks and function emergence can be challenging, language models, such as recurrent neural networks and long-short term memory networks, and, in particular, attention models, such as the transformer architecture, can solve many such complex problems. Application areas of such models include protein folding, molecular property prediction, prediction of material failure of complex nonlinear architected materials, and also generative strategies for materials discovery. We outline challenges and opportunities, especially focusing on extending the deep-rooted kinship of humans with symbolism toward generalizable artificial intelligence (AI) systems using neuro-symbolic AI, and outline how tools such as ChatGPT and DALL·E can drive materials discovery.

The prediction of stress, strain fields or fracture patterns, along with other physical quantities as a function of boundary conditions and material microstructure, is an important objective of various physics-based simulation methods like finite element models or molecular dynamics modeling. However, such simulations may be expensive and can be costly in their execution, limiting their application to solve inverse design problems, or in the discovery of novel materials that feature a desirable performance. In other fields of studies, such as in mechanobiology, field data can often not be calculated due to a lack of models to begin with; where experimental observations may be the only source of insight. Here we introduce a data-driven method to relate input microstructures and field data using an alternative approach, enabled by deep learning, to predict complex mechanical fields directly based on input microstructure data. We report two approaches to solve said problem, first using cycle-consistent adversarial neural networks trained on unpaired images of microstructures and physical fields, and second, using a Perceiver neural networks based on an attention mechanism based on paired images, implemented in a transformer neural network. Due to the cycle consistent nature of the GAN model we can immediately make forward (from microstructure to stress field) and backward (from stress field to microstructure) predictions, offering an avenue for inverse material design. The Perceiver model, implementing an efficient transformer model that can be scaled to very large input and output data, shows strong generalizability and can be successfully applied to scenarios distinct from those in the training set. We demonstrate the model in several sample microstructures including a set of small cracks (voids) to predict both stress fields and fracture patterns, as well as in a composite microstructure of a soft matrix and rigid inclusions, and in a series of solutions to the inverse problem facilitated by the cycle consistent GAN model. The applications of the models include biomimetic biotechnology (e.g. bone-tendon-inspired attachments and gradients), armor materials, mechanobiology, soft robotics, and lightweight resilient and tough structural systems.

Abstract Understanding fracture is critical to the design of resilient nanomaterials. Molecular dynamics offers a way to study fracture at an atomistic level, but is computationally expensive with limitations of scalability. In this work, we build upon machine-learning approaches for predicting nanoscopic fracture mechanisms including crack instabilities and branching as a function of crystal orientation. We focus on a particular technologically relevant material system, graphene, and apply a deep learning method to the study of such nanomaterials and explore the parameter space necessary for calibrating machine-learning predictions to meaningful results. Our results validate the ability of deep learning methods to quantitatively capture graphene fracture behavior, including its fractal dimension as a function of crystal orientation, and provide promise toward the wider application of deep learning to materials design, opening the potential for other 2D materials.

Mechanical strength of in situ assembled nuclear lamin filaments arranged in a 3D meshwork is unclear. Here, using mechanical, structural and simulation tools, the authors report the hierarchical organization of the lamin meshwork that imparts strength and toughness to lamin filaments at par with silk and Kevlar®

We describe a method to generate 3D architected materials based on mathematically parameterized human readable word input, offering a direct materialization of language. Our method uses a combination of a vector quantized generative adversarial network and contrastive language-image pre-training neural networks to generate images, which are translated into 3D architectures that are then 3D printed using fused deposition modeling into materials with varying rigidity. The novel materials are further analyzed in a metallic realization as an aluminum-based nano-architecture, using molecular dynamics modeling and thereby providing mechanistic insights into the physical behavior of the material under extreme compressive loading. This work offers a novel way to design, understand, and manufacture 3D architected materials designed from mathematically parameterized language input. Our work features, at its core, a generally applicable algorithm that transforms any 2D image data into hierarchical fully tileable, periodic architected materials. This method can have broader applications beyond language-based materials design and can render other avenues for the analysis and manufacturing of architected materials, including microstructure gradients through parametric modeling. As an emerging field, language-based design approaches can have a profound impact on end-to-end design environments and drive a new understanding of physical phenomena that intersect directly with human language and creativity. It may also be used to exploit information mined from diverse and complex databases and data sources.

Variational autoencoders (VAE) are machine learning models that can extract low dimensional representations of data from datasets of high complexity and volume. Importantly, they can be used for generative purposes to reconstruct complex data, such as images, from a low dimensional encoding of only a few variables. Long short-term memory (LSTM) neural networks are well suited to learning logical trajectory relationships within datasets. Using these two models in concert, we develop a VAE-LSTM approach to learn a classic mechanical materials design problem. Here, we focus on the compliance optimization of cantilever design, using a VAE to encode cantilever structures into a 2D latent space and a LSTM to learn trajectories in that latent space corresponding to the optimization process. Ultimately, we are able to clearly visualize the space of cantilever design, generate new design with extremely low density beyond the original dataset, and obtain optimal cantilever structures inspired by nature. We also demonstrate how the resulting designs can be manufactured using 3D printing, offering a rapid pathway from concept to prototype. The method we developed here can be generalized to other image-based datasets encapsulating changes from multiple factors. The ability offered by our approach to interpret complex behavior, via representations in simplified space, has great potential for application in the intelligent design and manufacturing of materials structure problems.

Transformer neural networks have become widely used in a variety of AI applications, enabling significant advances in Natural Language Processing (NLP) and computer vision. Here we demonstrate the use of transformer neural networks in the de novo design of architected materials using a unique approach based on text input that enables the design to be directed by descriptive text, such as “a regular lattice of steel”. Since transformer neural nets enable the conversion of data from distinct forms into one another, including text into images, such methods have the potential to be used as a natural-language-driven tool to develop complex materials designs. In this study we use the Contrastive Language-Image Pre-Training (CLIP) and VQGAN neural networks in an iterative process to generate images that reflect text prompt driven materials designs. We then use the resulting images to generate three-dimensional models that can be realized using additive manufacturing, resulting in physical samples of these text-based materials. We present several such word-to-matter examples, and analyze 3D printed material specimen through associated additional finite element analysis, especially focused on mechanical properties including mechanism design. As an emerging new field, such language-based design approaches can have profound impact, including the use of transformer neural nets to generate machine code for 3D printing, optimization of processing conditions, and other end-to-end design environments that intersect directly with human language.

Summary: Fire has fascinated humankind since the prehistoric era. Rooted in the interactions between sound and flames, here we report a method to use fire for a variety of purposes, including sonification, art, and the design and manufacturing nature-inspired materials. We present a method to sonify fire, thereby offering a translation from the silent nature of flames, to represent audible information and to generate de novo flame images. To realize material specimen derived from fire, we use the autoencoder to generate image stacks to yield continuous 3D geometries that are manufactured using 3D printing. This represents the first generation of nature-inspired materials from fire and can be a platform to be used for other natural phenomena in the quest for de novo architectures, geometries, and design ideas, thus creating additional directions in artistic and scientific research through the creative manipulation of data with structural similarities across fields.

Enamel is the hardest tissue in the body and has been widely studied, yet aspects of its structure remain unclear. Here, the authors report on a study of the orientation and alignment of enamel crystals and challenge previous assumptions.

Chromatophores in cephalopod skin are known for fast changes in coloration due to light-scattering pigment granules. Here, authors demonstrate structural coloration facilitated by reflectin in sheath cells and offer insights into the interplay between structural and pigmentary coloration elements.

The transfer process of as-grown graphene limits its electrical performance and reliability. Here, the authors develop a transfer approach using paraffin as a support layer and obtain wrinkle-reduced and clean large-area graphene retaining high mobility.

We report the use of a deep learning model to design de novo proteins, based on the interplay of elementary building blocks via hierarchical patterns. The deep neural network model is based on translating protein sequences and structural information into a musical score that features different pitches for each of the amino acids, and variations in note length and note volume reflecting secondary structure information and information about the chain length and distinct protein molecules. We train a deep learning model whose architecture is composed of several long short-term memory units from data consisting of musical representations of proteins classified by certain features, focused here on alpha-helix rich proteins. Using the deep learning model, we then generate de novo musical scores and translate the pitch information and chain lengths into sequences of amino acids. We use a Basic Local Alignment Search Tool to compare the predicted amino acid sequences against known proteins, and estimate folded protein structures using the Optimized protein fold RecognitION method (ORION) and MODELLER. We find that the method proposed here can be used to design de novo proteins that do not exist yet, and that the designed proteins fold into specified secondary structures. We validate the newly predicted protein by molecular dynamics equilibration in explicit water and subsequent characterization using a normal mode analysis. The method provides a tool to design novel protein materials that could find useful applications as materials in biology, medicine, and engineering.

Natural silk fibers are produced using a simple and green approach compared to alternative synthetic methods. Here, the authors show a bioinspired approach to spin regenerated silk fibers using anisotropic liquid crystals and dry spinning, resulting in remarkably robust fibers.

Collagen denaturation is thought to occur during tissue mechanical damage, but its role in damage initiation is still unclear. Here, the authors use a collagen hybridizing peptide to provide insights into the molecular mechanisms leading to collagen unfolding during tendon mechanical stretch.

Elastin provides elastic tissues with resilience through stretch and recoil cycles, and is primarily made of its extensively cross-linked monomer, tropoelastin. Here, we leverage the recently published full atomistic model of tropoelastin to assess how allysine modifications, which are essential to cross-linking, contribute to the dynamics and structural changes that occur in tropoelastin in the context of elastin assembly. We used replica exchange molecular dynamics to generate structural ensembles of allysine containing tropoelastin. We conducted principal component analysis on these ensembles and found that the molecule departs from the canonical structural ensemble. Furthermore, we showed that, while the canonical scissors-twist movement was retained, new movements emerged that deviated from those of the wild type protein, providing evidence for the involvement of a variety of molecular motions in elastin assembly. Additionally, we highlighted secondary structural changes and linked these perturbations to the longevity of specific salt bridges. We propose a model where allysines in tropoelastin contribute to hierarchical elastin assembly through global and local perturbations to molecular structure and dynamics. Keywords: Elastin, Assembly, Molecular dynamics, Replica exchange molecular dynamics

Thermal management is important for designing bio-nano interfaces for biosensing and thermotherapic applications. Here the authors perform simulations showing that nm-thick water layers between graphene and cell membranes display layered ordering, promoting interfacial thermal coupling and thermal dissipation.

We report a novel multi-paradigm multi-scale approach based on a combination of the first principles ReaxFF force field with an empirical Tersoff potential. Our hybrid multi-scale simulation model is computationally efficient and capable of treating thousands of atoms with QM accuracy, extending our ability to simulate the dynamical behavior of a wider range of chemically complex materials such as silicon, silica and metal-organic compounds. It is implemented in the Python based Computational Materials Design Facility (CMDF). We exemplify our method in a study focused on a systematic comparison of the fracture dynamics in silicon under mode II shear versus mode I tensile loading. We find that the mode II crack tends to branch at an angle of approximately 45 degrees once the crack speed approaches 38% of the Rayleigh-wave speed. In contrast, the mode I crack continuously propagates in the direction of the initial crack, and only makes a slight change of direction towards 10 degrees once fracture instabilities occur. Our results reveal fundamental differences of fracture dynamics under mode I versus mode II loading.