Search

This survey examines the broad suite of methods and models for combining machine learning with physics knowledge for prediction and forecast, with a focus on partial differential equations. These methods have attracted significant interest due to their potential impact on advancing scientific research and industrial practices by improving predictive models with small- or large-scale datasets and expressive predictive models with useful inductive biases. The survey has two parts. The first considers incorporating physics knowledge on an architectural level through objective functions, structured predictive models, and data augmentation. The second considers data as physics knowledge, which motivates looking at multi-task, meta, and contextual learning as an alternative approach to incorporating physics knowledge in a data-driven fashion. Finally, we also provide an industrial perspective on the application of these methods and a survey of the open-source ecosystem for physics-informed machine learning., Comment: 56 pages, 8 figures, 2 tables

With the widespread adoption of Large Language Models (LLMs), concerns about potential misuse have emerged. To this end, watermarking has been adapted to LLM, enabling a simple and effective way to detect and monitor generated text. However, while the existing methods can differentiate between watermarked and unwatermarked text with high accuracy, they often face a trade-off between the quality of the generated text and the effectiveness of the watermarking process. In this work, we present a novel type of LLM watermark, Sparse Watermark, which aims to mitigate this trade-off by applying watermarks to a small subset of generated tokens distributed across the text. The key strategy involves anchoring watermarked tokens to words that have specific Part-of-Speech (POS) tags. Our experimental results demonstrate that the proposed watermarking scheme achieves high detectability while generating text that outperforms previous LLM watermarking methods in quality across various tasks

The material removal rates during milling operations are affected by the selection of the cutting depth and spindle speed. Poor selection of these parameters can result in chatter or suboptimal material removal rates. Stability Lobe Diagrams (SLDs) are the well-known approach to selecting appropriate chatter-free values for these parameters. The Physics-based stability lobe diagram is usually generated using the structural dynamics and the cutting parameters. However, since the machine dynamics are measured in the static state of the machine (zero speed), the generated SLD is not reliable as the machine behavior may vary during the cutting operations. Besides, measuring structural dynamics parameters under cutting conditions is difficult and needs new equipment. This study proposes a new approach to determining in-process structural dynamics parameters based on a multivariate Newton-Raphson method. The physics-based model is combined with empirical records to extract reliable structural dynamics parameters inversely. Some examples based on synthetic data are presented to illustrate this inverse approach. Also, the performance of the algorithm is evaluated on an empirical data set and its ability to improve the stability boundary is verified. Furthermore, the sensitivity analysis is performed to quantify the exposure of the SLD to the changes in each structural dynamics parameter.

We propose a forward-backward splitting dynamical system for solving inclusion problems of the form $0\in A(x)+B(x)$ in Hilbert spaces, where $A$ is a maximal operator and $B$ is a single-valued operator. Involved operators are assumed to satisfy a generalized monotonicity condition, which is weaker than the standard monotone assumptions. Under mild conditions on parameters, we establish the fixed-time stability of the proposed dynamical system. In addition, we consider an explicit forward Euler discretization of the dynamical system leading to a new forward backward algorithm for which we present the convergence analysis. Applications to other optimization problems such as Constrained Optimization Problems (COPs), Mixed Variational Inequalities (MVIs), and Variational Inequalities (VIs) are presented and some numerical examples are given to illustrate the theoretical results.

Prompt learning methods are gaining increasing attention due to their ability to customize large vision-language models to new domains using pre-trained contextual knowledge and minimal training data. However, existing works typically rely on optimizing unified prompt inputs, often struggling with fine-grained classification tasks due to insufficient discriminative attributes. To tackle this, we consider a new framework based on a dual context of both domain-shared and class-specific contexts, where the latter is generated by Large Language Models (LLMs) such as GPTs. Such dual prompt methods enhance the model's feature representation by joining implicit and explicit factors encoded in LLM knowledge. Moreover, we formulate the Unbalanced Optimal Transport (UOT) theory to quantify the relationships between constructed prompts and visual tokens. Through partial matching, UOT can properly align discrete sets of visual tokens and prompt embeddings under different mass distributions, which is particularly valuable for handling irrelevant or noisy elements, ensuring that the preservation of mass does not restrict transport solutions. Furthermore, UOT's characteristics integrate seamlessly with image augmentation, expanding the training sample pool while maintaining a reasonable distance between perturbed images and prompt inputs. Extensive experiments across few-shot classification and adapter settings substantiate the superiority of our model over current state-of-the-art baselines., Comment: Version 1

Large language models (LLMs), even when specifically trained to process long input contexts, struggle to capture relevant information located in the middle of their input. This phenomenon has been known as the lost-in-the-middle problem. In this work, we make three contributions. First, we set out to understand the factors that cause this phenomenon. In doing so, we establish a connection between lost-in-the-middle to LLMs' intrinsic attention bias: LLMs exhibit a U-shaped attention bias where the tokens at the beginning and at the end of its input receive higher attention, regardless of their relevance. Second, we mitigate this positional bias through a calibration mechanism, found-in-the-middle, that allows the model to attend to contexts faithfully according to their relevance, even though when they are in the middle. Third, we show found-in-the-middle not only achieves better performance in locating relevant information within a long context, but also eventually leads to improved retrieval-augmented generation (RAG) performance across various tasks, outperforming existing methods by up to 15 percentage points. These findings open up future directions in understanding LLM attention bias and its potential consequences., Comment: ACL Findings 2024

Grounded generation aims to equip language models (LMs) with the ability to produce more credible and accountable responses by accurately citing verifiable sources. However, existing methods, by either feeding LMs with raw or preprocessed materials, remain prone to errors. To address this, we introduce CaLM, a novel verification framework. CaLM leverages the insight that a robust grounded response should be consistent with information derived solely from its cited sources. Our framework empowers smaller LMs, which rely less on parametric memory and excel at processing relevant information given a query, to validate the output of larger LMs. Larger LM responses that closely align with the smaller LMs' output, which relies exclusively on cited documents, are verified. Responses showing discrepancies are iteratively refined through a feedback loop. Experiments on three open-domain question-answering datasets demonstrate significant performance gains of 1.5% to 7% absolute average without any required model fine-tuning., Comment: ACL 2024 Camera Ready Version

Instruction tuning has emerged as the key in aligning large language models (LLMs) with specific task instructions, thereby mitigating the discrepancy between the next-token prediction objective and users' actual goals. To reduce the labor and time cost to collect or annotate data by humans, researchers start to explore the use of LLMs to generate instruction-aligned synthetic data. Recent works focus on generating diverse instructions and applying LLM to increase instruction complexity, often neglecting downstream use cases. It remains unclear how to tailor high-quality data to elicit better instruction-following abilities in different target instruction distributions and LLMs. To this end, we introduce CodecLM, a general framework for adaptively generating high-quality synthetic data for LLM alignment with different downstream instruction distributions and LLMs. Drawing on the Encode-Decode principles, we use LLMs as codecs to guide the data generation process. We first encode seed instructions into metadata, which are concise keywords generated on-the-fly to capture the target instruction distribution, and then decode metadata to create tailored instructions. We also introduce Self-Rubrics and Contrastive Filtering during decoding to tailor data-efficient samples. Extensive experiments on four open-domain instruction following benchmarks validate the effectiveness of CodecLM over the current state-of-the-arts., Comment: Accepted to Findings of NAACL 2024

Doped semiconductors are a central and crucial component of all integrated circuits. By using a combination of white light and a focused laser beam, and exploiting hBN defect states, heterostructures of hBN/Graphene/hBN are photodoped in-operando, reproducibly and reversibly. We demonstrate device geometries with spatially-defined doping type and magnitude. After each optical doping procedure, magnetotransport measurements including quantum Hall measurements are performed to characterize the device performance. In the unipolar (p+-p-p+ and n-n+-n) configurations, we observe quantization of the longitudinal resistance, proving well-defined doped regions and interfaces that are further analyzed by Landauer-Buttiker modeling. Our unique measurements and modeling of these optically doped devices reveal a complete separation of the p- and n-Landau level edge states. The non-interaction of the edge states results in an observed "insulating" state in devices with a bi-polar p-n-p configuration that is uncommon and has not been measured previously in graphene devices. This insulating state could be utilized in high-performance graphene electrical switches. These quantitative magnetotransport measurements confirm that these doping techniques can be applied to any 2D materials encapsulated within hBN layers, enabling versatile, rewritable circuit elements for future computing and memory applications.

A molecule's 2D representation consists of its atoms, their attributes, and the molecule's covalent bonds. A 3D (geometric) representation of a molecule is called a conformer and consists of its atom types and Cartesian coordinates. Every conformer has a potential energy, and the lower this energy, the more likely it occurs in nature. Most existing machine learning methods for molecular property prediction consider either 2D molecular graphs or 3D conformer structure representations in isolation. Inspired by recent work on using ensembles of conformers in conjunction with 2D graph representations, we propose $\mathrm{E}$(3)-invariant molecular conformer aggregation networks. The method integrates a molecule's 2D representation with that of multiple of its conformers. Contrary to prior work, we propose a novel 2D-3D aggregation mechanism based on a differentiable solver for the Fused Gromov-Wasserstein Barycenter problem and the use of an efficient conformer generation method based on distance geometry. We show that the proposed aggregation mechanism is $\mathrm{E}$(3) invariant and propose an efficient GPU implementation. Moreover, we demonstrate that the aggregation mechanism helps to significantly outperform state-of-the-art molecule property prediction methods on established datasets., Comment: Accepted at ICML 2024 (updated version)

The life cycle of most marine invertebrates includes a planktonic larval stage before metamorphosis to bottom-dwelling adulthood. During larval stage, ciliary-mediated activity enables feeding (capture unicellular algae) and transport of materials (oxygen) required for the larva's growth, development, and successful metamorphosis. Investigating the underlying hydrodynamics of these behaviors is valuable for addressing fundamental biological questions (e.g., phenotypic plasticity) and advancing engineering applications. In this work, we combined microfluidics and fluorescence microscopy as a miniaturized PIV (mPIV) to study ciliary-medicated hydrodynamics during suspension feeding in sand dollar larvae (Dendraster excentricus). First, we confirmed the approach's feasibility by examining the underlying hydrodynamics (vortex patterns) for low- and high-fed larvae. Next, ciliary hydrodynamics were tracked from 11 days post-fertilization (DPF) to 20 DPF for 21 low-fed larvae. Microfluidics enabled the examination of baseline activities (without external flow) and behaviors in the presence of environmental cues (external flow). A library of qualitative vortex patterns and quantitative hydrodynamics was generated and shared as a stand alone repository. Results from mPIV (velocities) were used to examine the role of ciliary activity in transporting materials (oxygen). Given the laminar flow and the viscosity-dominated environments surrounding the larvae, overcoming the diffusive boundary layer is critical for the organism's survival. Peclet number analysis for oxygen transport suggested that ciliary velocities help overcome the diffusion dominated transport (max Pe numbers between 30-60). Microfluidics serving as mPIV provided a scalable and accessible approach for investigating the ciliary hydrodynamics of marine organisms., Comment: 21 pages and 11 figures (videos not included)

Motion planning is still an open problem for many disciplines, e.g., robotics, autonomous driving, due to their need for high computational resources that hinder real-time, efficient decision-making. A class of methods striving to provide smooth solutions is gradient-based trajectory optimization. However, those methods usually suffer from bad local minima, while for many settings, they may be inapplicable due to the absence of easy-to-access gradients of the optimization objectives. In response to these issues, we introduce Motion Planning via Optimal Transport (MPOT) -- a \textit{gradient-free} method that optimizes a batch of smooth trajectories over highly nonlinear costs, even for high-dimensional tasks, while imposing smoothness through a Gaussian Process dynamics prior via the planning-as-inference perspective. To facilitate batch trajectory optimization, we introduce an original zero-order and highly-parallelizable update rule: the Sinkhorn Step, which uses the regular polytope family for its search directions. Each regular polytope, centered on trajectory waypoints, serves as a local cost-probing neighborhood, acting as a \textit{trust region} where the Sinkhorn Step "transports" local waypoints toward low-cost regions. We theoretically show that Sinkhorn Step guides the optimizing parameters toward local minima regions of non-convex objective functions. We then show the efficiency of MPOT in a range of problems from low-dimensional point-mass navigation to high-dimensional whole-body robot motion planning, evincing its superiority compared to popular motion planners, paving the way for new applications of optimal transport in motion planning., Comment: Published as a conference paper at NeurIPS 2023. Project website: https://sites.google.com/view/sinkhorn-step/

We propose a globally convergent computational technique for the nonlinear inverse problem of reconstructing the zero-order coefficient in a parabolic equation using partial boundary data. This technique is called the "reduced dimensional method". Initially, we use the polynomial-exponential basis to approximate the inverse problem as a system of 1D nonlinear equations. We then employ a Picard iteration based on the quasi-reversibility method and a Carleman weight function. We will rigorously prove that the sequence derived from this iteration converges to the accurate solution for that 1D system without requesting a good initial guess of the true solution. The key tool for the proof is a Carleman estimate. We will also show some numerical examples.

This paper aims to reconstruct the initial condition of a hyperbolic equation with an unknown damping coefficient. Our approach involves approximating the hyperbolic equation's solution by its truncated Fourier expansion in the time domain and using a polynomial-exponential basis. This truncation process facilitates the elimination of the time variable, consequently, yielding a system of quasi-linear elliptic equations. To globally solve the system without needing an accurate initial guess, we employ the Carleman contraction principle. We provide several numerical examples to illustrate the efficacy of our method. The method not only delivers precise solutions but also showcases remarkable computational efficiency.

Learning priors on trajectory distributions can help accelerate robot motion planning optimization. Given previously successful plans, learning trajectory generative models as priors for a new planning problem is highly desirable. Prior works propose several ways on utilizing this prior to bootstrapping the motion planning problem. Either sampling the prior for initializations or using the prior distribution in a maximum-a-posterior formulation for trajectory optimization. In this work, we propose learning diffusion models as priors. We then can sample directly from the posterior trajectory distribution conditioned on task goals, by leveraging the inverse denoising process of diffusion models. Furthermore, diffusion has been recently shown to effectively encode data multimodality in high-dimensional settings, which is particularly well-suited for large trajectory dataset. To demonstrate our method efficacy, we compare our proposed method - Motion Planning Diffusion - against several baselines in simulated planar robot and 7-dof robot arm manipulator environments. To assess the generalization capabilities of our method, we test it in environments with previously unseen obstacles. Our experiments show that diffusion models are strong priors to encode high-dimensional trajectory distributions of robot motions.

Microscopes are essential for biomechanics and hydrodynamical investigation of small aquatic organisms. We report a DIY microscope (GLUBscope) that enables the visualization of organisms from two orthogonal imaging planes (top and side views). Compared to conventional imaging systems, this approach provides a comprehensive visualization strategy of organisms, which could have complex shapes and morphologies. The microscope was constructed by combining custom 3D-printed parts and off-the-shelf components. The system is designed for modularity and reconfigurability. Open-source design files and build instructions are provided in this report. Additionally, proof of use experiments, particularly with Hydra and other organisms that combine the GLUBscope with an analysis pipeline, were demonstrated. Beyond the applications demonstrated, the system can be used or modified for various imaging applications.

Motivated by the Quantum Modularity Conjecture and its arithmetic aspects related to the Habiro ring of a number field, we define a map from the Kauffman bracket skein module of an integer homology 3-sphere to the Habiro ring, and use Witten's conjecture (now a theorem) to show that the image is an effectively computable module of finite rank that can be used to phrase the quantum modularity conjecture., Comment: 15 pages, 4 figures. Updated 2 references

Facial wound segmentation plays a crucial role in preoperative planning and optimizing patient outcomes in various medical applications. In this paper, we propose an efficient approach for automating 3D facial wound segmentation using a two-stream graph convolutional network. Our method leverages the Cir3D-FaIR dataset and addresses the challenge of data imbalance through extensive experimentation with different loss functions. To achieve accurate segmentation, we conducted thorough experiments and selected a high-performing model from the trained models. The selected model demonstrates exceptional segmentation performance for complex 3D facial wounds. Furthermore, based on the segmentation model, we propose an improved approach for extracting 3D facial wound fillers and compare it to the results of the previous study. Our method achieved a remarkable accuracy of 0.9999986\% on the test suite, surpassing the performance of the previous method. From this result, we use 3D printing technology to illustrate the shape of the wound filling. The outcomes of this study have significant implications for physicians involved in preoperative planning and intervention design. By automating facial wound segmentation and improving the accuracy of wound-filling extraction, our approach can assist in carefully assessing and optimizing interventions, leading to enhanced patient outcomes. Additionally, it contributes to advancing facial reconstruction techniques by utilizing machine learning and 3D bioprinting for printing skin tissue implants. Our source code is available at \url{https://github.com/SIMOGroup/WoundFilling3D}.

The objective of this article is to introduce a novel technique for computing numerical solutions to the nonlinear inverse heat conduction problem. This involves solving nonlinear parabolic equations with Cauchy data provided on one side $\Gamma$ of the boundary of the computational domain $\Omega$. The key step of our proposed method is the truncation of the Fourier series of the solution to the governing equation. The truncation technique enables us to derive a system of 1D ordinary differential equations. Then, we employ the well-known Runge-Kutta method to solve this system, which aids in addressing the nonlinearity and the lack of data on $\partial \Omega \setmunus \Gamma$. This new approach is called the dimensional reduction method. By converting the high-dimensional problem into a 1D problem, we achieve exceptional computational speed. Numerical results are provided to support the effectiveness of our approach.

Recent work on extending coreference resolution across domains and languages relies on annotated data in both the target domain and language. At the same time, pre-trained large language models (LMs) have been reported to exhibit strong zero- and few-shot learning abilities across a wide range of NLP tasks. However, prior work mostly studied this ability using artificial sentence-level datasets such as the Winograd Schema Challenge. In this paper, we assess the feasibility of prompt-based coreference resolution by evaluating instruction-tuned language models on difficult, linguistically-complex coreference benchmarks (e.g., CoNLL-2012). We show that prompting for coreference can outperform current unsupervised coreference systems, although this approach appears to be reliant on high-quality mention detectors. Further investigations reveal that instruction-tuned LMs generalize surprisingly well across domains, languages, and time periods; yet continued fine-tuning of neural models should still be preferred if small amounts of annotated examples are available.

Our objective is to calculate the derivatives of data corrupted by noise. This is a challenging task as even small amounts of noise can result in significant errors in the computation. This is mainly due to the randomness of the noise, which can result in high-frequency fluctuations. To overcome this challenge, we suggest an approach that involves approximating the data by eliminating high-frequency terms from the Fourier expansion of the given data with respect to the polynomial-exponential basis. This truncation method helps to regularize the issue, while the use of the polynomial-exponential basis ensures accuracy in the computation. We demonstrate the effectiveness of our approach through numerical examples in one and two dimensions.

We conducted a prospective study to measure the clinical impact of an explainable machine learning system on interobserver agreement in chest radiograph interpretation. The AI system, which we call as it VinDr-CXR when used as a diagnosis-supporting tool, significantly improved the agreement between six radiologists with an increase of 1.5% in mean Fleiss' Kappa. In addition, we also observed that, after the radiologists consulted AI's suggestions, the agreement between each radiologist and the system was remarkably increased by 3.3% in mean Cohen's Kappa. This work has been accepted for publication in IEEE Access and this paper is our short version submitted to the Midwest Machine Learning Symposium (MMLS 2023), Chicago, IL, USA., Comment: This work has been accepted for publication in IEEE Access. This is a short version submitted to the Midwest Machine Learning Symposium (MMLS 2023), Chicago, IL, USA

We establish the existence of several quantum trace maps. The simplest one is an algebra map between two quantizations of the algebra of regular functions on the $SL_n$-character variety of a surface $\mathfrak{S}$ equipped with an ideal triangulation $\lambda$. The first is the (stated) $SL_n$-skein algebra $\mathscr{S}(\mathfrak{S})$. The second $\overline{\mathcal{X}}(\mathfrak{S},\lambda)$ is the Fock and Goncharov's quantization of their $X$-moduli space. The quantum trace is an algebra homomorphism $\bar{tr}^X:\overline{\mathscr{S}}(\mathfrak{S})\to\overline{\mathcal{X}}(\mathfrak{S},\lambda)$ where the reduced skein algebra $\overline{\mathscr{S}}(\mathfrak{S})$ is a quotient of $\mathscr{S}(\mathfrak{S})$. When the quantum parameter is 1, the quantum trace $\bar{tr}^X$ coincides with the classical Fock-Goncharov homomorphism. This is a generalization of the Bonahon-Wong quantum trace map for the case $n=2$. We then define the extended Fock-Goncharov algebra $\mathcal{X}(\mathfrak{S},\lambda)$ and show that $\bar{tr}^X$ can be lifted to $tr^X:\mathscr{S}(\mathfrak{S})\to\mathcal{X}(\mathfrak{S},\lambda)$. We show that both $\bar{tr}^X$ and $tr^X$ are natural with respect to the change of triangulations. When each connected component of $\mathfrak{S}$ has non-empty boundary and no interior ideal point, we define a quantization of the Fock-Goncharov $A$-moduli space $\overline{\mathcal{A}}(\mathfrak{S},\lambda)$ and its extension $\mathcal{A}(\mathfrak{S},\lambda)$. We then show that there exist quantum traces $\bar{tr}^A:\overline{\mathscr{S}}(\mathfrak{S})\to\overline{\mathcal{A}}(\mathfrak{S},\lambda)$ and $tr^A:\mathscr{S}(\mathfrak{S})\hookrightarrow\mathcal{A}(\mathfrak{S},\lambda)$, where the second map is injective, while the first is injective at least when $\mathfrak{S}$ is a polygon. They are equivalent to the $X$-versions but have better algebraic properties., Comment: 111 pages, 35 figures