Search

The recently discovered nonlinear Hall (NLH) effect arises either without external magnetic field (type-I) or with an in-plane magnetic field (type-II). In this work we propose a new type of geometrical nonlinear Hall effect with an out-of-plane magnetic field (type-III) induced by the combination of Lorentz force and anomalous electronic velocity. The type-III NLH effect is proportional to the more refined structures of Bloch wave functions, i.e., the dipole moment of square of Berry curvature, thus becoming prominent near the band crossings or anticrossings. Our effective model analysis and first-principles calculations show that gate-tuned MnBi$_2$Te$_4$ thin film under uniaxial strain is an ideal platform to observe this effect. Especially, giant unidirectional magnetoresistance can occur in this material, based on which an efficient electrical transistor device prototype can be built. Finally a symmetry analysis indicates that type-III NLH effect has unique symmetry properties stemming from Berry curvature square dipole, which is different from other previously reported NLH effects and can exist in a wider class of magnetic crystals. Our study offers new paradigms for nonlinear electronics.

Multiscale dynamical systems, modeled by high-dimensional stiff ordinary differential equations (ODEs) with wide-ranging characteristic timescales, arise across diverse fields of science and engineering, but their numerical solvers often encounter severe efficiency bottlenecks. This paper introduces a novel DeePODE method, which consists of a global multiscale sampling method and a fitting by deep neural networks to handle multiscale systems. DeePODE's primary contribution is to address the multiscale challenge of efficiently uncovering representative training sets by combining the Monte Carlo method and the ODE system's intrinsic evolution without suffering from the ``curse of dimensionality''. The DeePODE method is validated in multiscale systems from diverse areas, including a predator-prey model, a power system oscillation, a battery electrolyte auto-ignition, and turbulent flames. Our methods exhibit strong generalization capabilities to unseen conditions, highlighting the power of deep learning in modeling intricate multiscale dynamical processes across science and engineering domains., Comment: 7 pages, 6 figures

In recent years, pre-trained large language models (LLMs) have achieved tremendous success in the field of Natural Language Processing (NLP). Prior studies have primarily focused on general and generic domains, with relatively less research on specialized LLMs in the medical field. The specialization and high accuracy requirements for diagnosis in the medical field, as well as the challenges in collecting large-scale data, have constrained the application and development of LLMs in medical scenarios. In the field of ophthalmology, clinical diagnosis mainly relies on doctors' interpretation of reports and making diagnostic decisions. In order to take advantage of LLMs to provide decision support for doctors, we collected three modalities of ophthalmic report data and fine-tuned the LLaMA2 model, successfully constructing an LLM termed the "Ophtha-LLaMA2" specifically tailored for ophthalmic disease diagnosis. Inference test results show that even with a smaller fine-tuning dataset, Ophtha-LLaMA2 performs significantly better in ophthalmic diagnosis compared to other LLMs. It demonstrates that the Ophtha-LLaMA2 exhibits satisfying accuracy and efficiency in ophthalmic disease diagnosis, making it a valuable tool for ophthalmologists to provide improved diagnostic support for patients. This research provides a useful reference for the application of LLMs in the field of ophthalmology, while showcasing the immense potential and prospects in this domain.

In this paper, we present a large-scale evaluation probing GPT-4V's capabilities and limitations for biomedical image analysis. GPT-4V represents a breakthrough in artificial general intelligence (AGI) for computer vision, with applications in the biomedical domain. We assess GPT-4V's performance across 16 medical imaging categories, including radiology, oncology, ophthalmology, pathology, and more. Tasks include modality recognition, anatomy localization, disease diagnosis, report generation, and lesion detection. The extensive experiments provide insights into GPT-4V's strengths and weaknesses. Results show GPT-4V's proficiency in modality and anatomy recognition but difficulty with disease diagnosis and localization. GPT-4V excels at diagnostic report generation, indicating strong image captioning skills. While promising for biomedical imaging AI, GPT-4V requires further enhancement and validation before clinical deployment. We emphasize responsible development and testing for trustworthy integration of biomedical AGI. This rigorous evaluation of GPT-4V on diverse medical images advances understanding of multimodal large language models (LLMs) and guides future work toward impactful healthcare applications.

In photoacoustic computed tomography (PACT) with short-pulsed laser excitation, wideband acoustic signals are generated in biological tissues with frequencies related to the effective shapes and sizes of the optically absorbing targets. Low-frequency photoacoustic signal components correspond to slowly varying spatial features and are often omitted during imaging due to the limited detection bandwidth of the ultrasound transducer, or during image reconstruction as undesired background that degrades image contrast. Here we demonstrate that low-frequency photoacoustic signals, in fact, contain functional and molecular information, and can be used to enhance structural visibility, improve quantitative accuracy, and reduce spare-sampling artifacts. We provide an in-depth theoretical analysis of low-frequency signals in PACT, and experimentally evaluate their impact on several representative PACT applications, such as mapping temperature in photothermal treatment, measuring blood oxygenation in a hypoxia challenge, and detecting photoswitchable molecular probes in deep organs. Our results strongly suggest that low-frequency signals are important for functional and molecular PACT.

Photoacoustic computed tomography (PACT) is a proven technology for imaging hemodynamics in deep brain of small animal models. PACT is inherently compatible with ultrasound (US) imaging, providing complementary contrast mechanisms. While PACT can quantify the brain's oxygen saturation of hemoglobin (sO$_2$), US imaging can probe the blood flow based on the Doppler effect. Further, by tracking gas-filled microbubbles, ultrasound localization microscopy (ULM) can map the blood flow velocity with sub-diffraction spatial resolution. In this work, we present a 3D deep-brain imaging system that seamlessly integrates PACT and ULM into a single device, 3D-PAULM. Using a low ultrasound frequency of 4 MHz, 3D-PAULM is capable of imaging the whole-brain hemodynamic functions with intact scalp and skull in a totally non-invasive manner. Using 3D-PAULM, we studied the mouse brain functions with ischemic stroke. Multi-spectral PACT, US B-mode imaging, microbubble-enhanced power Doppler (PD), and ULM were performed on the same mouse brain with intrinsic image co-registration. From the multi-modality measurements, we future quantified blood perfusion, sO$_2$, vessel density, and flow velocity of the mouse brain, showing stroke-induced ischemia, hypoxia, and reduced blood flow. We expect that 3D-PAULM can find broad applications in studying deep brain functions on small animal models., Comment: 10 pages

Prompts have been proven to play a crucial role in large language models, and in recent years, vision models have also been using prompts to improve scalability for multiple downstream tasks. In this paper, we focus on adapting prompt design based on instruction tuning into a visual transformer model for image classification which we called Instruction-ViT. The key idea is to implement multi-modal prompts (text or image prompt) related to category information to guide the fine-tuning of the model. Based on the experiments of several image captionining tasks, the performance and domain adaptability were improved. Our work provided an innovative strategy to fuse multi-modal prompts with better performance and faster adaptability for visual classification models.

Large language models (LLMs) such as ChatGPT have recently demonstrated significant potential in mathematical abilities, providing valuable reasoning paradigm consistent with human natural language. However, LLMs currently have difficulty in bridging perception, language understanding and reasoning capabilities due to incompatibility of the underlying information flow among them, making it challenging to accomplish tasks autonomously. On the other hand, abductive learning (ABL) frameworks for integrating the two abilities of perception and reasoning has seen significant success in inverse decipherment of incomplete facts, but it is limited by the lack of semantic understanding of logical reasoning rules and the dependence on complicated domain knowledge representation. This paper presents a novel method (ChatABL) for integrating LLMs into the ABL framework, aiming at unifying the three abilities in a more user-friendly and understandable manner. The proposed method uses the strengths of LLMs' understanding and logical reasoning to correct the incomplete logical facts for optimizing the performance of perceptual module, by summarizing and reorganizing reasoning rules represented in natural language format. Similarly, perceptual module provides necessary reasoning examples for LLMs in natural language format. The variable-length handwritten equation deciphering task, an abstract expression of the Mayan calendar decoding, is used as a testbed to demonstrate that ChatABL has reasoning ability beyond most existing state-of-the-art methods, which has been well supported by comparative studies. To our best knowledge, the proposed ChatABL is the first attempt to explore a new pattern for further approaching human-level cognitive ability via natural language interaction with ChatGPT.

Photoacoustic microscopy (PAM) is an emerging imaging method combining light and sound. However, limited by the laser's repetition rate, state-of-the-art high-speed PAM technology often sacrifices spatial sampling density (i.e., undersampling) for increased imaging speed over a large field-of-view. Deep learning (DL) methods have recently been used to improve sparsely sampled PAM images; however, these methods often require time-consuming pre-training and large training dataset with ground truth. Here, we propose the use of deep image prior (DIP) to improve the image quality of undersampled PAM images. Unlike other DL approaches, DIP requires neither pre-training nor fully-sampled ground truth, enabling its flexible and fast implementation on various imaging targets. Our results have demonstrated substantial improvement in PAM images with as few as 1.4$\%$ of the fully sampled pixels on high-speed PAM. Our approach outperforms interpolation, is competitive with pre-trained supervised DL method, and is readily translated to other high-speed, undersampling imaging modalities.

Shock wave lithotripsy (SWL) has been widely used for non-invasive treatment of kidney stones. Cavitation plays an important role in stone fragmentation, yet may also contribute to renal injury during SWL. It is therefore crucial to determine the spatiotemporal distributions of cavitation activities to maximize stone fragmentation while minimizing tissue injury. Traditional cavitation detection methods include high-speed optical imaging, active cavitation mapping (ACM), and passive cavitation mapping (PCM). While each of the three methods provides unique information about the dynamics of the bubbles, PCM has most practical applications in biological tissues. To image the dynamics of cavitation bubble collapse, we previously developed a sliding-window PCM (SW-PCM) method to identify each bubble collapse with high temporal and spatial resolution. To further validate and optimize the SW-PCM method, in this work, we have developed tri-modality cavitation imaging that includes 3D high-speed optical imaging, ACM, and PCM seamlessly integrated in a single system. Using the tri-modality system, we imaged and analyzed laser-induced single cavitation bubbles in both free and constricted space and shockwave-induced cavitation clusters. Collectively, our results have demonstrated the high reliability and spatial-temporal accuracy of the SW-PCM approach, which paves the way for future in vivo applications on large animals and humans in SWL.

The penetration depth of photoacoustic imaging in biological tissues has been fundamentally limited by the strong optical attenuation when light is delivered externally through the tissue surface. To address this issue, we previously reported internal-illumination photoacoustic imaging using a customized radial-emission optical fiber diffuser, which, however, has complex fabrication, high cost, and non-uniform light emission. To overcome these shortcomings, we have developed a new type of low-cost fiber diffusers based on a graded-scattering method in which the optical scattering of the fiber diffuser is gradually increased as the light travels. The graded scattering can compensate for the optical attenuation and provide relatively uniform light emission along the diffuser. We performed Monte Carlo numerical simulations to optimize several key design parameters, including the number of scattering segments, scattering anisotropy factor, divergence angle of the optical fiber, and reflective index of the surrounding medium. These optimized parameters collectively result in uniform light emission along the fiber diffuser and can be flexibly adjusted to accommodate different applications. We fabricated and characterized the prototype fiber diffuser made of agarose gel and intralipid. Equipped with the new fiber diffuser, we performed thorough proof-of-concept studies on ex vivo tissue phantoms and an in vivo swine model to demonstrate the deep-imaging capability (~10 cm achieved ex vivo) of photoacoustic tomography. We believe that the internal light delivery via the optimized fiber diffuser is an effective strategy to image deep targets (e.g., kidney) in large animals or humans.

Photoacoustic computed tomography (PACT) detects light-induced ultrasound waves to reconstruct the optical absorption contrast of the biological tissues. Due to its relatively deep penetration (several centimeters in soft tissue), high spatial resolution, and inherent functional sensitivity, PACT has great potential for imaging mouse brains with endogenous and exogenous contrasts, which is of immense interest to the neuroscience community. However, conventional PACT either assumes homogenous optical fluence within the brain or uses a simplified attenuation model for optical fluence estimation. Both approaches underestimate the complexity of the fluence heterogeneity and can result in poor quantitative imaging accuracy. To optimize the quantitative performance of PACT, we explore for the first time 3D Monte Carlo simulation to study the optical fluence distribution in a complete mouse brain model. We apply the MCX Monte Carlo simulation package on a digital mouse (Digimouse) brain atlas that has complete anatomy information. To evaluate the impact of the brain vasculature on light delivery, we also incorporate the whole-brain vasculature in the Digimouse atlas. The simulation results clearly show that the optical fluence in the mouse brain is heterogeneous at the global level and can decrease by a factor of five with increasing depth. Moreover, the strong absorption and scattering of the brain vasculature also induce the fluence disturbance at the local level. Our results suggest that both global and local fluence heterogeneity contributes to the reduced quantitative accuracy of the reconstructed PACT images of mouse brain., Comment: 16 pages, 6 figures, 1 table

One primary technical challenge in photoacoustic microscopy (PAM) is the necessary compromise between spatial resolution and imaging speed. In this study, we propose a novel application of deep learning principles to reconstruct undersampled PAM images and transcend the trade-off between spatial resolution and imaging speed. We compared various convolutional neural network (CNN) architectures, and selected a fully dense U-net (FD U-net) model that produced the best results. To mimic various undersampling conditions in practice, we artificially downsampled fully-sampled PAM images of mouse brain vasculature at different ratios. This allowed us to not only definitively establish the ground truth, but also train and test our deep learning model at various imaging conditions. Our results and numerical analysis have collectively demonstrated the robust performance of our model to reconstruct PAM images with as few as 2% of the original pixels, which may effectively shorten the imaging time without substantially sacrificing the image quality., Comment: 12 pages, 7 main figures, 3 supplemental figures (see last 2 pages)