Search

Multilingual automatic speech recognition (ASR) in the medical domain serves as a foundational task for various downstream applications such as speech translation, spoken language understanding, and voice-activated assistants. This technology enhances patient care by enabling efficient communication across language barriers, alleviating specialized workforce shortages, and facilitating improved diagnosis and treatment, particularly during pandemics. In this work, we introduce MultiMed, a collection of small-to-large end-to-end ASR models for the medical domain, spanning five languages: Vietnamese, English, German, French, and Mandarin Chinese, together with the corresponding real-world ASR dataset. To our best knowledge, MultiMed stands as the largest and the first multilingual medical ASR dataset, in terms of total duration, number of speakers, diversity of diseases, recording conditions, speaker roles, unique medical terms, accents, and ICD-10 codes. Secondly, we establish the empirical baselines, present the first reproducible study of multilinguality in medical ASR, conduct a layer-wise ablation study for end-to-end ASR training, and provide the first linguistic analysis for multilingual medical ASR. All code, data, and models are available online https://github.com/leduckhai/MultiMed/tree/master/MultiMed, Comment: Preprint

To protect patient safety, modern pharmaceutical complexity demands strict prescription verification. We offer a new approach - Rx Strategist - that makes use of knowledge graphs and different search strategies to enhance the power of Large Language Models (LLMs) inside an agentic framework. This multifaceted technique allows for a multi-stage LLM pipeline and reliable information retrieval from a custom-built active ingredient database. Different facets of prescription verification, such as indication, dose, and possible drug interactions, are covered in each stage of the pipeline. We alleviate the drawbacks of monolithic LLM techniques by spreading reasoning over these stages, improving correctness and reliability while reducing memory demands. Our findings demonstrate that Rx Strategist surpasses many current LLMs, achieving performance comparable to that of a highly experienced clinical pharmacist. In the complicated world of modern medications, this combination of LLMs with organized knowledge and sophisticated search methods presents a viable avenue for reducing prescription errors and enhancing patient outcomes., Comment: 17 Pages, 6 Figures, Under Review

Predicting program behavior without execution is a crucial and challenging task in software engineering. Traditional models often struggle to capture the dynamic dependencies and interactions within code. This paper introduces a novel machine learning-based framework called CodeFlow, designed to predict code coverage and detect runtime errors through Dynamic Dependencies Learning. By utilizing control flow graphs (CFGs), CodeFlow represents all possible execution paths and the relationships between different statements, providing a comprehensive understanding of program behavior. CodeFlow constructs CFGs to depict execution paths and learns vector representations for CFG nodes, capturing static control-flow dependencies. Additionally, it learns dynamic dependencies through execution traces, which reflect the impacts among statements during execution. This approach enables accurate prediction of code coverage and effective identification of runtime errors. Empirical evaluations demonstrate significant improvements in code coverage prediction accuracy and effective localization of runtime errors, outperforming existing models.

Underwater vision is crucial for autonomous underwater vehicles (AUVs), and enhancing degraded underwater images in real-time on a resource-constrained AUV is a key challenge due to factors like light absorption and scattering, or the sufficient model computational complexity to resolve such factors. Traditional image enhancement techniques lack adaptability to varying underwater conditions, while learning-based methods, particularly those using convolutional neural networks (CNNs) and generative adversarial networks (GANs), offer more robust solutions but face limitations such as inadequate enhancement, unstable training, or mode collapse. Denoising diffusion probabilistic models (DDPMs) have emerged as a state-of-the-art approach in image-to-image tasks but require intensive computational complexity to achieve the desired underwater image enhancement (UIE) using the recent UW-DDPM solution. To address these challenges, this paper introduces UW-DiffPhys, a novel physical-based and diffusion-based UIE approach. UW-DiffPhys combines light-computation physical-based UIE network components with a denoising U-Net to replace the computationally intensive distribution transformation U-Net in the existing UW-DDPM framework, reducing complexity while maintaining performance. Additionally, the Denoising Diffusion Implicit Model (DDIM) is employed to accelerate the inference process through non-Markovian sampling. Experimental results demonstrate that UW-DiffPhys achieved a substantial reduction in computational complexity and inference time compared to UW-DDPM, with competitive performance in key metrics such as PSNR, SSIM, UCIQE, and an improvement in the overall underwater image quality UIQM metric. The implementation code can be found at the following repository: https://github.com/bachzz/UW-DiffPhys

We study the ground state energy of a gas of spin $1/2$ fermions with repulsive short-range interactions. We derive an upper bound that agrees, at low density $\rho$, with the Huang-Yang conjecture. The latter captures the first three terms in an asymptotic low-density expansion, and in particular the Huang-Yang correction term of order $\rho^{7/3}$. Our trial state is constructed using an adaptation of the bosonic Bogoliubov theory to the Fermi system, where the correlation structure of fermionic particles is incorporated by quasi-bosonic Bogoliubov transformations. In the latter, it is important to consider a modified zero-energy scattering equation that takes into account the presence of the Fermi sea, in the spirit of the Bethe-Goldstone equation., Comment: 48 pages

We discuss in this article the creation and organization of a national core facility for bioimage analysis, based on a distributed team. F-BIAS federates bioimage analysts across France and relies on them to deliver services to the researchers of this territory. The main challenge in implementing this structure is to provide significant scientific value to its members, thereby encouraging their active participation and persuading their respective host teams to support their involvement. F-BIAS accomplished this by creating a professional network that mitigates the negative effects of isolation experienced by its members, who are often the sole bioimage analyst within their local teams, and fosters the development of their technical skills. In a second phase we capitalized on F-BIAS to create a virtual, remotely-operating core facility for bioimage analysis, offering consultations and collaborative project services to the scientific community of France. The services are organized so that they also contribute to the technical proficiency of the analysts. To promote the creation of similar structures, we present and discuss here the organization of this nationally distributed bioimage analysis service core, highlighting successes and challenges.

The capability of generating high-quality source code using large language models (LLMs) reduces software development time and costs. However, they often introduce security vulnerabilities due to training on insecure open-source data. This highlights the need for ensuring secure and functional code generation. This paper introduces PromSec, an algorithm for prom optimization for secure and functioning code generation using LLMs. In PromSec, we combine 1) code vulnerability clearing using a generative adversarial graph neural network, dubbed as gGAN, to fix and reduce security vulnerabilities in generated codes and 2) code generation using an LLM into an interactive loop, such that the outcome of the gGAN drives the LLM with enhanced prompts to generate secure codes while preserving their functionality. Introducing a new contrastive learning approach in gGAN, we formulate code-clearing and generation as a dual-objective optimization problem, enabling PromSec to notably reduce the number of LLM inferences. PromSec offers a cost-effective and practical solution for generating secure, functional code. Extensive experiments conducted on Python and Java code datasets confirm that PromSec effectively enhances code security while upholding its intended functionality. Our experiments show that while a state-of-the-art approach fails to address all code vulnerabilities, PromSec effectively resolves them. Moreover, PromSec achieves more than an order-of-magnitude reduction in operation time, number of LLM queries, and security analysis costs. Furthermore, prompts optimized with PromSec for a certain LLM are transferable to other LLMs across programming languages and generalizable to unseen vulnerabilities in training. This study is a step in enhancing the trustworthiness of LLMs for secure and functional code generation, supporting their integration into real-world software development., Comment: 15 pages, 19 figures, CCS 2024

The differences in H2O2 production between conventional (CONV) and ultra-high dose rate (UHDR) irradiations in water radiolysis are still not fully understood. The lower levels of this radiolytic species, as a critical end product of water radiolysis, are particularly relevant for investigating the connection between the high-density energy deposition during short-duration physical events (ionizations or excitations) and biological responses of the FLASH effect. In this study, we developed a new Geant4-DNA chemistry model to simulate radiolysis considering the time structure of the irradiation pulse at different absorbed doses to liquid water of 0.01, 0.1, 1, and 2 Gy under 1 MeV electron irradiation. The model allows the description of the beam's temporal structure, including the pulse duration, the pulse repetition frequency, and the pulse amplitude for the different beam irradiation conditions through a wide dose rate range, from 0.01 Gy/s up to about 105 Gy/s, at various oxygen concentrations. The preliminary results indicate a correlation between the temporal structure of the pulses and a significant reduction in the production of reactive oxygen species (ROS) at different dose rates., Comment: 27 pages, 14 figures including 3 figures in appendix

We study the size and shape statistics of ground state fuzzy spheres when projected onto the transverse plane, utilizing the regularized SU(N=2) matrix model in D=(1+3)-dimensional spacetime. We show that they appear as ellipses, from matrix/membrane correspondence. With our numerical and analytical approximation for the ground state wavefunction, we provide estimations for their expected surface areas, perimeters, eccentricities, and shape-parameters. These geometric constants of quantum membranes deviate drastically from classical mechanics.

Anomalous metallic behavior is ubiquitously observed near superconductor-insulator quantum critical points and, if persistent to zero temperature, poses a challenge to current theories of metals. One explanation for this behavior could be incomplete thermal equilibrium between the sample and the cryostat. However, despite decades of study, the actual temperature of an anomalous metallic sample has not been measured. We therefore introduce a new experimental probe by measuring microwave radiation emitted from the anomalous metal, using a two-dimensional array of superconductor-semiconductor hybrid Josephson junctions as a model system. The total emitted radiation exceeds the limits of thermodynamic equilibrium, but is well described by an elevated sample temperature. This extracted sample temperature matches the onset of anomalous metallic behavior. Additionally, we discover scaling behavior of radiative noise in the presence of an applied bias. Elements of our noise-scaling observations were predicted based on nonlinear critical field theories and gauge-gravity duality. This work shows that, in a prominent model system, anomalous metallic behavior is a non-equilibrium effect, and opens a new frontier in the study of universal, non-equilibrium phenomena near quantum criticality., Comment: 6 pages, 4 figures

This paper studies a class of linear parabolic equations in non-divergence form in which the leading coefficients are measurable and they can be singular or degenerate as a weight belonging to the $A_{1+\frac{1}{n}}$ class of Muckenhoupt weights. Krylov-Safonov Harnack inequality for solutions is proved under some smallness assumption on a weighted mean oscillation of the weight. The novelty of the paper is an introduction of a class of generic weighted parabolic cylinders in which several growth lemmas are proved. A perturbation method is used and the Aleksandrov-Bakelman-Pucci type maximum principle for parabolic operators is crucially applied to suitable barrier functions to prove the growth lemmas. As corollaries, H\"{o}lder regularity estimates of solutions with respect to a quasi-distance, and a Liouville type theorem are obtained in the paper., Comment: 45 pages, comments are welcomed

This paper introduces SGCode, a flexible prompt-optimizing system to generate secure code with large language models (LLMs). SGCode integrates recent prompt-optimization approaches with LLMs in a unified system accessible through front-end and back-end APIs, enabling users to 1) generate secure code, which is free of vulnerabilities, 2) review and share security analysis, and 3) easily switch from one prompt optimization approach to another, while providing insights on model and system performance. We populated SGCode on an AWS server with PromSec, an approach that optimizes prompts by combining an LLM and security tools with a lightweight generative adversarial graph neural network to detect and fix security vulnerabilities in the generated code. Extensive experiments show that SGCode is practical as a public tool to gain insights into the trade-offs between model utility, secure code generation, and system cost. SGCode has only a marginal cost compared with prompting LLMs. SGCode is available at: https://sgcode.codes/.

We do experiments on physical agents with dynamic binary ideologies, deep memories of previous probes of neigboring agents, but fixed personalities that interpret the memory content to make ideological decisions. We find experimentally a critical memory depth below which complete ideological polarization of the collective cannot occur, and above which it is inevitable, an emergent symmetry breaking that is memory depth dependent. Depending on the details of the personalities, the polarization can be static or dynamic in time, even in certain cases chaotic due to nonreciprocity in how the agents respond to other agents. Thus, agents with different personalities and depths of memory serve as a physics analog of the ideology dynamics among biased individuals, illuminating how decisions influenced by individual memories of past interactions can shape and influence subsequent polarization. Perhaps such applications of physics-based systems to political systems will help us to understand the ideological instabilities observed today.

Large Language Models (LLMs) have revolutionized software engineering (SE), demonstrating remarkable capabilities in various coding tasks. While recent efforts have produced autonomous software agents based on LLMs for end-to-end development tasks, these systems are typically designed for specific SE tasks. We introduce HyperAgent, a novel generalist multi-agent system designed to address a wide spectrum of SE tasks across different programming languages by mimicking human developers' workflows. Comprising four specialized agents - Planner, Navigator, Code Editor, and Executor. HyperAgent manages the full lifecycle of SE tasks, from initial conception to final verification. Through extensive evaluations, HyperAgent achieves state-of-the-art performance across diverse SE tasks: it attains a 25.01% success rate on SWE-Bench-Lite and 31.40% on SWE-Bench-Verified for GitHub issue resolution, surpassing existing methods. Furthermore, HyperAgent demonstrates SOTA performance in repository-level code generation (RepoExec), and in fault localization and program repair (Defects4J), often outperforming specialized systems. This work represents a significant advancement towards versatile, autonomous agents capable of handling complex, multi-step SE tasks across various domains and languages, potentially transforming AI-assisted software development practices.

Recent advances in deep learning (DL) have significantly impacted motor imagery (MI)-based brain-computer interface (BCI) systems, enhancing the decoding of electroencephalography (EEG) signals. However, most studies struggle to identify discriminative patterns across subjects during MI tasks, limiting MI classification performance. In this article, we propose MixNet, a novel classification framework designed to overcome this limitation by utilizing spectral-spatial signals from MI data, along with a multitask learning architecture named MIN2Net, for classification. Here, the spectral-spatial signals are generated using the filter-bank common spatial patterns (FBCSPs) method on MI data. Since the multitask learning architecture is used for the classification task, the learning in each task may exhibit different generalization rates and potential overfitting across tasks. To address this issue, we implement adaptive gradient blending, simultaneously regulating multiple loss weights and adjusting the learning pace for each task based on its generalization/overfitting tendencies. Experimental results on six benchmark data sets of different data sizes demonstrate that MixNet consistently outperforms all state-of-the-art algorithms in subject-dependent and -independent settings. Finally, the low-density EEG MI classification results show that MixNet outperforms all state-of-the-art algorithms, offering promising implications for Internet of Thing (IoT) applications, such as lightweight and portable EEG wearable devices based on low-density montages., Comment: Supplementary materials and source codes are available on-line at https://github.com/Max-Phairot-A/MixNet

Sphaleron electroweak phase transirion (EWPT) is calculated in two phase transition stages, thereby showing that the twin (or double) bubble nucleation structure of the phase transition and gravitational wave is in the investigation area of future detectors. The parameter $a$ or $\tan\beta$ is the ratio between two vacuum average values (VEV) although it does not affect the strengh of EWPT but affect the sphaleron energy. However, it only causes this energy to increase slightly. As $a$ increases, the maximum difference of sphaleron energy in one stage is about $6.15$ TeV. $a$ affects the expansion of bubbles during two phase transitions. The more $a$ increases, the more the expansion of two bubbles is at the same time. This ratio does not greatly affect the sphaleron energy but has a strong impact on gravitational waves. The larger the masses of the charged Higgs particles is, the greater the gravitational wave energy density ($\Omega h^2$) is. When the frequency is in the range $0-1.2$ mHz, $\Omega h^2$ will has a maximum value in the range $10^{-12}-10^{-11}$ for all values of $a$ so this can be detected in the future., Comment: 26 pages, 13 figures

General expressions for one-loop contributions associated with the lepton-flavor violating decays of the standard model-like Higgs boson $h\to e_b^\pm e_a^\mp$ and gauge boson $Z\to e^\pm_b e_a^\mp$ are introduced in the unitary gauge. The results are used to discuss these decays as new physics signals in a minimal left-right symmetric model containing only one bidoublet Higgs and a $SU(2)_R$ Higgs doublet accommodating data of neutrino oscillations and $(g-2)_{\mu}$. The numerical investigation indicates that some of these decay rates can reach the near future experimental sensitivities., Comment: 39 pages, 4 figures

In this paper, we present the results for one-loop induced processes $\gamma \gamma \rightarrow \phi_i\phi_j$ with CP-even Higgses $\phi_{i,j} \equiv h,~H$ at high energy photon-photon collision, within the frameworks of Inert Higgs Doublet Models and Two Higgs Doublet Models. Total cross-sections are shown as functions of center-of-mass energies. We find that the cross-sections for the computed processes in all the models under investigations are enhanced at around the threshold of singly charged Higgs pair ($\sim 2 M_{H^\pm}$). Furthermore, the enhancement factors defined as the ratio of cross-sections of $\gamma \gamma \rightarrow \phi_i\phi_j$ in the investigated models over the corresponding ones for $\gamma \gamma \rightarrow hh$ in the Standard Model, are examined in the model's parameter space. In the Inert Higgs Doublet Models, the factors are studied in the parameter space of $(M_{H^\pm},~\mu^2_2)$ and $(M_{H^\pm},~\lambda_2)$. In the Two Higgs Doublet Models, the factors are examined in the planes of $(M_{H^\pm},~t_{\beta})$ as well as in the space of charged Higgs mass $M_{H^\pm}$ and the soft-breaking $Z_2$ parameter $m_{12}^2$. Two scenarios of $c_{\beta-\alpha} > 0$ and $c_{\beta-\alpha} < 0$ are studied in further detail. The factors give a different behavior from considering these scenarios. As a result, discriminations for the above-mentioned scenarios can be performed at future colliders., Comment: 46 pages, 21 figures of data, 8 Tables

In recent years, the continuous wavelet transform (CWT) has been employed as a spectral feature extractor for acoustic recognition tasks in conjunction with machine learning and deep learning models. However, applying the CWT to each individual audio sample is computationally intensive. This paper proposes an approach that applies the CWT to a subset of samples, spaced according to a specified hop size. Experimental results demonstrate that this method significantly reduces computational costs while maintaining the robust performance of the trained models.

Infrared imaging offers resilience against changing lighting conditions by capturing object temperatures. Yet, in few scenarios, its lack of visual details compared to daytime visible images, poses a significant challenge for human and machine interpretation. This paper proposes a novel diffusion method, dubbed Temporally Consistent Patch Diffusion Models (TC-DPM), for infrared-to-visible video translation. Our method, extending the Patch Diffusion Model, consists of two key components. Firstly, we propose a semantic-guided denoising, leveraging the strong representations of foundational models. As such, our method faithfully preserves the semantic structure of generated visible images. Secondly, we propose a novel temporal blending module to guide the denoising trajectory, ensuring the temporal consistency between consecutive frames. Experiment shows that TC-PDM outperforms state-of-the-art methods by 35.3% in FVD for infrared-to-visible video translation and by 6.1% in AP50 for day-to-night object detection. Our code is publicly available at https://github.com/dzungdoan6/tc-pdm, Comment: Technical report

Active learning enhances annotation efficiency by selecting the most revealing samples for labeling, thereby reducing reliance on extensive human input. Previous methods in semantic segmentation have centered on individual pixels or small areas, neglecting the rich patterns in natural images and the power of advanced pre-trained models. To address these challenges, we propose three key contributions: Firstly, we introduce Entity-Superpixel Annotation (ESA), an innovative and efficient active learning strategy which utilizes a class-agnostic mask proposal network coupled with super-pixel grouping to capture local structural cues. Additionally, our method selects a subset of entities within each image of the target domain, prioritizing superpixels with high entropy to ensure comprehensive representation. Simultaneously, it focuses on a limited number of key entities, thereby optimizing for efficiency. By utilizing an annotator-friendly design that capitalizes on the inherent structure of images, our approach significantly outperforms existing pixel-based methods, achieving superior results with minimal queries, specifically reducing click cost by 98% and enhancing performance by 1.71%. For instance, our technique requires a mere 40 clicks for annotation, a stark contrast to the 5000 clicks demanded by conventional methods.

In the rapidly evolving landscape of Natural Language Processing (NLP) and text generation, the emergence of Retrieval Augmented Generation (RAG) presents a promising avenue for improving the quality and reliability of generated text by leveraging information retrieved from user specified database. Benchmarking is essential to evaluate and compare the performance of the different RAG configurations in terms of retriever and generator, providing insights into their effectiveness, scalability, and suitability for the specific domain and applications. In this paper, we present a comprehensive framework to generate a domain relevant RAG benchmark. Our framework is based on automatic question-answer generation with Human (domain experts)-AI Large Language Model (LLM) teaming. As a case study, we demonstrate the framework by introducing WeQA, a first-of-its-kind benchmark on the wind energy domain which comprises of multiple scientific documents/reports related to environmental impact of wind energy projects. Our framework systematically evaluates RAG performance using diverse metrics and multiple question types with varying complexity level. We also demonstrate the performance of different models on our benchmark.

Creating soft-Coulomb-type (SC) molecular potential within single-active-electron approximation (SAE) is essential since it allows solving time-dependent Schr\"odinger equations with fewer computational resources compared to other multielectron methods. The current available SC potentials can accurately reproduce the energy of the highest occupied molecular orbital (HOMO), which is sufficient for analyzing nonlinear effects in laser-molecule interactions like high-order harmonic generation (HHG). However, recent discoveries of significant effects of deep-lying molecular orbitals call for more precise potentials to analyze them. In this study, we present a fast and accurate method based on machine learning to construct SC potentials that simultaneously reproduce various molecular features, including energies, symmetries, and dipole moments of HOMO, HOMO-1, and HOMO-2. We use this ML model to create SC SAE potentials of the HCN molecule and then comprehensively analyze the fingerprints of lower-lying orbitals in HHG spectra emitted during the H-CN stretching. Our findings reveal that HOMO-1 plays a role in forming the second HHG plateau. Additionally, as the H-C distance increases, the plateau structure and the smoothness of HHG spectra are altered due to the redistribution of orbital electron density. These results are in line with other experimental and theoretical studies. Lastly, the machine learning approach using deconvolution and convolution neural networks in the present study is so general that it can be applied to construct molecular potential for other molecules and molecular dynamic processes.

X-ray diffraction is ideal for probing sub-surface state during complex or rapid thermomechanical loading of crystalline materials. However, challenges arise as the size of diffraction volumes increase due to spatial broadening and inability to deconvolute the effects of different lattice deformation mechanisms. Here we present a novel approach to use combinations of physics-based modeling and machine learning to deconvolve thermal and mechanical elastic strains for diffraction data analysis. The method builds on a previous effort to extract thermal strain distribution information from diffraction data. The new approach is applied to extract the evolution of thermomechanical state during laser melting of an Inconel 625 wall specimen which produces significant residual stress upon cooling. A combination of heat transfer and fluid flow, elasto-plasticity, and X-ray diffraction simulations are used to generate training data for machine-learning (Gaussian Process Regression, GPR) models that map diffracted intensity distributions to underlying thermomechanical strain fields. First-principles density functional theory is used to determine accurate temperature-dependent thermal expansion and elastic stiffness used in the elasto-plasticity modeling. The trained GPR models are found to be capable of deconvoluting the effects of thermal and mechanical strains, in addition to providing information about underlying strain distributions, even from complex diffraction patterns with irregularly shaped peaks.

Activation functions are non-linearities in neural networks that allow them to learn complex mapping between inputs and outputs. Typical choices for activation functions are ReLU, Tanh, Sigmoid etc., where the choice generally depends on the application domain. In this work, we propose a framework/strategy that unifies several works on activation functions and theoretically explains the performance benefits of these works. We also propose novel techniques that originate from the framework and allow us to obtain ``extensions'' (i.e. special generalizations of a given neural network) of neural networks through operations on activation functions. We theoretically and empirically show that ``extensions'' of neural networks have performance benefits compared to vanilla neural networks with insignificant space and time complexity costs on standard test functions. We also show the benefits of neural network ``extensions'' in the time-series domain on real-world datasets.

Anytime multi-agent path finding (MAPF) is a promising approach to scalable path optimization in multi-agent systems. MAPF-LNS, based on Large Neighborhood Search (LNS), is the current state-of-the-art approach where a fast initial solution is iteratively optimized by destroying and repairing selected paths of the solution. Current MAPF-LNS variants commonly use an adaptive selection mechanism to choose among multiple destroy heuristics. However, to determine promising destroy heuristics, MAPF-LNS requires a considerable amount of exploration time. As common destroy heuristics are non-adaptive, any performance bottleneck caused by these heuristics cannot be overcome via adaptive heuristic selection alone, thus limiting the overall effectiveness of MAPF-LNS in terms of solution cost. In this paper, we propose Adaptive Delay-based Destroy-and-Repair Enhanced with Success-based Self-Learning (ADDRESS) as a single-destroy-heuristic variant of MAPF-LNS. ADDRESS applies restricted Thompson Sampling to the top-K set of the most delayed agents to select a seed agent for adaptive LNS neighborhood generation. We evaluate ADDRESS in multiple maps from the MAPF benchmark set and demonstrate cost improvements by at least 50% in large-scale scenarios with up to a thousand agents, compared with the original MAPF-LNS and other state-of-the-art methods., Comment: arXiv admin note: text overlap with arXiv:2312.16767

The integration of vision-language models such as CLIP and Concept Bottleneck Models (CBMs) offers a promising approach to explaining deep neural network (DNN) decisions using concepts understandable by humans, addressing the black-box concern of DNNs. While CLIP provides both explainability and zero-shot classification capability, its pre-training on generic image and text data may limit its classification accuracy and applicability to medical image diagnostic tasks, creating a transfer learning problem. To maintain explainability and address transfer learning needs, CBM methods commonly design post-processing modules after the bottleneck module. However, this way has been ineffective. This paper takes an unconventional approach by re-examining the CBM framework through the lens of its geometrical representation as a simple linear classification system. The analysis uncovers that post-CBM fine-tuning modules merely rescale and shift the classification outcome of the system, failing to fully leverage the system's learning potential. We introduce an adaptive module strategically positioned between CLIP and CBM to bridge the gap between source and downstream domains. This simple yet effective approach enhances classification performance while preserving the explainability afforded by the framework. Our work offers a comprehensive solution that encompasses the entire process, from concept discovery to model training, providing a holistic recipe for leveraging the strengths of GPT, CLIP, and CBM., Comment: Accepted at MICCAI 2024, the 27th International Conference on Medical Image Computing and Computer Assisted Intervention

Rapid advances in the capabilities of large language models (LLMs) have raised widespread concerns regarding their potential for malicious use. Open-weight LLMs present unique challenges, as existing safeguards lack robustness to tampering attacks that modify model weights. For example, recent works have demonstrated that refusal and unlearning safeguards can be trivially removed with a few steps of fine-tuning. These vulnerabilities necessitate new approaches for enabling the safe release of open-weight LLMs. We develop a method, called TAR, for building tamper-resistant safeguards into open-weight LLMs such that adversaries cannot remove the safeguards even after thousands of steps of fine-tuning. In extensive evaluations and red teaming analyses, we find that our method greatly improves tamper-resistance while preserving benign capabilities. Our results demonstrate that tamper-resistance is a tractable problem, opening up a promising new avenue to improve the safety and security of open-weight LLMs., Comment: Website: https://www.tamper-resistant-safeguards.com

As artificial intelligence systems grow more powerful, there has been increasing interest in "AI safety" research to address emerging and future risks. However, the field of AI safety remains poorly defined and inconsistently measured, leading to confusion about how researchers can contribute. This lack of clarity is compounded by the unclear relationship between AI safety benchmarks and upstream general capabilities (e.g., general knowledge and reasoning). To address these issues, we conduct a comprehensive meta-analysis of AI safety benchmarks, empirically analyzing their correlation with general capabilities across dozens of models and providing a survey of existing directions in AI safety. Our findings reveal that many safety benchmarks highly correlate with upstream model capabilities, potentially enabling "safetywashing" -- where capability improvements are misrepresented as safety advancements. Based on these findings, we propose an empirical foundation for developing more meaningful safety metrics and define AI safety in a machine learning research context as a set of clearly delineated research goals that are empirically separable from generic capabilities advancements. In doing so, we aim to provide a more rigorous framework for AI safety research, advancing the science of safety evaluations and clarifying the path towards measurable progress.

In recent years, Multi-Agent Reinforcement Learning (MARL) has found application in numerous areas of science and industry, such as autonomous driving, telecommunications, and global health. Nevertheless, MARL suffers from, for instance, an exponential growth of dimensions. Inherent properties of quantum mechanics help to overcome these limitations, e.g., by significantly reducing the number of trainable parameters. Previous studies have developed an approach that uses gradient-free quantum Reinforcement Learning and evolutionary optimization for variational quantum circuits (VQCs) to reduce the trainable parameters and avoid barren plateaus as well as vanishing gradients. This leads to a significantly better performance of VQCs compared to classical neural networks with a similar number of trainable parameters and a reduction in the number of parameters by more than 97 \% compared to similarly good neural networks. We extend an approach of K\"olle et al. by proposing a Gate-Based, a Layer-Based, and a Prototype-Based concept to mutate and recombine VQCs. Our results show the best performance for mutation-only strategies and the Gate-Based approach. In particular, we observe a significantly better score, higher total and own collected coins, as well as a superior own coin rate for the best agent when evaluated in the Coin Game environment.

Vision-and-language pretraining (VLP) in the medical field utilizes contrastive learning on image-text pairs to achieve effective transfer across tasks. Yet, current VLP approaches with the masked modelling strategy face two challenges when applied to the medical domain. First, current models struggle to accurately reconstruct key pathological features due to the scarcity of medical data. Second, most methods only adopt either paired image-text or image-only data, failing to exploit the combination of both paired and unpaired data. To this end, this paper proposes a XLIP (Masked modelling for medical Language-Image Pre-training) framework to enhance pathological learning and feature learning via unpaired data. First, we introduce the attention-masked image modelling (AttMIM) and entity-driven masked language modelling module (EntMLM), which learns to reconstruct pathological visual and textual tokens via multi-modal feature interaction, thus improving medical-enhanced features. The AttMIM module masks a portion of the image features that are highly responsive to textual features. This allows XLIP to improve the reconstruction of highly similar image data in medicine efficiency. Second, our XLIP capitalizes unpaired data to enhance multimodal learning by introducing disease-kind prompts. The experimental results show that XLIP achieves SOTA for zero-shot and fine-tuning classification performance on five datasets. Our code will be available at https://github.com/White65534/XLIP

Birds, bats and many insects can tuck their wings against their bodies at rest and deploy them to power flight. Whereas birds and bats use well-developed pectoral and wing muscles and tendons, how insects control these movements remains unclear, as mechanisms of wing deployment and retraction vary among insect species. Beetles (Coleoptera) display one of the most complex wing mechanisms. For example, in rhinoceros beetles, the wing deployment initiates by fully opening the elytra and partially releasing the hindwings from the abdomen. Subsequently, the beetle starts flapping, elevates the hindwings at the bases, and unfolds the wingtips in an origami-like fashion. Whilst the origami-like fold have been extensively explored, limited attention has been given to the hindwing base deployment and retraction, which are believed to be driven by thoracic muscles. Using high-speed cameras and robotic flapping-wing models, here we demonstrate that rhinoceros beetles can effortlessly elevate the hindwings to flight position without the need for muscular activity. We show that opening the elytra triggers a spring-like partial release of the hindwings from the body, allowing the clearance needed for subsequent flapping motion that brings the hindwings into flight position. The results also show that after flight, beetles can leverage the elytra to push the hindwings back into the resting position, further strengthening the hypothesis of a passive deployment mechanism. Finally, we validate the hypothesis with a flapping microrobot that passively deploys its wings for stable controlled flight and retracts them neatly upon landing, which offers a simple yet effective approach to the design of insect-like flying micromachines., Comment: 20 pages, 10 figures

Transparency in AI decision-making is crucial in healthcare due to the severe consequences of errors, and this is important for building trust among AI and users in sentiment analysis task. Incorporating reasoning capabilities helps Large Language Models (LLMs) understand human emotions within broader contexts, handle nuanced and ambiguous language, and infer underlying sentiments that may not be explicitly stated. In this work, we introduce a new task - Sentiment Reasoning - for both speech and text modalities, along with our proposed multimodal multitask framework and dataset. Our study showed that rationale-augmented training enhances model performance in sentiment classification across both human transcript and ASR settings. Also, we found that the generated rationales typically exhibit different vocabularies compared to human-generated rationales, but maintain similar semantics. All code, data (English-translated and Vietnamese) and models are published online: https://github.com/leduckhai/MultiMed, Comment: Preprint, 18 pages

Continuing advances in Large Language Models (LLMs) in artificial intelligence offer important capacities in intuitively accessing and using medical knowledge in many contexts, including education and training as well as assessment and treatment. Most of the initial literature on LLMs in medicine has emphasized that LLMs are unsuitable for medical use because they are non-deterministic, may provide incorrect or harmful responses, and cannot be regulated to assure quality control. If these issues could be corrected, optimizing LLM technology could benefit patients and physicians by providing affordable, point-of-care medical knowledge. Our proposed framework refines LLM responses by restricting their primary knowledge base to domain-specific datasets containing validated medical information. Additionally, we introduce an actor-critic LLM prompting protocol based on active inference principles of human cognition, where a Therapist agent initially responds to patient queries, and a Supervisor agent evaluates and adjusts responses to ensure accuracy and reliability. We conducted a validation study where expert cognitive behaviour therapy for insomnia (CBT-I) therapists evaluated responses from the LLM in a blind format. Experienced human CBT-I therapists assessed responses to 100 patient queries, comparing LLM-generated responses with appropriate and inappropriate responses crafted by experienced CBT-I therapists. Results showed that LLM responses received high ratings from the CBT-I therapists, often exceeding those of therapist-generated appropriate responses. This structured approach aims to integrate advanced LLM technology into medical applications, meeting regulatory requirements for establishing the safe and effective use of special purpose validated LLMs in medicine., Comment: 25 pages, 4 figures

Energy based models (EBMs) are appealing for their generality and simplicity in data likelihood modeling, but have conventionally been difficult to train due to the unstable and time-consuming implicit MCMC sampling during contrastive divergence training. In this paper, we present a novel energy-based generative framework, Variational Potential Flow (VAPO), that entirely dispenses with implicit MCMC sampling and does not rely on complementary latent models or cooperative training. The VAPO framework aims to learn a potential energy function whose gradient (flow) guides the prior samples, so that their density evolution closely follows an approximate data likelihood homotopy. An energy loss function is then formulated to minimize the Kullback-Leibler divergence between density evolution of the flow-driven prior and the data likelihood homotopy. Images can be generated after training the potential energy, by initializing the samples from Gaussian prior and solving the ODE governing the potential flow on a fixed time interval using generic ODE solvers. Experiment results show that the proposed VAPO framework is capable of generating realistic images on various image datasets. In particular, our proposed framework achieves competitive FID scores for unconditional image generation on the CIFAR-10 and CelebA datasets.

Creating secure and resilient applications with large language models (LLM) requires anticipating, adjusting to, and countering unforeseen threats. Red-teaming has emerged as a critical technique for identifying vulnerabilities in real-world LLM implementations. This paper presents a detailed threat model and provides a systematization of knowledge (SoK) of red-teaming attacks on LLMs. We develop a taxonomy of attacks based on the stages of the LLM development and deployment process and extract various insights from previous research. In addition, we compile methods for defense and practical red-teaming strategies for practitioners. By delineating prominent attack motifs and shedding light on various entry points, this paper provides a framework for improving the security and robustness of LLM-based systems., Comment: Preprint. Under review

Reinforcement learning from human feedback plays a crucial role in aligning language models towards human preferences, traditionally represented through comparisons between pairs or sets of responses within a given context. While many studies have enhanced algorithmic techniques to optimize learning from such data, this work shifts focus to improving preference learning through a data-centric approach. Specifically, we propose enriching existing preference datasets with machine-generated rationales that explain the reasons behind choices. We develop a simple and principled framework to augment current preference learning methods with rationale information. Our comprehensive analysis highlights how rationales enhance learning efficiency. Extensive experiments reveal that rationale-enriched preference learning offers multiple advantages: it improves data efficiency, accelerates convergence to higher-performing models, and reduces verbosity bias and hallucination. Furthermore, this framework is versatile enough to integrate with various preference optimization algorithms. Overall, our findings highlight the potential of re-imagining data design for preference learning, demonstrating that even freely available machine-generated rationales can significantly boost performance across multiple dimensions. The code repository is available at https: //github.com/reds-lab/preference-learning-with-rationales, Comment: Data-Centric Human Preference Learning with Rationales

This paper addresses the challenge of improving learning-based monocular visual odometry (VO) in underwater environments by integrating principles of underwater optical imaging to manipulate optical flow estimation. Leveraging the inherent properties of underwater imaging, the novel wflow-TartanVO is introduced, enhancing the accuracy of VO systems for autonomous underwater vehicles (AUVs). The proposed method utilizes a normalized medium transmission map as a weight map to adjust the estimated optical flow for emphasizing regions with lower degradation and suppressing uncertain regions affected by underwater light scattering and absorption. wflow-TartanVO does not require fine-tuning of pre-trained VO models, thus promoting its adaptability to different environments and camera models. Evaluation of different real-world underwater datasets demonstrates the outperformance of wflow-TartanVO over baseline VO methods, as evidenced by the considerably reduced Absolute Trajectory Error (ATE). The implementation code is available at: https://github.com/bachzz/wflow-TartanVO

Electronic health records (EHRs) are multimodal by nature, consisting of structured tabular features like lab tests and unstructured clinical notes. In real-life clinical practice, doctors use complementary multimodal EHR data sources to get a clearer picture of patients' health and support clinical decision-making. However, most EHR predictive models do not reflect these procedures, as they either focus on a single modality or overlook the inter-modality interactions/redundancy. In this work, we propose MEDFuse, a Multimodal EHR Data Fusion framework that incorporates masked lab-test modeling and large language models (LLMs) to effectively integrate structured and unstructured medical data. MEDFuse leverages multimodal embeddings extracted from two sources: LLMs fine-tuned on free clinical text and masked tabular transformers trained on structured lab test results. We design a disentangled transformer module, optimized by a mutual information loss to 1) decouple modality-specific and modality-shared information and 2) extract useful joint representation from the noise and redundancy present in clinical notes. Through comprehensive validation on the public MIMIC-III dataset and the in-house FEMH dataset, MEDFuse demonstrates great potential in advancing clinical predictions, achieving over 90% F1 score in the 10-disease multi-label classification task.

Magnetic Johnson noise is an important consideration for many applications involving precision magnetometry, and its significance will only increase in the future with improvements in measurement sensitivity. The fluctuation-dissipation theorem can be utilized to derive analytic expressions for magnetic Johnson noise in certain situations. But when used in conjunction with finite element analysis tools, the combined approach is particularly powerful as it provides a practical means to calculate the magnetic Johnson noise arising from conductors of arbitrary geometry and permeability. In this paper, we demonstrate this method to be one of the most comprehensive approaches presently available to calculate thermal magnetic noise. In particular, its applicability is shown to not be limited to cases where the noise is evaluated at a point in space but also can be expanded to include cases where the magnetic field detector has a more general shape, such as a finite size loop, a gradiometer, or a detector that consists of a polarized atomic species trapped in a volume. Furthermore, some physics insights gained through studies made using this method are discussed

We propose a forward-backward splitting dynamical system for solving inclusion problems of the form $0\in A(x)+B(x)$ in Hilbert spaces, where $A$ is a maximal operator and $B$ is a single-valued operator. Involved operators are assumed to satisfy a generalized monotonicity condition, which is weaker than the standard monotone assumptions. Under mild conditions on parameters, we establish the fixed-time stability of the proposed dynamical system. In addition, we consider an explicit forward Euler discretization of the dynamical system leading to a new forward backward algorithm for which we present the convergence analysis. Applications to other optimization problems such as Constrained Optimization Problems (COPs), Mixed Variational Inequalities (MVIs), and Variational Inequalities (VIs) are presented and some numerical examples are given to illustrate the theoretical results.

Large Language Models (LLMs) have been applied to many research problems across various domains. One of the applications of LLMs is providing question-answering systems that cater to users from different fields. The effectiveness of LLM-based question-answering systems has already been established at an acceptable level for users posing questions in popular and public domains such as trivia and literature. However, it has not often been established in niche domains that traditionally require specialized expertise. To this end, we construct the NEPAQuAD1.0 benchmark to evaluate the performance of three frontier LLMs -- Claude Sonnet, Gemini, and GPT-4 -- when answering questions originating from Environmental Impact Statements prepared by U.S. federal government agencies in accordance with the National Environmental Environmental Act (NEPA). We specifically measure the ability of LLMs to understand the nuances of legal, technical, and compliance-related information present in NEPA documents in different contextual scenarios. For example, we test the LLMs' internal prior NEPA knowledge by providing questions without any context, as well as assess how LLMs synthesize the contextual information present in long NEPA documents to facilitate the question/answering task. We compare the performance of the long context LLMs and RAG powered models in handling different types of questions (e.g., problem-solving, divergent). Our results suggest that RAG powered models significantly outperform the long context models in the answer accuracy regardless of the choice of the frontier LLM. Our further analysis reveals that many models perform better answering closed questions than divergent and problem-solving questions., Comment: 14 pages

We revisit data selection in a modern context of finetuning from a fundamental perspective. Extending the classical wisdom of variance minimization in low dimensions to high-dimensional finetuning, our generalization analysis unveils the importance of additionally reducing bias induced by low-rank approximation. Inspired by the variance-bias tradeoff in high dimensions from the theory, we introduce Sketchy Moment Matching (SkMM), a scalable data selection scheme with two stages. (i) First, the bias is controlled using gradient sketching that explores the finetuning parameter space for an informative low-dimensional subspace $\mathcal{S}$; (ii) then the variance is reduced over $\mathcal{S}$ via moment matching between the original and selected datasets. Theoretically, we show that gradient sketching is fast and provably accurate: selecting $n$ samples by reducing variance over $\mathcal{S}$ preserves the fast-rate generalization $O(\dim(\mathcal{S})/n)$, independent of the parameter dimension. Empirically, we concretize the variance-bias balance via synthetic experiments and demonstrate the effectiveness of SkMM for finetuning in real vision tasks.

In this paper, we present the submission to the 5th Annual Smoky Mountains Computational Sciences Data Challenge, Challenge 3. This is the solution for semantic segmentation problem in both real-world and synthetic images from a vehicle s forward-facing camera. We concentrate in building a robust model which performs well across various domains of different outdoor situations such as sunny, snowy, rainy, etc. In particular, our method is developed with two main directions: model development and domain adaptation. In model development, we use the High Resolution Network (HRNet) as the baseline. Then, this baseline s result is processed by two coarse-to-fine models: Object-Contextual Representations (OCR) and Hierarchical Multi-scale Attention (HMA) to get the better robust feature. For domain adaption, we implement the Domain-Based Batch Normalization (DNB) to reduce the distribution shift from diverse domains. Our proposed method yield 81.259 mean intersection-over-union (mIoU) in validation set. This paper studies the effectiveness of employing real-world and synthetic data to handle the domain adaptation in semantic segmentation problem., Comment: 13 pages

The spatial distribution of the gaseous components of the Milky Way is of great importance for a number of different fields, e.g. Galactic structure, star formation and cosmic rays. However, obtaining distance information to gaseous clouds in the interstellar medium from Doppler-shifted line emission is notoriously difficult given our unique vantage point in the Galaxy. It requires precise knowledge of gas velocities and generally suffers from distance ambiguities. Previous works often assumed the optically thin limit (no absorption), a fixed velocity field, and lack resolution overall. We aim to overcome these issues and improve previous reconstructions of the gaseous constituents of the interstellar medium of the Galaxy. We use 3D Gaussian processes to model correlations in the interstellar medium, including correlations between different lines of sight, and enforce a spatially coherent structure in the prior. For modelling the transport of radiation from the emitting gas to us as observers, we take absorption effects into account. A special numerical grid ensures high resolution nearby. We infer the spatial distributions of HI, CO, their emission line-widths, and the Galactic velocity field in a joint Bayesian inference. We further constrain these fields with complementary data from Galactic masers and young stellar object clusters. Our main result consists of a set of samples that implicitly contain statistical uncertainties. The resulting maps are spatially coherent and reproduce the data with high fidelity. We confirm previous findings regarding the warping and flaring of the Galactic disc. A comparison with 3D dust maps reveals a good agreement on scales larger than approximately 400 pc. While our results are not free of artefacts, they present a big step forward in obtaining high quality 3D maps of the interstellar medium., Comment: 21 pages, 22 figures

Cosmic rays (CRs) play a major role in the dynamics of the interstellar medium (ISM). Their interactions and transport ionize, heat, and push the ISM thereby coupling different regions of it. The spatial distribution of CRs depends on the distribution of their sources as well as the ISM constituents they interact with, such as gas, starlight, and magnetic fields. Particularly, gas interacts closely with CRs, influencing CR fluxes and gamma -ray emission. We illustrate the influence of 3D gas structures on CR transport and gamma -ray emission. We use the PICARD code and multiple samples of recent 3D reconstructions of the HI and H$_2$ Galactic gas constituents to investigate the impact on the transport of CRs and emission of gamma -rays. We find the necessary transport parameters to reproduce local measurements of CR fluxes, and see that they depend on the local distribution of gas density and structure. The distribution of CR fluxes exhibits energy-dependent structures that vary for all CR species due to their corresponding loss processes. Regions of enhanced secondary (primary) species are spatially correlated (anti-correlated) with the gas density. We observe a high sensitivity of the gamma -ray emission on the contrast of gas structures, as those determine the 3D spatial distributions of hadronic interactions and bremsstrahlung. We find that corresponding gas-induced structures in the distribution of CR electrons are also visible in Inverse Compton (IC) emission. Due to the aforementioned sensitivity, the analysis of CR data for CR sources and transport parameters requires the usage of accurate 3D gas maps.

We derive an analytical expression for $s$-state exciton energies in monolayer transition-metal dichalcogenides (TMDCs): $E_{\text{ns}}=-{\text{Ry}}^*\times P_n/{(n-1/2+0.479\, r^*_0/\kappa)^2}$, $n=1,2,...$, where $r^*_0$ and $\kappa$ are the dimensionless screening length and dielectric constant of the surrounding medium; $\text{Ry}^*$ is an effective Rydberg energy scaled by the dielectric constant and exciton reduced mass; $P_n(r^*_0/\kappa)$ is a function of variables $n$ and $r^*_0/\kappa$. Its values are around 1.0 so we consider it a term that corrects the Rydberg energy. Despite the simple form, the suggested formula gives exciton energies with high precision compared to the exact numerical solutions that accurately describe recent experimental data for a large class of TMDC materials, including WSe$_2$, WS$_2$, MoSe$_2$, MoS$_2$, and MoTe$_2$. To achieve these results, we have developed a so-called regulated perturbation theory by combining the conventional perturbation method with several elements of the Feranchuk-Komarov operator method, including the Levi-Civita transformation, the algebraic calculation technique via the annihilation and creation operators, and the introduction of a free parameter to optimize the convergence rate of the perturbation series. This universal form of exciton energies could be helpful in various physical analyses, including retrieval of the material parameters such as reduced exciton mass and screening length from the available measured exciton energies., Comment: 5 pages, 1 figure, 3 tables, 1 supplementary