Search

[This corrects the article DOI: 10.3389/fimmu.2023.1207108.].

Frame-level micro- and macro-expression spotting methods require time-consuming frame-by-frame observation during annotation. Meanwhile, video-level spotting lacks sufficient information about the location and number of expressions during training, resulting in significantly inferior performance compared with fully-supervised spotting. To bridge this gap, we propose a point-level weakly-supervised expression spotting (PWES) framework, where each expression requires to be annotated with only one random frame (i.e., a point). To mitigate the issue of sparse label distribution, the prevailing solution is pseudo-label mining, which, however, introduces new problems: localizing contextual background snippets results in inaccurate boundaries and discarding foreground snippets leads to fragmentary predictions. Therefore, we design the strategies of multi-refined pseudo label generation (MPLG) and distribution-guided feature contrastive learning (DFCL) to address these problems. Specifically, MPLG generates more reliable pseudo labels by merging class-specific probabilities, attention scores, fused features, and point-level labels. DFCL is utilized to enhance feature similarity for the same categories and feature variability for different categories while capturing global representations across the entire datasets. Extensive experiments on the CAS(ME)^2, CAS(ME)^3, and SAMM-LV datasets demonstrate PWES achieves promising performance comparable to that of recent fully-supervised methods.

As transformer-based language models are trained on increasingly large datasets and with vast numbers of parameters, finding more efficient alternatives to the standard Transformer has become very valuable. While many efficient Transformers and Transformer alternatives have been proposed, none provide theoretical guarantees that they are a suitable replacement for the standard Transformer. This makes it challenging to identify when to use a specific model and what directions to prioritize for further investigation. In this paper, we aim to understand the capabilities and limitations of efficient Transformers, specifically the Sparse Transformer and the Linear Transformer. We focus on their reasoning capability as exhibited by Chain-of-Thought (CoT) prompts and follow previous works to model them as Dynamic Programming (DP) problems. Our results show that while these models are expressive enough to solve general DP tasks, contrary to expectations, they require a model size that scales with the problem size. Nonetheless, we identify a class of DP problems for which these models can be more efficient than the standard Transformer. We confirm our theoretical results through experiments on representative DP tasks, adding to the understanding of efficient Transformers' practical strengths and weaknesses.

With the development of AI-Generated Content (AIGC), text-to-audio models are gaining widespread attention. However, it is challenging for these models to generate audio aligned with human preference due to the inherent information density of natural language and limited model understanding ability. To alleviate this issue, we formulate the BATON, a framework designed to enhance the alignment between generated audio and text prompt using human preference feedback. Our BATON comprises three key stages: Firstly, we curated a dataset containing both prompts and the corresponding generated audio, which was then annotated based on human feedback. Secondly, we introduced a reward model using the constructed dataset, which can mimic human preference by assigning rewards to input text-audio pairs. Finally, we employed the reward model to fine-tune an off-the-shelf text-to-audio model. The experiment results demonstrate that our BATON can significantly improve the generation quality of the original text-to-audio models, concerning audio integrity, temporal relationship, and alignment with human preference.

In this work, we leverage the intrinsic segmentation of language sequences and design a new positional encoding method called Bilevel Positional Encoding (BiPE). For each position, our BiPE blends an intra-segment encoding and an inter-segment encoding. The intra-segment encoding identifies the locations within a segment and helps the model capture the semantic information therein via absolute positional encoding. The inter-segment encoding specifies the segment index, models the relationships between segments, and aims to improve extrapolation capabilities via relative positional encoding. Theoretical analysis shows this disentanglement of positional information makes learning more effective. The empirical results also show that our BiPE has superior length extrapolation capabilities across a wide range of tasks in diverse text modalities., Comment: 17 pages, 7 figures, 8 tables; Working in Progress

Utilization of inter-base station cooperation for information processing has shown great potential in enhancing the overall quality of communication services (QoS) in wireless communication networks. Nevertheless, such cooperations require the knowledge of channel state information (CSI) at base stations (BSs), which is assumed to be perfectly known. However, CSI errors are inevitable in practice which necessitates beamforming techniques that can achieve robust performance in the presence of channel estimation errors. Existing approaches relax the robust beamforming design problems into semidefinite programming (SDP), which can only achieve a solution that is far from being optimal. To this end, this paper views robust beamforming design problems from a bilevel optimization perspective. In particular, we focus on maximizing the worst-case weighted sum-rate (WSR) in the downlink multi-cell multi-user multiple-input single-output (MISO) system considering bounded CSI errors. We first reformulate this problem into a bilevel optimization problem and then develop an efficient algorithm based on the cutting plane method. A distributed optimization algorithm has also been developed to facilitate the parallel processing in practical settings. Numerical results are provided to confirm the effectiveness of the proposed algorithm in terms of performance and complexity, particularly in the presence of CSI uncertainties., Comment: accepted at AAAI2024

Exploration remains a critical issue in deep reinforcement learning for an agent to attain high returns in unknown environments. Although the prevailing exploration Random Network Distillation (RND) algorithm has been demonstrated to be effective in numerous environments, it often needs more discriminative power in bonus allocation. This paper highlights the "bonus inconsistency" issue within RND, pinpointing its primary limitation. To address this issue, we introduce the Distributional RND (DRND), a derivative of the RND. DRND enhances the exploration process by distilling a distribution of random networks and implicitly incorporating pseudo counts to improve the precision of bonus allocation. This refinement encourages agents to engage in more extensive exploration. Our method effectively mitigates the inconsistency issue without introducing significant computational overhead. Both theoretical analysis and experimental results demonstrate the superiority of our approach over the original RND algorithm. Our method excels in challenging online exploration scenarios and effectively serves as an anti-exploration mechanism in D4RL offline tasks. Our code is publicly available at https://github.com/yk7333/DRND., Comment: ICML 2024 accepted

Recent advancements in generative artificial intelligence (AI) have raised concerns about their potential to create convincing fake social media accounts, but empirical evidence is lacking. In this paper, we present a systematic analysis of Twitter(X) accounts using human faces generated by Generative Adversarial Networks (GANs) for their profile pictures. We present a dataset of 1,353 such accounts and show that they are used to spread scams, spam, and amplify coordinated messages, among other inauthentic activities. Leveraging a feature of GAN-generated faces -- consistent eye placement -- and supplementing it with human annotation, we devise an effective method for identifying GAN-generated profiles in the wild. Applying this method to a random sample of active Twitter users, we estimate a lower bound for the prevalence of profiles using GAN-generated faces between 0.021% and 0.044% -- around 10K daily active accounts. These findings underscore the emerging threats posed by multimodal generative AI. We release the source code of our detection method and the data we collect to facilitate further investigation. Additionally, we provide practical heuristics to assist social media users in recognizing such accounts., Comment: 18 pages, 6 figures

Social media platforms allow users to create polls to gather public opinion on diverse topics. However, we know little about what such polls are used for and how reliable they are, especially in significant contexts like elections. Focusing on the 2020 presidential elections in the U.S., this study shows that outcomes of election polls on Twitter deviate from election results despite their prevalence. Leveraging demographic inference and statistical analysis, we find that Twitter polls are disproportionately authored by older males and exhibit a large bias towards candidate Donald Trump relative to representative mainstream polls. We investigate potential sources of biased outcomes from the point of view of inauthentic, automated, and counter-normative behavior. Using social media experiments and interviews with poll authors, we identify inconsistencies between public vote counts and those privately visible to poll authors, with the gap potentially attributable to purchased votes. We also find that Twitter accounts participating in election polls are more likely to be bots, and election poll outcomes tend to be more biased, before the election day than after. Finally, we identify instances of polls spreading voter fraud conspiracy theories and estimate that a couple thousand of such polls were posted in 2020. The study discusses the implications of biased election polls in the context of transparency and accountability of social media platforms., Comment: 14 pages, 10 figures

The primary goal of online change detection (OCD) is to promptly identify changes in the data stream. OCD problem find a wide variety of applications in diverse areas, e.g., security detection in smart grids and intrusion detection in communication networks. Prior research usually assumes precise knowledge of the system parameters. Nevertheless, this presumption often proves unattainable in practical scenarios due to factors such as estimation errors, system updates, etc. This paper aims to take the first attempt to develop a triadic-OCD framework with certifiable robustness, provable optimality, and guaranteed convergence. In addition, the proposed triadic-OCD algorithm can be realized in a fully asynchronous distributed manner, easing the necessity of transmitting the data to a single server. This asynchronous mechanism could also mitigate the straggler issue that faced by traditional synchronous algorithm. Moreover, the non-asymptotic convergence property of Triadic-OCD is theoretically analyzed, and its iteration complexity to achieve an $\epsilon$-optimal point is derived. Extensive experiments have been conducted to elucidate the effectiveness of the proposed method., Comment: Accepted at ICML2024

Hao Zhang,1,2 Kai Yang,3 Qin Wang,2 Lei Jin,4 Ling-Mei Wang,2 Xing-Yu Fan,2 Xiu Wang,2 Qiang Zhou,2 Yuan-Hong Xu1 1Department of Clinical Laboratory, First Affiliated Hospital of Anhui Medical University, Hefei, Anhui, 230022, People’s Republic of China; 2Department of Clinical Laboratory, Second Affiliated Hospital of Anhui Medical University, Hefei, Anhui, 230601, People’s Republic of China; 3Department of Medical Technology, Anhui Medical College, Hefei, Anhui, 230601, People’s Republic of China; 4Department of Infectious Diseases, Second Affiliated Hospital of Anhui Medical University, Hefei, Anhui, 230601, People’s Republic of ChinaCorrespondence: Yuan-Hong Xu, Department of Clinical Laboratory, the First Hospital of Anhui Medical University, No. 218, Jixi Road, Hefei, Anhui, 230022, People’s Republic of China, Email xyhong1964@163.com Qiang Zhou, Department of Clinical Laboratory, the Second Hospital of Anhui Medical University, No. 678, Furong Road, Hefei, 230601, Anhui, People’s Republic of China, Email zhouqiang1973@163.comPurpose: Hepatitis B virus-related acute-on-chronic liver failure (HBV-ACLF) is a serious medical condition with a high short-term mortality rate, making accurate prognostic assessment essential for informed clinical decision-making. In this study, we aimed to develop a simple and effective prognostic model for predicting short-term mortality in patients with HBV-ACLF.Patients and Methods: To achieve our objective, we enrolled both a cross-sectional cohort (n = 291) and a retrospective cohort (n = 185) in this study. We collected laboratory and clinical data from these cohorts and performed univariate and multivariate logistic regression analyses to identify independent predictors of short-term mortality. Subsequently, we developed a novel prognostic score for HBV-ACLF, which was validated and assessed using receiver operating characteristic (ROC) curve analysis to determine its performance.Results: Our analysis revealed that

Bioinspired iron-catechol cross-links have shown remarkable success in increasing the mechanical properties of polymer networks, in part due to clustering of Fe3+-catechol domains which act as secondary network reinforcing sites. We report a versatile synthetic procedure to prepare modular PEG-acrylate networks with independently tunable covalent bis(acrylate) and supramolecular Fe3+-catechol cross-linking. Initial control of network structure is achieved through radical polymerization and cross-linking, followed by postpolymerization incorporation of catechol units via quantitative active ester chemistry and subsequent complexation with iron salts. By tuning the ratio of each building block, dual cross-linked networks reinforced by clustered iron-catechol domains are prepared and exhibit a wide range of properties (Young's moduli up to ∼245 MPa), well beyond the values achieved through purely covalent cross-linking. This stepwise approach to mixed covalent and metal-ligand cross-linked networks also permits local patterning of PEG-based films through masking techniques forming distinct hard, soft, and gradient regions.

Kai Yang,1 Beibei Sun,2 Shicheng Zhang,3 Ying Pan,1 Jun Fang4 1Department of Medical Technology, Anhui Medical College, Hefei, Anhui, 230601, People’s Republic of China; 2Department of Clinical Laboratory, the Second Hospital of Anhui Medical University, Hefei, Anhui, 230601, People’s Republic of China; 3School of Public Health and Health Management, Anhui Medical College, Hefei, Anhui, 230601, People’s Republic of China; 4Faculty of Pharmaceutical Science, Sojo University, Kumamoto, 860-0082, JapanCorrespondence: Jun Fang, Faculty of Pharmaceutical Science, Sojo University, Ikeda 4-22-1, Kumamoto, 860-0082, Japan, Email fangjun@ph.sojo-u.ac.jpPurpose: The clinical significance of the red blood cell distribution width (RDW)-coefficient of variation (RDW-CV) has been recognized in numerous diseases, but few studies have investigated the usefulness of RDW-standard deviation (RDW-SD). This study aimed to compare the utility of RDW-SD and RDW-CV in evaluating liver fibrosis stage in patients with chronic hepatitis B (CHB).Patients and Methods: In this retrospective study, we enrolled 720 treatment-naïve CHB patients and 578 healthy controls, and evaluated their clinical parameters. In CHB patients, the associations between RDW-CV and liver fibrosis stage were analyzed as compared to RDW-SD using one-way analysis of variance (ANOVA), Spearman’s rank correlation, student’s t-test, binary logistic regression, and receiver operating characteristic (ROC) curve.Results: RDW-SD, rather than RDW-CV was significantly elevated in CHB patients compared with healthy controls. Correlation analysis showed a stronger association between RDW-SD and liver fibrosis stage than RDW-CV in CHB patients. RDW-CV and RDW-SD are both independent predictors of significant fibrosis. For the diagnosis of significant fibrosis, the area under the receiver operating characteristic curve (AUC) for RDW-CV was 0.599, while for RDW-SD, it was 0.706. RDW-to-platelet ratio (RPR), a novel index fo

Solar eruptive activities could occur in weak magnetic field environments and over large spatial scales, especially relevant to eruptions involving intermediate or quiescent solar filaments. To handle the large scales, we implement and apply a flux rope embedding method using regularized Biot-Savart laws in the spherical coordinate system. Combined with a potential field source surface model and a magneto-frictional method, a nonlinear force-free field comprising a flux rope embedded in a potential field is constructed. Using the combined nonlinear force-free field as the initial condition, we then perform a zero-$\beta$ data-constrained magnetohydrodynamic (MHD) simulation for an M8.7 flare at 03:38 UT on 2012 January 23. The MHD model reproduces the eruption process, flare ribbon evolution (represented by the quasi-separatrix layer evolution) and kinematics of the flux rope. This approach could potentially model global-scale eruptions from weak field regions., Comment: 23 pages, 7 figures, accepted for publicaiton in ApJ

Protostellar outflows are considered a signpost of star formation. These outflows can cause shocks in the molecular gas and are typically traced by the line wings of certain molecules. HNCO (4--3) has been regarded as a shock tracer because of the high abundance in shocked regions. Here we present the first imaging results of HNCO (4--3) line wings toward nine sources in a sample of twenty three massive star-forming regions using the IRAM 30\,m telescope. We adopt the velocity range of the full width of HC$_{3}$N (10--9) and H$^{13}$CO$^+$ (1--0) emissions as the central emission values, beyond which the emission from HNCO (4--3) is considered to be from line wings. The spatial distributions of the red- and/or blue-lobes of HNCO (4--3) emission nicely associate with those lobes of HCO$^{+}$ (1--0) in most of the sources. High intensity ratios of HNCO (4--3) to HCO$^+$ (1--0) are obtained in the line wings. The derived column density ratios of HNCO to HCO$^+$ are consistent with those previously observed towards massive star-forming regions. These results provide direct evidence that HNCO could trace outflow in massive star-forming regions. This work also implies that the formation of some HNCO molecules is related to shock, either on the grain surface or within the shocked gas., Comment: 18 pages, 4 tables, 4 figures, and accepted for publication in ApJ

Fu X, Li H, Yang K, Chen D, Tang H. Onco Targets Ther. 2016;9:2237–2245. The Editor and Publisher of OncoTargets and Therapy are retracting the published article. Concerns were raised regarding the alleged duplication of tumor images in Figure 4A with those from other articles by the same authors. Specifically, The tumor images in Figure 4A appear to have been duplicated with those from Figure 5A from Li H, Fu X, Yang K, Chen D, Tang H, Zhao Q. The clock gene PER1 suppresses expression of tumor-related genes in human oral squamous cell carcinoma. Oncotarget. 2016;7:20574–20583. https://doi.org/10.18632/oncotarget.7827; Figure 4A from Zhao Q, Zheng G, Yang K, et al. The clock gene PER1 plays an important role in regulating the clock gene network in human oral squamous cell carcinoma cells. Oncotarget. 2016;7:70290–70302. https://doi.org/10.18632/oncotarget.11844 and Figure 6A from Su X, Chen D, Yang K, et al. The circadian clock gene PER2 plays an important role in tumor suppression through regulating tumor-associated genes in human oral squamous cell carcinoma. Oncology Reports. 2017;38:472–480. https://doi.org/10.3892/or.2017.5653. In addition, the image from Figure 4C, SCC15 has been duplicated with the image from Figure 4D, Control-shRNA from Zhao Q et al (2016). The authors responded to our queries but were unable to provide an explanation for the duplicated images nor were they able to provide the original data from their study. As the findings could not be verified the Editor and Publisher made the decision to retract the article and the authors were notified of this. We have been informed in our decision-making by our editorial policies and the COPE guidelines. The retracted article will remain online to maintain the scholarly record, but it will be digitally watermarked on each page as “Retracted”.

Infection with the intracellular bacterium Coxiella (C.) burnetii can cause chronic Q fever with severe complications and limited treatment options. Here, we identify the enzyme cis-aconitate decarboxylase 1 (ACOD1 or IRG1) and its product itaconate as protective host immune pathway in Q fever. Infection of mice with C. burnetii induced expression of several anti-microbial candidate genes, including Acod1. In macrophages, Acod1 was essential for restricting C. burnetii replication, while other antimicrobial pathways were dispensable. Intratracheal or intraperitoneal infection of Acod1(-/-) mice caused increased C. burnetii burden, weight loss and stronger inflammatory gene expression. Exogenously added itaconate restored pathogen control in Acod1(-/-) mouse macrophages and blocked replication in human macrophages. In axenic cultures, itaconate directly inhibited growth of C. burnetii. Finally, treatment of infected Acod1(-/-) mice with itaconate efficiently reduced the tissue pathogen load. Thus, ACOD1-derived itaconate is a key factor in the macrophage-mediated defense against C. burnetii and may be exploited for novel therapeutic approaches in chronic Q fever.

Several approximate inference methods have been proposed for deep discrete latent variable models. However, non-parametric methods which have previously been successfully employed for classical sparse coding models have largely been unexplored in the context of deep models. We propose a non-parametric iterative algorithm for learning discrete latent representations in such deep models. Additionally, to learn scale invariant discrete features, we propose local data scaling variables. Lastly, to encourage sparsity in our representations, we propose a Beta-Bernoulli process prior on the latent factors. We evaluate our spare coding model coupled with different likelihood models. We evaluate our method across datasets with varying characteristics and compare our results to current amortized approximate inference methods.

IoT time series analysis has found numerous applications in a wide variety of areas, ranging from health informatics to network security. Nevertheless, the complex spatial temporal dynamics and high dimensionality of IoT time series make the analysis increasingly challenging. In recent years, the powerful feature extraction and representation learning capabilities of deep learning (DL) have provided an effective means for IoT time series analysis. However, few existing surveys on time series have systematically discussed unsupervised DL-based methods. To fill this void, we investigate unsupervised deep learning for IoT time series, i.e., unsupervised anomaly detection and clustering, under a unified framework. We also discuss the application scenarios, public datasets, existing challenges, and future research directions in this area., Comment: 22 pages, 8 figures, has been published by IEEE Internet of Things Journal

Mounting compact and lightweight base stations on unmanned aerial vehicles (UAVs) is a cost-effective and flexible solution to provide seamless coverage on the existing terrestrial networks. While the coverage probability in UAV-assisted cellular networks has been widely investigated, it provides only the first-order statistic of signal-to-interference-plus-noise ratio (SINR). In this paper, to analyze high-order statistics of SINR and characterize the disparity among individual links, we provide a meta distribution (MD)-based analytical framework for UAV-assisted cellular networks, in which the probabilistic line-of-sight channel and realistic antenna pattern are taken into account for air-to-ground transmissions. To accurately characterize the interference from UAVs, we relax the widely applied uniform off-boresight angle (OBA) assumption and derive the exact distribution of OBA. Using stochastic geometry, for both steerable and vertical antenna scenarios, we obtain mathematical expressions for the moments of condition success probability, the SINR MD, and the mean local delay. Moreover, we study the asymptotic behavior of the moments as network density approaches infinity. Numerical results validate the tightness of the theoretical results and show that the uniform OBA assumption underestimates the network performance, especially in the regime of moderate altitude of UAV. We also show that when UAVs are equipped with steerable antennas, the network coverage and user fairness can be optimized simultaneously by carefully adjusting the UAV parameters., Comment: 30 pages, 9 figures

Social media are utilized by millions of citizens to discuss important political issues. Politicians use these platforms to connect with the public and broadcast policy positions. Therefore, data from social media has enabled many studies of political discussion. While most analyses are limited to data from individual platforms, people are embedded in a larger information ecosystem spanning multiple social networks. Here we describe and provide access to the Indiana University 2022 U.S. Midterms Multi-Platform Social Media Dataset (MEIU22), a collection of social media posts from Twitter, Facebook, Instagram, Reddit, and 4chan. MEIU22 links to posts about the midterm elections based on a comprehensive list of keywords and tracks the social media accounts of 1,011 candidates from October 1 to December 25, 2022. We also publish the source code of our pipeline to enable similar multi-platform research projects., Comment: 8 pages, 3 figures, forthcoming in ICWSM23

Sihua Wen,1,* Kai Yang,1,* Yunfeng Bai,2 Yanan Wu,1 Ding Liu,1 Xu Wu,1 Xiaofei Zhang,1 Jing Sun1 1Department of Pharmaceutics, The Key Laboratory of Basic and New Drug Research of Traditional Chinese Medicine, Shaanxi University of Chinese Medicine, Xianyang, People’s Republic of China; 2Shaanxi Dongtai Pharmaceutical Co., Ltd, Shaanxi, People’s Republic of China*These authors contributed equally to this workCorrespondence: Xiaofei Zhang; Jing Sun, Department of Pharmaceutics, College of Pharmacy, Shaanxi University of Chinese Medicine, Xianyang, People’s Republic of China, Tel +86 177 7003 7322, Fax +86 029-38185333, Email 2051028@sntcm.edu.cn; sunjing202203@126.comPurpose: To study the active components, drug targets and mechanism of Schisandra chinensis (S.chinensis) combined with coenzyme Q10 (CQ10) in the treatment of heart failure (HF).Methods: Network pharmacology combined with the gene expression omnibus chip method to analyze the main pathways by which S.chinensis combined with CQ10 functioned to treat heart failure. Subsequently, the biological activities of the major pathway key proteins and their corresponding compounds were verified by molecular docking techniques. Finally, the molecular mechanism of S. chinensis combined with CQ10 for the treatment of heart failure was verified using a rat heart failure model induced by isoproterenol hydrochloride and using hematoxylin-eosin staining, TUNEL, immunohistochemistry and Western blot.Results: Network pharmacology combined with experimental validation suggests that the mechanism of action of S.chinensis combined with CQ10 in the treatment of heart failure may involve CQ10, Citral, Schisandrone, Schisanhenol B, Gomisin O, Schisandrin C and other components, which may synergistically inhibit the PI3K-AKT signaling pathway and affect the expression of AKT1, PIK3CG and other targets on this pathway. In addition, S. chinensis combined with CQ10 could effectively improve the cardiac c

Most micro- and macro-expression spotting methods in untrimmed videos suffer from the burden of video-wise collection and frame-wise annotation. Weakly-supervised expression spotting (WES) based on video-level labels can potentially mitigate the complexity of frame-level annotation while achieving fine-grained frame-level spotting. However, we argue that existing weakly-supervised methods are based on multiple instance learning (MIL) involving inter-modality, inter-sample, and inter-task gaps. The inter-sample gap is primarily from the sample distribution and duration. Therefore, we propose a novel and simple WES framework, MC-WES, using multi-consistency collaborative mechanisms that include modal-level saliency, video-level distribution, label-level duration and segment-level feature consistency strategies to implement fine frame-level spotting with only video-level labels to alleviate the above gaps and merge prior knowledge. The modal-level saliency consistency strategy focuses on capturing key correlations between raw images and optical flow. The video-level distribution consistency strategy utilizes the difference of sparsity in temporal distribution. The label-level duration consistency strategy exploits the difference in the duration of facial muscles. The segment-level feature consistency strategy emphasizes that features under the same labels maintain similarity. Experimental results on three challenging datasets -- CAS(ME)$^2$, CAS(ME)$^3$, and SAMM-LV -- demonstrate that MC-WES is comparable to state-of-the-art fully-supervised methods.

In recent years, AI models that mine intrinsic patterns from molecular structures and protein sequences have shown promise in accelerating drug discovery. However, these methods partly lag behind real-world pharmaceutical approaches of human experts that additionally grasp structured knowledge from knowledge bases and unstructured knowledge from biomedical literature. To bridge this gap, we propose KEDD, a unified, end-to-end, and multimodal deep learning framework that optimally incorporates both structured and unstructured knowledge for vast AI drug discovery tasks. The framework first extracts underlying characteristics from heterogeneous inputs, and then applies multimodal fusion for accurate prediction. To mitigate the problem of missing modalities, we leverage multi-head sparse attention and a modality masking mechanism to extract relevant information robustly. Benefiting from integrated knowledge, our framework achieves a deeper understanding of molecule entities, brings significant improvements over state-of-the-art methods on a wide range of tasks and benchmarks, and reveals its promising potential in assisting real-world drug discovery., Comment: 10 pages, 6 figures

The research fields of parametric face model and 3D face reconstruction have been extensively studied. However, a critical question remains unanswered: how to tailor the face model for specific reconstruction settings. We argue that reconstruction with multi-view uncalibrated images demands a new model with stronger capacity. Our study shifts attention from data-dependent 3D Morphable Models (3DMM) to an understudied human-designed skinning model. We propose Adaptive Skinning Model (ASM), which redefines the skinning model with more compact and fully tunable parameters. With extensive experiments, we demonstrate that ASM achieves significantly improved capacity than 3DMM, with the additional advantage of model size and easy implementation for new topology. We achieve state-of-the-art performance with ASM for multi-view reconstruction on the Florence MICC Coop benchmark. Our quantitative analysis demonstrates the importance of a high-capacity model for fully exploiting abundant information from multi-view input in reconstruction. Furthermore, our model with physical-semantic parameters can be directly utilized for real-world applications, such as in-game avatar creation. As a result, our work opens up new research direction for parametric face model and facilitates future research on multi-view reconstruction., Comment: 18 pages

Although large language models (LLMs) have shown exceptional performance in various natural language processing tasks, they are prone to hallucinations. State-of-the-art chatbots, such as the new Bing, attempt to mitigate this issue by gathering information directly from the internet to ground their answers. In this setting, the capacity to distinguish trustworthy sources is critical for providing appropriate accuracy contexts to users. Here we assess whether ChatGPT, a prominent LLM, can evaluate the credibility of news outlets. With appropriate instructions, ChatGPT can provide ratings for a diverse set of news outlets, including those in non-English languages and satirical sources, along with contextual explanations. Our results show that these ratings correlate with those from human experts (Spearmam's $\rho=0.54, p<0.001$). These findings suggest that LLMs could be an affordable reference for credibility ratings in fact-checking applications. Future LLMs should enhance their alignment with human expert judgments of source credibility to improve information accuracy., Comment: 10 pages, 3 figures

Research on social bots aims at advancing knowledge and providing solutions to one of the most debated forms of online manipulation. Yet, social bot research is plagued by widespread biases, hyped results, and misconceptions that set the stage for ambiguities, unrealistic expectations, and seemingly irreconcilable findings. Overcoming such issues is instrumental towards ensuring reliable solutions and reaffirming the validity of the scientific method. In this contribution, we review some recent results in social bots research, highlighting and revising factual errors as well as methodological and conceptual biases. More importantly, we demystify common misconceptions, addressing fundamental points on how social bots research is discussed. Our analysis surfaces the need to discuss research about online disinformation and manipulation in a rigorous, unbiased, and responsible way. This article bolsters such effort by identifying and refuting common fallacious arguments used by both proponents and opponents of social bots research, as well as providing directions toward sound methodologies for future research in the field.

To investigate whether disk-mediated accretion is the primary mechanism in high-mass star formation, we have established a survey of a large sample of massive dense cores within a giant molecular cloud. We used high angular resolution ($\sim 1.8''$) observations with SMA to study the dust emission and molecular line emission of about 50 massive dense cores in Cygnus-X. At a typical distance of 1.4 kpc for Cygnus-X, these massive dense cores are resolved into $\sim 2000$ au condensations. We combined the CO outflow emission and gas kinematics traced by several high-density tracers to search for disk candidates. We extracted hundreds of dust condensations from the SMA 1.3 mm dust continuum emission. The CO data show bipolar or unipolar outflow signatures toward 49 dust condensations. Among them, only 27 sources are detected in dense gas tracers, which reveals the gas kinematics, and nine sources show evidence of rotating envelopes, suggesting the existence of embedded accretion disks. The position-velocity diagrams along the velocity gradient of all rotating condensations suggest that four condensations are possible to host Keplerian-like disks. A detailed investigation of the 27 sources detected in dense gas tracers suggests that the nine disk candidates are at earlier evolutionary stages compared to the remaining 18 sources. Non-detection of rotating disks in our sample may be due to several factors, including an unknown inclination angle of the rotation axis and an early evolutionary stage of the central source, and the latter could be important, considering that young and powerful outflows could confuse the observational evidence for rotation. The detection rate of disk candidates in our sample is 1/3, which confirms that disk accretion is a viable mechanism for high-mass star formation, although it may not be the only one., Comment: 19 pages, 12 figures, accepted for publication in A&A

We identify 182 flares on 158 stars within 100 pc of the Sun in both the near-ultraviolet (NUV: 1750-2750 \r{A}) and far-ultraviolet (FUV: 1350-1750 \r{A}) using high-cadence light curves from the Galaxy Evolution Explorer (GALEX). Ultraviolet (UV) emission from stellar flares plays a crucial role in determining the habitability of exoplanetary systems. However, whether such UV emission promotes or threatens such life depends strongly on the energetics of these flares. Most studies assessing the effect of flares on planetary habitability assume a 9000 K blackbody spectral energy distribution that produces more NUV flux than FUV flux ($R \equiv F_{\rm FUV} / F_{\rm NUV} \approx \frac{1}{6}$). Instead, we observe the opposite with the excess FUV reaching $R \approx \frac{1}{2} - 2$, roughly $3-12$ times the expectation of a 9000 K blackbody. The ratio of FUV to NUV time-integrated flare energies is 3.0 times higher on average than would be predicted by a constant 9000 K blackbody during the flare. Finally, we find that the FUV/NUV ratio at peak tentatively correlates ($\sim 2 \sigma$ significance) both with total UV flare energy and with the G - RP color of the host star. On average, we observe higher FUV/NUV ratios at peak in $E_{\text{UV}}>10^{32}$ erg flares and in flares on fully convective stars., Comment: Submitted to MNRAS, comments welcome

Deuterated molecules are valuable probes for investigating the evolution and the kinematics in the earliest stages of star formation. In this study, we conduct a comprehensive investigation by performing a single point survey of 101 starless clump candidates, and carrying out on-the-fly (OTF) observations of 11 selected sources, focusing on deuterated molecular lines using the IRAM 30-m telescope. In the single-point observation, we make 46 detections for DCO$^{+}$ J=1$-$0, 12 for DCN J=1$-$0, 51 for DNC J=1$-$0, 7 for N$_{2}$D$^{+}$ J=1$-$0, 20 for DCO$^{+}$ J=2$-$1, and 10 for DCN J=2$-$1. The starless clump candidates (SCCs) with deuterated molecule detections exhibit lower median kinetic temperatures and narrower H$_{2}$CO (1$_{(0,1)}$$-$0$_{(0,0)}$) median full width at half maximum (FWHM) compared to those without such detections, while simultaneously displaying similar median values of 1.1mm intensity, mass, and distance. Furthermore, our OTF observations reveal that deuterated molecules predominantly have peaks near the 1.1mm continuum peaks, with the DCO$^{+}$ J=1$-$0 emission demonstrating higher intensity in the deuterated peak region compared to the DCN and DNC J=1$-$0 emissions. Additionally, the majority of emissions from deuterated molecules and $^{13}$C$-$isotopologues exhibit peak positions close to those of the 1.1mm continuum peaks. By analyzing the 20"$\times$20" regions with strongest deuterated emissions in the OTF observations, we estimated deuterated abundances of 0.004$-$0.045, 0.011$-$0.040, and 0.004$-$0.038 for D$_{\rm frac}$(HCN), D$_{\rm frac}$(HCO$^{+}$), and D$_{\rm frac}$(HNC), respectively. The differential detection of deuterated molecular lines in our OTF observations could be attributed to variations in critical densities and formation pathways., Comment: 75 pages, 56 figures, 8 tables; accepted for publication in ApJS

The SiO emissions are usually used to trace high-velocity outflow shocks in star-forming regions. However, several studies have found low-velocity and widespread SiO emissions not associated with outflows in molecular clouds. We aim to detect and characterize the SiO emissions in massive dense cores (MDCs), and explore the properties of the central sources of SiO emission. We present high-angular-resolution ($\sim$1.5$^{\prime\prime}$) observations of the SiO (5$-$4) line made with the Submillimeter Array towards a sample of 48 MDCs in the Cygnus-X star-forming complex. We studied the SiO emission structures, including their morphologies, kinematics, and energetics, and investigated their relationship with the evolution of the central sources. The SiO (5$-$4) emission is detected in 16 out of 48 MDCs. We identify 14 bipolar and 18 unipolar SiO (5$-$4) outflows associated with 29 dust condensations. Most outflows (24 out of 32) are associated with excess Spitzer 4.5 $\mu$m emissions. We also find diffuse low-velocity ($\Delta{v}$ $\le$ 1.2 km s$^{-1}$) SiO (5$-$4) emission closely surrounding the dust condensations in two MDCs, and suggest that it may originate from decelerated outflow shocks or large-scale shocks from global cloud collapse. We find that the SMA SiO (5$-$4) emission in MDCs is mostly associated with outflows. Probably due to the relatively high excitation of SiO (5$-$4) compared to SiO (2$-$1) and due to the spatial filtering effect, we do not detect large-scale low-velocity SiO (5$-$4) emission, but detect more compact low-velocity emission in close proximity to the dust condensations. We group the sources into different evolutionary stages based on the infrared emission, radio continuum emission, and gas temperature properties of the outflow central sources, and find that the 24 $\mu$m luminosity tends to increase with evolution., Comment: 47 pages, 24 figures,4 tables, accepted for publication in A&A

To understand the relation between NH$_2$D and its physical environment, we mapped ortho-NH$_2$D $1_{11}^s-1_{01}^a$ at 85.9 GHz toward 24 Galactic late-stage massive star-forming regions with Institut de Radioastronomie Millim$ \'e$trique (IRAM) 30-m telescope. Ortho-NH$_2$D $1_{11}^s-1_{01}^a$ was detected in 18 of 24 sources. Comparing with the distribution of H$^{13}$CN 1-0 as a dense gas tracer and radio recombination line H42$\alpha$, ortho-NH$_2$D $1_{11}^s-1_{01}^a$ present complex and diverse spatial distribution in these targets. 11 of the 18 targets, present a different distribution between ortho-NH$_2$D $1_{11}^s-1_{01}^a$ and H$^{13}$CN 1-0, while no significant difference between these two lines can be found in the other 7 sources, mainly due to limited spatial resolution and sensitivity. Moreover, with H42$\alpha$ tracing massive young stellar objects, ortho-NH$_2$D $1_{11}^s-1_{01}^a$ seems to show a relatively weak emission near the massive young stellar objects., Comment: 30 pages, 20 figures, 4 tables. Accepted to MNRAS

M-dwarf flares observed by the \textit{Transiting Exoplanet Survey Satellite} (\textit{TESS}) sometimes exhibit a "peak-bump" light-curve morphology, characterized by a secondary, gradual peak well after the main, impulsive peak. A similar "late phase" is frequently detected in solar flares observed in the extreme-ultraviolet from longer hot coronal loops distinct from the impulsive flare structures. White-light emission has also been observed in off-limb solar flare loops. Here, we perform a suite of one-dimensional hydrodynamic loop simulations for M-dwarf flares inspired by these solar examples. Our results suggest that coronal plasma condensation following impulsive flare heating can yield high electron number density in the loop, allowing it to contribute significantly to the optical light curves via free-bound and free-free emission mechanisms. Our simulation results qualitatively agree with \textit{TESS} observations: the longer evolutionary time scale of coronal loops produces a distinct, secondary emission peak; its intensity increases with the injected flare energy. We argue that coronal plasma condensation is a possible mechanism for the \textit{TESS} late-phase flares., Comment: 31 pages, 13 figures, accepted for publication in ApJ

This paper provides a comprehensive review of advanced radio frequency (RF) filter technologies available in miniature chip or integrated circuit (IC) form for wireless communication applications. The RF filter technologies were organized according to the timeline of their introduction, in conjunction with each generation of wireless (cellular) communication standards (1G to 5G). This approach enabled a clear explanation of the corresponding invention history, working principles, typical applications and future development trends. The article covered commercially successful acoustic filter technologies, including the widely used surface acoustic wave (SAW) and bulk acoustic wave (BAW) filters, as well as electromagnetic filter technologies based on low-temperature co-fired ceramic (LTCC) and integrated passive device (IPD). Additionally, emerging filter technologies such as IHP-SAW, suspended thin-film lithium niobate (LiNbO3 or LN) resonant devices and hybrid were also discussed. In order to achieve higher performance, smaller form factor and lower cost for the wireless communication industry, it is believed that fundamental breakthroughs in materials and fabrication techniques are necessary for the future development of RF filters.

While social media are a key source of data for computational social science, their ease of manipulation by malicious actors threatens the integrity of online information exchanges and their analysis. In this Chapter, we focus on malicious social bots, a prominent vehicle for such manipulation. We start by discussing recent studies about the presence and actions of social bots in various online discussions to show their real-world implications and the need for detection methods. Then we discuss the challenges of bot detection methods and use Botometer, a publicly available bot detection tool, as a case study to describe recent developments in this area. We close with a practical guide on how to handle social bots in social media research., Comment: This is a draft of the chapter. The final version will be available in the Handbook of Computational Social Science edited by Taha Yasseri, forthcoming 2024, Edward Elgar Publishing Ltd. The material cannot be used for any other purpose without further permission of the publisher and is for private use only

This report introduces a new family of multimodal models, Gemini, that exhibit remarkable capabilities across image, audio, video, and text understanding. The Gemini family consists of Ultra, Pro, and Nano sizes, suitable for applications ranging from complex reasoning tasks to on-device memory-constrained use-cases. Evaluation on a broad range of benchmarks shows that our most-capable Gemini Ultra model advances the state of the art in 30 of 32 of these benchmarks - notably being the first model to achieve human-expert performance on the well-studied exam benchmark MMLU, and improving the state of the art in every one of the 20 multimodal benchmarks we examined. We believe that the new capabilities of the Gemini family in cross-modal reasoning and language understanding will enable a wide variety of use cases. We discuss our approach toward post-training and deploying Gemini models responsibly to users through services including Gemini, Gemini Advanced, Google AI Studio, and Cloud Vertex AI.

Trilevel learning, also called trilevel optimization (TLO), has been recognized as a powerful modelling tool for hierarchical decision process and widely applied in many machine learning applications, such as robust neural architecture search, hyperparameter optimization, and domain adaptation. Tackling TLO problems has presented a great challenge due to their nested decision-making structure. In addition, existing works on TLO face the following key challenges: 1) they all focus on the non-distributed setting, which may lead to privacy breach; 2) they do not offer any non-asymptotic convergence analysis which characterizes how fast an algorithm converges. To address the aforementioned challenges, this paper proposes an asynchronous federated trilevel optimization method to solve TLO problems. The proposed method utilizes $\mu$-cuts to construct a hyper-polyhedral approximation for the TLO problem and solve it in an asynchronous manner. We demonstrate that the proposed $\mu$-cuts are applicable to not only convex functions but also a wide range of non-convex functions that meet the $\mu$-weakly convex assumption. Furthermore, we theoretically analyze the non-asymptotic convergence rate for the proposed method by showing its iteration complexity to obtain $\epsilon$-stationary point is upper bounded by $\mathcal{O}(\frac{1}{\epsilon^2})$. Extensive experiments on real-world datasets have been conducted to elucidate the superiority of the proposed method, e.g., it has a faster convergence rate with a maximum acceleration of approximately 80$\%$., Comment: Accepted at AAAI 2024

While researchers often study message features like moral content in text, such as party manifestos and social media, their quantification remains a challenge. Conventional human coding struggles with scalability and intercoder reliability. While dictionary-based methods are cost-effective and computationally efficient, they often lack contextual sensitivity and are limited by the vocabularies developed for the original applications. In this paper, we present an approach to construct vec-tionary measurement tools that boost validated dictionaries with word embeddings through nonlinear optimization. By harnessing semantic relationships encoded by embeddings, vec-tionaries improve the measurement of message features from text, especially those in short format, by expanding the applicability of original vocabularies to other contexts. Importantly, a vec-tionary can produce additional metrics to capture the valence and ambivalence of a message feature beyond its strength in texts. Using moral content in tweets as a case study, we illustrate the steps to construct the moral foundations vec-tionary, showcasing its ability to process texts missed by conventional dictionaries and word embedding methods and to produce measurements better aligned with crowdsourced human assessments. Furthermore, additional metrics from the vec-tionary unveiled unique insights that facilitated predicting outcomes such as message retransmission.

Uplift modeling refers to the set of machine learning techniques that a manager may use to estimate customer uplift, that is, the net effect of an action on some customer outcome. By identifying the subset of customers for whom a treatment will have the greatest effect, uplift models assist decision-makers in optimizing resource allocations and maximizing overall returns. Accurately estimating customer uplift poses practical challenges, as it requires assessing the difference between two mutually exclusive outcomes for each individual. In this paper, we propose two innovative adaptations of the well-established Gradient Boosting Decision Trees (GBDT) algorithm, which learn the causal effect in a sequential way and overcome the counter-factual nature. Both approaches innovate existing techniques in terms of ensemble learning method and learning objectives, respectively. Experiments on large-scale datasets demonstrate the usefulness of the proposed methods, which often yielding remarkable improvements over base models. To facilitate the application, we develop the UTBoost, an end-to-end tree boosting system specifically designed for uplift modeling. The package is open source and has been optimized for training speed to meet the needs of real industrial applications., Comment: 11 pages, 3 figures

In field environments, numerous robots necessitate manual intervention for restoration of functionality post a turnover, resulting in diminished operational efficiency. This study presents an innovative design solution for a reversible omnidirectional mobile robot denoted as CuRobot, featuring a cube structure, thereby facilitating uninterrupted omnidirectional movement even in the event of flipping. The incorporation of eight conical wheels at the cube vertices ensures consistent omnidirectional motion no matter which face of the cube contacts the ground. Additionally, a kinematic model is formulated for CuRobot, accompanied by the development of a trajectory tracking controller utilizing model predictive control. Through simulation experiments, the correlation between trajectory tracking accuracy and the robot's motion direction is examined. Furthermore, the robot's proficiency in omnidirectional mobility and sustained movement post-flipping is substantiated via both simulation and prototype experiments. This design reduces the inefficiencies associated with manual intervention, thereby increasing the operational robustness of robots in field environments.

Using reinforcement learning with human feedback (RLHF) has shown significant promise in fine-tuning diffusion models. Previous methods start by training a reward model that aligns with human preferences, then leverage RL techniques to fine-tune the underlying models. However, crafting an efficient reward model demands extensive datasets, optimal architecture, and manual hyperparameter tuning, making the process both time and cost-intensive. The direct preference optimization (DPO) method, effective in fine-tuning large language models, eliminates the necessity for a reward model. However, the extensive GPU memory requirement of the diffusion model's denoising process hinders the direct application of the DPO method. To address this issue, we introduce the Direct Preference for Denoising Diffusion Policy Optimization (D3PO) method to directly fine-tune diffusion models. The theoretical analysis demonstrates that although D3PO omits training a reward model, it effectively functions as the optimal reward model trained using human feedback data to guide the learning process. This approach requires no training of a reward model, proving to be more direct, cost-effective, and minimizing computational overhead. In experiments, our method uses the relative scale of objectives as a proxy for human preference, delivering comparable results to methods using ground-truth rewards. Moreover, D3PO demonstrates the ability to reduce image distortion rates and generate safer images, overcoming challenges lacking robust reward models. Our code is publicly available at https://github.com/yk7333/D3PO., Comment: CVPR 2024 accepted; huggingface daily paper

Crocodiles, known as one of the oldest and most resilient species on Earth, have demonstrated remarkable locomotor abilities both on land and in water, evolving over millennia to adapt to diverse environments. In this paper, we draw inspiration from crocodiles and introduce a highly biomimetic crocodile robot equipped with multiple degrees of freedom and articulated trunk joints. This design is based on a comprehensive analysis of the structural and motion characteristics observed in real crocodiles. The bionic crocodile robot has the problem of limb-torso incoordination during movement, in order to solve this problem, we apply the D-H method for both forward and inverse kinematics analysis of the robot's legs and spine. Through a series of simulation experiments, we investigate the robot's stability of motion, fault tolerance, and adaptability to the environment in two motor pattern: with and without the involvement of the spine and tail in its movements. Experiment results demonstrate that the bionic crocodile robot exhibits superior motion performance when the spine and tail cooperate with the extremities. This research not only showcases the potential of biomimicry in robotics but also underscores the significance of understanding how nature's designs can inform and enhance our technological innovations.

Stable imaging in adverse environments (e.g., total darkness) makes thermal infrared (TIR) cameras a prevalent option for night scene perception. However, the low contrast and lack of chromaticity of TIR images are detrimental to human interpretation and subsequent deployment of RGB-based vision algorithms. Therefore, it makes sense to colorize the nighttime TIR images by translating them into the corresponding daytime color images (NTIR2DC). Despite the impressive progress made in the NTIR2DC task, how to improve the translation performance of small object classes is under-explored. To address this problem, we propose a generative adversarial network incorporating feedback-based object appearance learning (FoalGAN). Specifically, an occlusion-aware mixup module and corresponding appearance consistency loss are proposed to reduce the context dependence of object translation. As a representative example of small objects in nighttime street scenes, we illustrate how to enhance the realism of traffic light by designing a traffic light appearance loss. To further improve the appearance learning of small objects, we devise a dual feedback learning strategy to selectively adjust the learning frequency of different samples. In addition, we provide pixel-level annotation for a subset of the Brno dataset, which can facilitate the research of NTIR image understanding under multiple weather conditions. Extensive experiments illustrate that the proposed FoalGAN is not only effective for appearance learning of small objects, but also outperforms other image translation methods in terms of semantic preservation and edge consistency for the NTIR2DC task., Comment: 14 pages, 14 figures. arXiv admin note: text overlap with arXiv:2208.02960

The use of machine learning (ML) models to assess and score textual data has become increasingly pervasive in an array of contexts including natural language processing, information retrieval, search and recommendation, and credibility assessment of online content. A significant disruption at the intersection of ML and text are text-generating large-language models such as generative pre-trained transformers (GPTs). We empirically assess the differences in how ML-based scoring models trained on human content assess the quality of content generated by humans versus GPTs. To do so, we propose an analysis framework that encompasses essay scoring ML-models, human and ML-generated essays, and a statistical model that parsimoniously considers the impact of type of respondent, prompt genre, and the ML model used for assessment model. A rich testbed is utilized that encompasses 18,460 human-generated and GPT-based essays. Results of our benchmark analysis reveal that transformer pretrained language models (PLMs) more accurately score human essay quality as compared to CNN/RNN and feature-based ML methods. Interestingly, we find that the transformer PLMs tend to score GPT-generated text 10-15\% higher on average, relative to human-authored documents. Conversely, traditional deep learning and feature-based ML models score human text considerably higher. Further analysis reveals that although the transformer PLMs are exclusively fine-tuned on human text, they more prominently attend to certain tokens appearing only in GPT-generated text, possibly due to familiarity/overlap in pre-training. Our framework and results have implications for text classification settings where automated scoring of text is likely to be disrupted by generative AI., Comment: Data available at: https://github.com/nd-hal/automated-ML-scoring-versus-generation

Fact checking can be an effective strategy against misinformation, but its implementation at scale is impeded by the overwhelming volume of information online. Recent artificial intelligence (AI) language models have shown impressive ability in fact-checking tasks, but how humans interact with fact-checking information provided by these models is unclear. Here, we investigate the impact of fact-checking information generated by a popular large language model (LLM) on belief in, and sharing intent of, political news in a preregistered randomized control experiment. Although the LLM performs reasonably well in debunking false headlines, we find that it does not significantly affect participants' ability to discern headline accuracy or share accurate news. Subsequent analysis reveals that the AI fact-checker is harmful in specific cases: it decreases beliefs in true headlines that it mislabels as false and increases beliefs in false headlines that it is unsure about. On the positive side, the AI fact-checking information increases sharing intents for correctly labeled true headlines. When participants are given the option to view LLM fact checks and choose to do so, they are significantly more likely to share both true and false news but only more likely to believe false news. Our findings highlight an important source of potential harm stemming from AI applications and underscore the critical need for policies to prevent or mitigate such unintended consequences.

Foundation models (FMs) have exhibited remarkable performance across a wide range of downstream tasks in many domains. Nevertheless, general-purpose FMs often face challenges when confronted with domain-specific problems, due to their limited access to the proprietary training data in a particular domain. In biomedicine, there are various biological modalities, such as molecules, proteins, and cells, which are encoded by the language of life and exhibit significant modality gaps with human natural language. In this paper, we introduce BioMedGPT, an open multimodal generative pre-trained transformer (GPT) for biomedicine, to bridge the gap between the language of life and human natural language. BioMedGPT allows users to easily ``communicate'' with diverse biological modalities through free text, which is the first of its kind. BioMedGPT aligns different biological modalities with natural language via a large generative language model, namely, BioMedGPT-LM. We publish BioMedGPT-10B, which unifies the feature spaces of molecules, proteins, and natural language via encoding and alignment. Through fine-tuning, BioMedGPT-10B outperforms or is on par with human and significantly larger general-purpose foundation models on the biomedical QA task. It also demonstrates promising performance in the molecule QA and protein QA tasks, which could greatly accelerate the discovery of new drugs and therapeutic targets. In addition, BioMedGPT-LM-7B is the first large generative language model based on Llama2 in the biomedical domain, therefore is commercial friendly. Both BioMedGPT-10B and BioMedGPT-LM-7B are open-sourced to the research community. In addition, we publish the datasets that are meticulously curated for the alignment of multi-modalities, i.e., PubChemQA and UniProtQA. All the models, codes, and datasets are available at \url{https://github.com/PharMolix/OpenBioMed}., Comment: 12 pages, 4 figures

Traditional Graph Neural Network (GNN), as a graph representation learning method, is constrained by label information. However, Graph Contrastive Learning (GCL) methods, which tackle the label problem effectively, mainly focus on the feature information of the global graph or small subgraph structure (e.g., the first-order neighborhood). In the paper, we propose a Local Structure-aware Graph Contrastive representation Learning method (LS-GCL) to model the structural information of nodes from multiple views. Specifically, we construct the semantic subgraphs that are not limited to the first-order neighbors. For the local view, the semantic subgraph of each target node is input into a shared GNN encoder to obtain the target node embeddings at the subgraph-level. Then, we use a pooling function to generate the subgraph-level graph embeddings. For the global view, considering the original graph preserves indispensable semantic information of nodes, we leverage the shared GNN encoder to learn the target node embeddings at the global graph-level. The proposed LS-GCL model is optimized to maximize the common information among similar instances at three various perspectives through a multi-level contrastive loss function. Experimental results on five datasets illustrate that our method outperforms state-of-the-art graph representation learning approaches for both node classification and link prediction tasks.

Large language models (LLMs) exhibit impressive capabilities in generating realistic text across diverse subjects. Concerns have been raised that they could be utilized to produce fake content with a deceptive intention, although evidence thus far remains anecdotal. This paper presents a case study about a Twitter botnet that appears to employ ChatGPT to generate human-like content. Through heuristics, we identify 1,140 accounts and validate them via manual annotation. These accounts form a dense cluster of fake personas that exhibit similar behaviors, including posting machine-generated content and stolen images, and engage with each other through replies and retweets. ChatGPT-generated content promotes suspicious websites and spreads harmful comments. While the accounts in the AI botnet can be detected through their coordination patterns, current state-of-the-art LLM content classifiers fail to discriminate between them and human accounts in the wild. These findings highlight the threats posed by AI-enabled social bots.

Distributionally Robust Optimization (DRO), which aims to find an optimal decision that minimizes the worst case cost over the ambiguity set of probability distribution, has been widely applied in diverse applications, e.g., network behavior analysis, risk management, etc. However, existing DRO techniques face three key challenges: 1) how to deal with the asynchronous updating in a distributed environment; 2) how to leverage the prior distribution effectively; 3) how to properly adjust the degree of robustness according to different scenarios. To this end, we propose an asynchronous distributed algorithm, named Asynchronous Single-looP alternatIve gRadient projEction (ASPIRE) algorithm with the itErative Active SEt method (EASE) to tackle the federated distributionally robust optimization (FDRO) problem. Furthermore, a new uncertainty set, i.e., constrained D-norm uncertainty set, is developed to effectively leverage the prior distribution and flexibly control the degree of robustness. Finally, our theoretical analysis elucidates that the proposed algorithm is guaranteed to converge and the iteration complexity is also analyzed. Extensive empirical studies on real-world datasets demonstrate that the proposed method can not only achieve fast convergence, and remain robust against data heterogeneity as well as malicious attacks, but also tradeoff robustness with performance., Comment: arXiv admin note: substantial text overlap with arXiv:2210.07588

Molecular knowledge resides within three different modalities of information sources: molecular structures, biomedical documents, and knowledge bases. Effective incorporation of molecular knowledge from these modalities holds paramount significance in facilitating biomedical research. However, existing multimodal molecular foundation models exhibit limitations in capturing intricate connections between molecular structures and texts, and more importantly, none of them attempt to leverage a wealth of molecular expertise derived from knowledge graphs. In this study, we introduce MolFM, a multimodal molecular foundation model designed to facilitate joint representation learning from molecular structures, biomedical texts, and knowledge graphs. We propose cross-modal attention between atoms of molecular structures, neighbors of molecule entities and semantically related texts to facilitate cross-modal comprehension. We provide theoretical analysis that our cross-modal pre-training captures local and global molecular knowledge by minimizing the distance in the feature space between different modalities of the same molecule, as well as molecules sharing similar structures or functions. MolFM achieves state-of-the-art performance on various downstream tasks. On cross-modal retrieval, MolFM outperforms existing models with 12.13% and 5.04% absolute gains under the zero-shot and fine-tuning settings, respectively. Furthermore, qualitative analysis showcases MolFM's implicit ability to provide grounding from molecular substructures and knowledge graphs. Code and models are available on https://github.com/BioFM/OpenBioMed., Comment: 31 pages, 15 figures, and 15 tables