Search

The need for improved diagnostic methods in ophthalmology is acute, especially in the less developed regions with limited access to specialists and advanced equipment. Therefore, we introduce VisionUnite, a novel vision-language foundation model for ophthalmology enhanced with clinical knowledge. VisionUnite has been pretrained on an extensive dataset comprising 1.24 million image-text pairs, and further refined using our proposed MMFundus dataset, which includes 296,379 high-quality fundus image-text pairs and 889,137 simulated doctor-patient dialogue instances. Our experiments indicate that VisionUnite outperforms existing generative foundation models such as GPT-4V and Gemini Pro. It also demonstrates diagnostic capabilities comparable to junior ophthalmologists. VisionUnite performs well in various clinical scenarios including open-ended multi-disease diagnosis, clinical explanation, and patient interaction, making it a highly versatile tool for initial ophthalmic disease screening. VisionUnite can also serve as an educational aid for junior ophthalmologists, accelerating their acquisition of knowledge regarding both common and rare ophthalmic conditions. VisionUnite represents a significant advancement in ophthalmology, with broad implications for diagnostics, medical education, and understanding of disease mechanisms.

Noise suppression in seismic data processing is a crucial research focus for enhancing subsequent imaging and reservoir prediction. Deep learning has shown promise in computer vision and holds significant potential for seismic data processing. However, supervised learning, which relies on clean labels to train network prediction models, faces challenges due to the unavailability of clean labels for seismic exploration data. In contrast, self-supervised learning substitutes traditional supervised learning with surrogate tasks by different auxiliary means, exploiting internal input data information. Inspired by Self2Self with Dropout, this paper presents a self-supervised learning-based noise suppression method called Self-Supervised Deep Convolutional Networks (SSDCN), specifically designed for Common Reflection Point (CRP) gathers. We utilize pairs of Bernoulli-sampled instances of the input noisy image as surrogate tasks to leverage its inherent structure. Furthermore, SSDCN incorporates geological knowledge through the normal moveout correction technique, which capitalizes on the approximately horizontal behavior and strong self-similarity observed in useful signal events within CRP gathers. By exploiting the discrepancy in self-similarity between the useful signals and noise in CRP gathers, SSDCN effectively extracts self-similarity features during training iterations, prioritizing the extraction of useful signals to achieve noise suppression. Experimental results on synthetic and actual CRP gathers demonstrate that SSDCN achieves high-fidelity noise suppression.

Recently, transformer-based models have demonstrated remarkable performance on audio-visual segmentation (AVS) tasks. However, their expensive computational cost makes real-time inference impractical. By characterizing attention maps of the network, we identify two key obstacles in AVS models: 1) attention dissipation, corresponding to the over-concentrated attention weights by Softmax within restricted frames, and 2) inefficient, burdensome transformer decoder, caused by narrow focus patterns in early stages. In this paper, we introduce AVESFormer, the first real-time Audio-Visual Efficient Segmentation transformer that achieves fast, efficient and light-weight simultaneously. Our model leverages an efficient prompt query generator to correct the behaviour of cross-attention. Additionally, we propose ELF decoder to bring greater efficiency by facilitating convolutions suitable for local features to reduce computational burdens. Extensive experiments demonstrate that our AVESFormer significantly enhances model performance, achieving 79.9% on S4, 57.9% on MS3 and 31.2% on AVSS, outperforming previous state-of-the-art and achieving an excellent trade-off between performance and speed. Code can be found at https://github.com/MarkXCloud/AVESFormer.git.

Fine-grained sentiment analysis involves extracting and organizing sentiment elements from textual data. However, existing approaches often overlook issues of category semantic inclusion and overlap, as well as inherent structural patterns within the target sequence. This study introduces a generative sentiment analysis model. To address the challenges related to category semantic inclusion and overlap, a latent category distribution variable is introduced. By reconstructing the input of a variational autoencoder, the model learns the intensity of the relationship between categories and text, thereby improving sequence generation. Additionally, a trie data structure and constrained decoding strategy are utilized to exploit structural patterns, which in turn reduces the search space and regularizes the generation process. Experimental results on the Restaurant-ACOS and Laptop-ACOS datasets demonstrate a significant performance improvement compared to baseline models. Ablation experiments further confirm the effectiveness of latent category distribution and constrained decoding strategy.

Interrogation of biological pathways is an integral part of omics data analysis. Large language models (LLMs) enable the generation of custom pathways and gene sets tailored to specific scientific questions. These targeted sets are significantly smaller than traditional pathway enrichment analysis libraries, reducing multiple hypothesis testing and potentially enhancing statistical power. Lomics (Large Language Models for Omics Studies) v1.0 is a python-based bioinformatics toolkit that streamlines the generation of pathways and gene sets for transcriptomic analysis. It operates in three steps: 1) deriving relevant pathways based on the researcher's scientific question, 2) generating valid gene sets for each pathway, and 3) outputting the results as .GMX files. Lomics also provides explanations for pathway selections. Consistency and accuracy are ensured through iterative processes, JSON format validation, and HUGO Gene Nomenclature Committee (HGNC) gene symbol verification. Lomics serves as a foundation for integrating LLMs into omics research, potentially improving the specificity and efficiency of pathway analysis.

Structured Sentiment Analysis (SSA) was cast as a problem of bi-lexical dependency graph parsing by prior studies. Multiple formulations have been proposed to construct the graph, which share several intrinsic drawbacks: (1) The internal structures of spans are neglected, thus only the boundary tokens of spans are used for relation prediction and span recognition, thus hindering the model's expressiveness; (2) Long spans occupy a significant proportion in the SSA datasets, which further exacerbates the problem of internal structure neglect. In this paper, we treat the SSA task as a dependency parsing task on partially-observed dependency trees, regarding flat spans without determined tree annotations as latent subtrees to consider internal structures of spans. We propose a two-stage parsing method and leverage TreeCRFs with a novel constrained inside algorithm to model latent structures explicitly, which also takes advantages of joint scoring graph arcs and headed spans for global optimization and inference. Results of extensive experiments on five benchmark datasets reveal that our method performs significantly better than all previous bi-lexical methods, achieving new state-of-the-art.

Document-level event extraction aims to extract structured event information from unstructured text. However, a single document often contains limited event information and the roles of different event arguments may be biased due to the influence of the information source. This paper addresses the limitations of traditional document-level event extraction by proposing the task of cross-document event extraction (CDEE) to integrate event information from multiple documents and provide a comprehensive perspective on events. We construct a novel cross-document event extraction dataset, namely CLES, which contains 20,059 documents and 37,688 mention-level events, where over 70% of them are cross-document. To build a benchmark, we propose a CDEE pipeline that includes 5 steps, namely event extraction, coreference resolution, entity normalization, role normalization and entity-role resolution. Our CDEE pipeline achieves about 72% F1 in end-to-end cross-document event extraction, suggesting the challenge of this task. Our work builds a new line of information extraction research and will attract new research attention., Comment: ACL2024(Findings)

Existing cross-document event coreference resolution models, which either compute mention similarity directly or enhance mention representation by extracting event arguments (such as location, time, agent, and patient), lacking the ability to utilize document-level information. As a result, they struggle to capture long-distance dependencies. This shortcoming leads to their underwhelming performance in determining coreference for the events where their argument information relies on long-distance dependencies. In light of these limitations, we propose the construction of document-level Rhetorical Structure Theory (RST) trees and cross-document Lexical Chains to model the structural and semantic information of documents. Subsequently, cross-document heterogeneous graphs are constructed and GAT is utilized to learn the representations of events. Finally, a pair scorer calculates the similarity between each pair of events and co-referred events can be recognized using standard clustering algorithm. Additionally, as the existing cross-document event coreference datasets are limited to English, we have developed a large-scale Chinese cross-document event coreference dataset to fill this gap, which comprises 53,066 event mentions and 4,476 clusters. After applying our model on the English and Chinese datasets respectively, it outperforms all baselines by large margins.

Deep learning-based ISP algorithms have demonstrated significant potential in raw2rgb reconstruction. However, existing networks have not fully considered the specific characteristics of raw data, such as black level and CFA, which can negatively impact texture and color if mishandled. Moreover, uneven exposure in raw data is also not considered carefully, leading to adverse effects on contrast and brightness. In this paper, we introduce RMFA-Net to tackle these problems. We perform implicit black level correction to mitigate color shifts in dim scenes. To preserve high-frequency information and prevent misalignment, we propose a novel Three-Channel-Split mode. To address the issue of uneven exposure, we designed an explicit tone mapping module based on the Retinex theory. We train and evaluate our models using the dataset released by the Mobile AI 2022 Learned Smartphone ISP Challenge. It is demonstrated that RMFA-Net outperforms previous algorithms, achieving a PSNR score of over 25 dB, surpassing the state-of-the-art by +1 dB. Furthermore, we developed a lightweight version, RMFANet-tiny, for engineering deployment while still maintaining strong performance, surpassing the SOTA by +0.5 dB.

We demonstrate through high-fidelity particle-in-cell simulations a simple approach for efficiently generating 20+ GeV electron beams with the necessary charge, energy spread, and emittance for use as the injector for an electron arm of a future linear collider or a next generation XFEL. The self-focusing of an unmatched, relatively low quality, drive beam results in self-injection by elongating the wakefield excited in the nonlinear blowout regime. Over pump depletion distances, the drive beam dynamics and self-loading from the injected beam leads to extremely high quality and high energy output beams. For plasma densities of $10^{18} \ \text{cm}^{-3}$, PIC simulation results indicate that self-injected beams with $0.52 \ \text{nC}$ of charge can be accelerated to $\sim 20$ GeV energies with projected energy spreads, $\lesssim 1\%$ within the beam core, slice normalized emittances as low as $110 \ \text{nm}$, a peak normalized brightness $\gtrsim 10^{19} \ \text{A}/\text{m}^2/\text{rad}^2$, and energy transfer efficiencies $\gtrsim 54\%$.

In the field of information extraction (IE), tasks across a wide range of modalities and their combinations have been traditionally studied in isolation, leaving a gap in deeply recognizing and analyzing cross-modal information. To address this, this work for the first time introduces the concept of grounded Multimodal Universal Information Extraction (MUIE), providing a unified task framework to analyze any IE tasks over various modalities, along with their fine-grained groundings. To tackle MUIE, we tailor a multimodal large language model (MLLM), Reamo, capable of extracting and grounding information from all modalities, i.e., recognizing everything from all modalities at once. Reamo is updated via varied tuning strategies, equipping it with powerful capabilities for information recognition and fine-grained multimodal grounding. To address the absence of a suitable benchmark for grounded MUIE, we curate a high-quality, diverse, and challenging test set, which encompasses IE tasks across 9 common modality combinations with the corresponding multimodal groundings. The extensive comparison of Reamo with existing MLLMs integrated into pipeline approaches demonstrates its advantages across all evaluation dimensions, establishing a strong benchmark for the follow-up research. Our resources are publicly released at https://haofei.vip/MUIE.

We explore the bound neutrons decay into invisible particles (e.g., $n\rightarrow 3 \nu$ or $nn \rightarrow 2 \nu$) in the JUNO liquid scintillator detector. The invisible decay includes two decay modes: $ n \rightarrow { inv} $ and $ nn \rightarrow { inv} $. The invisible decays of $s$-shell neutrons in $^{12}{\rm C}$ will leave a highly excited residual nucleus. Subsequently, some de-excitation modes of the excited residual nuclei can produce a time- and space-correlated triple coincidence signal in the JUNO detector. Based on a full Monte Carlo simulation informed with the latest available data, we estimate all backgrounds, including inverse beta decay events of the reactor antineutrino $\bar{\nu}_e$, natural radioactivity, cosmogenic isotopes and neutral current interactions of atmospheric neutrinos. Pulse shape discrimination and multivariate analysis techniques are employed to further suppress backgrounds. With two years of exposure, JUNO is expected to give an order of magnitude improvement compared to the current best limits. After 10 years of data taking, the JUNO expected sensitivities at a 90% confidence level are $\tau/B( n \rightarrow { inv} ) > 5.0 \times 10^{31} \, {\rm yr}$ and $\tau/B( nn \rightarrow { inv} ) > 1.4 \times 10^{32} \, {\rm yr}$., Comment: 28 pages, 7 figures, 4 tables

The production of light nuclei in isobaric $^{96}_{44}$Ru + $^{96}_{44}$Ru and $^{96}_{40}$Zr + $^{96}_{40}$Zr collisions, ranging from $\sqrt{s_{NN}}$ = 7.7 to 200 GeV, are studied using the string melting version of A Multi Phase Transport (AMPT) model in combination with a coalescence approach to light nuclei production. From the calculated yields, transverse momentum $p_{T}$ spectra, and rapidity dependences of light nuclei $p$, $n$, $d$, $t$, ${}^{3}$He, we find that the Ru+Ru/Zr+Zr ratios for the yields of these particles exceed unity with the inclusion of a quadrupole deformation $\beta_{ 2 }$ and octupole deformation $\beta_{ 3 }$ as well as the neutron skins. We also find that heavier particles have a larger deviation from unity. Furthermore, we find that as the collision energy increases, the influence of isospin effects on the production of light nuclei in isobar collisions gradually decreases, while the influence of nuclear structure becomes more significant, particularly evident from the energy dependence of the deuteron ratio, which is unaffected by isospin effects., Comment: 9 pages, 5 figures

In the General Next-to-Minimal Supersymmetric Standard Model (GNMSSM), singlet particles may form a secluded sector of dark matter (DM), in which Singlino-like DM could achieve the observed relic abundance through various channels such as $\tilde{\chi}_1^0 \tilde{\chi}_1^0 \to h_s h_s, A_s A_s, h_s A_s$, where $h_s$ and $A_s$ represent singlet-dominated CP-even and CP-odd Higgs bosons. We provide analytical formulas for both the spin-independent and spin-dependent cross sections of Singlino DM scattering with nucleons, illustrating their dependence on the model's parameters in a clear manner. We also present analytic expressions for the annihilation cross sections of these three important channels. Based on these preparations, we conducted Bayesian analyses of the GNMSSM and concluded that the theory significantly favored Singlino-dominated DM over Bino-like DM across a much broader range of parameters. The combined results from our numerical analyses and the formulas distinctly highlight crucial aspects of DM physics within the GNMSSM., Comment: 44 pages, 8 tables, 6 figures

This study documents several correlations observed during the first run of the plasma wakefield acceleration experiment E300 conducted at FACET-II, using a single drive electron bunch. The established correlations include those between the measured maximum energy loss of the drive electron beam and the integrated betatron x-ray signal, the calculated total beam energy deposited in the plasma and the integrated x-ray signal, among three visible light emission measuring cameras, and between the visible plasma light and x-ray signal. The integrated x-ray signal correlates almost linearly with both the maximum energy loss of the drive beam and the energy deposited into the plasma, demonstrating its usability as a measure of energy transfer from the drive beam to the plasma. Visible plasma light is found to be a useful indicator of the presence of wake at three locations that overall are two meters apart. Despite the complex dynamics and vastly different timescales, the x-ray radiation from the drive bunch and visible light emission from the plasma may prove to be effective non-invasive diagnostics for monitoring the energy transfer from the beam to the plasma in future high-repetition-rate experiments., Comment: 20 pages, 6 figures

We interact with the world with our hands and see it through our own (egocentric) perspective. A holistic 3Dunderstanding of such interactions from egocentric views is important for tasks in robotics, AR/VR, action recognition and motion generation. Accurately reconstructing such interactions in 3D is challenging due to heavy occlusion, viewpoint bias, camera distortion, and motion blur from the head movement. To this end, we designed the HANDS23 challenge based on the AssemblyHands and ARCTIC datasets with carefully designed training and testing splits. Based on the results of the top submitted methods and more recent baselines on the leaderboards, we perform a thorough analysis on 3D hand(-object) reconstruction tasks. Our analysis demonstrates the effectiveness of addressing distortion specific to egocentric cameras, adopting high-capacity transformers to learn complex hand-object interactions, and fusing predictions from different views. Our study further reveals challenging scenarios intractable with state-of-the-art methods, such as fast hand motion, object reconstruction from narrow egocentric views, and close contact between two hands and objects. Our efforts will enrich the community's knowledge foundation and facilitate future hand studies on egocentric hand-object interactions., Comment: Accepted to ECCV 2024

Headline generation aims to summarize a long document with a short, catchy title that reflects the main idea. This requires accurately capturing the core document semantics, which is challenging due to the lengthy and background information-rich na ture of the texts. In this work, We propose using a unified semantic discourse structure (S3) to represent document semantics, achieved by combining document-level rhetorical structure theory (RST) trees with sentence-level abstract meaning representation (AMR) graphs to construct S3 graphs. The hierarchical composition of sentence, clause, and word intrinsically characterizes the semantic meaning of the overall document. We then develop a headline generation framework, in which the S3 graphs are encoded as contextual features. To consolidate the efficacy of S3 graphs, we further devise a hierarchical structure pruning mechanism to dynamically screen the redundant and nonessential nodes within the graph. Experimental results on two headline generation datasets demonstrate that our method outperforms existing state-of-art methods consistently. Our work can be instructive for a broad range of document modeling tasks, more than headline or summarization generation.

In federated learning, data heterogeneity significantly impacts performance. A typical solution involves segregating these parameters into shared and personalized components, a concept also relevant in multi-task learning. Addressing this, we propose "Loop Improvement" (LI), a novel method enhancing this separation and feature extraction without necessitating a central server or data interchange among participants. Our experiments reveal LI's superiority in several aspects: In personalized federated learning environments, LI consistently outperforms the advanced FedALA algorithm in accuracy across diverse scenarios. Additionally, LI's feature extractor closely matches the performance achieved when aggregating data from all clients. In global model contexts, employing LI with stacked personalized layers and an additional network also yields comparable results to combined client data scenarios. Furthermore, LI's adaptability extends to multi-task learning, streamlining the extraction of common features across tasks and obviating the need for simultaneous training. This approach not only enhances individual task performance but also achieves accuracy levels on par with classic multi-task learning methods where all tasks are trained simultaneously. LI integrates a loop topology with layer-wise and end-to-end training, compatible with various neural network models. This paper also delves into the theoretical underpinnings of LI's effectiveness, offering insights into its potential applications. The code is on https://github.com/axedge1983/LI, Comment: 11 pages, 11 figures

Very recently, the Belle~II Collaboration presented a measurement for the decays $B^+\to\bar{D}^{(*)0} K^+\bar{K}^0$ and $B^0\to D^{(*)-}K^+\bar{K}^0$, the bulk of observed $m(K^+ K_S^0)$ distributions showing low-mass structures in all four channels. In this work, we study the contributions of $\rho(770,1450)^+$, $a_2(1320)^+$ and $a_0(980,1450)^+$ resonances to these decay processes. The intermediate states $\rho(770,1450)^+$ are found to dominate the low-mass distribution of kaon pairs roughly contributing to half of the total branching fraction in each of the four decay channels. The contribution of the tensor $a_2(1320)^+$ meson is found to be negligible. Near the threshold of the kaon pair, the state $a_0(980)^+$ turns out to be much less important than expected, not being able to account for the enhancement of events in that energy region observed in the $B^+\to\bar{D}^{(*)0} K^+\bar{K}^0$ decays. Further studies both from the theoretical and experimental sides are needed to elucidate the role of the non-resonant contributions governing the formation of $K^+\bar{K}^0$ pairs near their threshold in these decay processes., Comment: 18 pages, 4 figures

Multimodal Named Entity Recognition (MNER) is a pivotal task designed to extract named entities from text with the support of pertinent images. Nonetheless, a notable paucity of data for Chinese MNER has considerably impeded the progress of this natural language processing task within the Chinese domain. Consequently, in this study, we compile a Chinese Multimodal NER dataset (CMNER) utilizing data sourced from Weibo, China's largest social media platform. Our dataset encompasses 5,000 Weibo posts paired with 18,326 corresponding images. The entities are classified into four distinct categories: person, location, organization, and miscellaneous. We perform baseline experiments on CMNER, and the outcomes underscore the effectiveness of incorporating images for NER. Furthermore, we conduct cross-lingual experiments on the publicly available English MNER dataset (Twitter2015), and the results substantiate our hypothesis that Chinese and English multimodal NER data can mutually enhance the performance of the NER model.

The dominant mechanism forming multiple stellar systems in the high-mass regime (M$_\ast \gtrsim $ 8 $M_{\odot}$) remained unknown because direct imaging of multiple protostellar systems at early phases of high-mass star formation is very challenging. High-mass stars are expected to form in clustered environments containing binaries and higher-order multiplicity systems. So far only a few high-mass protobinary systems, and no definitive higher-order multiples, have been detected. Here we report the discovery of one quintuple, one quadruple, one triple and four binary protostellar systems simultaneously forming in a single high-mass protocluster, G333.23--0.06, using Atacama Large Millimeter/submillimeter Array high-resolution observations. We present a new example of a group of gravitationally bound binary and higher-order multiples during their early formation phases in a protocluster. This provides the clearest direct measurement of the initial configuration of primordial high-order multiple systems, with implications for the in situ multiplicity and its origin. We find that the binary and higher-order multiple systems, and their parent cores, show no obvious sign of disk-like kinematic structure. We conclude that the observed fragmentation into binary and higher-order multiple systems can be explained by core fragmentation, indicating its crucial role in establishing the multiplicity during high-mass star cluster formation., Comment: Published in Nature Astronomy at https://www.nature.com/articles/s41550-023-02181-9. 27 pages, 12 figures, 1 table

It has been more than 30 years since the enigmatic 21 {\mu}m emission feature was first discovered in protoplanetary nebulae (PPNs). Although dozens of different dust carrier candidates have been proposed, there is as yet no widely accepted one. We present the results of molecular observations toward 21{\mu}m objects using the 10m Submillimeter Telescope of Arizona Radio Observatory at the 1.3 mm band and the 13.7 m telescope of Purple Mountain Observatory at the 3mm band, aiming to investigate whether the gas-phase environments of these unusual sources have some peculiarities compared to normal PPNs. We detect 31 emission lines belonging to seven different molecular species, most of which are the first detection in 21 {\mu}m PPNs. The observations provide clues on the identification of the 21 {\mu}m feature. We report a correlation study between the fractional abundance of gas-phase molecules and the strengths of the 21 {\mu}m emission. Our study shows that given the small sample size, the 21 {\mu}m feature has weak or no correlations with the gas-phase molecules. Future radio observations of high spatial and spectral resolution toward a large sample are desirable to elucidate the 21 {\mu}m emission phenomena., Comment: 40 pages, 10 figures, 11 tables, accepted for publication in The Astronomical Journal

With the proliferation of dialogic data across the Internet, the Dialogue Commonsense Multi-choice Question Answering (DC-MCQ) task has emerged as a response to the challenge of comprehending user queries and intentions. Although prevailing methodologies exhibit effectiveness in addressing single-choice questions, they encounter difficulties in handling multi-choice queries due to the heightened intricacy and informational density. In this paper, inspired by the human cognitive process of progressively excluding options, we propose a three-step Reverse Exclusion Graph-of-Thought (ReX-GoT) framework, including Option Exclusion, Error Analysis, and Combine Information. Specifically, our ReX-GoT mimics human reasoning by gradually excluding irrelevant options and learning the reasons for option errors to choose the optimal path of the GoT and ultimately infer the correct answer. By progressively integrating intricate clues, our method effectively reduces the difficulty of multi-choice reasoning and provides a novel solution for DC-MCQ. Extensive experiments on the CICERO and CICERO$_{v2}$ datasets validate the significant improvement of our approach on DC-MCQ task. On zero-shot setting, our model outperform the best baseline by 17.67% in terms of F1 score for the multi-choice task. Most strikingly, our GPT3.5-based ReX-GoT framework achieves a remarkable 39.44% increase in F1 score., Comment: This paper has been accepted by the 38th Annual AAAI Conference on Artificial Intelligence (AAAI'24, FEBRUARY 20-27, 2024, VANCOUVER, CANADA)