Search

Recent vision foundation models can extract universal representations and show impressive abilities in various tasks. However, their application on object detection is largely overlooked, especially without fine-tuning them. In this work, we show that frozen foundation models can be a versatile feature enhancer, even though they are not pre-trained for object detection. Specifically, we explore directly transferring the high-level image understanding of foundation models to detectors in the following two ways. First, the class token in foundation models provides an in-depth understanding of the complex scene, which facilitates decoding object queries in the detector's decoder by providing a compact context. Additionally, the patch tokens in foundation models can enrich the features in the detector's encoder by providing semantic details. Utilizing frozen foundation models as plug-and-play modules rather than the commonly used backbone can significantly enhance the detector's performance while preventing the problems caused by the architecture discrepancy between the detector's backbone and the foundation model. With such a novel paradigm, we boost the SOTA query-based detector DINO from 49.0% AP to 51.9% AP (+2.9% AP) and further to 53.8% AP (+4.8% AP) by integrating one or two foundation models respectively, on the COCO validation set after training for 12 epochs with R50 as the detector's backbone., Comment: Accepted to NeurIPS 2024

Social media is often the first place where communities discuss the latest societal trends. Prior works have utilized this platform to extract epidemic-related information (e.g. infections, preventive measures) to provide early warnings for epidemic prediction. However, these works only focused on English posts, while epidemics can occur anywhere in the world, and early discussions are often in the local, non-English languages. In this work, we introduce the first multilingual Event Extraction (EE) framework SPEED++ for extracting epidemic event information for a wide range of diseases and languages. To this end, we extend a previous epidemic ontology with 20 argument roles; and curate our multilingual EE dataset SPEED++ comprising 5.1K tweets in four languages for four diseases. Annotating data in every language is infeasible; thus we develop zero-shot cross-lingual cross-disease models (i.e., training only on English COVID data) utilizing multilingual pre-training and show their efficacy in extracting epidemic-related events for 65 diverse languages across different diseases. Experiments demonstrate that our framework can provide epidemic warnings for COVID-19 in its earliest stages in Dec 2019 (3 weeks before global discussions) from Chinese Weibo posts without any training in Chinese. Furthermore, we exploit our framework's argument extraction capabilities to aggregate community epidemic discussions like symptoms and cure measures, aiding misinformation detection and public attention monitoring. Overall, we lay a strong foundation for multilingual epidemic preparedness., Comment: Accepted at EMNLP 2024

In this work, a formalism is presented for the helicity amplitude analysis of the decays $\psi(2S) \to \gamma_1 \chi_{cJ},~ \chi_{cJ} \to \gamma_2 V (V=\rho^0,~\phi,~\omega)$~(the subscript 1,2 is used to distinguish the two radiative photons), and the polarization expressions of the P-wave charmonia $\chi_{cJ}$ and the vector mesons $\rho^0, \phi, \omega$ for experimental measurements at Electron-Positron Collider. In addition, we derive the formulae of the angular distributions of the $\chi_{c1,2} \to \gamma V$ to extract the degree of transverse polarization $P_T$ of $e^+ e^-$ pairs with symmetric beam energy as well as the ratios of two helicity amplitudes $x$ (in $\chi_{c1}$ decays) and $x,~y$ (in $\chi_{c2}$ decays) representing the relative magnitudes of transverse to longitudinal polarization amplitude, and validate it by performing the Monte Carlo simulation. Finally, the statistical sensitivity of $P_T$, $x$ and $y$ are estimated based on the large $\psi(2S)$ data samples collected at the current and proposed future $e^+e^-$ collider experiment., Comment: 26 pages, 13 figures. Submitted to CPC

Multi-channel EEG signals are commonly used for the diagnosis and assessment of diseases such as epilepsy. Currently, various EEG diagnostic algorithms based on deep learning have been developed. However, most research efforts focus solely on diagnosing and classifying current signal data but do not consider the prediction of future trends for early warning. Additionally, since multi-channel EEG can be essentially regarded as the spatio-temporal signal data received by detectors at different locations in the brain, how to construct spatio-temporal information representations of EEG signals to facilitate future trend prediction for multi-channel EEG becomes an important problem. This study proposes a multi-signal prediction algorithm based on generative diffusion models (EEG-DIF), which transforms the multi-signal forecasting task into an image completion task, allowing for comprehensive representation and learning of the spatio-temporal correlations and future developmental patterns of multi-channel EEG signals. Here, we employ a publicly available epilepsy EEG dataset to construct and validate the EEG-DIF. The results demonstrate that our method can accurately predict future trends for multi-channel EEG signals simultaneously. Furthermore, the early warning accuracy for epilepsy seizures based on the generated EEG data reaches 0.89. In general, EEG-DIF provides a novel approach for characterizing multi-channel EEG signals and an innovative early warning algorithm for epilepsy seizures, aiding in optimizing and enhancing the clinical diagnosis process. The code is available at https://github.com/JZK00/EEG-DIF., Comment: 9 pages, 4 figures, 3 tables, accepted by ACM BCB 2024

Ultra-long gamma-ray bursts (ULGRBs) are characterized by exceptionally long-duration central engine activities, with characteristic timescales exceeding 1000 seconds. We present ground-based optical afterglow observations of the ultra-long gamma-ray burst GRB 211024B, detected by \textit{Swift}. Its X-ray light curve exhibits a characteristic ``internal plateau" with a shallow decay phase lasting approximately $\sim 15$ ks, followed by a steep decline ($\alpha_{\rm drop}\sim-7.5$). Moreover, the early optical emission predicted by the late r-band optical afterglow is significantly higher than the observed value, indicating an external shock with energy injection. To explain these observations, we propose a magnetar central engine model. The magnetar collapse into a black hole due to spin-down or hyperaccretion, leading to the observed steep break in the X-ray light curve. The afterglow model fitting reveals that the afterglow injection luminosity varies with different assumptions of the circumburst medium density, implying different potential energy sources. For the interstellar medium (ISM) case with a fixed injection end time, the energy may originate from the magnetar's dipole radiation. However, in other scenarios, relativistic jets produced by the magnetar/black hole system could be the primary energy source., Comment: 18 pages, 7 figures, accepted by ApJ

Virtual Reality (VR) museums provide immersive visiting experiences. Despite growing efforts in VR museum design optimization, limited research addresses its efficacy for older adults. We sought to investigate the challenges of and preferences for VR museum visits among older adults through a user-centered participatory workshop. Our preliminary findings illuminate issues regarding spatial navigation, interpretive descriptions, collective aspiration for augmented multi-sensory interactions, and imagined content visualization. Based on our preliminary findings, we discuss potential design principles for enhancing the accessibility of VR museums for older adults., Comment: # indicates equal contribution

In engineering design, one of the most daunting problems in the design-through-analysis workflow is to deal with trimmed NURBS (Non-Uniform Rational B-Splines), which often involve topological/geometric issues and lead to inevitable gaps and overlaps in the model. Given the dominance of the trimming technology in CAD systems, reconstructing such a model as a watertight representation is highly desired. While remarkable progress has been made in recent years, especially with the advancement of isogeometric analysis (IGA), there still lack a fully automatic and scalable tool to achieve this reconstruction goal. To address this issue, we present a semi-automatic and scalable reparameterization pipeline based on a scalable and feature-aligned meshing tool, QuadriFlow [1]. On top of it, we provide support for open surfaces to deal with engineering shell structures, and perform sophisticated patch simplification to remove undesired tiny/slender patches. As a result, we obtain a watertight spline surface (multi-patch NURBS or unstructured splines) with a simple quadrilateral layout. Through several challenging models from industry applications, we demonstrate the efficacy and efficiency of the proposed pipeline as well as its integration with IGA. Our source code is publicly available on GitHub [2].

Large Language Models (LLMs) have demonstrated remarkable performance across various domains, including healthcare. However, their ability to effectively represent structured non-textual data, such as the alphanumeric medical codes used in records like ICD-10 or SNOMED-CT, is limited and has been particularly exposed in recent research. This paper examines the challenges LLMs face in processing medical codes due to the shortcomings of current tokenization methods. As a result, we introduce the UniStruct architecture to design a multimodal medical foundation model of unstructured text and structured data, which addresses these challenges by adapting subword tokenization techniques specifically for the structured medical codes. Our approach is validated through model pre-training on both an extensive internal medical database and a public repository of structured medical records. Trained on over 1 billion tokens on the internal medical database, the proposed model achieves up to a 23% improvement in evaluation metrics, with around 2% gain attributed to our proposed tokenization. Additionally, when evaluated on the EHRSHOT public benchmark with a 1/1000 fraction of the pre-training data, the UniStruct model improves performance on over 42% of the downstream tasks. Our approach not only enhances the representation and generalization capabilities of patient-centric models but also bridges a critical gap in representation learning models' ability to handle complex structured medical data, alongside unstructured text., Comment: NeurIPS 2024 Workshop on Unifying Representations in Neural Models (UniReps 2024)

We explore the potential of conducting an experiment on the Chinese Space Station (CSS) to constrain beyond-the-standard-model (BSM) long-range spin- and velocity-dependent interactions, which are mediated by the exchange of an ultralight $\left(m_{z^{\prime}}<10^{-10}\text{eV}\right)$ or massless intermediate vector boson. We demonstrate that the proposed experiment on the CSS offers several advantages compared to ground-based experiments. The high speed can enhance the sensitivity to velocity-dependent interactions. The periodicity allows efficient extraction of signals from background noises, thereby strengthening the experiment's accuracy. Combining these advantages, one can improve the existing bounds on such interactions by up to five orders of magnitude. With advancements in sensor technology, we anticipate a further enhancement of four orders of magnitude, resulting in a total potential improvement of up to nine orders of magnitude., Comment: 6 pages, 4 figures; comments welcome

Utilizing a data set of $6.7$ fb$^{-1}$ from electron-positron collisions recorded by the BESIII detector at the BEPCII storage ring, a search is conducted for the processes $e^{+}e^{-} \to \phi \chi_{c0}$ and $\phi\eta_{c2}(1D)$ across center-of-mass energies from 4.47 to 4.95 GeV. In the absence of any significant signals, upper limits are set. These include limits on the Born cross sections for $e^{+}e^{-} \to \phi \chi_{c0}$, as well as the product of the Born cross section for $e^{+}e^{-} \to \phi\eta_{c2}(1D)$ and a sum of five branching fractions. Furthermore, the product of the electronic width of $Y(4660)$ and the branching fraction of the $Y(4660) \to \phi\chi_{c0}$, denoted as $\Gamma^{Y(4660)}_{e^{+}e^{-}} \mathcal{B}_{Y(4660) \to \phi\chi_{c0}}$, is determined to be $< 0.40$ eV at the 90\% confidence level., Comment: 14 pages, 6 figures

The Segment Anything Model (SAM) has demonstrated strong performance in image segmentation of natural scene images. However, its effectiveness diminishes markedly when applied to specific scientific domains, such as Scanning Probe Microscope (SPM) images. This decline in accuracy can be attributed to the distinct data distribution and limited availability of the data inherent in the scientific images. On the other hand, the acquisition of adequate SPM datasets is both time-intensive and laborious as well as skill-dependent. To address these challenges, we propose an Adaptive Prompt Learning with SAM (APL-SAM) framework tailored for few-shot SPM image segmentation. Our approach incorporates two key innovations to enhance SAM: 1) An Adaptive Prompt Learning module leverages few-shot embeddings derived from limited support set to learn adaptively central representatives, serving as visual prompts. This innovation eliminates the need for time-consuming online user interactions for providing prompts, such as exhaustively marking points and bounding boxes slice by slice; 2) A multi-source, multi-level mask decoder specifically designed for few-shot SPM image segmentation is introduced, which can effectively capture the correspondence between the support and query images. To facilitate comprehensive training and evaluation, we introduce a new dataset, SPM-Seg, curated for SPM image segmentation. Extensive experiments on this dataset reveal that the proposed APL-SAM framework significantly outperforms the original SAM, achieving over a 30% improvement in terms of Dice Similarity Coefficient with only one-shot guidance. Moreover, APL-SAM surpasses state-of-the-art few-shot segmentation methods and even fully supervised approaches in performance. Code and dataset used in this study will be made available upon acceptance., Comment: 10 pages, 7 figures

Unlearning in Large Language Models (LLMs) is essential for ensuring ethical and responsible AI use, especially in addressing privacy leak, bias, safety, and evolving regulations. Existing approaches to LLM unlearning often rely on retain data or a reference LLM, yet they struggle to adequately balance unlearning performance with overall model utility. This challenge arises because leveraging explicit retain data or implicit knowledge of retain data from a reference LLM to fine-tune the model tends to blur the boundaries between the forgotten and retain data, as different queries often elicit similar responses. In this work, we propose eliminating the need to retain data or the reference LLM for response calibration in LLM unlearning. Recognizing that directly applying gradient ascent on the forget data often leads to optimization instability and poor performance, our method guides the LLM on what not to respond to, and importantly, how to respond, based on the forget data. Hence, we introduce Forget data only Loss AjustmenT (FLAT), a "flat" loss adjustment approach which addresses these issues by maximizing f-divergence between the available template answer and the forget answer only w.r.t. the forget data. The variational form of the defined f-divergence theoretically provides a way of loss adjustment by assigning different importance weights for the learning w.r.t. template responses and the forgetting of responses subject to unlearning. Empirical results demonstrate that our approach not only achieves superior unlearning performance compared to existing methods but also minimizes the impact on the model's retained capabilities, ensuring high utility across diverse tasks, including copyrighted content unlearning on Harry Potter dataset and MUSE Benchmark, and entity unlearning on the TOFU dataset., Comment: Paper under review

In this work, we present a comprehensive analysis of causal image modeling and introduce the Adventurer series models where we treat images as sequences of patch tokens and employ uni-directional language models to learn visual representations. This modeling paradigm allows us to process images in a recurrent formulation with linear complexity relative to the sequence length, which can effectively address the memory and computation explosion issues posed by high-resolution and fine-grained images. In detail, we introduce two simple designs that seamlessly integrate image inputs into the causal inference framework: a global pooling token placed at the beginning of the sequence and a flipping operation between every two layers. Extensive empirical studies demonstrate the significant efficiency and effectiveness of this causal image modeling paradigm. For example, our base-sized Adventurer model attains a competitive test accuracy of 84.0% on the standard ImageNet-1k benchmark with 216 images/s training throughput, which is 5.3 times more efficient than vision transformers to achieve the same result.

Instruction tuning is critical for adapting large language models (LLMs) to downstream tasks, and recent studies have demonstrated that small amounts of human-curated data can outperform larger datasets, challenging traditional data scaling laws. While LLM-based data quality rating systems offer a cost-effective alternative to human annotation, they often suffer from inaccuracies and biases, even in powerful models like GPT-4. In this work, we introduce DS2, a Diversity-aware Score curation method for Data Selection. By systematically modeling error patterns through a score transition matrix, DS2 corrects LLM-based scores and promotes diversity in the selected data samples. Our approach shows that a curated subset (just 3.3% of the original dataset) outperforms full-scale datasets (300k samples) across various machine-alignment benchmarks, and matches or surpasses human-aligned datasets such as LIMA with the same sample size (1k samples). These findings challenge conventional data scaling assumptions, highlighting that redundant, low-quality samples can degrade performance and reaffirming that "more can be less."

The first very long baseline interferometry (VLBI) detections at 870$\mu$m wavelength (345$\,$GHz frequency) are reported, achieving the highest diffraction-limited angular resolution yet obtained from the surface of the Earth, and the highest-frequency example of the VLBI technique to date. These include strong detections for multiple sources observed on inter-continental baselines between telescopes in Chile, Hawaii, and Spain, obtained during observations in October 2018. The longest-baseline detections approach 11$\,$G$\lambda$ corresponding to an angular resolution, or fringe spacing, of 19$\mu$as. The Allan deviation of the visibility phase at 870$\mu$m is comparable to that at 1.3$\,$mm on the relevant integration time scales between 2 and 100$\,$s. The detections confirm that the sensitivity and signal chain stability of stations in the Event Horizon Telescope (EHT) array are suitable for VLBI observations at 870$\mu$m. Operation at this short wavelength, combined with anticipated enhancements of the EHT, will lead to a unique high angular resolution instrument for black hole studies, capable of resolving the event horizons of supermassive black holes in both space and time., Comment: Corresponding author: S. Doeleman

For 6-DoF grasp detection, simulated data is expandable to train more powerful model, but it faces the challenge of the large gap between simulation and real world. Previous works bridge this gap with a sim-to-real way. However, this way explicitly or implicitly forces the simulated data to adapt to the noisy real data when training grasp detectors, where the positional drift and structural distortion within the camera noise will harm the grasp learning. In this work, we propose a Real-to-Sim framework for 6-DoF Grasp detection, named R2SGrasp, with the key insight of bridging this gap in a real-to-sim way, which directly bypasses the camera noise in grasp detector training through an inference-time real-to-sim adaption. To achieve this real-to-sim adaptation, our R2SGrasp designs the Real-to-Sim Data Repairer (R2SRepairer) to mitigate the camera noise of real depth maps in data-level, and the Real-to-Sim Feature Enhancer (R2SEnhancer) to enhance real features with precise simulated geometric primitives in feature-level. To endow our framework with the generalization ability, we construct a large-scale simulated dataset cost-efficiently to train our grasp detector, which includes 64,000 RGB-D images with 14.4 million grasp annotations. Sufficient experiments show that R2SGrasp is powerful and our real-to-sim perspective is effective. The real-world experiments further show great generalization ability of R2SGrasp. Project page is available on https://isee-laboratory.github.io/R2SGrasp.

There is a growing interest in expanding the input capacity of language models (LMs) across various domains. However, simply increasing the context window does not guarantee robust performance across diverse long-input processing tasks, such as understanding extensive documents and extracting detailed information from lengthy and noisy data. In response, we introduce SEGMENT+, a general framework that enables LMs to handle extended inputs within limited context windows efficiently. SEGMENT+ utilizes structured notes and a filtering module to manage information flow, resulting in a system that is both controllable and interpretable. Our extensive experiments across various model sizes, focusing on long-document question-answering and Needle-in-a-Haystack tasks, demonstrate the effectiveness of SEGMENT+ in improving performance., Comment: EMNLP 2024

Using (2712.4 $\pm$ 14.3)$\times 10^{6}$ $\psi(3686)$ events collected with the BESIII detector at BEPCII, a partial wave analysis of the decay $\psi(3686) \to \phi \eta \eta' $ is performed with the covariant tensor approach. An axial-vector state with a mass near 2.3 $\rm GeV/c^2$ is observed for the first time. Its mass and width are measured to be 2316 $\pm 9_{\mathrm{stat}} \pm 30_{\mathrm{syst}}\,\rm MeV/c^2$ and 89 $\pm 15_{\mathrm{stat}} \pm 26_{\mathrm{syst}}\,\rm MeV$, respectively. The product branching fractions of $\mathcal{B}(\psi(3686) \to X(2300) \eta') \mathcal{B}(X(2300)\to \phi \eta)$ and $\mathcal{B}(\psi(3686) \to X(2300) \eta)\mathcal{B}(X(2300)\to \phi \eta')$ are determined to be (4.8 $\pm 1.3_{\mathrm{stat}} \pm 0.7_{\mathrm{syst}})\times 10^{-6}$ and (2.2 $\pm 0.7_{\mathrm{stat}} \pm 0.7_{\mathrm{syst}})\times 10^{-6}$, respectively. The branching fraction $\mathcal{B}(\psi(3686) \to \phi \eta \eta')$ is measured for the first time to be (3.14$\pm0.17_{\mathrm{stat}}\pm0.24_{\mathrm{syst}})\times10^{-5}$. The first uncertainties are statistical and the second are systematic.

Polarization measurements provide insight into the magnetic field, a critical aspect of the dynamics and emission properties around the compact object. In this paper, we present the polarized magnetic field of the Crab outer torus and the Vela arc utilizing Imaging X-ray Polarimetry Explorer observation data. The polarization angle (PA) measured for the Crab outer torus exhibits two monotonic evolutions along the azimuth angle, which are consistent with the normal line of the elliptical ring. There is a slight increase in PA along the azimuth angle for both the inner arc and the outer arc of the Vela nebula. The polarized magnetic vector along the outer torus of the Crab nebula shows the polarized magnetic field aligns with Crab outer torus structure. The PA variation along the Crab outer torus suggests a bulk flow speed of 0.8c. Meanwhile, the Vela nebula polarized magnetic field does not exactly align with the Vela arc structure. We noted that the Crab nebula possesses a polarized toroidal magnetic field, where as the Vela nebula exhibits an incomplete toroidal magnetic field., Comment: 8 pages, 5 figures

Understanding long text is of great demands in practice but beyond the reach of most language-image pre-training (LIP) models. In this work, we empirically confirm that the key reason causing such an issue is that the training images are usually paired with short captions, leaving certain tokens easily overshadowed by salient tokens. Towards this problem, our initial attempt is to relabel the data with long captions, however, directly learning with which may lead to performance degradation in understanding short text (e.g., in the image classification task). Then, after incorporating corner tokens to aggregate diverse textual information, we manage to help the model catch up to its original level of short text understanding yet greatly enhance its capability of long text understanding. We further look into whether the model can continuously benefit from longer captions and notice a clear trade-off between the performance and the efficiency. Finally, we validate the effectiveness of our approach using a self-constructed large-scale dataset, which consists of 100M long caption oriented text-image pairs. It is noteworthy that, on the task of long-text image retrieval, we beat the competitor using long captions with 11.1% improvement (i.e., from 72.62% to 83.72%). We will release the code, the model, and the new dataset to facilitate the reproducibility and further research. The project page is available at https://wuw2019.github.io/lot-lip.

In this study, energy-energy correlator (EEC) distributions for the $\rm D^0$, $\rm B^0$-tagged, inclusive, and the PYTHIA generated pure quark jets are computed in pp, p+Pb, and Pb+Pb collisions at \sqrts =5.02 TeV for a same jet transverse momentum interval 15-30 GeV. We find the number of particles per jet determines the height of the EEC distribution in pp baseline. The averaged energy weight distribution resulted in a shift of the originally larger angular distributed particle distribution to a smaller RL, thereby obtaining an EEC distribution. The EEC distributions for all quark-tagged jets in A+A exhibit a noticeable shift towards larger RL region, suggesting that the jets will be more widely distributed compared to those in pp collisions. The jet quenching effect will cause the pair angular distribution to shift towards larger values and increase the number of particles per jet. This redistribution of energy within the jets suggests that the already reduced jet energy is redistributed among a larger number of particles, leading to a reduced energy weight per pair. The enhancement of the number of particles per jet and the reduced averaged energy weight interplay with each other to form the medium modification of EEC., Comment: 9 pages, 5 figures

In recent advancements, multimodal large language models (MLLMs) have been fine-tuned on specific medical image datasets to address medical visual question answering (Med-VQA) tasks. However, this common approach of task-specific fine-tuning is costly and necessitates separate models for each downstream task, limiting the exploration of zero-shot capabilities. In this paper, we introduce MC-CoT, a modular cross-modal collaboration Chain-of-Thought (CoT) framework designed to enhance the zero-shot performance of MLLMs in Med-VQA by leveraging large language models (LLMs). MC-CoT improves reasoning and information extraction by integrating medical knowledge and task-specific guidance, where LLM provides various complex medical reasoning chains and MLLM provides various observations of medical images based on instructions of the LLM. Our experiments on datasets such as SLAKE, VQA-RAD, and PATH-VQA show that MC-CoT surpasses standalone MLLMs and various multimodality CoT frameworks in recall rate and accuracy. These findings highlight the importance of incorporating background information and detailed guidance in addressing complex zero-shot Med-VQA tasks., Comment: 21 pages, 14 figures, 6 tables

We report the detection of an extended very-high-energy (VHE) gamma-ray source coincident with the locations of middle-aged (62.4~\rm kyr) pulsar PSR J0248+6021, by using the LHAASO-WCDA data of live 796 days and LHAASO-KM2A data of live 1216 days. A significant excess of \gray induced showers is observed both by WCDA in energy bands of 1-25~\rm TeV and KM2A in energy bands of $>$ 25~\rm TeV with 7.3 $\sigma$ and 13.5 $\sigma$, respectively. The best-fit position derived through WCDA data is R.A. = 42.06$^\circ \pm$ 0.12$^\circ$ and Dec. = 60.24$^\circ \pm $ 0.13$^\circ$ with an extension of 0.69$^\circ\pm$0.15$^\circ$ and that of the KM2A data is R.A.= 42.29$^\circ \pm $ 0.13$^\circ$ and Dec. = 60.38$^\circ \pm$ 0.07$^\circ$ with an extension of 0.37$^\circ\pm$0.07$^\circ$. No clear extended multiwavelength counterpart of this LHAASO source has been found from the radio band to the GeV band. The most plausible explanation of the VHE \gray emission is the inverse Compton process of highly relativistic electrons and positrons injected by the pulsar. These electrons/positrons are hypothesized to be either confined within the pulsar wind nebula or to have already escaped into the interstellar medium, forming a pulsar halo., Comment: 12 pages, 10 figures, Accepted by Sci. China-Phys. Mech. Astron

Quantum secret sharing (QSS) enables a dealer to securely distribute keys to multiple players. Device-independent (DI) QSS can resist all possible attacks from practical imperfect devices and provide QSS the highest level of security in theory. However, DI QSS requires high-performance devices, especially for low-noise channels, which is a big challenge for its experimental demonstration. We propose a DI QSS protocol with the advanced random key generation basis strategy, which combines the random key generation basis with the noise preprocessing and postselection strategies. We develop the methods to simplify Eve's conditional entropy bound and numerically simulate the key generation rate in an acceptable time. Our DI QSS protocol has some advantages. First, it can increase the noise tolerance threshold from initial 7.147% to 9.231% (29.16% growth), and reduce the global detection efficiency threshold from 96.32% to 93.41%. The maximal distance between any two users increases to 1.43 km, which is about 5.5 times of the initial value. Second, by randomly selecting two basis combinations to generate the key, our DI QSS protocol can reduce the entanglement resource consumption. Our protocol has potential for DI QSS's experimental demonstration and application in the future., Comment: 16 pages, 6 figures, 1 table

In this paper, we leverage the insights from Quon, a picture language for quantum information, to develop a new class of quantum error-correcting codes termed Variational Graphical Quantum Error Correction~(VGQEC) codes. The VGQEC codes feature adjustable configuration parameters that play a pivotal role in determining the error-correcting capability of the codes. This key feature offers remarkable flexibility in customizing high-quality quantum error-correcting codes for various noise models. For instance, we will present a specific VGQEC code that exhibits a seamless transition of parameters, enabling the smooth transformation of the code from the five-qubit repetition code to the [[5,1,3]] code, and furthermore, the new VGQEC code has a better performance than the above two well-known codes under certain noise models. Meanwhile, we also propose a general physical scheme to implement and optimize VGQEC codes in realistic quantum devices. Lastly, we apply our approach to amplitude damping noise, and by numerical calculations, we discover an unexpected novel three-qubit code that can effectively mitigate the noise.

Bounds of total curvature and entropy are two common conditions placed on mean curvature flows. We show that these two hypotheses are equivalent for the class of ancient complete embedded smooth planar curve shortening flows, which are one-dimensional mean curvature flows. As an application, we give a short proof of the uniqueness of tangent flow at infinity of an ancient smooth complete non-compact curve shortening flow with finite entropy embedded in $\mathbb{R}^2$.

We present the Melody Guided Music Generation (MMGen) model, the first novel approach using melody to guide the music generation that, despite a pretty simple method and extremely limited resources, achieves excellent performance. Specifically, we first align the melody with audio waveforms and their associated descriptions using the multimodal alignment module. Subsequently, we condition the diffusion module on the learned melody representations. This allows MMGen to generate music that matches the style of the provided audio while also producing music that reflects the content of the given text description. To address the scarcity of high-quality data, we construct a multi-modal dataset, MusicSet, which includes melody, text, and audio, and will be made publicly available. We conduct extensive experiments which demonstrate the superiority of the proposed model both in terms of experimental metrics and actual performance quality., Comment: 9 pages, 1 figure, 2 tables

We attempt to measure the DIBs $\lambda$5780, $\lambda$5797 and $\lambda$6614 in over two million low-resolution spectra of cool stars from LAMOST. Based on the DIB measurements, the correlation between DIBs and extinction, the kinematics of DIBs, and the Galactic distribution of DIBs are reviewed and investigated from the perspective of statistics. A pipeline is developed to measure the DIBs $\lambda$5780, $\lambda$5797 and $\lambda$6614 in the LAMOST low-resolution spectra. We obtain the DIB measurements of spectra of late-type stars from LAMOST, and screen out 176,831, 13,473 and 110,152 high-quality (HQ) measurements of the DIBs $\lambda$5780, $\lambda$5797 and $\lambda$6614, respectively, corresponding to 142,074, 11,480 and 85,301 unique sources. Utilizing these HQ measurements, we present the Galactic maps of the DIBs $\lambda$5780 and $\lambda$6614 in the northern sky for the first time. The central wavelengths of the DIBs $\lambda$5780, $\lambda$5797 and $\lambda$6614 in air are determined to be 5780.48 $\pm$ 0.01, 5796.94 $\pm$ 0.02 and 6613.64 $\pm$ 0.01 {\AA}, respectively, based on their kinematics. The equivalent widths of these three DIBs per unit extinction are statistically fitted to be 0.565, 0.176 and 0.256 {\AA}/mag. As a part of our work, three catalogs of the HQ measurements for the DIBs $\lambda$5780, $\lambda$5797 and $\lambda$6614 are provided online. To the best of our knowledge, this is the largest number of measurements of these three DIBs to date. It is also the first time that the Galactic maps of the DIBs $\lambda$5780 and $\lambda$6614 in the northern hemisphere are presented, and the central wavelengths of the DIBs $\lambda$5780, $\lambda$5797 and $\lambda$6614 are estimated from the kinematics., Comment: 19 pages, 5 tables, 13 figures, 1 appendix, accepted for publication in A&A

Holography is an essential technique of generating three-dimensional images. Recently, quantum holography with undetected photons (QHUP) has emerged as a groundbreaking method capable of capturing complex amplitude images. Despite its potential, the practical application of QHUP has been limited by susceptibility to phase disturbances, low interference visibility, and limited spatial resolution. Deep learning, recognized for its ability in processing complex data, holds significant promise in addressing these challenges. In this report, we present an ample advancement in QHUP achieved by harnessing the power of deep learning to extract images from single-shot holograms, resulting in vastly reduced noise and distortion, alongside a notable enhancement in spatial resolution. The proposed and demonstrated deep learning QHUP (DL-QHUP) methodology offers a transformative solution by delivering high-speed imaging, improved spatial resolution, and superior noise resilience, making it suitable for diverse applications across an array of research fields stretching from biomedical imaging to remote sensing. DL-QHUP signifies a crucial leap forward in the realm of holography, demonstrating its immense potential to revolutionize imaging capabilities and pave the way for advancements in various scientific disciplines. The integration of DL-QHUP promises to unlock new possibilities in imaging applications, transcending existing limitations and offering unparalleled performance in challenging environments.

Document content analysis has been a crucial research area in computer vision. Despite significant advancements in methods such as OCR, layout detection, and formula recognition, existing open-source solutions struggle to consistently deliver high-quality content extraction due to the diversity in document types and content. To address these challenges, we present MinerU, an open-source solution for high-precision document content extraction. MinerU leverages the sophisticated PDF-Extract-Kit models to extract content from diverse documents effectively and employs finely-tuned preprocessing and postprocessing rules to ensure the accuracy of the final results. Experimental results demonstrate that MinerU consistently achieves high performance across various document types, significantly enhancing the quality and consistency of content extraction. The MinerU open-source project is available at https://github.com/opendatalab/MinerU., Comment: MinerU Technical Report

This comprehensive study evaluates the performance of OpenAI's o1-preview large language model across a diverse array of complex reasoning tasks, spanning multiple domains, including computer science, mathematics, natural sciences, medicine, linguistics, and social sciences. Through rigorous testing, o1-preview demonstrated remarkable capabilities, often achieving human-level or superior performance in areas ranging from coding challenges to scientific reasoning and from language processing to creative problem-solving. Key findings include: -83.3% success rate in solving complex competitive programming problems, surpassing many human experts. -Superior ability in generating coherent and accurate radiology reports, outperforming other evaluated models. -100% accuracy in high school-level mathematical reasoning tasks, providing detailed step-by-step solutions. -Advanced natural language inference capabilities across general and specialized domains like medicine. -Impressive performance in chip design tasks, outperforming specialized models in areas such as EDA script generation and bug analysis. -Remarkable proficiency in anthropology and geology, demonstrating deep understanding and reasoning in these specialized fields. -Strong capabilities in quantitative investing. O1 has comprehensive financial knowledge and statistical modeling skills. -Effective performance in social media analysis, including sentiment analysis and emotion recognition. The model excelled particularly in tasks requiring intricate reasoning and knowledge integration across various fields. While some limitations were observed, including occasional errors on simpler problems and challenges with certain highly specialized concepts, the overall results indicate significant progress towards artificial general intelligence.

Resilience serves to assess the ability of financial markets to resist external shocks. The intensity and duration, used to indicate resilience, are calculated for China's financial markets in this paper, focusing on the performance of each financial market during and after several crises. Given that climate issues have been recognized as an important source of risk by financial markets, we also investigate the spillover effects and mechanism of China's climate policy uncertainty on its financial markets resilience. We have found that the two resilience indicators of each market have a relatively consistent trend, but spillovers among markets have different sensitivities to the both. In addition, China's climate policy uncertainty shocks its financial markets resilience by increasing the investor sentiment index and the non-performing loan ratio of commercial banks and by reducing the capital and financial account balance. It is further found that China's financial markets' consensus on the unswerving implementation of climate policy, which provides the reference for other countries on how to balance climate policies introduction and financial markets development.

Knot data analysis (KDA), which studies data with curve-type structures such as knots, links, and tangles, has emerging as a promising geometric topology approach in data science. While evolutionary Khovanov homology has been developed to analyze the global persistent topological features of links, it has limitations in capturing the local topological characteristics of knots and links. To address this challenge, we introduce the persistent Khovanov homology of tangles, providing a new mathematical framework for characterizing local features in curve-type data. While tangle homology is inherently abstract, we provide a concrete functor which maps the category of tangles to the category of modules, enabling the computation of tangle homology. Additionally, we employ planar algebra to construct a category of tangles without invoking fixed boundaries, thereby giving rise to a persistent Khovanov homology functor that enables practical applications. This framework provides a theoretical foundation and practical strategies for the topological analysis of curve-type data.

Thanks to the rapidly increasing time-domain facilities, we are entering a golden era of research on gamma-ray bursts (GRBs). In this Letter, we report our observations of GRB 240529A with the Burst Optical Observer and Transient Exploring System, the 1.5-meter telescope at Observatorio Sierra Nevada, the 2.5-meter Wide Field Survey Telescope of China, the Large Binocular Telescope, and the Telescopio Nazionale Galileo. The prompt emission of GRB 240529A shows two comparable energetic episodes separated by a quiescence time of roughly 400 s. Combining all available data on the GRB Coordinates Network, we reveal the simultaneous apparent X-ray plateau and optical re-brightening around $10^3-10^4$ s after the burst. Rather than the energy injection from the magnetar as widely invoked for similar GRBs, the multi-wavelength emissions could be better explained as two shocks launched from the central engine separately. The optical peak time and our numerical modeling suggest that the initial bulk Lorentz factor of the later shock is roughly 50, which indicates that the later jet should be accretion-driven and have a higher mass loading than a typical one. The quiescence time between the two prompt emission episodes may be caused by the transition between different accretion states of a central magnetar or black hole, or the fall-back accretion process. A sample of similar bursts with multiple emission episodes in the prompt phase and sufficient follow-up could help to probe the underlying physics of GRB central engines., Comment: Resubmitted to ApJL after addressing the referee's comments; comments are welcome

Due to their substantial sizes, large language models (LLMs) are typically deployed within a single-backbone multi-tenant framework. In this setup, a single instance of an LLM backbone must cater to multiple users or tasks through the application of various parameter-efficient fine-tuning (PEFT) models. Despite the availability of numerous effective PEFT techniques such as LoRA, there remains a need for a PEFT approach that achieves both high efficiency during inference and competitive performance on downstream tasks. In this research, we introduce a new and straightforward PEFT methodology named \underline{P}rompt D\underline{E}pen\underline{D}ent \underline{R}epresentation M\underline{O}dification (PEDRO). The proposed method involves integrating a lightweight vector generator into each Transformer layer, which generates vectors contingent upon the input prompts. These vectors then modify the hidden representations created by the LLM through a dot product operation, thereby influencing the semantic output and generated content of the model. Extensive experimentation across a variety of tasks indicates that: (a) PEDRO surpasses recent PEFT benchmarks when using a similar number of tunable parameters. (b) Under the single-backbone multi-tenant deployment model, PEDRO exhibits superior efficiency compared to LoRA, indicating significant industrial potential., Comment: arXiv admin note: text overlap with arXiv:2405.18203

In recent years, live video streaming has gained widespread popularity across various social media platforms. Quality of experience (QoE), which reflects end-users' satisfaction and overall experience, plays a critical role for media service providers to optimize large-scale live compression and transmission strategies to achieve perceptually optimal rate-distortion trade-off. Although many QoE metrics for video-on-demand (VoD) have been proposed, there remain significant challenges in developing QoE metrics for live video streaming. To bridge this gap, we conduct a comprehensive study of subjective and objective QoE evaluations for live video streaming. For the subjective QoE study, we introduce the first live video streaming QoE dataset, TaoLive QoE, which consists of $42$ source videos collected from real live broadcasts and $1,155$ corresponding distorted ones degraded due to a variety of streaming distortions, including conventional streaming distortions such as compression, stalling, as well as live streaming-specific distortions like frame skipping, variable frame rate, etc. Subsequently, a human study was conducted to derive subjective QoE scores of videos in the TaoLive QoE dataset. For the objective QoE study, we benchmark existing QoE models on the TaoLive QoE dataset as well as publicly available QoE datasets for VoD scenarios, highlighting that current models struggle to accurately assess video QoE, particularly for live content. Hence, we propose an end-to-end QoE evaluation model, Tao-QoE, which integrates multi-scale semantic features and optical flow-based motion features to predicting a retrospective QoE score, eliminating reliance on statistical quality of service (QoS) features., Comment: 14 pages, 5 figures

This paper describes the submissions of Huawei Translation Services Center(HW-TSC) to WMT24 chat translation shared task on English$\leftrightarrow$Germany (en-de) bidirection. The experiments involved fine-tuning models using chat data and exploring various strategies, including Minimum Bayesian Risk (MBR) decoding and self-training. The results show significant performance improvements in certain directions, with the MBR self-training method achieving the best results. The Large Language Model also discusses the challenges and potential avenues for further research in the field of chat translation., Comment: 7 pages, 6 Tables, WMT24

This article introduces the submission status of the Translation into Low-Resource Languages of Spain task at (WMT 2024) by Huawei Translation Service Center (HW-TSC). We participated in three translation tasks: spanish to aragonese (es-arg), spanish to aranese (es-arn), and spanish to asturian (es-ast). For these three translation tasks, we use training strategies such as multilingual transfer, regularized dropout, forward translation and back translation, labse denoising, transduction ensemble learning and other strategies to neural machine translation (NMT) model based on training deep transformer-big architecture. By using these enhancement strategies, our submission achieved a competitive result in the final evaluation., Comment: 6 pages,wmt24. arXiv admin note: substantial text overlap with arXiv:2409.14842; text overlap with arXiv:2409.14800

This paper introduces the submission by Huawei Translation Center (HW-TSC) to the WMT24 Indian Languages Machine Translation (MT) Shared Task. To develop a reliable machine translation system for low-resource Indian languages, we employed two distinct knowledge transfer strategies, taking into account the characteristics of the language scripts and the support available from existing open-source models for Indian languages. For Assamese(as) and Manipuri(mn), we fine-tuned the existing IndicTrans2 open-source model to enable bidirectional translation between English and these languages. For Khasi (kh) and Mizo (mz), We trained a multilingual model as a baseline using bilingual data from these four language pairs, along with an additional about 8kw English-Bengali bilingual data, all of which share certain linguistic features. This was followed by fine-tuning to achieve bidirectional translation between English and Khasi, as well as English and Mizo. Our transfer learning experiments produced impressive results: 23.5 BLEU for en-as, 31.8 BLEU for en-mn, 36.2 BLEU for as-en, and 47.9 BLEU for mn-en on their respective test sets. Similarly, the multilingual model transfer learning experiments yielded impressive outcomes, achieving 19.7 BLEU for en-kh, 32.8 BLEU for en-mz, 16.1 BLEU for kh-en, and 33.9 BLEU for mz-en on their respective test sets. These results not only highlight the effectiveness of transfer learning techniques for low-resource languages but also contribute to advancing machine translation capabilities for low-resource Indian languages., Comment: 6 pages, wmt24. arXiv admin note: substantial text overlap with arXiv:2409.14800

Breast cancer is a significant threat to human health. Contrastive learning has emerged as an effective method to extract critical lesion features from mammograms, thereby offering a potent tool for breast cancer screening and analysis. A crucial aspect of contrastive learning involves negative sampling, where the selection of appropriate hard negative samples is essential for driving representations to retain detailed information about lesions. In contrastive learning, it is often assumed that features can sufficiently capture semantic content, and that each minibatch inherently includes ideal hard negative samples. However, the characteristics of breast lumps challenge these assumptions. In response, we introduce ManiNeg, a novel approach that leverages manifestations as proxies to mine hard negative samples. Manifestations, which refer to the observable symptoms or signs of a disease, provide a knowledge-driven and robust basis for choosing hard negative samples. This approach benefits from its invariance to model optimization, facilitating efficient sampling. To support ManiNeg and future research endeavors, we developed the MVKL dataset, which includes multi-view mammograms, corresponding reports, meticulously annotated manifestations, and pathologically confirmed benign-malignant outcomes. We evaluate ManiNeg on the benign and malignant classification task. Our results demonstrate that ManiNeg not only improves representation in both unimodal and multimodal contexts but also shows generalization across datasets. The MVKL dataset and our codes are publicly available at https://github.com/wxwxwwxxx/ManiNeg.

This paper presents the submission of Huawei Translation Services Center (HW-TSC) to machine translation tasks of the 20th China Conference on Machine Translation (CCMT 2024). We participate in the bilingual machine translation task and multi-domain machine translation task. For these two translation tasks, we use training strategies such as regularized dropout, bidirectional training, data diversification, forward translation, back translation, alternated training, curriculum learning, and transductive ensemble learning to train neural machine translation (NMT) models based on the deep Transformer-big architecture. Furthermore, to explore whether large language model (LLM) can help improve the translation quality of NMT systems, we use supervised fine-tuning to train llama2-13b as an Automatic post-editing (APE) model to improve the translation results of the NMT model on the multi-domain machine translation task. By using these plyometric strategies, our submission achieves a competitive result in the final evaluation., Comment: 13 pages, 2 figures, 6 Tables, CCMT2024. arXiv admin note: substantial text overlap with arXiv:2409.14800

The quantity of processed data is crucial for advancing the field of singing voice synthesis. While there are tools available for lyric or note transcription tasks, they all need pre-processed data which is relatively time-consuming (e.g., vocal and accompaniment separation). Besides, most of these tools are designed to address a single task and struggle with aligning lyrics and notes (i.e., identifying the corresponding notes of each word in lyrics). To address those challenges, we first design a pipeline by optimizing existing tools and annotating numerous lyric-note pairs of songs. Then, based on the annotated data, we train a unified SongTrans model that can directly transcribe lyrics and notes while aligning them simultaneously, without requiring pre-processing songs. Our SongTrans model consists of two modules: (1) the \textbf{Autoregressive module} predicts the lyrics, along with the duration and note number corresponding to each word in a lyric. (2) the \textbf{Non-autoregressive module} predicts the pitch and duration of the notes. Our experiments demonstrate that SongTrans achieves state-of-the-art (SOTA) results in both lyric and note transcription tasks. Furthermore, it is the first model capable of aligning lyrics with notes. Experimental results demonstrate that the SongTrans model can effectively adapt to different types of songs (e.g., songs with accompaniment), showcasing its versatility for real-world applications.

Understanding atomic hydrogen (H) diffusion in multi-principal element alloys (MPEAs) is essential for advancing clean energy technologies such as H transport, storage, and nuclear fusion applications. However, the vast compositional space and the intricate chemical environments inherent in MPEAs pose significant obstacles. In this work, we address this challenge by developing a multifaceted machine learning framework that integrates machine-learning force field, neural network-driven kinetic Monte Carlo, and machine-learning symbolic regression. This framework allows for accurate investigation of H diffusion across the entire compositional space of body-centered cubic (BCC) refractory MoNbTaW alloys, achieving density functional theory accuracy. For the first time, we discover that H diffusion in MPEAs exhibits super-Arrhenius behavior, described by the Vogel-Fulcher-Tammann model, where the Vogel temperature correlates with the 5th percentile of H solution energy spectrum. We also derive robust analytical expressions that can be used to predict H diffusivity in general BCC MPEAs. Our findings further elucidate that chemical short-range order (SRO) generally does not impact H diffusion, except it enhances diffusion when "H-favoring" elements (notably Nb and Ta) are present in low concentrations. These findings not only enhance our understanding of H diffusion dynamics in general MPEAs but also guide the development of advanced MPEAs in H-related applications by manipulating element type, composition and SRO.

The energy-energy correlator (EEC) is considered as a powerful probe of jet substructure, especially a better probe of certain soft and collinear features. To study the utility of such observable for quark vs gluon discrimination of jet quenching phenomenon, this work first predicts the energy-energy correlators of inclusive jets in central~($0-10\%$) Pb+Pb collisions at $\sqrt{\rm s}$ = 5.02 TeV for jet transverse momentum interval $40 - 60$ GeV. The Pb+Pb EEC distribution shifts to larger $R_{\rm L}$ and smaller $R_{\rm L}$ simultaneously. The shift towards larger $R_{\rm L}$ is attributed to the energy loss effect when the jet evolves in the hot/dense medium and the shift towards smaller $R_{\rm L}$ is due to the selection bias effects. Moreover, we find the EEC distribution for pure quark jets in nucleus-nucleus (A+A) collisions will only be suffering even stronger enhancement at $R_{\rm L}> 0.2$, and the EEC distribution for pure gluon jets in A+A collisions will be observed shifting toward smaller and larger $R_{\rm L}$ at the same time. The jet quenching patterns (A+A/p+p) of the quark jets and the gluon jets can then be separated. We also find that the differences are mainly determined by the initial EEC distribution in p+p, and are not affected much by the energy loss differences between quark and gluon. Inclusive jets are dominated by gluon jets, and photon-tagged jets are used to represent quark jets, we propose this double-ratio measurement to demonstrate the quark/gluon discrimination for the jet quenching phenomenon of jet substructures., Comment: 9 pages, 8 figures

Probing beam measurement (PBM)-based hybrid beamforming provides a feasible solution for cell-free MIMO. In this letter, we propose a novel probing beam optimization framework where three collaborative modules respectively realize PBM augmentation, sum-rate prediction and probing beam optimization. Specifically, the PBM augmentation model integrates the conditional variational auto-encoder (CVAE) and mixture density networks and adopts correlated PBM distribution with full-covariance, for which a Cholesky-decomposition based training is introduced to address the issues of covariance legality and numerical stability. Simulations verify the better performance of the proposed augmentation model compared to the traditional CVAE and the efficiency of proposed optimization framework.

Accurate segmentation of various regions within the prostate is pivotal for diagnosing and treating prostate-related diseases. However, the scarcity of labeled data, particularly in specialized medical fields like prostate imaging, poses a significant challenge. Segment Anything Model (SAM) is a new large model for natural image segmentation, but there are some challenges in medical imaging. In order to better utilize the powerful feature extraction capability of SAM as well as to address the problem of low data volume for medical image annotation, we use Low-Rank Adaptation (LoRA) and semi-supervised learning methods of Monte Carlo guided interpolation consistency (MCIC) to enhance the fine-tuned SAM. We propose Monte Carlo-guided Interpolation Consistency Segment Anything Model (MCICSAM) for application to semi-supervised learning based prostate region segmentation. In the unlabeled data section, MCIC performs two different interpolation transformations on the input data and incorporates Monte Carlo uncertainty analysis in the output, forcing the model to be consistent in its predictions. The consistency constraints imposed on these interpolated samples allow the model to fit the distribution of unlabeled data better, ultimately improving its performance in semi-supervised scenarios. We use Dice and Hausdorff Distance at 95th percentile (HD95) to validate model performance. MCICSAM yieldes Dice with 79.38% and 89.95%, along with improves HD95 values of 3.12 and 2.27 for transition zone and transition zone. At the same time MCICSAM demonstrates strong generalizability. This method is expected to bring new possibilities in the field of prostate image segmentation., Comment: 13 pages, 5 figures

Existing vision-language understanding benchmarks largely consist of images of objects in their usual contexts. As a consequence, recent multimodal large language models can perform well with only a shallow visual understanding by relying on background language biases. Thus, strong performance on these benchmarks does not necessarily correlate with strong visual understanding. In this paper, we release JourneyBench, a comprehensive human-annotated benchmark of generated images designed to assess the model's fine-grained multimodal reasoning abilities across five tasks: complementary multimodal chain of thought, multi-image VQA, imaginary image captioning, VQA with hallucination triggers, and fine-grained retrieval with sample-specific distractors. Unlike existing benchmarks, JourneyBench explicitly requires fine-grained multimodal reasoning in unusual imaginary scenarios where language bias and holistic image gist are insufficient. We benchmark state-of-the-art models on JourneyBench and analyze performance along a number of fine-grained dimensions. Results across all five tasks show that JourneyBench is exceptionally challenging for even the best models, indicating that models' visual reasoning abilities are not as strong as they first appear. We discuss the implications of our findings and propose avenues for further research.

Federated learning (FL) has rapidly evolved as a promising paradigm that enables collaborative model training across distributed participants without exchanging their local data. Despite its broad applications in fields such as computer vision, graph learning, and natural language processing, the development of a data projection model that can be effectively used to visualize data in the context of FL is crucial yet remains heavily under-explored. Neighbor embedding (NE) is an essential technique for visualizing complex high-dimensional data, but collaboratively learning a joint NE model is difficult. The key challenge lies in the objective function, as effective visualization algorithms like NE require computing loss functions among pairs of data. In this paper, we introduce \textsc{FedNE}, a novel approach that integrates the \textsc{FedAvg} framework with the contrastive NE technique, without any requirements of shareable data. To address the lack of inter-client repulsion which is crucial for the alignment in the global embedding space, we develop a surrogate loss function that each client learns and shares with each other. Additionally, we propose a data-mixing strategy to augment the local data, aiming to relax the problems of invisible neighbors and false neighbors constructed by the local $k$NN graphs. We conduct comprehensive experiments on both synthetic and real-world datasets. The results demonstrate that our \textsc{FedNE} can effectively preserve the neighborhood data structures and enhance the alignment in the global embedding space compared to several baseline methods.