Search

How to better evaluate the capabilities of Large Language Models (LLMs) is the focal point and hot topic in current LLMs research. Previous work has noted that due to the extremely high cost of iterative updates of LLMs, they are often unable to answer the latest dynamic questions well. To promote the improvement of Chinese LLMs' ability to answer dynamic questions, in this paper, we introduce CDQA, a Chinese Dynamic QA benchmark containing question-answer pairs related to the latest news on the Chinese Internet. We obtain high-quality data through a pipeline that combines humans and models, and carefully classify the samples according to the frequency of answer changes to facilitate a more fine-grained observation of LLMs' capabilities. We have also evaluated and analyzed mainstream and advanced Chinese LLMs on CDQA. Extensive experiments and valuable insights suggest that our proposed CDQA is challenging and worthy of more further study. We believe that the benchmark we provide will become one of the key data resources for improving LLMs' Chinese question-answering ability in the future., Comment: Work in progress!

Hypothesis: Immiscible liquids are commonly used to achieve unique functions in many applications, where the breakup of compound droplets in airflow is an important process. Due to the existence of the liquid-liquid interface, compound droplets are expected to form different deformation and breakup morphologies compared with single-component droplets. Experiments: We investigate experimentally the deformation and breakup of compound droplets in airflow. The deformation characteristics of compound droplets are quantitatively analyzed and compared with single-component droplets. Theoretical models are proposed to analyze the transition between breakup morphologies. Findings: The breakup modes of compound droplets are classified into shell retraction, shell breakup, and core-shell breakup based on the location where the breakup occurs. The comparison with single-component droplets reveals that the compound droplet is stretched more in the flow direction and expands less in the cross-flow direction, and these differences occur when the core of the compound droplet protrudes into the airflow. The transition conditions between different breakup modes are obtained theoretically. In addition, the eccentricity of the compound droplet can lead to the formation of the thick ligament or the two stamens in the droplet middle.

The atomisation of suspension containing liquid and dispersed particles is prevalent in many applications. Previous studies of droplet breakup mainly focused on homogeneous fluids, and the heterogeneous effect of particles on the breakup progress is unclear. In this study, the breakup of particle-laden droplets in airflow is investigated experimentally. Combining synchronised high-speed images from the side view and the 45$^\circ$ view, we compare the morphology of particle-laden droplets with that of homogeneous fluids in different breakup modes. The results show that the higher effective viscosity of particle-laden droplets affects the initial deformation, and the heterogeneous effect of particles appears in the later breakup stage. To evaluate the heterogeneous effect of particles quantitatively, we eliminate the effect of the higher effective viscosity of particle-laden droplets by comparing cases corresponding to the same inviscid Weber number. The quantitative comparison reveals that the heterogeneous effect of particles accelerates the fragmentation of liquid film and promotes localised rapid piercing. A correlation length that depends on the particle diameter and the volume fraction is proposed to characterise the length scale of the concentration fluctuation under the combined effect of the initial flattening and later stretching during the droplet breakup process. Based on this correlation length, the fragment size distributions are analysed, and the scaling results agree well with the experimental data., Comment: 29 pages, 19 figures

The Open-Domain Question Answering (ODQA) task involves retrieving and subsequently generating answers from fine-grained relevant passages within a database. Current systems leverage Pretrained Language Models (PLMs) to model the relationship between questions and passages. However, the diversity in surface form expressions can hinder the model's ability to capture accurate correlations, especially within complex contexts. Therefore, we utilize Abstract Meaning Representation (AMR) graphs to assist the model in understanding complex semantic information. We introduce a method known as Graph-as-Token (GST) to incorporate AMRs into PLMs. Results from Natural Questions (NQ) and TriviaQA (TQ) demonstrate that our GST method can significantly improve performance, resulting in up to 2.44/3.17 Exact Match score improvements on NQ/TQ respectively. Furthermore, our method enhances robustness and outperforms alternative Graph Neural Network (GNN) methods for integrating AMRs. To the best of our knowledge, we are the first to employ semantic graphs in ODQA., Comment: Accepted by ACL2023 findings, reviewer scores: 4 4 4

This study focuses on the evaluation of the Open Question Answering (Open-QA) task, which can directly estimate the factuality of large language models (LLMs). Current automatic evaluation methods have shown limitations, indicating that human evaluation still remains the most reliable approach. We introduce a new task, Evaluating QA Evaluation (QA-Eval) and the corresponding dataset EVOUNA, designed to assess the accuracy of AI-generated answers in relation to standard answers within Open-QA. Our evaluation of these methods utilizes human-annotated results to measure their performance. Specifically, the work investigates methods that show high correlation with human evaluations, deeming them more reliable. We also discuss the pitfalls of current methods and methods to improve LLM-based evaluators. We believe this new QA-Eval task and corresponding dataset EVOUNA will facilitate the development of more effective automatic evaluation tools and prove valuable for future research in this area. All resources are available at \url{https://github.com/wangcunxiang/QA-Eval} and it is under the Apache-2.0 License., Comment: Accepted by Neurips-2023 Datasets and Benchmarks track; 28 pages

The deformation and breakup of droplets in airflows is important in spray and atomisation processes, but the shear effect in non-uniform airflow is rarely reported. In this study, the deformation and breakup of droplets in a shear flow of air is investigated experimentally using high-speed imaging, digital image processing, and particle image velocimetry. The results show that in airflow with a strong shear effect, the droplet breakup exhibits unique features due to the uplift and stretching produced by the interaction between the deformed droplet and the shear layer. The breakup process can be divided into three stages according to the droplet morphology and the breakup mechanism, namely the sheet breakup, the swing breakup, and the rim breakup stages. Theoretical analysis reveals that the swing breakup is governed by the transverse Rayleigh-Taylor instability. A regime map of the droplet breakup is produced, and the transitions between different regimes are obtained theoretically. The stretching liquid film during the droplet deformation and the fragment size distribution after droplet breakup are analysed quantitatively, and the results show that they are determined by the competition of breakup at different stages affected by the shear. Finally, the effect of the droplet viscosity is investigated, and the viscosity inhibits the droplet breakup in a strong shear airflow., Comment: 28 pages, 19 figures

The breakup of non-Newtonian droplets is ubiquitous in numerous applications. Although the non-Newtonian property can significantly change the droplet breakup process, most previous studies consider Newtonian droplets, and the effects of the non-Newtonian properties on the breakup process are still unclear. This study focuses on the secondary breakup of shear-thinning droplets by numerical simulation. The volume of fluid method is used to capture interface dynamics on adaptive grids. To compare shear-thinning droplets and Newtonian droplets, a new definition of the Ohnesorge number is proposed by considering the characteristic shear rate in the droplet induced by the airflow. The results show that compared with the Newtonian fluid, the shear-thinning properties can change the effective viscosity distribution inside the droplet, alter the local deformation, change the droplet morphology, and affect the transition in the droplet breakup regime., Comment: 14 pages, 15 figures

The deformation and breakup of droplets in air flows is important in many applications of spray and atomization processes. However, the shear effect of airflow has never been reported. In this study, the deformation and breakup of droplets in the shear flow of air is investigated experimentally using high-speed imaging, digital image processing, and particle image velocimetry. We identify a new breakup mode of droplets, i.e., the butterfly breakup, in which the strong aerodynamic pressure on the lower part of the droplet leads to the deflection of the droplet and then the formation of a butterfly-shaped bag. A regime map of the droplet breakup is produced, and the transitions between different modes are obtained based on scaling analysis. The elongation and the fragmentation of the droplet rim are analyzed, and the results show that they are significantly affected by the shear via the formation and the growth of nodes on the rim.

Mobile traffic is projected to increase 1000 times from 2010 to 2020. This poses significant challenges on the 5th generation (5G) wireless communication system design, including network structure, air interface, key transmission schemes, multiple access, and duplexing schemes. In this paper, full duplex networking issues are discussed, aiming to provide some insights on the design and possible future deployment for 5G. Particularly, the interference scenarios in full duplex are analyzed, followed by discussions on several candidate interference mitigation approaches, interference proof frame structures, transceiver structures for channel reciprocity recovery, and super full duplex base station where each sector operates in time division duplex (TDD) mode. The extension of TDD and frequency division duplex (FDD) to full duplex is also examined. It is anticipated that with future standardization and deployment of full duplex systems, TDD and FDD will be harmoniously integrated, supporting all the existing half duplex mobile phones efficiently, and leading to a substantially enhanced 5G system performance., Comment: This paper has been withdrawn by the author due to there are some errors in this paper