Search

Data annotation refers to the labeling or tagging of textual data with relevant information. A large body of works have reported positive results on leveraging LLMs as an alternative to human annotators. However, existing studies focus on classic NLP tasks, and the extent to which LLMs as data annotators perform in domains requiring expert knowledge remains underexplored. In this work, we investigate comprehensive approaches across three highly specialized domains and discuss practical suggestions from a cost-effectiveness perspective. To the best of our knowledge, we present the first systematic evaluation of LLMs as expert-level data annotators., Comment: Accepted to WiML @ NeurIPS 2024 (extended version)

The advancement of text generation models has granted us the capability to produce coherent and convincing text on demand. Yet, in real-life circumstances, individuals do not continuously generate text or voice their opinions. For instance, consumers pen product reviews after weighing the merits and demerits of a product, and professional analysts issue reports following significant news releases. In essence, opinion expression is typically prompted by particular reasons or signals. Despite long-standing developments in opinion mining, the appropriate timing for expressing an opinion remains largely unexplored. To address this deficit, our study introduces an innovative task - the identification of news-triggered opinion expressing timing. We ground this task in the actions of professional stock analysts and develop a novel dataset for investigation. Our approach is decision-focused, leveraging text generation models to steer the classification model, thus enhancing overall performance. Our experimental findings demonstrate that the text generated by our model contributes fresh insights from various angles, effectively aiding in identifying the optimal timing for opinion expression., Comment: Accepted: COLING-2025

Training large language models (LLMs) from scratch is an expensive endeavor, particularly as world knowledge continually evolves. To maintain relevance and accuracy of LLMs, model editing has emerged as a pivotal research area. While these methods hold promise, they can also produce unintended side effects. Their underlying factors and causes remain largely unexplored. This paper delves into a critical factor-question type-by categorizing model editing questions. Our findings reveal that the extent of performance degradation varies significantly across different question types, providing new insights for experimental design in knowledge editing. Furthermore, we investigate whether insights from smaller models can be extrapolated to larger models. Our results indicate discrepancies in findings between models of different sizes, suggesting that insights from smaller models may not necessarily apply to larger models. Additionally, we examine the impact of batch size on side effects, discovering that increasing the batch size can mitigate performance drops.

In diverse professional environments, ranging from academic conferences to corporate earnings calls, the ability to anticipate audience questions stands paramount. Traditional methods, which rely on manual assessment of an audience's background, interests, and subject knowledge, often fall short - particularly when facing large or heterogeneous groups, leading to imprecision and inefficiency. While NLP has made strides in text-based question generation, its primary focus remains on academic settings, leaving the intricate challenges of professional domains, especially earnings call conferences, underserved. Addressing this gap, our paper pioneers the multi-question generation (MQG) task specifically designed for earnings call contexts. Our methodology involves an exhaustive collection of earnings call transcripts and a novel annotation technique to classify potential questions. Furthermore, we introduce a retriever-enhanced strategy to extract relevant information. With a core aim of generating a spectrum of potential questions that analysts might pose, we derive these directly from earnings call content. Empirical evaluations underscore our approach's edge, revealing notable excellence in the accuracy, consistency, and perplexity of the questions generated.

Understanding the duration of news events' impact on the stock market is crucial for effective time-series forecasting, yet this facet is largely overlooked in current research. This paper addresses this research gap by introducing a novel dataset, the Impact Duration Estimation Dataset (IDED), specifically designed to estimate impact duration based on investor opinions. Our research establishes that pre-finetuning language models with IDED can enhance performance in text-based stock movement predictions. In addition, we juxtapose our proposed pre-finetuning task with sentiment analysis pre-finetuning, further affirming the significance of learning impact duration. Our findings highlight the promise of this novel research direction in stock movement prediction, offering a new avenue for financial forecasting. We also provide the IDED and pre-finetuned language models under the CC BY-NC-SA 4.0 license for academic use, fostering further exploration in this field., Comment: NTCIR-18 FinArg-2 Dataset

In the era of rapid Internet and social media platform development, individuals readily share their viewpoints online. The overwhelming quantity of these posts renders comprehensive analysis impractical. This necessitates an efficient recommendation system to filter and present significant, relevant opinions. Our research introduces a dual-pronged argument mining technique to improve recommendation system effectiveness, considering both professional and amateur investor perspectives. Our first strategy involves using the discrepancy between target and closing prices as an opinion indicator. The second strategy applies argument mining principles to score investors' opinions, subsequently ranking them by these scores. Experimental results confirm the effectiveness of our approach, demonstrating its ability to identify opinions with higher profit potential. Beyond profitability, our research extends to risk analysis, examining the relationship between recommended opinions and investor behaviors. This offers a holistic view of potential outcomes following the adoption of these recommended opinions.

The rapid advancement of drone technology has made accurate Unmanned Aerial Vehicle (UAV) detection essential for surveillance, security, and airspace management. This paper presents a novel trajectory-guided approach, the Patch Intensity Convergence (PIC) technique, which generates high-fidelity bounding boxes for UAV detection without manual labeling. This technique forms the foundation of UAVDB, a dedicated database designed specifically for UAV detection. Unlike datasets that often focus on large UAVs or simple backgrounds, UAVDB utilizes high-resolution RGB video to capture UAVs at various scales, from hundreds of pixels to near-single-digit sizes. This extensive scale variation enables robust evaluation of detection algorithms under diverse conditions. Using the PIC technique, bounding boxes can be efficiently generated from trajectory or position data. We benchmark UAVDB using state-of-the-art (SOTA) YOLO series detectors, providing a comprehensive performance analysis. Our results demonstrate UAVDB's potential as a critical resource for advancing UAV detection, particularly in high-resolution and long-distance tracking scenarios., Comment: 8 pages, 6 figures, 3 tables

Topology and superconductivity, two distinct phenomena offer unique insight into quantum properties and their applications in quantum technologies, spintronics, and sustainable energy technologies if system can be found where they coexist. Tin (Sn) plays a pivotal role here as an element due to its two structural phases, $\alpha$-Sn and $\beta$-Sn, exhibiting topological characteristics ($\alpha$-Sn) and superconductivity ($\beta$-Sn). In this study we show how precise control of $\alpha$ and $\beta$ phases of Sn thin films can be achieved by using molecular beam epitaxy grown buffer layers with systematic control over the lattice parameter. The resulting Sn films showed either $\beta$-Sn or $\alpha$-Sn phases as the lattice constant of the buffer layer was varied from 6.10 A to 6.48 A, covering the range between GaSb (closely matched to InAs) and InSb. The crystal structures of the $\alpha$- and $\beta$-Sn films were characterized by x-ray diffraction and confirmed by Raman spectroscopy and scanning transmission electron microscopy. The smooth and continuous surface morphology of the Sn films was validated using atomic force microscopy. The characteristics of $\alpha$- and $\beta$-Sn phases were further verified using electrical transport measurements by observing resistance drop near 3.7 K for superconductivity of the $\beta$-Sn phase and Shubnikov-de Haas oscillations for the $\alpha$-Sn phase. Density functional theory calculations showed that the stability of the Sn phases is highly dependent on lattice strain, with $\alpha$-Sn being more stable under tensile strain and $\beta$-Sn becoming favorable under compressive strain, which is in good agreement with experimental observations. Hence, this study sheds light on controlling Sn phases through lattice engineering, enabling innovative applications in quantum technologies and beyond.

The field of spintronics has seen a surge of interest in altermagnetism due to novel predictions and many possible applications. MnTe is a leading altermagnetic candidate that is of significant interest across spintronics due to its layered antiferromagnetic structure, high Neel temperature (TN ~ 310 K) and semiconducting properties. We present results on molecular beam epitaxy (MBE) grown MnTe/InP(111) films. Here, it is found that the electronic and magnetic properties are driven by the natural stoichiometry of MnTe. Electronic transport and in situ angle-resolved photoemission spectroscopy show the films are natively metallic with the Fermi level in the valence band and the band structure is in good agreement with first principles calculations for altermagnetic spin-splitting. Neutron diffraction confirms that the film is antiferromagnetic with planar anisotropy and polarized neutron reflectometry indicates weak ferromagnetism, which is linked to a slight Mn-richness that is intrinsic to the MBE grown samples. When combined with the anomalous Hall effect, this work shows that the electronic response is strongly affected by the ferromagnetic moment. Altogether, this highlights potential mechanisms for controlling altermagnetic ordering for diverse spintronic applications., Comment: Accepted in Advanced Functional Materials

In this paper, we investigate the phenomena of "selection biases" in Large Language Models (LLMs), focusing on problems where models are tasked with choosing the optimal option from an ordered sequence. We delve into biases related to option order and token usage, which significantly impact LLMs' decision-making processes. We also quantify the impact of these biases through an extensive empirical analysis across multiple models and tasks. Furthermore, we propose mitigation strategies to enhance model performance. Our key contributions are threefold: 1) Precisely quantifying the influence of option order and token on LLMs, 2) Developing strategies to mitigate the impact of token and order sensitivity to enhance robustness, and 3) Offering a detailed analysis of sensitivity across models and tasks, which informs the creation of more stable and reliable LLM applications for selection problems., Comment: Accepted as a long findings paper at ACL 2024

Realizing topological superconductivity by integrating high-transition-temperature ($T_C$) superconductors with topological insulators can open new paths for quantum computing applications. Here, we report a new approach for increasing the superconducting transition temperature ($T_{C}^{onset}$) by interfacing the unconventional superconductor Fe(Te,Se) with the topological insulator Bi-Te system in the low-Se doping regime, near where superconductivity vanishes in the bulk. The critical finding is that the $T_{C}^{onset}$ of Fe(Te,Se) increases from nominally non-superconducting to as high as 12.5 K when $Bi_2Te_3$ is replaced with the topological phase $Bi_4Te_3$. Interfacing Fe(Te,Se) with $Bi_4Te_3$ is also found to be critical for stabilizing superconductivity in monolayer films where $T_{C}^{onset}$ can be as high as 6 K. Measurements of the electronic and crystalline structure of the $Bi_4Te_3$ layer reveal that a large electron transfer, epitaxial strain, and novel chemical reduction processes are critical factors for the enhancement of superconductivity. This novel route for enhancing $T_C$ in an important epitaxial system provides new insight on the nature of interfacial superconductivity and a platform to identify and utilize new electronic phases., Comment: 18 pages, 5 figures, accepted by Advanced Materials

Automated data labeling techniques are crucial for accelerating the development of deep learning models, particularly in complex medical imaging applications. However, ensuring accuracy and efficiency remains challenging. This paper presents iterative refinement strategies for automated data labeling in facial landmark diagnosis to enhance accuracy and efficiency for deep learning models in medical applications, including dermatology, plastic surgery, and ophthalmology. Leveraging feedback mechanisms and advanced algorithms, our approach iteratively refines initial labels, reducing reliance on manual intervention while improving label quality. Through empirical evaluation and case studies, we demonstrate the effectiveness of our proposed strategies in deep learning tasks across medical imaging domains. Our results highlight the importance of iterative refinement in automated data labeling to enhance the capabilities of deep learning systems in medical imaging applications.

In the realm of continual learning, the presence of noisy labels within data streams represents a notable obstacle to model reliability and fairness. We focus on the data stream scenario outlined in pertinent literature, characterized by fuzzy task boundaries and noisy labels. To address this challenge, we introduce a novel and intuitive sampling method called Noisy Test Debiasing (NTD) to mitigate noisy labels in evolving data streams and establish a fair and robust continual learning algorithm. NTD is straightforward to implement, making it feasible across various scenarios. Our experiments benchmark four datasets, including two synthetic noise datasets (CIFAR10 and CIFAR100) and real-world noise datasets (mini-WebVision and Food-101N). The results validate the efficacy of NTD for online continual learning in scenarios with noisy labels in data streams. Compared to the previous leading approach, NTD achieves a training speedup enhancement over two times while maintaining or surpassing accuracy levels. Moreover, NTD utilizes less than one-fifth of the GPU memory resources compared to previous leading methods.

In this paper, we address the hallucination problem commonly found in natural language generation tasks. Language models often generate fluent and convincing content but can lack consistency with the provided source, resulting in potential inaccuracies. We propose a new decoding method called Fidelity-Enriched Contrastive Search (FECS), which augments the contrastive search framework with context-aware regularization terms. FECS promotes tokens that are semantically similar to the provided source while penalizing repetitiveness in the generated text. We demonstrate its effectiveness across two tasks prone to hallucination: abstractive summarization and dialogue generation. Results show that FECS consistently enhances faithfulness across various language model sizes while maintaining output diversity comparable to well-performing decoding algorithms., Comment: Accepted as a short paper at EMNLP 2023

Headline generation, a key task in abstractive summarization, strives to condense a full-length article into a succinct, single line of text. Notably, while contemporary encoder-decoder models excel based on the ROUGE metric, they often falter when it comes to the precise generation of numerals in headlines. We identify the lack of datasets providing fine-grained annotations for accurate numeral generation as a major roadblock. To address this, we introduce a new dataset, the NumHG, and provide over 27,000 annotated numeral-rich news articles for detailed investigation. Further, we evaluate five well-performing models from previous headline generation tasks using human evaluation in terms of numerical accuracy, reasonableness, and readability. Our study reveals a need for improvement in numerical accuracy, demonstrating the potential of the NumHG dataset to drive progress in number-focused headline generation and stimulate further discussions in numeral-focused text generation., Comment: NumEval@SemEval-2024 Dataset

We explore the extension of chain-of-thought (CoT) prompting to medical reasoning for the task of automatic diagnosis. Motivated by doctors' underlying reasoning process, we present Diagnostic-Reasoning CoT (DR-CoT). Empirical results demonstrate that by simply prompting large language models trained only on general text corpus with two DR-CoT exemplars, the diagnostic accuracy improves by 15% comparing to standard prompting. Moreover, the gap reaches a pronounced 18% in out-domain settings. Our findings suggest expert-knowledge reasoning in large language models can be elicited through proper promptings., Comment: Accepted as a Tiny Paper at ICLR 2023 (10 pages, 5 figures)

This article introduces a novel approach to shuttlecock hitting event detection. Instead of depending on generic methods, we capture the hitting action of players by reasoning over a sequence of images. To learn the features of hitting events in a video clip, we specifically utilize a deep learning model known as SwingNet. This model is designed to capture the relevant characteristics and patterns associated with the act of hitting in badminton. By training SwingNet on the provided video clips, we aim to enable the model to accurately recognize and identify the instances of hitting events based on their distinctive features. Furthermore, we apply the specific video processing technique to extract the prior features from the video, which significantly reduces the learning difficulty for the model. The proposed method not only provides an intuitive and user-friendly approach but also presents a fresh perspective on the task of detecting badminton hitting events. The source code will be available at https://github.com/TW-yuhsi/A-New-Perspective-for-Shuttlecock-Hitting-Event-Detection.

Large language models (LLMs) have exhibited striking in-context learning (ICL) ability to adapt to target tasks with a few input-output demonstrations. For better ICL, different methods are proposed to select representative demonstrations from existing training corpora. However, such settings are not aligned with real-world practices, as end-users usually query LMs without access to demonstration pools. In this work, we introduce Self-ICL -- a simple framework which bootstraps LMs' intrinsic capabilities to perform zero-shot ICL. Given a test input, Self-ICL first prompts the model to generate pseudo-inputs. Next, the model predicts pseudo-labels for the pseudo-inputs via zero-shot prompting. Finally, we perform ICL for the test input with the pseudo-input-label pairs as demonstrations. Evaluation on 23 BIG-Bench Hard tasks shows Self-ICL outperforms zero-shot baselines on both average accuracy and head-to-head comparison. Moreover, with zero-shot chain-of-thought, Self-ICL achieves results comparable to using real demonstrations. Additionally, we conduct a range of analyses to validate Self-ICL's effectiveness and provide insights for its behaviors under different settings., Comment: Accepted as a long paper at EMNLP 2023

Language models (LMs) that jointly generate end-task answers as well as free-text rationales are known as self-rationalization models. Recent works demonstrate great performance gain for self-rationalization by few-shot prompting LMs with rationale-augmented exemplars. However, the ability to benefit from explanations only emerges with large-scale LMs, which have poor accessibility. In this work, we explore the less-studied setting of leveraging explanations for small LMs to improve few-shot self-rationalization. We first revisit the relationship between rationales and answers. Inspired by the implicit mental process of how human beings assess explanations, we present a novel approach, Zero-shot Augmentation of Rationale-Answer pairs (ZARA), to automatically construct pseudo-parallel data for self-training by reducing the problem of plausibility judgement to natural language inference. Experimental results show ZARA achieves SOTA performance on the FEB benchmark, for both the task accuracy and the explanation metric. In addition, we conduct human and quantitative evaluation validating ZARA's ability to automatically identify plausible and accurate rationale-answer pairs., Comment: Accepted as a long paper at EMNLP Findings 2023

Lifelogging has gained more attention due to its wide applications, such as personalized recommendations or memory assistance. The issues of collecting and extracting personal life events have emerged. People often share their life experiences with others through conversations. However, extracting life events from conversations is rarely explored. In this paper, we present Life Event Dialog, a dataset containing fine-grained life event annotations on conversational data. In addition, we initiate a novel conversational life event extraction task and differentiate the task from the public event extraction or the life event extraction from other sources like microblogs. We explore three information extraction (IE) frameworks to address the conversational life event extraction task: OpenIE, relation extraction, and event extraction. A comprehensive empirical analysis of the three baselines is established. The results suggest that the current event extraction model still struggles with extracting life events from human daily conversations. Our proposed life event dialog dataset and in-depth analysis of IE frameworks will facilitate future research on life event extraction from conversations., Comment: Accepted to EACL 2023 Findings

Fake news and misinformation spread rapidly on the Internet. How to identify it and how to interpret the identification results have become important issues. In this paper, we propose a Dual Co-Attention Network (Dual-CAN) for fake news detection, which takes news content, social media replies, and external knowledge into consideration. Our experimental results support that the proposed Dual-CAN outperforms current representative models in two benchmark datasets. We further make in-depth discussions by comparing how models work in both datasets with empirical analysis of attention weights., Comment: EACL 2023 Findings

Making our research results positively impact on society and environment is one of the goals our community has been pursuing recently. Although financial technology (FinTech) is one of the popular application fields, we notice that there is no discussion on how NLP can help in FinTech for the social good. When mentioning FinTech for social good, people are talking about financial inclusion and green finance. However, the role of NLP in these directions only gets limited discussions. To fill this gap, this paper shares our idea of how we can use NLP in FinTech for social good. We hope readers can rethink the relationship between finance and NLP based on our sharing, and further join us in improving the financial literacy of individual investors and improving the supports for impact investment.

We propose a post-processor, called NeighborTrack, that leverages neighbor information of the tracking target to validate and improve single-object tracking (SOT) results. It requires no additional data or retraining. Instead, it uses the confidence score predicted by the backbone SOT network to automatically derive neighbor information and then uses this information to improve the tracking results. When tracking an occluded target, its appearance features are untrustworthy. However, a general siamese network often cannot tell whether the tracked object is occluded by reading the confidence score alone, because it could be misled by neighbors with high confidence scores. Our proposed NeighborTrack takes advantage of unoccluded neighbors' information to reconfirm the tracking target and reduces false tracking when the target is occluded. It not only reduces the impact caused by occlusion, but also fixes tracking problems caused by object appearance changes. NeighborTrack is agnostic to SOT networks and post-processing methods. For the VOT challenge dataset commonly used in short-term object tracking, we improve three famous SOT networks, Ocean, TransT, and OSTrack, by an average of ${1.92\%}$ EAO and ${2.11\%}$ robustness. For the mid- and long-term tracking experiments based on OSTrack, we achieve state-of-the-art ${72.25\%}$ AUC on LaSOT and ${75.7\%}$ AO on GOT-10K. Code duplication can be found in https://github.com/franktpmvu/NeighborTrack., Comment: This paper was accepted by 9th International Workshop on Computer Vision in Sports (CVsports) 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)

We reconstruct the information quantities in the holographic relaxed superconductor system and discuss how these quantities behave under non-symmetry configuration. We then combine the effect of the superconductor and the momentum relaxation, and find out that the superconductor effect will change how the momentum relaxation affect these information quantities.

In this work, we consider a multi-user mobile edge computing system with multiple computing access points (CAPs). Each mobile user has multiple dependent tasks that must be processed in a round-by-round schedule. In every round, a user may process their individual task locally, or choose to offload their task to one of the $M$ CAPs or the remote cloud server, in order to possibly reduce their processing cost. We aim to jointly optimize the offloading decisions of the users and the resource allocation decisions for each CAP over a global objective function, defined as a weighted sum of total energy consumption and the round time. We first present a centralized heuristic solution, termed MCAP, where the original problem is relaxed to a semi-definite program (SDP) to probabilistically generate the offloading decision. Then, recognizing that the users often exhibit selfish behavior to reduce their individual cost, we propose a game-theoretical approach, termed MCAP-NE, which allows us to compute a Nash Equilibrium (NE) through a finite improvement method starting from the previous SDP solution. This approach leads to a solution from which the users have no incentive to deviate, with substantially reduced NE computation time. In simulation, we compare the system cost of the NE solution with those of MCAP, MCAP-NE, a random mapping, and the optimal solution, showing that our NE solution attains near optimal performance under a wide set of parameter settings, as well as demonstrating the advantages of using MCAP to produce the initial point for MCAP-NE., Comment: 13 pages, 11 figures

Opinion mining is a prevalent research issue in many domains. In the financial domain, however, it is still in the early stages. Most of the researches on this topic only focus on the coarse-grained market sentiment analysis, i.e., 2-way classification for bullish/bearish. Thanks to the recent financial technology (FinTech) development, some interdisciplinary researchers start to involve in the in-depth analysis of investors' opinions. In this position paper, we first define the financial opinions from both coarse-grained and fine-grained points of views, and then provide an overview on the issues already tackled. In addition to listing research issues of the existing topics, we further propose a road map of fine-grained financial opinion mining for future researches, and point out several challenges yet to explore. Moreover, we provide possible directions to deal with the proposed research issues.

With the advance of science and technology, people are used to record their daily life events via writing blogs, uploading social media posts, taking photos, or filming videos. Such rich repository personal information is useful for supporting human living assistance. The main challenge is how to store and manage personal knowledge from various sources. In this position paper, we propose a research agenda on mining personal knowledge from various sources of lifelogs, personal knowledge base construction, and information recall for assisting people to recall their experiences.

Financial Technology (FinTech) is one of the worldwide rapidly-rising topics in the past five years according to the statistics of FinTech from Google Trends. In this position paper, we focus on the researches applying natural language processing (NLP) technologies in the finance domain. Our goal is to indicate the position we are now and provide the blueprint for future researches. We go through the application scenarios from three aspects including Know Your Customer (KYC), Know Your Product (KYP), and Satisfy Your Customer (SYC). Both formal documents and informal textual data are analyzed to understand corporate customers and personal customers. Furthermore, we talk over how to dynamically update the features of products from the prospect and the risk points of view. Finally, we discuss satisfying the customers in both B2C and C2C business models. After summarizing the past and the recent challenges, we highlight several promising future research directions in the trend of FinTech and the open finance tendency.

We present an approach to overcoming challenges in dynamical dexterity for robots through tunable origami structures. Our work leverages a one-parameter family of flat sheet crease patterns that folds into origami bellows, whose axial compliance can be tuned to select desired stiffness. Concentrically arranged cylinder pairs reliably manifest additive stiffness, extending the tunable range by nearly an order of magnitude and achieving bulk axial stiffness spanning 200-1500 N/m using 8 mil thick polyester-coated paper. Accordingly, we design origami energy-storing springs with a stiffness of 1035 N/m each and incorporate them into a three degree-of-freedom (DOF) tendon-driven spatial pointing mechanism that exhibits trajectory tracking accuracy less than 15% rms error within a ~2 cm^3 volume. The origami springs can sustain high power throughput, enabling the robot to achieve asymptotically stable juggling for both highly elastic (1~kg resilient shot put ball) and highly damped ("medicine ball") collisions in the vertical direction with apex heights approaching 10 cm. The results demonstrate that "soft" robotic mechanisms are able to perform a controlled, dynamically actuated task., Comment: This paper is submitted to the IEEE Robotics and Automation Letters, in review