17,534 results on '"Cheng, Hao"'
Search Results
102. Registered report adoption in academic journals: assessing rates in different research domains
- Author
-
Lin, Ting-Yu, Cheng, Hao-Chien, Cheng, Li-Fu, and Hung, Tsung-Min
- Published
- 2024
- Full Text
- View/download PDF
103. Optimum splitting computing for DNN training through next generation smart networks: a multi-tier deep reinforcement learning approach
- Author
-
Lien, Shao-Yu, Yeh, Cheng-Hao, and Deng, Der-Jiunn
- Published
- 2024
- Full Text
- View/download PDF
104. The costs and opportunities of overload: exploring the double-edged sword effect of unreasonable tasks on thriving at work
- Author
-
Li, Zhen, Cheng, Hao, Gao, Rong, Teng, Rongrong, Zhao, Junshu, Yue, Longhua, Li, Fangfang, and Liao, Qianyi
- Published
- 2024
- Full Text
- View/download PDF
105. Gaining the Sparse Rewards by Exploring Lottery Tickets in Spiking Neural Network
- Author
-
Cheng, Hao, Cao, Jiahang, Xiao, Erjia, Sun, Mengshu, and Xu, Renjing
- Subjects
Computer Science - Neural and Evolutionary Computing ,Computer Science - Computer Vision and Pattern Recognition - Abstract
Deploying energy-efficient deep learning algorithms on computational-limited devices, such as robots, is still a pressing issue for real-world applications. Spiking Neural Networks (SNNs), a novel brain-inspired algorithm, offer a promising solution due to their low-latency and low-energy properties over traditional Artificial Neural Networks (ANNs). Despite their advantages, the dense structure of deep SNNs can still result in extra energy consumption. The Lottery Ticket Hypothesis (LTH) posits that within dense neural networks, there exist winning Lottery Tickets (LTs), namely sub-networks, that can be obtained without compromising performance. Inspired by this, this paper delves into the spiking-based LTs (SLTs), examining their unique properties and potential for extreme efficiency. Then, two significant sparse \textbf{\textit{Rewards}} are gained through comprehensive explorations and meticulous experiments on SLTs across various dense structures. Moreover, a sparse algorithm tailored for spiking transformer structure, which incorporates convolution operations into the Patch Embedding Projection (ConvPEP) module, has been proposed to achieve Multi-level Sparsity (MultiSp). MultiSp refers to (1) Patch number sparsity; (2) ConvPEP weights sparsity and binarization; and (3) ConvPEP activation layer binarization. Extensive experiments demonstrate that our method achieves extreme sparsity with only a slight performance decrease, paving the way for deploying energy-efficient neural networks in robotics and beyond., Comment: This paper is under submission
- Published
- 2023
106. RBFormer: Improve Adversarial Robustness of Transformer by Robust Bias
- Author
-
Cheng, Hao, Duan, Jinhao, Li, Hui, Zhang, Lyutianyang, Cao, Jiahang, Wang, Ping, Zhang, Jize, Xu, Kaidi, and Xu, Renjing
- Subjects
Computer Science - Computer Vision and Pattern Recognition - Abstract
Recently, there has been a surge of interest and attention in Transformer-based structures, such as Vision Transformer (ViT) and Vision Multilayer Perceptron (VMLP). Compared with the previous convolution-based structures, the Transformer-based structure under investigation showcases a comparable or superior performance under its distinctive attention-based input token mixer strategy. Introducing adversarial examples as a robustness consideration has had a profound and detrimental impact on the performance of well-established convolution-based structures. This inherent vulnerability to adversarial attacks has also been demonstrated in Transformer-based structures. In this paper, our emphasis lies on investigating the intrinsic robustness of the structure rather than introducing novel defense measures against adversarial attacks. To address the susceptibility to robustness issues, we employ a rational structure design approach to mitigate such vulnerabilities. Specifically, we enhance the adversarial robustness of the structure by increasing the proportion of high-frequency structural robust biases. As a result, we introduce a novel structure called Robust Bias Transformer-based Structure (RBFormer) that shows robust superiority compared to several existing baseline structures. Through a series of extensive experiments, RBFormer outperforms the original structures by a significant margin, achieving an impressive improvement of +16.12% and +5.04% across different evaluation criteria on CIFAR-10 and ImageNet-1k, respectively., Comment: BMVC 2023
- Published
- 2023
107. AceGPT, Localizing Large Language Models in Arabic
- Author
-
Huang, Huang, Yu, Fei, Zhu, Jianqing, Sun, Xuening, Cheng, Hao, Song, Dingjie, Chen, Zhihong, Alharthi, Abdulmohsen, An, Bang, He, Juncai, Liu, Ziche, Zhang, Zhiyi, Chen, Junying, Li, Jianquan, Wang, Benyou, Zhang, Lian, Sun, Ruoyu, Wan, Xiang, Li, Haizhou, and Xu, Jinchao
- Subjects
Computer Science - Computation and Language - Abstract
This paper is devoted to the development of a localized Large Language Model (LLM) specifically for Arabic, a language imbued with unique cultural characteristics inadequately addressed by current mainstream models. Significant concerns emerge when addressing cultural sensitivity and local values. To address this, the paper proposes a comprehensive solution that includes further pre-training with Arabic texts, Supervised Fine-Tuning (SFT) utilizing native Arabic instructions, and GPT-4 responses in Arabic, alongside Reinforcement Learning with AI Feedback (RLAIF) employing a reward model attuned to local culture and values. The goal is to cultivate culturally cognizant and value-aligned Arabic LLMs capable of accommodating the diverse, application-specific needs of Arabic-speaking communities. Comprehensive evaluations reveal that the resulting model, dubbed `AceGPT', sets the state-of-the-art standard for open Arabic LLMs across various benchmarks. Codes, data, and models are in https://github.com/FreedomIntelligence/AceGPT., Comment: Accepted to NAACL main conference. https://github.com/FreedomIntelligence/AceGPT
- Published
- 2023
108. Privacy Amplification Against Quantum Side Information Via Regular Random Binning
- Author
-
Shen, Yu-Chen, Gao, Li, and Cheng, Hao-Chung
- Subjects
Quantum Physics - Abstract
We consider privacy amplification against quantum side information by using regular random binning as an effective extractor. For constant-type sources, we obtain error exponent and strong converse bounds in terms of the so-called quantum Augustin information. Via type decomposition, we then recover the error exponent for independent and identically distributed sources proved by Dupuis [arXiv:2105.05342]. As an application, we obtain an achievable secrecy exponent for classical-quantum wiretap channel coding in terms of the Augustin information, which solves an open problem in [IEEE Trans.~Inf.~Theory, 65(12):7985, 2019]. Our approach is to establish an operational equivalence between privacy amplification and quantum soft covering; this may be of independent interest., Comment: 10 pages, submitted to 2023 IEEE Allerton Conference
- Published
- 2023
109. ImGeoNet: Image-induced Geometry-aware Voxel Representation for Multi-view 3D Object Detection
- Author
-
Tu, Tao, Chuang, Shun-Po, Liu, Yu-Lun, Sun, Cheng, Zhang, Ke, Roy, Donna, Kuo, Cheng-Hao, and Sun, Min
- Subjects
Computer Science - Computer Vision and Pattern Recognition - Abstract
We propose ImGeoNet, a multi-view image-based 3D object detection framework that models a 3D space by an image-induced geometry-aware voxel representation. Unlike previous methods which aggregate 2D features into 3D voxels without considering geometry, ImGeoNet learns to induce geometry from multi-view images to alleviate the confusion arising from voxels of free space, and during the inference phase, only images from multiple views are required. Besides, a powerful pre-trained 2D feature extractor can be leveraged by our representation, leading to a more robust performance. To evaluate the effectiveness of ImGeoNet, we conduct quantitative and qualitative experiments on three indoor datasets, namely ARKitScenes, ScanNetV2, and ScanNet200. The results demonstrate that ImGeoNet outperforms the current state-of-the-art multi-view image-based method, ImVoxelNet, on all three datasets in terms of detection accuracy. In addition, ImGeoNet shows great data efficiency by achieving results comparable to ImVoxelNet with 100 views while utilizing only 40 views. Furthermore, our studies indicate that our proposed image-induced geometry-aware representation can enable image-based methods to attain superior detection accuracy than the seminal point cloud-based method, VoteNet, in two practical scenarios: (1) scenarios where point clouds are sparse and noisy, such as in ARKitScenes, and (2) scenarios involve diverse object classes, particularly classes of small objects, as in the case in ScanNet200., Comment: ICCV'23; project page: https://ttaoretw.github.io/imgeonet/
- Published
- 2023
110. Tracing the Influence of Predecessors on Trajectory Prediction
- Author
-
Liu, Mengmeng, Cheng, Hao, and Yang, Michael Ying
- Subjects
Computer Science - Robotics - Abstract
In real-world traffic scenarios, agents such as pedestrians and car drivers often observe neighboring agents who exhibit similar behavior as examples and then mimic their actions to some extent in their own behavior. This information can serve as prior knowledge for trajectory prediction, which is unfortunately largely overlooked in current trajectory prediction models. This paper introduces a novel Predecessor-and-Successor (PnS) method that incorporates a predecessor tracing module to model the influence of predecessors (identified from concurrent neighboring agents) on the successor (target agent) within the same scene. The method utilizes the moving patterns of these predecessors to guide the predictor in trajectory prediction. PnS effectively aligns the motion encodings of the successor with multiple potential predecessors in a probabilistic manner, facilitating the decoding process. We demonstrate the effectiveness of PnS by integrating it into a graph-based predictor for pedestrian trajectory prediction on the ETH/UCY datasets, resulting in a new state-of-the-art performance. Furthermore, we replace the HD map-based scene-context module with our PnS method in a transformer-based predictor for vehicle trajectory prediction on the nuScenes dataset, showing that the predictor maintains good prediction performance even without relying on any map information.
- Published
- 2023
111. Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignment
- Author
-
Liu, Yang, Yao, Yuanshun, Ton, Jean-Francois, Zhang, Xiaoying, Guo, Ruocheng, Cheng, Hao, Klochkov, Yegor, Taufiq, Muhammad Faaiz, and Li, Hang
- Subjects
Computer Science - Artificial Intelligence ,Computer Science - Machine Learning - Abstract
Ensuring alignment, which refers to making models behave in accordance with human intentions [1,2], has become a critical task before deploying large language models (LLMs) in real-world applications. For instance, OpenAI devoted six months to iteratively aligning GPT-4 before its release [3]. However, a major challenge faced by practitioners is the lack of clear guidance on evaluating whether LLM outputs align with social norms, values, and regulations. This obstacle hinders systematic iteration and deployment of LLMs. To address this issue, this paper presents a comprehensive survey of key dimensions that are crucial to consider when assessing LLM trustworthiness. The survey covers seven major categories of LLM trustworthiness: reliability, safety, fairness, resistance to misuse, explainability and reasoning, adherence to social norms, and robustness. Each major category is further divided into several sub-categories, resulting in a total of 29 sub-categories. Additionally, a subset of 8 sub-categories is selected for further investigation, where corresponding measurement studies are designed and conducted on several widely-used LLMs. The measurement results indicate that, in general, more aligned models tend to perform better in terms of overall trustworthiness. However, the effectiveness of alignment varies across the different trustworthiness categories considered. This highlights the importance of conducting more fine-grained analyses, testing, and making continuous improvements on LLM alignment. By shedding light on these key dimensions of LLM trustworthiness, this paper aims to provide valuable insights and guidance to practitioners in the field. Understanding and addressing these concerns will be crucial in achieving reliable and ethically sound deployment of LLMs in various applications., Comment: Fixed several typos
- Published
- 2023
112. An End-to-End Framework of Road User Detection, Tracking, and Prediction from Monocular Images
- Author
-
Cheng, Hao, Liu, Mengmeng, and Chen, Lin
- Subjects
Computer Science - Computer Vision and Pattern Recognition - Abstract
Perception that involves multi-object detection and tracking, and trajectory prediction are two major tasks of autonomous driving. However, they are currently mostly studied separately, which results in most trajectory prediction modules being developed based on ground truth trajectories without taking into account that trajectories extracted from the detection and tracking modules in real-world scenarios are noisy. These noisy trajectories can have a significant impact on the performance of the trajectory predictor and can lead to serious prediction errors. In this paper, we build an end-to-end framework for detection, tracking, and trajectory prediction called ODTP (Online Detection, Tracking and Prediction). It adopts the state-of-the-art online multi-object tracking model, QD-3DT, for perception and trains the trajectory predictor, DCENet++, directly based on the detection results without purely relying on ground truth trajectories. We evaluate the performance of ODTP on the widely used nuScenes dataset for autonomous driving. Extensive experiments show that ODPT achieves high performance end-to-end trajectory prediction. DCENet++, with the enhanced dynamic maps, predicts more accurate trajectories than its base model. It is also more robust when compared with other generative and deterministic trajectory prediction models trained on noisy detection results.
- Published
- 2023
113. ReCLIP: Refine Contrastive Language Image Pre-Training with Source Free Domain Adaptation
- Author
-
Hu, Xuefeng, Zhang, Ke, Xia, Lu, Chen, Albert, Luo, Jiajia, Sun, Yuyin, Wang, Ken, Qiao, Nan, Zeng, Xiao, Sun, Min, Kuo, Cheng-Hao, and Nevatia, Ram
- Subjects
Computer Science - Computer Vision and Pattern Recognition ,Computer Science - Machine Learning - Abstract
Large-scale Pre-Training Vision-Language Model such as CLIP has demonstrated outstanding performance in zero-shot classification, e.g. achieving 76.3% top-1 accuracy on ImageNet without seeing any example, which leads to potential benefits to many tasks that have no labeled data. However, while applying CLIP to a downstream target domain, the presence of visual and text domain gaps and cross-modality misalignment can greatly impact the model performance. To address such challenges, we propose ReCLIP, the first source-free domain adaptation method for vision-language models, which does not require any source data or target labeled data. ReCLIP first learns a projection space to mitigate the misaligned visual-text embeddings and learns pseudo labels, and then deploys cross-modality self-training with the pseudo labels, to update visual and text encoders, refine labels and reduce domain gaps and misalignments iteratively. With extensive experiments, we demonstrate ReCLIP reduces the average error rate of CLIP from 30.17% to 25.06% on 22 image classification benchmarks. Code available at https://github.com/michiganleon/ReCLIP_WACV., Comment: Accepted as Oral Paper by 2024 IEEE CVF Winter Conference on Applications of Computer Vision (WACV)
- Published
- 2023
114. LaplaceConfidence: a Graph-based Approach for Learning with Noisy Labels
- Author
-
Chen, Mingcai, Du, Yuntao, Tang, Wei, Zhang, Baoming, Cheng, Hao, Qian, Shuwei, and Wang, Chongjun
- Subjects
Computer Science - Machine Learning - Abstract
In real-world applications, perfect labels are rarely available, making it challenging to develop robust machine learning algorithms that can handle noisy labels. Recent methods have focused on filtering noise based on the discrepancy between model predictions and given noisy labels, assuming that samples with small classification losses are clean. This work takes a different approach by leveraging the consistency between the learned model and the entire noisy dataset using the rich representational and topological information in the data. We introduce LaplaceConfidence, a method that to obtain label confidence (i.e., clean probabilities) utilizing the Laplacian energy. Specifically, it first constructs graphs based on the feature representations of all noisy samples and minimizes the Laplacian energy to produce a low-energy graph. Clean labels should fit well into the low-energy graph while noisy ones should not, allowing our method to determine data's clean probabilities. Furthermore, LaplaceConfidence is embedded into a holistic method for robust training, where co-training technique generates unbiased label confidence and label refurbishment technique better utilizes it. We also explore the dimensionality reduction technique to accommodate our method on large-scale noisy datasets. Our experiments demonstrate that LaplaceConfidence outperforms state-of-the-art methods on benchmark datasets under both synthetic and real-world noise.
- Published
- 2023
115. A Novel Multi-Task Model Imitating Dermatologists for Accurate Differential Diagnosis of Skin Diseases in Clinical Images
- Author
-
Zhou, Yan-Jie, Liu, Wei, Gao, Yuan, Xu, Jing, Lu, Le, Duan, Yuping, Cheng, Hao, Jin, Na, Man, Xiaoyong, Zhao, Shuang, and Wang, Yu
- Subjects
Computer Science - Computer Vision and Pattern Recognition ,Computer Science - Artificial Intelligence - Abstract
Skin diseases are among the most prevalent health issues, and accurate computer-aided diagnosis methods are of importance for both dermatologists and patients. However, most of the existing methods overlook the essential domain knowledge required for skin disease diagnosis. A novel multi-task model, namely DermImitFormer, is proposed to fill this gap by imitating dermatologists' diagnostic procedures and strategies. Through multi-task learning, the model simultaneously predicts body parts and lesion attributes in addition to the disease itself, enhancing diagnosis accuracy and improving diagnosis interpretability. The designed lesion selection module mimics dermatologists' zoom-in action, effectively highlighting the local lesion features from noisy backgrounds. Additionally, the presented cross-interaction module explicitly models the complicated diagnostic reasoning between body parts, lesion attributes, and diseases. To provide a more robust evaluation of the proposed method, a large-scale clinical image dataset of skin diseases with significantly more cases than existing datasets has been established. Extensive experiments on three different datasets consistently demonstrate the state-of-the-art recognition performance of the proposed approach., Comment: MICCAI 2023 early accept
- Published
- 2023
116. Does Collaborative Human-LM Dialogue Generation Help Information Extraction from Human Dialogues?
- Author
-
Lu, Bo-Ru, Haduong, Nikita, Lee, Chia-Hsuan, Wu, Zeqiu, Cheng, Hao, Koester, Paul, Utke, Jean, Yu, Tao, Smith, Noah A., and Ostendorf, Mari
- Subjects
Computer Science - Computation and Language - Abstract
The capabilities of pretrained language models have opened opportunities to explore new application areas, but applications involving human-human interaction are limited by the fact that most data is protected from public release for privacy reasons. Problem-solving human dialogues in real applications can be much more complex than existing Wizard-of-Oz collections, preventing successful domain transfer. To support information extraction (IE) for a private call center dataset, we introduce a human-in-the-loop dialogue generation framework capable of synthesizing realistic dialogues. In IE experiments with auto insurance call center dialogues, we observe 25\% relative improvement in $F_1$ after augmenting a small set of real human conversations with synthetic data. We release code and our synthetic dataset to illustrate the complexity of real-world call center conversations and encourage development of complex dialogue datasets that are more representative of natural data.
- Published
- 2023
117. Shifting Attention to Relevance: Towards the Predictive Uncertainty Quantification of Free-Form Large Language Models
- Author
-
Duan, Jinhao, Cheng, Hao, Wang, Shiqi, Zavalny, Alex, Wang, Chenan, Xu, Renjing, Kailkhura, Bhavya, and Xu, Kaidi
- Subjects
Computer Science - Computation and Language ,Computer Science - Artificial Intelligence ,Computer Science - Machine Learning - Abstract
Large Language Models (LLMs) show promising results in language generation and instruction following but frequently "hallucinate", making their outputs less reliable. Despite Uncertainty Quantification's (UQ) potential solutions, implementing it accurately within LLMs is challenging. Our research introduces a simple heuristic: not all tokens in auto-regressive LLM text equally represent the underlying meaning, as "linguistic redundancy" often allows a few keywords to convey the essence of long sentences. However, current methods underestimate this inequality when assessing uncertainty, causing tokens with limited semantics to be equally or excessively weighted in UQ. To correct this, we propose Shifting Attention to more Relevant (SAR) components at both token- and sentence-levels for better UQ. We conduct extensive experiments involving a range of popular "off-the-shelf" LLMs, such as Vicuna, WizardLM, and LLaMA-2-chat, with model sizes extending up to 33B parameters. We evaluate various free-form question-answering tasks, encompassing domains such as reading comprehension, science Q&A, and medical Q&A. Our experimental results, coupled with a comprehensive demographic analysis, demonstrate the superior performance of SAR. The code is available at https://github.com/jinhaoduan/SAR., Comment: To appear in ACL 2024
- Published
- 2023
118. SERS biosensors based on catalytic hairpin self-assembly and hybridization chain reaction cascade signal amplification strategies for ultrasensitive microRNA-21 detection
- Author
-
Chen, Qiying, Cao, Jinru, Kong, Hongxing, Chen, Ruijue, Wang, Ying, Zhou, Pei, Huang, Wenyi, Cheng, Hao, Li, Lijun, Gao, Si, and Feng, Jun
- Published
- 2024
- Full Text
- View/download PDF
119. Learning functional conservation between human and pig to decipher evolutionary mechanisms underlying gene expression and complex traits.
- Author
-
Li, Jinghui, Zhao, Tianjing, Guan, Dailu, Pan, Zhangyuan, Bai, Zhonghao, Teng, Jinyan, Zhang, Zhe, Zheng, Zhili, Zeng, Jian, Zhou, Huaijun, Fang, Lingzhao, and Cheng, Hao
- Subjects
complex trait ,deep learning ,functional conservation ,gene expression ,human ,pig - Abstract
Assessment of genomic conservation between humans and pigs at the functional level can improve the potential of pigs as a human biomedical model. To address this, we developed a deep learning-based approach to learn the genomic conservation at the functional level (DeepGCF) between species by integrating 386 and 374 functional profiles from humans and pigs, respectively. DeepGCF demonstrated better prediction performance compared with the previous method. In addition, the resulting DeepGCF score captures the functional conservation between humans and pigs by examining chromatin states, sequence ontologies, and regulatory variants. We identified a core set of genomic regions as functionally conserved that plays key roles in gene regulation and is enriched for the heritability of complex traits and diseases in humans. Our results highlight the importance of cross-species functional comparison in illustrating the genetic and evolutionary basis of complex phenotypes.
- Published
- 2023
120. Interpreting single-step genomic evaluation as a neural network of three layers: pedigree, genotypes, and phenotypes.
- Author
-
Zhao, Tianjing and Cheng, Hao
- Subjects
Humans ,Pedigree ,Algorithms ,Models ,Genetic ,Genotype ,Phenotype ,Genomics ,Neural Networks ,Computer ,Polymorphism ,Single Nucleotide - Abstract
The single-step approach has become the most widely-used methodology for genomic evaluations when only a subset of phenotyped individuals in the pedigree are genotyped, where the genotypes for non-genotyped individuals are imputed based on gene contents (i.e., genotypes) of genotyped individuals through their pedigree relationships. We proposed a new method named single-step neural network with mixed models (NNMM) to represent single-step genomic evaluations as a neural network of three sequential layers: pedigree, genotypes, and phenotypes. These three sequential layers of information create a unified network instead of two separate steps, allowing the unobserved gene contents of non-genotyped individuals to be sampled based on pedigree, observed genotypes of genotyped individuals, and phenotypes. In addition to imputation of genotypes using all three sources of information, including phenotypes, genotypes, and pedigree, single-step NNMM provides a more flexible framework to allow nonlinear relationships between genotypes and phenotypes, and for individuals to be genotyped with different single-nucleotide polymorphism (SNP) panels. The single-step NNMM has been implemented in the software package JWAS.
- Published
- 2023
121. CDPK protein in cotton: genomic-wide identification, expression analysis, and conferring resistance to heat stress
- Author
-
Wen-Ben Lv, Cheng-Cheng Miao, Cheng-Hao Du, Ya-Ting Cui, Man Liu, Mei-Chen Shen, Anane Gideon Owusu, Ning Guo, Da-Hui Li, and Jun-Shan Gao
- Subjects
Cotton ,CDPK gene family ,Heat stress ,Tolerance ,Botany ,QK1-989 - Abstract
Abstract Background Calcium-dependent protein kinase (CDPK) plays a key role in cotton tolerance to abiotic stress. However, its role in cotton heat stress tolerance is not well understood. Here, we characterize the GhCDPK gene family and their expression profiles with the aim of identifying CDPK genes associated with heat stress tolerance. Results This study revealed 48 GhCDPK members in the cotton genome, distributed on 18 chromosomes. Tree phylogenetic analysis showed three main clustering groups of the GhCDPKs. Cis-elements revealed many abiotic stress and phytohormone pathways conserved promoter regions. Similarly, analysis of the transcription factor binding sites (TFBDS) in the GhCDPK genes showed many stress and hormone related sites. The expression analysis based on qRT-PCR showed that GhCDPK16 was highly responsive to high-temperature stress. Subsequent protein-protein interactions of GhCDPK16 revealed predictable interaction with ROS generating, calcium binding, and ABA signaling proteins. Overexpression of GhCDPK16 in cotton and Arabidopsis improved thermotolerance by lowering ROS compound buildup. Under heat stress, GhCDPK16 transgenic lines upregulated heat-inducible genes GhHSP70, GHSP17.3, and GhGR1, as demonstrated by qRT-PCR analysis. Contrarily, GhCDPK16 knockout lines in cotton exhibited an increase in ROS accumulation. Furthermore, antioxidant enzyme activity was dramatically boosted in the GhCDPK16-ox transgenic lines. Conclusions The collective findings demonstrated that GhCDPK16 could be a viable gene to enhance thermotolerance in cotton and, therefore, a potential candidate gene for improving heat tolerance in cotton.
- Published
- 2024
- Full Text
- View/download PDF
122. Global, regional, and national burden and trends of migraine among youths and young adults aged 15–39 years from 1990 to 2021: findings from the global burden of disease study 2021
- Author
-
Zhi-feng Chen, Xiang-meng Kong, Cheng-hao Yang, Xin-yu Li, Hong Guo, and Zhao-wei Wang
- Subjects
Migraine ,Global burden of disease ,Disability-adjusted life years ,Socio-demographic index ,Medicine - Abstract
Abstract Background Migraine, a widespread neurological condition, substantially affects the quality of life, particularly for adolescents and young adults. While its impact is significant, there remains a paucity of comprehensive global research on the burden of migraine in younger demographics. Our study sought to elucidate the global prevalence, incidence, and disability-adjusted life-years (DALYs) associated with migraine in the 15–39 age group from 1990 to 2021, utilizing data from the Global Burden of Disease (GBD) 2021 study. Methods Our comprehensive study analyzed migraine data from the GBD 2021 report, examining the prevalence, incidence, and DALYs across 204 countries and territories over a 32-year span. We stratified the information by age, sex, year, geographical region, and Socio-demographic Index (SDI). To evaluate temporal trends in these metrics, we employed the estimated annual percentage change (EAPC) calculation. Results Between 1990 and 2021, the worldwide prevalence of migraine among 15–39 year-olds increased substantially. By 2021, an estimated 593.8 million cases were reported, representing a 39.52% rise from 425.6 million cases in 1990. Global trends showed increases in age-standardized prevalence rate, incidence rate, and DALY rate for migraine during this period. The EAPC were positive for all three metrics: 0.09 for ASPR, 0.03 for ASIR, and 0.09 for DALY rate. Regions with medium SDI reported the highest absolute numbers of prevalent cases, incident cases, and DALYs in 2021. However, high SDI regions demonstrated the most elevated rates overall. Across the globe, migraine prevalence peaked in the 35–39 age group. Notably, female rates consistently exceeded male rates across all age categories. Conclusion The global impact of migraine on youths and young adults has grown considerably from 1990 to 2021, revealing notable variations across SDI regions, countries, age groups, and sexes. This escalating burden necessitates targeted interventions and public health initiatives, especially in areas and populations disproportionately affected by migraine.
- Published
- 2024
- Full Text
- View/download PDF
123. Global, regional, national epidemiology and trends of neglected tropical diseases in youths and young adults aged 15–39 years from 1990 to 2019: findings from the global burden of disease study 2019
- Author
-
Jia-jie Lv, Yi-chi Zhang, Xin-yu Li, Cheng-hao Yang, and Xuhui Wang
- Subjects
Global burden of disease ,Neglected tropical diseases ,Disability-adjusted life years ,Global Burden of Disease ,Social development index ,Public aspects of medicine ,RA1-1270 - Abstract
Abstract Background In recent years, the escalating concern for neglected tropical diseases (NTDs) has been recognized as a pressing global health issue. This concern is acutely manifested in low- and middle-income countries, where there is an escalating prevalence among adolescents and young adults. The burgeoning of these conditions threatens to impair patients' occupational capabilities and overall life quality. Despite the considerable global impact of NTDs, comprehensive studies focusing on their impact in younger populations remain scarce. Our study aims to describe the global prevalence of neglected tropical diseases among people aged 15 to 39 years over the 30-year period from 1990 to 2019, and to project the disease burden of the disease up to 2040. Methods Annual data on incident cases, mortality, and disability-adjusted life years (DALYs) for NTDs were procured from the Global Burden of Disease Study 2019 (GBD 2019). These data were stratified by global and regional distribution, country, social development index (SDI), age, and sex. We computed age-standardized rates (ASRs) and the numbers of incident cases, mortalities, and DALYs from 1990 to 2019. The estimated annual percentage change (EAPC) in the ASRs was calculated to evaluate evolving trends. Results In 2019, it was estimated that there were approximately 552 million NTD cases globally (95% Uncertainty Interval [UI]: 519.9 million to 586.3 million), a 29% decrease since 1990. South Asia reported the highest NTD prevalence, with an estimated 171.7 million cases (95% UI: 150.4 million to 198.6 million). Among the five SDI categories, the prevalence of NTDs was highest in the moderate and low SDI regions in 1990 (approximately 270.5 million cases) and 2019 (approximately 176.5 million cases). Sub-Saharan Africa recorded the most significant decline in NTD cases over the past three decades. Overall, there was a significant inverse correlation between the disease burden of NTDs and SDI. Conclusion NTDs imposed over half a billion incident cases and 10.8 million DALYs lost globally in 2019—exerting an immense toll rivaling major infectious and non-communicable diseases. Encouraging declines in prevalence and disability burdens over the past three decades spotlight the potential to accelerate progress through evidence-based allocation of resources. Such strategic integration could substantially enhance public awareness about risk factors and available treatment options.
- Published
- 2024
- Full Text
- View/download PDF
124. Grain and chaff separation detection method based on machine vision
- Author
-
LI Xin, QI Jiamin, CHENG Hao, and WANG Yanchun
- Subjects
grain and chaff separation ,machine vision ,image processing ,feature extraction ,Food processing and manufacture ,TP368-456 - Abstract
[Objective] To solve the problem of poor manual detection accuracy of traditional grain and chaff separator and improve production efficiency. [Methods] An image detection method based on machine vision was proposed, which realized the feature recognition and separation of grain rough through multi-stage progressive fusion of different image algorithms. The acquired images were selected in the ROI region and enhanced by Retinex algorithm. The Otsu algorithm was used to segment the image, and then the median filtering wwas combined with morphology to remove the image noise. The improved Canny algorithm was used to detect edge features of binary images, and the position information of the contour of the valley rough image was extracted by combining the Hough transform. Finally, the state estimation of the position information was performed by using the Kalman filter, and the best predicted value of the separated position was obtained, while the position offset error was reduced. [Results] The average detection error of the system was 3.14 mm, a decrease of 1.82 mm compared to before filtering, and the average standard deviation of filtering error was 0.8 mm. [Conclusion] This method can effectively detect the grain rough feature information and improve the separation accuracy.
- Published
- 2024
- Full Text
- View/download PDF
125. The burden of non-alcoholic fatty liver disease among working-age people in the Western Pacific Region, 1990–2019: an age–period–cohort analysis of the Global Burden of Disease study
- Author
-
Jia-jie Lv, Yi-chi Zhang, Xin-yu Li, Hong Guo, and Cheng-hao Yang
- Subjects
Global burden of disease ,Non-alcoholic fatty liver disease ,Disability-adjusted life years ,Age-period-cohort analysis ,Trend analysis ,Public aspects of medicine ,RA1-1270 - Abstract
Abstract Background The growing prevalence of non-alcoholic fatty liver disease (NAFLD) in younger populations, particularly those of working age (15–64 years), has become a public health concern. Being diagnosed at a younger age implies a greater likelihood of accruing disability-adjusted life years (DALYs) later in life due to potential progression to conditions such as cirrhosis or hepatocellular carcinoma. This study aims to analyze NAFLD prevalence trends over three decades globally, regionally, and nationally, with a focus on age, period, and birth cohort associations. Methods Global, regional, and country time trends in the prevalence of NAFLD among working-age people from 1990 to 2019: Age-period-cohort analysis based on Global Burden of Disease Study 2019 estimates and 95% uncertainty interval (UI) of NAFLD prevalence in the working age population was extracted from the Global Burden of Diseases, Injuries and Risk Factors Study 2019. Age-period-cohort models were used to estimate the prevalence within each age group from 1990 to 2019 (local drift, % per year), fitted longitudinal age-specific rates adjusted for period bias (age effect), and period/cohort relative risk (period/cohort effect). Results The global age-standardized prevalence (ASPR) of NAFLD increased significantly from 1990 (14,477.6 per 100 000) to 2019 (19,837.6 per 100 000). In the Western Pacific, there were 42,903.8 NAFLD cases in 2019, 54.15% higher than in 1990. The ASPR also increased significantly in the region over the past three decades. At the national level, Palau had the highest ASPR while Brunei Darussalam had the lowest. Age-period-cohort analysis showed that in the Western Pacific, unlike globally, the risk of NAFLD declined after age 60–64 years. Relative to 1980–1989, incidence and DALY risks decreased but prevalence increased in subsequent birth cohorts. Future predictions indicate an upward trend in NAFLD burden, especially among women and medium (SDI) regions like China. Conclusion Non-alcoholic fatty liver disease imparts an immense health burden that continues to grow globally and in the Asia Pacific region. Our work highlights working age adults as an at-risk group and calls attention to socioeconomic gradients within Western Pacific countries. Upward future projections demonstrate that NAFLD prevention is an urgent priority.
- Published
- 2024
- Full Text
- View/download PDF
126. Phenolics from the Leaves and Stems of Caesalpinia enneaphylla
- Author
-
Sun, Qiong-Hui, Yang, Shun-Yi, Yu, Li-Mei, Yan, Wen, Cheng, Hao, Liu, Bo, Aisa, Haji Akber, and Chen, Ye-Gao
- Published
- 2024
- Full Text
- View/download PDF
127. Thompson sampling for improved exploration in GFlowNets
- Author
-
Rector-Brooks, Jarrid, Madan, Kanika, Jain, Moksh, Korablyov, Maksym, Liu, Cheng-Hao, Chandar, Sarath, Malkin, Nikolay, and Bengio, Yoshua
- Subjects
Computer Science - Machine Learning - Abstract
Generative flow networks (GFlowNets) are amortized variational inference algorithms that treat sampling from a distribution over compositional objects as a sequential decision-making problem with a learnable action policy. Unlike other algorithms for hierarchical sampling that optimize a variational bound, GFlowNet algorithms can stably run off-policy, which can be advantageous for discovering modes of the target distribution. Despite this flexibility in the choice of behaviour policy, the optimal way of efficiently selecting trajectories for training has not yet been systematically explored. In this paper, we view the choice of trajectories for training as an active learning problem and approach it using Bayesian techniques inspired by methods for multi-armed bandits. The proposed algorithm, Thompson sampling GFlowNets (TS-GFN), maintains an approximate posterior distribution over policies and samples trajectories from this posterior for training. We show in two domains that TS-GFN yields improved exploration and thus faster convergence to the target distribution than the off-policy exploration strategies used in past work., Comment: Structured Probabilistic Inference and Generative Modeling (SPIGM) workshop @ ICML 2023
- Published
- 2023
128. Spiking Denoising Diffusion Probabilistic Models
- Author
-
Cao, Jiahang, Wang, Ziqing, Guo, Hanzhong, Cheng, Hao, Zhang, Qiang, and Xu, Renjing
- Subjects
Computer Science - Neural and Evolutionary Computing ,Computer Science - Computer Vision and Pattern Recognition - Abstract
Spiking neural networks (SNNs) have ultra-low energy consumption and high biological plausibility due to their binary and bio-driven nature compared with artificial neural networks (ANNs). While previous research has primarily focused on enhancing the performance of SNNs in classification tasks, the generative potential of SNNs remains relatively unexplored. In our paper, we put forward Spiking Denoising Diffusion Probabilistic Models (SDDPM), a new class of SNN-based generative models that achieve high sample quality. To fully exploit the energy efficiency of SNNs, we propose a purely Spiking U-Net architecture, which achieves comparable performance to its ANN counterpart using only 4 time steps, resulting in significantly reduced energy consumption. Extensive experimental results reveal that our approach achieves state-of-the-art on the generative tasks and substantially outperforms other SNN-based generative models, achieving up to 12x and 6x improvement on the CIFAR-10 and the CelebA datasets, respectively. Moreover, we propose a threshold-guided strategy that can further improve the performances by 2.69% in a training-free manner. The SDDPM symbolizes a significant advancement in the field of SNN generation, injecting new perspectives and potential avenues of exploration. Our code is available at https://github.com/AndyCao1125/SDDPM., Comment: Accepted by WACV 2024
- Published
- 2023
129. Explainable Recommendation with Personalized Review Retrieval and Aspect Learning
- Author
-
Cheng, Hao, Wang, Shuo, Lu, Wensheng, Zhang, Wei, Zhou, Mingyang, Lu, Kezhong, and Liao, Hao
- Subjects
Computer Science - Social and Information Networks - Abstract
Explainable recommendation is a technique that combines prediction and generation tasks to produce more persuasive results. Among these tasks, textual generation demands large amounts of data to achieve satisfactory accuracy. However, historical user reviews of items are often insufficient, making it challenging to ensure the precision of generated explanation text. To address this issue, we propose a novel model, ERRA (Explainable Recommendation by personalized Review retrieval and Aspect learning). With retrieval enhancement, ERRA can obtain additional information from the training sets. With this additional information, we can generate more accurate and informative explanations. Furthermore, to better capture users' preferences, we incorporate an aspect enhancement component into our model. By selecting the top-n aspects that users are most concerned about for different items, we can model user representation with more relevant details, making the explanation more persuasive. To verify the effectiveness of our model, extensive experiments on three datasets show that our model outperforms state-of-the-art baselines (for example, 3.4% improvement in prediction and 15.8% improvement in explanation for TripAdvisor).
- Published
- 2023
- Full Text
- View/download PDF
130. Multi-Fidelity Active Learning with GFlowNets
- Author
-
Hernandez-Garcia, Alex, Saxena, Nikita, Jain, Moksh, Liu, Cheng-Hao, and Bengio, Yoshua
- Subjects
Computer Science - Machine Learning ,Quantitative Biology - Biomolecules - Abstract
In the last decades, the capacity to generate large amounts of data in science and engineering applications has been growing steadily. Meanwhile, machine learning has progressed to become a suitable tool to process and utilise the available data. Nonetheless, many relevant scientific and engineering problems present challenges where current machine learning methods cannot yet efficiently leverage the available data and resources. For example, in scientific discovery, we are often faced with the problem of exploring very large, structured and high-dimensional spaces. Moreover, the high fidelity, black-box objective function is often very expensive to evaluate. Progress in machine learning methods that can efficiently tackle such challenges would help accelerate currently crucial areas such as drug and materials discovery. In this paper, we propose a multi-fidelity active learning algorithm with GFlowNets as a sampler, to efficiently discover diverse, high-scoring candidates where multiple approximations of the black-box function are available at lower fidelity and cost. Our evaluation on molecular discovery tasks shows that multi-fidelity active learning with GFlowNets can discover high-scoring candidates at a fraction of the budget of its single-fidelity counterpart while maintaining diversity, unlike RL-based alternatives. These results open new avenues for multi-fidelity active learning to accelerate scientific discovery and engineering design., Comment: Published in Transactions on Machine Learning Research (TMLR) 07/2024 https://openreview.net/forum?id=dLaazW9zuF
- Published
- 2023
131. Augmenting Language Models with Long-Term Memory
- Author
-
Wang, Weizhi, Dong, Li, Cheng, Hao, Liu, Xiaodong, Yan, Xifeng, Gao, Jianfeng, and Wei, Furu
- Subjects
Computer Science - Computation and Language - Abstract
Existing large language models (LLMs) can only afford fix-sized inputs due to the input length limit, preventing them from utilizing rich long-context information from past inputs. To address this, we propose a framework, Language Models Augmented with Long-Term Memory (LongMem), which enables LLMs to memorize long history. We design a novel decoupled network architecture with the original backbone LLM frozen as a memory encoder and an adaptive residual side-network as a memory retriever and reader. Such a decoupled memory design can easily cache and update long-term past contexts for memory retrieval without suffering from memory staleness. Enhanced with memory-augmented adaptation training, LongMem can thus memorize long past context and use long-term memory for language modeling. The proposed memory retrieval module can handle unlimited-length context in its memory bank to benefit various downstream tasks. Typically, LongMem can enlarge the long-form memory to 65k tokens and thus cache many-shot extra demonstration examples as long-form memory for in-context learning. Experiments show that our method outperforms strong long-context models on ChapterBreak, a challenging long-context modeling benchmark, and achieves remarkable improvements on memory-augmented in-context learning over LLMs. The results demonstrate that the proposed method is effective in helping language models to memorize and utilize long-form contents. Our code is open-sourced at https://aka.ms/LongMem.
- Published
- 2023
132. RGBlimp: Robotic Gliding Blimp -- Design, Modeling, Development, and Aerodynamics Analysis
- Author
-
Cheng, Hao, Sha, Zeyu, Zhu, Yongjian, and Zhang, Feitian
- Subjects
Computer Science - Robotics ,Electrical Engineering and Systems Science - Systems and Control - Abstract
A miniature robotic blimp, as one type of lighter-than-air aerial vehicle, has attracted increasing attention in the science and engineering field for its long flight duration and safe aerial locomotion. While a variety of miniature robotic blimps have been developed over the past decade, most of them utilize the buoyant lift and neglect the aerodynamic lift in their design, thus leading to a mediocre aerodynamic performance. This letter proposes a new design of miniature robotic blimp that combines desirable features of both a robotic blimp and a fixed-wing glider, named the Robotic Gliding Blimp, or RGBlimp. This robot, equipped with an envelope filled with helium and a pair of wings, uses an internal moving mass and a pair of propellers for its locomotion control. This letter presents the design, dynamic modeling, prototyping, and system identification of the RGBlimp. To the best of the authors' knowledge, this is the first effort to systematically design and develop such a miniature robotic blimp with hybrid lifts and moving mass control. Experimental results are presented to validate the design and the dynamic model of the RGBlimp. Analysis of the RGBlimp aerodynamics is conducted which confirms the performance improvement of the proposed RGBlimp in aerodynamic efficiency and flight stability.
- Published
- 2023
- Full Text
- View/download PDF
133. DOS: Diverse Outlier Sampling for Out-of-Distribution Detection
- Author
-
Jiang, Wenyu, Cheng, Hao, Chen, Mingcai, Wang, Chongjun, and Wei, Hongxin
- Subjects
Computer Science - Machine Learning - Abstract
Modern neural networks are known to give overconfident prediction for out-of-distribution inputs when deployed in the open world. It is common practice to leverage a surrogate outlier dataset to regularize the model during training, and recent studies emphasize the role of uncertainty in designing the sampling strategy for outlier dataset. However, the OOD samples selected solely based on predictive uncertainty can be biased towards certain types, which may fail to capture the full outlier distribution. In this work, we empirically show that diversity is critical in sampling outliers for OOD detection performance. Motivated by the observation, we propose a straightforward and novel sampling strategy named DOS (Diverse Outlier Sampling) to select diverse and informative outliers. Specifically, we cluster the normalized features at each iteration, and the most informative outlier from each cluster is selected for model training with absent category loss. With DOS, the sampled outliers efficiently shape a globally compact decision boundary between ID and OOD data. Extensive experiments demonstrate the superiority of DOS, reducing the average FPR95 by up to 25.79% on CIFAR-100 with TI-300K., Comment: Accepted by ICLR 2024
- Published
- 2023
134. Lattice-Aided Extraction of Spread-Spectrum Hidden Data
- Author
-
Yang, Fan, Lyu, Shanxiang, Cheng, Hao, Wen, Jinming, and Chen, Hao
- Subjects
Computer Science - Cryptography and Security - Abstract
This paper discusses the problem of extracting spread spectrum hidden data from the perspective of lattice decoding. Since the conventional blind extraction scheme multi-carrier iterative generalize least-squares (M-IGLS) and non-blind extraction scheme minimum mean square error (MMSE) suffer from performance degradation when the carriers lack sufficient orthogonality, we present two novel schemes from the viewpoint of lattice decoding, namely multi-carrier iterative successive interference cancellation (M-ISIC) and sphere decoding (SD). The better performance of M-ISIC and SD are confirmed by both theoretical justification and numerical simulations., Comment: 10 pages
- Published
- 2023
135. Self-Verification Improves Few-Shot Clinical Information Extraction
- Author
-
Gero, Zelalem, Singh, Chandan, Cheng, Hao, Naumann, Tristan, Galley, Michel, Gao, Jianfeng, and Poon, Hoifung
- Subjects
Computer Science - Computation and Language ,Computer Science - Machine Learning - Abstract
Extracting patient information from unstructured text is a critical task in health decision-support and clinical research. Large language models (LLMs) have shown the potential to accelerate clinical curation via few-shot in-context learning, in contrast to supervised learning which requires much more costly human annotations. However, despite drastic advances in modern LLMs such as GPT-4, they still struggle with issues regarding accuracy and interpretability, especially in mission-critical domains such as health. Here, we explore a general mitigation framework using self-verification, which leverages the LLM to provide provenance for its own extraction and check its own outputs. This is made possible by the asymmetry between verification and generation, where the latter is often much easier than the former. Experimental results show that our method consistently improves accuracy for various LLMs in standard clinical information extraction tasks. Additionally, self-verification yields interpretations in the form of a short text span corresponding to each output, which makes it very efficient for human experts to audit the results, paving the way towards trustworthy extraction of clinical information in resource-constrained scenarios. To facilitate future research in this direction, we release our code and prompts.
- Published
- 2023
136. Is Silent eHMI Enough? A Passenger-Centric Study on Effective eHMI for Autonomous Personal Mobility Vehicles in the Field
- Author
-
Liu, Hailong, Li, Yang, Zeng, Zhe, Cheng, Hao, Peng, Chen, and Wada, Takahiro
- Subjects
Computer Science - Human-Computer Interaction - Abstract
Autonomous Personal Mobility Vehicle (APMV) is a miniaturized autonomous vehicle designed to provide short-distance mobility to everyone in pedestrian-rich environments. By the characteristic of the open design, passengers on the APMV are exposed to the communication between the eHMI deployed on APMVs and pedestrians. Therefore, to ensure an optimal passenger experience, eHMI designs for APMVs must consider the potential impact of APMV-pedestrian communications on passengers' subjective feelings. To this end, this study discussed three external human-machine interface (eHMI) designs, i.e., 1) graphical user interface (GUI)-based eHMI with text message (eHMI-T), 2) multimodal user interface (MUI)-based eHMI with neutral voice (eHMI-NV), and 3) MUI-based eHMI with affective voice (eHMI-AV), from the perspective of APMV passengers in the communication between APMV and pedestrians. In the riding field experiment (N=24), we found that eHMI-T may be less suitable for APMVs. This conclusion was drawn based on passengers' feedback, as they expressed an awkward feeling during the "silent time" when the eHMI-T provided information only to pedestrians but not to passengers. Additionally, these two MUI-based eHMIs with voice cues had their own advantages, i.e., eHMI-NV has an advantage in pragmatic quality, while eHMI-AV has an advantage in hedonic quality. The study also highlights the necessity of considering passengers' personalities when desig
- Published
- 2023
- Full Text
- View/download PDF
137. Pre-training Multi-task Contrastive Learning Models for Scientific Literature Understanding
- Author
-
Zhang, Yu, Cheng, Hao, Shen, Zhihong, Liu, Xiaodong, Wang, Ye-Yi, and Gao, Jianfeng
- Subjects
Computer Science - Computation and Language ,Computer Science - Digital Libraries ,Computer Science - Information Retrieval ,Computer Science - Machine Learning - Abstract
Scientific literature understanding tasks have gained significant attention due to their potential to accelerate scientific discovery. Pre-trained language models (LMs) have shown effectiveness in these tasks, especially when tuned via contrastive learning. However, jointly utilizing pre-training data across multiple heterogeneous tasks (e.g., extreme multi-label paper classification, citation prediction, and literature search) remains largely unexplored. To bridge this gap, we propose a multi-task contrastive learning framework, SciMult, with a focus on facilitating common knowledge sharing across different scientific literature understanding tasks while preventing task-specific skills from interfering with each other. To be specific, we explore two techniques -- task-aware specialization and instruction tuning. The former adopts a Mixture-of-Experts Transformer architecture with task-aware sub-layers; the latter prepends task-specific instructions to the input text so as to produce task-aware outputs. Extensive experiments on a comprehensive collection of benchmark datasets verify the effectiveness of our task-aware specialization strategy, where we outperform state-of-the-art scientific pre-trained LMs. Code, datasets, and pre-trained models can be found at https://scimult.github.io/., Comment: 17 pages; Accepted to Findings of EMNLP 2023 (Project Page: https://scimult.github.io/)
- Published
- 2023
138. Chain-of-Skills: A Configurable Model for Open-domain Question Answering
- Author
-
Ma, Kaixin, Cheng, Hao, Zhang, Yu, Liu, Xiaodong, Nyberg, Eric, and Gao, Jianfeng
- Subjects
Computer Science - Computation and Language - Abstract
The retrieval model is an indispensable component for real-world knowledge-intensive tasks, e.g., open-domain question answering (ODQA). As separate retrieval skills are annotated for different datasets, recent work focuses on customized methods, limiting the model transferability and scalability. In this work, we propose a modular retriever where individual modules correspond to key skills that can be reused across datasets. Our approach supports flexible skill configurations based on the target domain to boost performance. To mitigate task interference, we design a novel modularization parameterization inspired by sparse Transformer. We demonstrate that our model can benefit from self-supervised pretraining on Wikipedia and fine-tuning using multiple ODQA datasets, both in a multi-task fashion. Our approach outperforms recent self-supervised retrievers in zero-shot evaluations and achieves state-of-the-art fine-tuned retrieval performance on NQ, HotpotQA and OTT-QA., Comment: ACL 2023
- Published
- 2023
139. Evaluating the Efficacy of Length-Controllable Machine Translation
- Author
-
Cheng, Hao, Zhang, Meng, Wang, Weixuan, Li, Liangyou, Liu, Qun, and Zhang, Zhihua
- Subjects
Computer Science - Computation and Language ,Computer Science - Artificial Intelligence - Abstract
Length-controllable machine translation is a type of constrained translation. It aims to contain the original meaning as much as possible while controlling the length of the translation. We can use automatic summarization or machine translation evaluation metrics for length-controllable machine translation, but this is not necessarily suitable and accurate. This work is the first attempt to evaluate the automatic metrics for length-controllable machine translation tasks systematically. We conduct a rigorous human evaluation on two translation directions and evaluate 18 summarization or translation evaluation metrics. We find that BLEURT and COMET have the highest correlation with human evaluation and are most suitable as evaluation metrics for length-controllable machine translation.
- Published
- 2023
140. End-to-end Training and Decoding for Pivot-based Cascaded Translation Model
- Author
-
Cheng, Hao, Zhang, Meng, Li, Liangyou, Liu, Qun, and Zhang, Zhihua
- Subjects
Computer Science - Computation and Language ,Computer Science - Artificial Intelligence - Abstract
Utilizing pivot language effectively can significantly improve low-resource machine translation. Usually, the two translation models, source-pivot and pivot-target, are trained individually and do not utilize the limited (source, target) parallel data. This work proposes an end-to-end training method for the cascaded translation model and configures an improved decoding algorithm. The input of the pivot-target model is modified to weighted pivot embedding based on the probability distribution output by the source-pivot model. This allows the model to be trained end-to-end. In addition, we mitigate the inconsistency between tokens and probability distributions while using beam search in pivot decoding. Experiments demonstrate that our method enhances the quality of translation.
- Published
- 2023
141. Improve Video Representation with Temporal Adversarial Augmentation
- Author
-
Duan, Jinhao, Fan, Quanfu, Cheng, Hao, Shi, Xiaoshuang, and Xu, Kaidi
- Subjects
Computer Science - Computer Vision and Pattern Recognition ,Computer Science - Artificial Intelligence - Abstract
Recent works reveal that adversarial augmentation benefits the generalization of neural networks (NNs) if used in an appropriate manner. In this paper, we introduce Temporal Adversarial Augmentation (TA), a novel video augmentation technique that utilizes temporal attention. Unlike conventional adversarial augmentation, TA is specifically designed to shift the attention distributions of neural networks with respect to video clips by maximizing a temporal-related loss function. We demonstrate that TA will obtain diverse temporal views, which significantly affect the focus of neural networks. Training with these examples remedies the flaw of unbalanced temporal information perception and enhances the ability to defend against temporal shifts, ultimately leading to better generalization. To leverage TA, we propose Temporal Video Adversarial Fine-tuning (TAF) framework for improving video representations. TAF is a model-agnostic, generic, and interpretability-friendly training strategy. We evaluate TAF with four powerful models (TSM, GST, TAM, and TPN) over three challenging temporal-related benchmarks (Something-something V1&V2 and diving48). Experimental results demonstrate that TAF effectively improves the test accuracy of these models with notable margins without introducing additional parameters or computational costs. As a byproduct, TAF also improves the robustness under out-of-distribution (OOD) settings. Code is available at https://github.com/jinhaoduan/TAF., Comment: To be appeared in IJCAI 2023
- Published
- 2023
142. The Mutual Information In The Vicinity of Capacity-Achieving Input Distributions
- Author
-
Nakiboğlu, Barış and Cheng, Hao-Chung
- Subjects
Computer Science - Information Theory ,Quantum Physics - Abstract
The mutual information is bounded from above by a decreasing affine function of the square of the distance between the input distribution and the set of all capacity-achieving input distributions $\Pi_{\mathcal{A}}$, on small enough neighborhoods of $\Pi_{\mathcal{A}}$, using an identity due to Tops{\o}e and the Pinsker's inequality, assuming that the input set of the channel is finite and the constraint set $\mathcal{A}$ is polyhedral, i.e., can be described by (possibly multiple but) finitely many linear constraints. Counterexamples demonstrating nonexistence of such a quadratic bound are provided for the case of infinitely many linear constraints and the case of infinite input sets. Using Taylor's theorem with the remainder term, rather than the Pinsker's inequality and invoking Moreau's decomposition theorem the exact characterization of the slowest decrease of the mutual information with the distance to $\Pi_{\mathcal{A}}$ is determined on small neighborhoods of $\Pi_{\mathcal{A}}$. Corresponding results for classical-quantum channels are established under separable output Hilbert space assumption for the quadratic bound and under finite-dimensional output Hilbert space assumption for the exact characterization. Implications of these observations for the channel coding problem and applications of the proof techniques to related problems are discussed., Comment: 17 pages, presented at ISIT 2023, submitted to IEEE Transactions on Information Theory on June 13, 2023, revised on September 6, 2024
- Published
- 2023
143. Quantum Broadcast Channel Simulation via Multipartite Convex Splitting
- Author
-
Cheng, Hao-Chung, Gao, Li, and Berta, Mario
- Subjects
Quantum Physics ,Computer Science - Information Theory ,Mathematical Physics - Abstract
We show that the communication cost of quantum broadcast channel simulation under free entanglement assistance between the sender and the receivers is asymptotically characterized by an efficiently computable single-letter formula in terms of the channel's multipartite mutual information. Our core contribution is a new one-shot achievability result for multipartite quantum state splitting via multipartite convex splitting. As part of this, we face a general instance of the quantum joint typicality problem with arbitrarily overlapping marginals. The crucial technical ingredient to sidestep this difficulty is a conceptually novel multipartite mean-zero decomposition lemma, together with employing recently introduced complex interpolation techniques for sandwiched R\'enyi divergences. Moreover, we establish an exponential convergence of the simulation error when the communication costs are within the interior of the capacity region. As the costs approach the boundary of the capacity region moderately quickly, we show that the error still vanishes asymptotically., Comment: The idea of the mean-zero decomposition lemma is independently and concurrently discovered for multipartite decoupling by Pau Colomer Saus and Andreas Winter (arXiv:2304.12114). v2: References updated
- Published
- 2023
144. Tight One-Shot Analysis for Convex Splitting with Applications in Quantum Information Theory
- Author
-
Cheng, Hao-Chung and Gao, Li
- Subjects
Quantum Physics ,Computer Science - Information Theory ,Mathematical Physics - Abstract
Convex splitting is a powerful technique in quantum information theory used in proving the achievability of numerous information-processing protocols such as quantum state redistribution and quantum network channel coding. In this work, we establish a one-shot error exponent and a one-shot strong converse for convex splitting with trace distance as an error criterion. Our results show that the derived error exponent (strong converse exponent) is positive if and only if the rate is in (outside) the achievable region. This leads to new one-shot exponent results in various tasks such as communication over quantum wiretap channels, secret key distillation, one-way quantum message compression, quantum measurement simulation, and quantum channel coding with side information at the transmitter. We also establish a near-optimal one-shot characterization of the sample complexity for convex splitting, which yields matched second-order asymptotics. This then leads to stronger one-shot analysis in many quantum information-theoretic tasks., Comment: v2: Adding an additivity property (Proposition 23)
- Published
- 2023
145. Chameleon: Plug-and-Play Compositional Reasoning with Large Language Models
- Author
-
Lu, Pan, Peng, Baolin, Cheng, Hao, Galley, Michel, Chang, Kai-Wei, Wu, Ying Nian, Zhu, Song-Chun, and Gao, Jianfeng
- Subjects
Computer Science - Computation and Language ,Computer Science - Artificial Intelligence ,Computer Science - Computer Vision and Pattern Recognition ,Computer Science - Machine Learning - Abstract
Large language models (LLMs) have achieved remarkable progress in solving various natural language processing tasks due to emergent reasoning abilities. However, LLMs have inherent limitations as they are incapable of accessing up-to-date information (stored on the Web or in task-specific knowledge bases), using external tools, and performing precise mathematical and logical reasoning. In this paper, we present Chameleon, an AI system that mitigates these limitations by augmenting LLMs with plug-and-play modules for compositional reasoning. Chameleon synthesizes programs by composing various tools (e.g., LLMs, off-the-shelf vision models, web search engines, Python functions, and heuristic-based modules) for accomplishing complex reasoning tasks. At the heart of Chameleon is an LLM-based planner that assembles a sequence of tools to execute to generate the final response. We showcase the effectiveness of Chameleon on two multi-modal knowledge-intensive reasoning tasks: ScienceQA and TabMWP. Chameleon, powered by GPT-4, achieves an 86.54% overall accuracy on ScienceQA, improving the best published few-shot result by 11.37%. On TabMWP, GPT-4-powered Chameleon improves the accuracy by 17.0%, lifting the state of the art to 98.78%. Our analysis also shows that the GPT-4-powered planner exhibits more consistent and rational tool selection via inferring potential constraints from instructions, compared to a ChatGPT-powered planner. The project is available at https://chameleon-llm.github.io., Comment: 32 pages, 10 figures, 24 tables. Accepted to NeurIPS 2023
- Published
- 2023
146. Harnessing the deep learning power of foundation models in single-cell omics
- Author
-
Ma, Qin, Jiang, Yi, Cheng, Hao, and Xu, Dong
- Published
- 2024
- Full Text
- View/download PDF
147. An Effective Algorithm Based on Space Net Optimization for Multi-Objective Optimization.
- Author
-
Chun-Wei Tsai, Cheng-Hao Lin, and Wei-Hong Wang
- Published
- 2024
- Full Text
- View/download PDF
148. ReCLIP: Refine Contrastive Language Image Pre-Training with Source Free Domain Adaptation.
- Author
-
Xuefeng Hu, Ke Zhang, Lu Xia, Albert Chen, Jiajia Luo, Yuyin Sun, Ken Wang, Nan Qiao 0009, Xiao Zeng, Min Sun, Cheng-Hao Kuo, and Ram Nevatia
- Published
- 2024
- Full Text
- View/download PDF
149. PyCIM: A Python Framework for the Dynamics of Coherent Ising Machine
- Author
-
Li, Peixiang, Cheng, Hao, Liu, Yingwen, Wang, Dongyang, Wu, Junjie, Goos, Gerhard, Series Editor, Hartmanis, Juris, Founding Editor, Bertino, Elisa, Editorial Board Member, Gao, Wen, Editorial Board Member, Steffen, Bernhard, Editorial Board Member, Yung, Moti, Editorial Board Member, Huang, De-Shuang, editor, Zhang, Chuanlei, editor, and Guo, Jiayang, editor
- Published
- 2024
- Full Text
- View/download PDF
150. Study on Influence of Contact Structure on Phase Shift Time of Magnetic Field in Vacuum Under Various Current Frequencies
- Author
-
Cheng, Hao, Huang, Peicheng, Zhang, Yirui, Ma, Hui, Liu, Zhiyuan, Geng, Yingsan, Angrisani, Leopoldo, Series Editor, Arteaga, Marco, Series Editor, Chakraborty, Samarjit, Series Editor, Chen, Jiming, Series Editor, Chen, Shanben, Series Editor, Chen, Tan Kay, Series Editor, Dillmann, Rüdiger, Series Editor, Duan, Haibin, Series Editor, Ferrari, Gianluigi, Series Editor, Ferre, Manuel, Series Editor, Jabbari, Faryar, Series Editor, Jia, Limin, Series Editor, Kacprzyk, Janusz, Series Editor, Khamis, Alaa, Series Editor, Kroeger, Torsten, Series Editor, Li, Yong, Series Editor, Liang, Qilian, Series Editor, Martín, Ferran, Series Editor, Ming, Tan Cher, Series Editor, Minker, Wolfgang, Series Editor, Misra, Pradeep, Series Editor, Mukhopadhyay, Subhas, Series Editor, Ning, Cun-Zheng, Series Editor, Nishida, Toyoaki, Series Editor, Oneto, Luca, Series Editor, Panigrahi, Bijaya Ketan, Series Editor, Pascucci, Federica, Series Editor, Qin, Yong, Series Editor, Seng, Gan Woon, Series Editor, Speidel, Joachim, Series Editor, Veiga, Germano, Series Editor, Wu, Haitao, Series Editor, Zamboni, Walter, Series Editor, Tan, Kay Chen, Series Editor, Yang, Qingxin, editor, Li, Zewen, editor, and Luo, An, editor
- Published
- 2024
- Full Text
- View/download PDF
Catalog
Discovery Service for Jio Institute Digital Library
For full access to our library's resources, please sign in.