Search

Your search keyword '"Zhang, Jiayin"' showing total 804 results
Start Over Author "Zhang, Jiayin"
804 results on '"Zhang, Jiayin"'

6. Benchmarking Foundation Models with Language-Model-as-an-Examiner

Author

Bai, Yushi, Ying, Jiahao, Cao, Yixin, Lv, Xin, He, Yuze, Wang, Xiaozhi, Yu, Jifan, Zeng, Kaisheng, Xiao, Yijia, Lyu, Haozhe, Zhang, Jiayin, Li, Juanzi, and Hou, Lei

Subjects
Computer Science - Computation and Language, Computer Science - Machine Learning
Abstract

Numerous benchmarks have been established to assess the performance of foundation models on open-ended question answering, which serves as a comprehensive test of a model's ability to understand and generate language in a manner similar to humans. Most of these works focus on proposing new datasets, however, we see two main issues within previous benchmarking pipelines, namely testing leakage and evaluation automation. In this paper, we propose a novel benchmarking framework, Language-Model-as-an-Examiner, where the LM serves as a knowledgeable examiner that formulates questions based on its knowledge and evaluates responses in a reference-free manner. Our framework allows for effortless extensibility as various LMs can be adopted as the examiner, and the questions can be constantly updated given more diverse trigger topics. For a more comprehensive and equitable evaluation, we devise three strategies: (1) We instruct the LM examiner to generate questions across a multitude of domains to probe for a broad acquisition, and raise follow-up questions to engage in a more in-depth assessment. (2) Upon evaluation, the examiner combines both scoring and ranking measurements, providing a reliable result as it aligns closely with human annotations. (3) We additionally propose a decentralized Peer-examination method to address the biases in a single examiner. Our data and benchmarking results are available at: http://lmexam.xlore.cn., Comment: NeurIPS 2023 Datasets and Benchmarks

Published
2023

7. CGSL: Collaborative Graph and Segment Learning Based Aspect-Level Sentiment Analysis Model

Author

Rao, Guozheng, Tian, Kaijia, Yu, Mufan, Zhang, Jiayin, Zhang, Li, Wang, Xin, Goos, Gerhard, Series Editor, Hartmanis, Juris, Founding Editor, Bertino, Elisa, Editorial Board Member, Gao, Wen, Editorial Board Member, Steffen, Bernhard, Editorial Board Member, Yung, Moti, Editorial Board Member, Zhang, Wenjie, editor, Tung, Anthony, editor, Zheng, Zhonglong, editor, Yang, Zhengyi, editor, Wang, Xiaoyang, editor, and Guo, Hongjie, editor

Published
2024
Full Text
View/download PDF

21. Diagnostic and prognostic performance of artificial intelligence-based fully-automated on-site CT-FFR in patients with CAD

Author

Guo, Bangjun, Jiang, Mengchun, Guo, Xiang, Tang, Chunxiang, Zhong, Jian, Lu, Mengjie, Liu, Chunyu, Zhang, Xiaolei, Qiao, Hongyan, Zhou, Fan, Xu, Pengpeng, Xue, Yi, Zheng, Minwen, Hou, Yang, Wang, Yining, Zhang, Jiayin, Zhang, Bo, Zhang, Daimin, Xu, Lei, Hu, Xiuhua, Zhou, Changsheng, Li, Jianhua, Yang, Zhiwen, Mao, Xinsheng, Lu, Guangming, and Zhang, Longjiang

Published
2024
Full Text
View/download PDF

Catalog

Books, media, physical & digital resources
See catalog results