Back to Search
Start Over
Man versus machine? Self-reports versus algorithmic measurement of publications
- Source :
- PLoS ONE, PLoS ONE, Vol 16, Iss 9, p e0257309 (2021)
- Publication Year :
- 2021
- Publisher :
- Public Library of Science, 2021.
-
Abstract
- This paper uses newly available data from Web of Science on publications matched to researchers in Survey of Doctorate Recipients to compare the quality of scientific publication data collected by surveys versus algorithmic approaches. We illustrate the different types of measurement errors in self-reported and machine-generated data by estimating how publication measures from the two approaches are related to career outcomes (e.g., salaries and faculty rankings). We find that the potential biases in the self-reports are smaller relative to the algorithmic data. Moreover, the errors in the two approaches are quite intuitive: the measurement errors in algorithmic data are mainly due to the accuracy of matching, which primarily depends on the frequency of names and the data that was available to make matches, while the noise in self reports increases over the career as researchers’ publication records become more complex, harder to recall, and less immediately relevant for career progress. At a methodological level, we show how the approaches can be evaluated using accepted statistical methods without gold standard data. We also provide guidance on how to use the new linked data.
- Subjects :
- Male
Computer science
Economics
Social Sciences
Surveys
Machine Learning
Mathematical and Statistical Techniques
Surveys and Questionnaires
Salaries
Publication data
media_common
Measurement
Multidisciplinary
Careers
Applied Mathematics
Simulation and Modeling
Instrumental variable
Statistics
Publications
Linked data
Faculty
Research Personnel
Research Design
Physical Sciences
Medicine
Engineering and Technology
Educational Status
Female
Algorithms
Research Article
Employment
Matching (statistics)
Computer and Information Sciences
Web of science
Universities
Science
media_common.quotation_subject
Research and Analysis Methods
Instrumental Variable Analysis
Machine Learning Algorithms
Artificial Intelligence
Humans
Quality (business)
Statistical Methods
Occupations
Signal to Noise Ratio
Information retrieval
Survey Research
Models, Statistical
Recall
Labor Economics
Signal Processing
Noise (video)
Self Report
Mathematics
Subjects
Details
- Language :
- English
- ISSN :
- 19326203
- Volume :
- 16
- Issue :
- 9
- Database :
- OpenAIRE
- Journal :
- PLoS ONE
- Accession number :
- edsair.doi.dedup.....4369b76b88999fb73c987b5c990e6cde