Search

In this essay, we argue that colleges of education, particularly those at research-intensive institutions, favor simplistic notions of scholarly impact and that this trend has concerning implications for the field, for researchers, and for the public at large. After describing the challenges and shortcomings of the current models of research assessment in education, we outline an alternative proposal in which trustworthiness and usability of research would complement traditional metrics of scholarly relevance. This proposal encourages a twofold approach to research assessment that involves (1) a more thorough analysis of the limitations and problems generated by the use of simplistic notions of scholarly impact, and (2) a commitment to the implementation of more equitable systems based on a broader range of assessment measures to assess faculty research contributions.

National culture has been overlooked in discussions related to research output and impact owing to individual, socio-political structure, and economic factors. This study shows the relationships between the dimensions of cultural value orientation of the nation and research output & impact. More than 60 countries were included, and Spearman correlation analysis was employed. The variables were taken from Geert Hofstede and Scimago Journal & Country Rank worksheets. This study found that (1) Power distance - the positive inclination of the culture toward power disparities among people - is negatively correlated with research impact; (2) Individualism - the level of independence a society keeps up among its individuals - are positively correlated with research output and research impact; (3) Indulgence - the degree to which society members do not attempt to control their urges - is positively correlated with research impact; and (4) after controlling the Log GDP per capita, uncertainty avoidance - the manner in which that a society seeks to manage the actuality that the future can never be controlled - is negatively correlated with research impact.

National culture has been overlooked in discussions related to research performance and impact owing to individual, socio-political structure, and economic factors. This study shows the relationships between the dimensions of cultural value orientation of the nation and research performance & impact. More than 60 countries were included and Spearman correlation analysis was employed. The variables were taken from Geert Hofstede and Scimago Journal & Country Rank worksheets. This study found that (1) Power distance - the positive inclination of the culture toward power disparities among people - is negatively correlated with research impact; (2) Individualism - the level of independence a society keeps up among its individuals - are positively correlated with research performance and research impact; (3) Indulgence - the degree to which society members do not attempt to control their urges - is positively correlated with research impact; and (4) after controlling the Log GDP per capita, uncertainty avoidance - the manner in which that a society seeks to manage the actuality that the future can never be controlled - is negatively correlated with research impact.

National culture has been overlooked in discussions related to research productivity and impact owing to individual, socio-political structure, and economic factors. This study shows the relationships between the dimensions of cultural value orientation of the nation and research performance indicators. More than 60 countries were included and Pearson correlation analysis was employed. The variables were taken from Geert Hofstede and Scimago Journal & Country Rank worksheets. This study found that (1) Individualism has significant correlations with the majority of the indicators; (2) Power distance and indulgence correlate with a country’s research impact in the form of citation per document; (3) Masculinity, long term orientation, and uncertainty avoidance do not correlate with the indicators. Owing to the fact that the national culture is relatively enduring, countries need to measure their elasticity of hopes and action plans in an effort to boost research productivity and impact, by integrating the national culture in the estimate.

Background: Decisions about which applications to fund are generally based on the mean scores of a panel of peer reviewers. As well as the mean, a large disagreement between peer reviewers may also be worth considering, as it may indicate a high-risk application with a high return. Methods: We examined the peer reviewers' scores for 227 funded applications submitted to the American Institute of Biological Sciences between 1999 and 2006. We examined the mean score and two measures of reviewer disagreement: the standard deviation and range. The outcome variable was the relative citation ratio, which is the number of citations from all publications associated with the application, standardised by field and publication year. Results: There was a clear increase in relative citations for applications with a better mean. There was no association between relative citations and either of the two measures of disagreement. Conclusions: We found no evidence that reviewer disagreement was able to identify applications with a higher than average return. However, this is the first study to empirically examine this association, and it would be useful to examine whether reviewer disagreement is associated with research impact in other funding schemes and in larger sample sizes.

Background: Decisions about which applications to fund are generally based on the mean scores of a panel of peer reviewers. As well as the mean, a large disagreement between peer reviewers may also be worth considering, as it may indicate a high-risk application with a high return. Methods: We examined the peer reviewers' scores for 227 funded applications submitted to the American Institute of Biological Sciences between 1999 and 2006. We examined the mean score and two measures of reviewer disagreement: the standard deviation and range. The outcome variable was the relative citation ratio, which is the number of citations from all publications associated with the application, standardised by field and publication year. Results: There was a clear increase in relative citations for applications with a higher mean. There was no association between relative citations and either of the two measures of disagreement. Conclusions: We found no evidence that reviewer disagreement was able to identify applications with a higher than average return. However, this is the first study to empirically examine this association, and it would be useful to examine whether reviewer disagreement is associated with research impact in other funding schemes and in larger sample sizes.

Data from open access biomolecular data resources, such as the European Nucleotide Archive and the Protein Data Bank are extensively reused within life science research for comparative studies, method development and to derive new scientific insights. Indicators that estimate the extent and utility of such secondary use of research data need to reflect this complex and highly variable data usage. By linking open access scientific literature, via Europe PubMedCentral, to the metadata in biological data resources we separate data citations associated with a deposition statement from citations that capture the subsequent, long-term, reuse of data in academia and industry. We extend this analysis to begin to investigate citations of biomolecular resources in patent documents. We find citations in more than 8,000 patents from 2014, demonstrating substantial use and an important role for data resources in defining biological concepts in granted patents to both academic and industrial innovators. Combined together our results indicate that the citation patterns in biomedical literature and patents vary, not only due to citation practice but also according to the data resource cited. The results guard against the use of simple metrics such as citation counts and show that indicators of data use must not only take into account citations within the biomedical literature but also include reuse of data in industry and other parts of society by including patents and other scientific and technical documents such as guidelines, reports and grant applications.

Quantitative data are crucial in the assessment of research impact in the academic world. However, as a young university created in 2009, King Abdullah University of Science and Technology (KAUST) needs to aggregate bibliometrics from researchers coming from diverse origins, not necessarily with the proper affiliations. In this context, the University has launched an institutional repository in September 2012 with the objectives of creating a home for the intellectual outputs of KAUST researchers. Later, the university adopted the first mandated institutional open access policy in the Arab region, effective June 31, 2014. Several projects were then initiated in order to accurately identify the research being done by KAUST authors and bring it into the repository in accordance with the open access policy. Integration with ORCID has been a key element in this process and the best way to ensure data quality for researcher’s scientific contributions. It included the systematic inclusion and creation, if necessary, of ORCID identifiers in the existing repository system, an institutional membership in ORCID, and the creation of dedicated integration tools. In addition and in cooperation with the Office of Research Evaluation, the Library worked at implementing a Current Research Information System (CRIS) as a standardized common resource to monitor KAUST research outputs. We will present our findings about the CRIS implementation, the ORCID API, the repository statistics as well as our approach in conducting the assessment of research impact in terms of usage by the global research community.

National culture has been overlooked in discussions related to research productivity and impact owing to individual, socio-political structure, and economic factors. This study shows the relationships between the dimensions of cultural value orientation of the nation and research performance indicators. More than 60 countries were included and Pearson correlation analysis was employed. The variables were taken from Geert Hofstede and Scimago Journal & Country Rank worksheets. This study found that (1) Individualism has significant correlations with the majority of the indicators; (2) Power distance and indulgence correlate with a country's research impact in the form of citation per document; (3) Masculinity, long term orientation, and uncertainty avoidance do not correlate with the indicators. Owing to the fact that the national culture is relatively enduring, countries need to measure their elasticity of hopes and action plans in an effort to boost research productivity and impact, by integrating the national culture in the estimate., Competing Interests: No competing interests were disclosed., (Copyright: © 2019 Abraham J.)

Background : Decisions about which applications to fund are generally based on the mean scores of a panel of peer reviewers. As well as the mean, a large disagreement between peer reviewers may also be worth considering, as it may indicate a high-risk application with a high return. Methods : We examined the peer reviewers' scores for 227 funded applications submitted to the American Institute of Biological Sciences between 1999 and 2006. We examined the mean score and two measures of reviewer disagreement: the standard deviation and range. The outcome variable was the relative citation ratio, which is the number of citations from all publications associated with the application, standardised by field and publication year. Results : There was a clear increase in relative citations for applications with a better mean. There was no association between relative citations and either of the two measures of disagreement. Conclusions : We found no evidence that reviewer disagreement was able to identify applications with a higher than average return. However, this is the first study to empirically examine this association, and it would be useful to examine whether reviewer disagreement is associated with research impact in other funding schemes and in larger sample sizes., Competing Interests: No competing interests were disclosed.

Data from open access biomolecular data resources, such as the European Nucleotide Archive and the Protein Data Bank are extensively reused within life science research for comparative studies, method development and to derive new scientific insights. Indicators that estimate the extent and utility of such secondary use of research data need to reflect this complex and highly variable data usage. By linking open access scientific literature, via Europe PubMedCentral, to the metadata in biological data resources we separate data citations associated with a deposition statement from citations that capture the subsequent, long-term, reuse of data in academia and industry.  We extend this analysis to begin to investigate citations of biomolecular resources in patent documents. We find citations in more than 8,000 patents from 2014, demonstrating substantial use and an important role for data resources in defining biological concepts in granted patents to both academic and industrial innovators. Combined together our results indicate that the citation patterns in biomedical literature and patents vary, not only due to citation practice but also according to the data resource cited. The results guard against the use of simple metrics such as citation counts and show that indicators of data use must not only take into account citations within the biomedical literature but also include reuse of data in industry and other parts of society by including patents and other scientific and technical documents such as guidelines, reports and grant applications.

Quantitative data are crucial in the assessment of research impact in the academic world. However, as a young university created in 2009, King Abdullah University of Science and Technology (KAUST) needs to aggregate bibliometrics from researchers coming from diverse origins, not necessarily with the proper affiliations. In this context, the University has launched an institutional repository in September 2012 with the objectives of creating a home for the intellectual outputs of KAUST researchers. Later, the university adopted the first mandated institutional open access policy in the Arab region, effective June 31, 2014. Several projects were then initiated in order to accurately identify the research being done by KAUST authors and bring it into the repository in accordance with the open access policy. Integration with ORCID has been a key element in this process and the best way to ensure data quality for researcher's scientific contributions. It included the systematic inclusion and creation, if necessary, of ORCID identifiers in the existing repository system, an institutional membership in ORCID, and the creation of dedicated integration tools. In addition and in cooperation with the Office of Research Evaluation, the Library worked at implementing a Current Research Information System (CRIS) as a standardized common resource to monitor KAUST research outputs. We will present our findings about the CRIS implementation, the ORCID API, the repository statistics as well as our approach in conducting the assessment of research impact in terms of usage by the global research community.