Search

This paper examines the consequences of a culture of “personal ethics” when using new methodologies, such as the use of social media (SM) sites as a source of data for research. Using SM research as an example, this paper explores the practices of a number of actors and researchers within the “Ethics Ecosystem” which as a network governs ethically responsible research behaviour. In the case of SM research, the ethical use of this data is currently in dispute, as even though it is seemingly publically available, concerns relating to privacy, vulnerability, potential harm and consent blur the lines of responsible ethical research behaviour. The findings point to the dominance of a personal, bottom-up, researcher-led, ‘ethical barometer’ for making decisions regarding the permissibility of using SM data. We show that the use of different barometers by different researchers can lead to wide disparities in ethical practice - disparities which are compounded by the lack of firm guidelines for responsible practice of SM research. This has widespread consequences on the development of shared norms and understandings at all levels, and by all actors within the Ethics Ecosystem, and risks inconsistencies in their approaches to ethical decision-making. This paper argues that this governance of ethical behaviour by individual researchers perpetuates a negative cycle of academic practice that is dependent on subjective judgements by researchers themselves, rather than governed by more formalised academic institutions such as the research ethics committee and funding council guidelines.

In order to examine potential effects of methodological choices influencing developments in relative citation scores for countries, a fixed journal set comprising of 3232 journals continuously indexed in the Web of Science from 1981 to 2014 is constructed. From this restricted set, a citation database depicting the citing relations between the journal publications is formed and relative citation scores based on full and fractional counting are calculated for the whole period. Previous longitudinal studies of citation impact show stable rankings between countries. To examine such findings coming from a dynamic set of journals for potential “database effects”, we compare them to our fixed set. We find that relative developments in impact scores, country profiles and rankings are both very stable and very similar within and between the two journal sets as well as counting methods. We do see a small “inflation factor” as citation scores generally are somewhat lower for high-performing countries in the fixed set compared to the dynamic set. Consequently, using an ever-decreasing set of journals compared to the dynamic set, we are still able to reproduce accurately the developments in impact scores and the rankings between the countries found in the dynamic set. Hence, potential effects of methodological choices seem to be of limited importance compared to the stability of citation networks.

In this study, the possibilities to extend the basis for research performance exercises with editorial material are explored. While this document type has been traditionally not considered as an important type of scientific communication in research performance assessment procedures, there is a perception from researchers that editorial materials should be considered as relevant document types as important sources for the dissemination of scientific knowledge. In a number of these cases, some of the mentioned editorial materials are actually `highly cited'. This lead to a thorough scrutiny of editorials or editorial material over the period 1992---2001, for all citation indexes of Thomson Scientific. The relevance of editorial materials through three quantitative bibliometric characteristics of scientific publications, namely page length, number of references, and the number of received citations, are thoroughly analyzed.

In this study we show that it is possible to identify top-cited publications other than Web of Science (WoS) publications, particularly non-journal publications, within fields in the social and behavioral sciences. We analyzed references in publications that were themselves highly cited, with at least one European address. Books represent between 62 (psychology) and 81% (political science) of the non-WoS references, journal articles 15-24%. Books (economics, political science) and manuals (psychology) account for the most highly cited publications. Between 50 (psychology, political science) and 71% (economics) of the top-ranked most cited publications originated from the US versus between 18 (economics) and 38% (psychology) from Europe. Finally, it is discussed how the methods and procedures of the study can be optimized.

In this study we have focused on long term developments of various types of scientific publishing, and the field-normalized impact generated by these various types. The types of scientific output distinguished are output resulting from international cooperation, national cooperation, and single address publications, in which no apparent cooperation is found. A fourth type is distinguished by focusing on first authorship, within the international cooperation output. Changes in especially the share of a country’s output from first-authored international cooperation and the share of single address publications can be regarded as indicators of strength and/or weakness of a science system.

In this study we combine the registered output of a whole university in the Netherlands with data retrieved from the Web of Science. The initial research question was: is it possible to show the impact of the university in its' full broadness, taking into account the variety of disciplines covered in the research profile of the university? In order to answer this question, we analyzed the output of the university as registered in the CRIS system METIS, over the years 2004---2009. The registration covers a wide variety of scholarly outputs, and these are all taken into account in the analysis. In the study we conduct analyses on the coverage of the output of the university, both from the perspective of the output itself, towards the Web of Science ("external"), as well as from the Web of Science perspective itself ("internal"). This provides us with the necessary information to be able to draw clear conclusions on the validity of the usage of standard bibliometric methodologies in the research assessment of universities with such a research profile.

Bio-pharmaceutical R&D is increasingly an international affair. Research articles published in the peer-reviewed international scientific and technical journals represent quantifiable research outputs of bio-pharmaceutical firms. Large-scale systemic measurements of worldwide trends and sectoral patterns within bio-pharmaceutical science can be gauged from these articles, where coauthored research papers are assumed to reflect research cooperation and associated knowledge flows and exchanges. We focus our attention on the largest science-based multinational enterprises (MNEs), those that produce relatively large quantities of research articles. The study deals with the worldwide output of research articles that are co-produced by corporate researchers during the years 1996–2001.

Today's theories and models on innovation stress the importance of scientific capabilities and science-technology proximity, especially in new emerging fields of economic activity. Inthis contribution we examine the relationship between national scientific capabilities, the science intensity of technology and technological performance within six emergent industrial fields. Our findings reveal that national technological performance is positively associated with scientific capabilities. Countries performing better on a technological level are characterized both by larger numbers of publications and by numbers of involved institutions that exceed average expected values. The latter observation holds for both companies and knowledge generating institutes actively involved in scientific activities. As such, our findings seem to suggest beneficial effects of scientific capabilities shouldered by a multitude of organizations. In addition, higher numbers of patent activity coincide with higher levels of science intensity pointing out the relevance of science 'proximity' when developing technology in newer, emerging fields. Limitations and directions for further research are discussed.

The paper discusses an application of bibliometric techniques in the social sciences. While the interest of policy makers is growing, the topic is getting more and more attention from bibliometricians. However, many efforts are put into developing tools to measure scientific output and impact outside the world of the Social Sciences Citation Index, while the use of the SSCI for bibliometric applications is covered with obscurity and myths. This study attempts to clarify some of the topics mentioned against the application of the SSCI for evaluation purposes. The study will cover topics like the existing publication and citation culture within the social sciences, the effect of variable citation windows, and the (geographical) origin of citation flows.

In this study, journal impact factors play a central role. In addition to this important bibliometric indicator, which evolves around the average impact of a journal in a two-year timeframe, related aspects of journal impact measurement are studied. Aspects like the output volume, the percentage of publications not cited, and the citation frequency distribution within a set timeframe are researched, and put in perspective with the 'classical' journal Impact Factor. In this study it is shown that these aspects of journal impact measurement play a significant role, and are strongly inter-related. Especially the separation between journals on the basis of the differences in output volume seems to be relevant, as can be concluded from the different results in the analysis of journal impact factors, the degree of uncitedness, and the share of a journal its contents above or below the impact factor value.

Evaluation studies of scientific performance conducted during the past years more and more focus on the identification of research of the 'highest quality', 'top' research, or 'scientific excellence'. This shift in focus has lead to the development of new bibliometric methodologies and indicators. Technically, it meant a shift from bibliometric impact scores based on average values such as the average impact of all papers published by some unit to be evaluated towards indicators reflecting the topof the citation distribution, such as the number of 'highly cited' or 'top' articles. In this study we present a comparative analysis of a number of standard and new indicators of research performance or 'scientific excellence', using techniques applied in studies conducted by CWTS in recent years. It will be shown that each type of indicator reflects a particular dimension of the general concept of research performance. Consequently, the application of one single indicator only may provide an incomplete picture of a unit's performance. It is argued that one needs to combine the various types of indicators in order to offer policy makers and evaluators valid and useful assessment tools.

In an evaluation of physics research programs in the Netherlands, held in 1996, assessments of research by expert panels were supplemented with bibliometric analysis. This latter analysis included the calculation of several bibliometric indicators, among which some taking journal impact measures as a baseline. Final outcomes of this evaluation provided an opportunity to reexamine the results of this assessment from the perspective of the degree of interdisciplinarity of programs involved. In this paper we discuss results of this latter analysis, in particular with respect to the relation between several citation based indicators and interdisciplinary research in Dutch physics.

This paper discusses development and application of journal impact indicators in a number of bibliometric studies commissioned by Dutch organizations and institutions, and conducted in our institute during the past five years. An outline is given of the research questions addressed in these studies and their policy context. For each study the appropriateness of the use of journal impact indicators produced by the Institute for Scientific Information (ISI) is evaluated. Alternative journal impact measures were developed which are shown to be more appropriate in the particular research and policy contexts than the ISI measures. These measures were considered to be highly useful by the users. The studies have revealed methodological flaws of the ISI journal impact factors.

In this paper we report on the results of an exploratory study of knowledge exchange between disciplines and subfields of science, based on bibliometric methods. The goal of this analysis is twofold. Firstly, we consider knowledge exchange between disciplines at a global level, by analysing cross-disciplinary citations in journal articles, based on the world publication output in 1999. Among others a central position of the Basic Life Sciences within the Life Sciences and of Physics within the Exact Sciences is shown. Limitations of analyses of interdisciplinary impact at the journal level are discussed. A second topic is a discussion of measures which may be used to quantify the rate of knowledge transfer between fields and the importance of work in a given field or for other disciplines. Two measures are applied, which appear to be proper indicators of impact of research on other fields. These indicators of interdisciplinary impact may be applied at other institutional levels as well.

This paper introduces a citation-based "systems approach" for analyzing the various institutional and cognitive dimensions of scientific excellence within national research systems. The methodology, covering several aggregate levels, focuses on the most highly cited research papers in the international journal literature. The distribution of these papers across institutions and disciplines enables objective comparisons their (possible) international-level scientific excellence. By way of example, we present key results from a recent series of analyses of the research system in the Netherlands in the mid 1990s, focussing on the performance of the universities across the various major scientific disciplines within the context of the entire system"s scientific performance. Special attention is paid to the contribution in the world"s top 1% and top 10% most highly cited research papers. The findings indicate that these high performance papers provide a useful analytical framework - both in terms of transparency, cognitive and institutional differentiation, as well as its scope for domestic and international comparisons - providing new indicators for identifying "world class" scientific excellence at the aggregate level. The average citation scores of these academic "Centres of Scientific Excellence" appear to be an inadequate predictor of their production of highly cited papers. However, further critical reflection and in-depth validation studies are needed to establish the true potential of this approach for science policy analyses and evaluation of research performance.

As part of a larger project to investigate knowledge flows between fields of science, westudied the differences in speed of knowledge transfer within and across disciplines. The agedistribution of references in three selections of articles was analysed, including almost 800.000references in journal publications of the United Kingdom in 1992, 700.000 references inpublications of Germany in 1992, and more than 11 million references in the world total ofpublications in 1998.

Empirical evidence presented in this paper shows that the utmost care must be taken ininterpreting bibliometric data in a comparative evaluation of national research systems. From theresults of recent studies, the authors conclude that the value of impact indicators of researchactivities at the level of an institution or a country strongly depend upon whether one includes orexcludes research publications in SCI covered journals written in other languages than in English.Additional material was gathered to show the distribution of SCI papers among publicationlanguages. Finally, the authors make suggestions for further research on how to deal with this typeof problems in future national research performance studies.

Patent citations to the research literature offer a way for identifying and comparing contributions of scientific and technical knowledge to technological development. This case study applies this approach through a series of analyses of citations to Dutch research papers listed on Dutch-invented and foreign patents granted in the US during the years 1987–1996.

This paper reviews a range of studies conducted by the authors on indicators reflecting scholarly journal impact. A critical examination of the journal impact data in theJournal Citation Reports (JCR), published by the Institute for Scientific Information (ISI) has shown that the JCR impact factor is inaccurate and biased towards journals revealing a rapid maturing or decline in impact. In addition, it was found that the JCR cited half life is an inappropriate measure of decline of journal impact. More appropriate impact measures of scholarly journals are proposed. A new classification system is explored, describing both maturing and decline of journal impact as measured through citations. Suggestions for future research are made, analysing in more detail the distribution of citations among papers in a journal.

It is shown that the Journal Impact Factor as published by ISI — an indicator increasingly used as an measure for the quality of scientific journals — is misleading when two leading journals in chemistry,Angew. Chem., andJ. Am. Chem. Soc., are compared. A detailed analysis of the various kinds publications in both journals over the period 1982–1994 shows that the overall impact factors based on publications and citations in two consecutive years forJACS communications (5.27 for 1993) are significantly higher than those ofAngew. Chem. (3.26 for 1993). Even when all types of articles, i.e. including reviews, are included in the impact factors,JACS has a higher score thanAngew. Chem. (5.07 vs. 4.03 in 1993). Critical and accurate analyses of citation figures is required when such data are used in science policy decisions, such as library subscriptions. It is proposed that when IF values for several journals are compared, only similar publication types are considered.

This paper gives an outline of a new bibliometric database based upon all articles published by authors from the Netherlands, and processed during the time period 1980–1993 by the Institute for Scientific Information (ISI) for theScience Citation Index (SCI),Social Science Citation Index (SSCI) andArts & Humanities Citation Index (AH detailed citation data on ISI journals and subfields; and a classification system of publishing main organizations, appearing in the addresses. Moreover, an overview is given of the types of bibliometric indicators that were constructed. Their relationship to indicators developed by other researchers in the field is discussed. Finally, two applications are given in order to illustrate the potentials of the database and of the bibliometric indicators derived from it. The first represents a synthesis of ‘classical’ macro indicator studies at the one hand, and bibliometric analyses of research groups or institutes at the other. The second application gives for the first time a detailed analysis of a country's publication output per institutional sector.

This article presents results of a study on the applicability of journal mapping of knowledge domains beyond the databases produced by the Institute for Scientific Information (ISI). The utility and validity of this generalisation is discussed with an emphasis on its added value in comparison to ‘traditional’ ISI-based journal maps, i.e. those restricted to (predominantly English-language) ISI-covered journals, and journal-to-journal citation data retrieved from ISI'sJournal Citation Reports. The mapping methodology was applied to Manufacturing technology and management — a multidisciplinary domain situated on the interface of science and technology. TheInternational Journal of Production Economics was singled out as a special case for the validation study of the maps. Results of this study, involving several subject experts, indicate that a journal content-based map was not only far superior to the journal citation map, but also outperformed the map derived from the combination of both types of data. The selection of periodicals from other databases such as COMPENDEX produced a substantial number of additional titles of which only half were also covered by ISI.

The obsolescence and "durability" of scientific literature have been important elements of debate during many years, especially regarding the proper calculation of bibliometric indicators. The effects of "delayed recognition" on impact indicators have importance and are of interest not only to bibliometricians but also among research managers and scientists themselves. It has been suggested that the "Mendel syndrome" is a potential drawback when assessing individual researchers through impact measures. If publications from particular researchers need more time than "normal" to be properly acknowledged by their colleagues, the impact of these researchers may be underestimated with common citation windows. In this paper, we answer the question whether the bibliometric indicators for scientists can be significantly affected by the Mendel syndrome. Applying a methodology developed previously for the classification of papers according to their durability (Costas et al., J Am Soc Inf Sci Technol 61(8):1564---1581, 2010a; J Am Soc Inf Sci Technol 61(2):329---339, 2010b), the scientific production of 1,064 researchers working at the Spanish Council for Scientific Research (CSIC) in three different research areas has been analyzed. Cases of potential "Mendel syndrome" are rarely found among researchers and these cases do not significantly outperform the impact of researchers with a standard pattern of reception in their citations. The analysis of durability could be included as a parameter for the consideration of the citation windows used in the bibliometric analysis of individuals.

This paper presents a selection of results of a comprehensive quantitative, research literature-based study of Dutch energy research. The primary goal of this paper is to provide an overview of what bibliometric data from ISI and non-ISI databases may offer to describe the state of affairs in a scientific field. It illustrates the added value of combining bibliometric indicators of publication output, international visibility, international co-operation, and interdisciplinarity in a study of nuclear energy research in the 1980's when its budget decreased dramatically.

We present an empirical comparison between two normalization mechanisms for citation-based indicators of research performance. These mechanisms aim to normalize citation counts for the field and the year in which a publication was published. One mechanism is applied in the current so-called crown indicator of our institute. The other mechanism is applied in the new crown indicator that our institute is currently exploring. We find that at high aggregation levels, such as at the level of large research institutions or at the level of countries, the differences between the two mechanisms are very small. At lower aggregation levels, such as at the level of research groups or at the level of journals, the differences between the two mechanisms are somewhat larger. We pay special attention to the way in which recent publications are handled. These publications typically have very low citation counts and should therefore be handled with special care.

21 pages, 37 graphs, 5 tables., This paper focuses on the study of self-citations at the meso and micro (individual) levels, on the basis of an analysis of the production (1994–2004) of individual researchers working at the Spanish CSIC in the areas of Biology and Biomedicine and Material Sciences. Two different types of self-citations are described: author self-citations (citations received from the author him/herself) and co-author self-citations (citations received from the researchers’ co-authors but without his/her participation). Self-citations do not play a decisive role in the high citation scores of documents either at the individual or at the meso level, which are mainly due to external citations. At micro-level, the percentage of self-citations does not change by professional rank or age, but differences in the relative weight of author and co-author self-citations have been found. The percentage of co-author self-citations tends to decrease with age and professional rank while the percentage of author self-citations shows the opposite trend. Suppressing author selfcitations from citation counts to prevent overblown self-citation practices may result in a higher reduction of citation numbers of old scientists and, particularly, of those in the highest categories. Author and co-author self-citations provide valuable information on the scientific communication process, but external citations are the most relevant for evaluative purposes. As a final recommendation, studies considering self-citations at the individual level should make clear whether author or total self-citations are used as these can affect researchers differently., This study was completed thanks to an I3P-CSIC grant at CINDOC (now IEDCYT) and also thanks to a research stay grant at the CWTS in Leiden (The Netherlands).

The Relative Specialization Index (RSI) is an indicator that measures the research profile of a country by comparing the share of a given field in the publications of a given country with the share of the same field in the world total of publications. If measured over time, this indicator may be influenced in the world total by the increased representation of certain other countries with different research profiles. As a case, we study the effect on the RSI for The Netherlands of the increased representation of China in the ISI Web of Science. Although the booming of China is visible in the RSI for The Netherlands, especially in the last decade and in fields where the countries have opposite specializations, the basic research profile as measured by the RSI remains the same. We conclude that the indicator is robust with regard to booming countries, and that it may suffice to observe the general changes in the research profile of the database if the RSI for a country is studied over time.