Search

In 2011, the programme for Severo Ochoa Centers of Excellence and Mar\'ia de Maeztu Units of Excellence was launched for the first time. Since this programme has become one of the axes of the Spanish scientific policy. 186 million euros have been distributed and 26 centers and 16 units have been accredited. One of the most relevant criteria for submission is the need for guarantor researchers to have a Normalized Impact of 1.5. In this work, we critically analyze the origin of this bibliometric indicator rooted in the 1980s, the different variants that have been proposed and the limitations its use in this programme have. Finally, we offer a series of practical recommendations for a more accurate use of normalized impact indicators for evaluative purposes., Comment: Paper published in Spanish language in El profesional de la informaci\'on

The phenomenon of rankings is intimately related with the government interest in fiscalizing the research outputs of universities. New forms of managerialism have been introduced into the higher education system, leading to an increasing interest from funding bodies in developing external evaluation tools to allocate funds. Rankings rely heavily on bibliometric indicators. But bibliometricians have been very critical with their use. Among other, they have pointed out the over-simplistic view rankings represent when analyzing the research output of universities, as they consider them as homogeneous ignoring disciplinary differences. Although many university rankings now include league tables by fields, reducing the complex framework of universities' research activity to a single dimension leads to poor judgment and decision making. This is partly because of the influence disciplinary specialization has on research evaluation. This chapter analyzes from a methodological perspective how rankings suppress disciplinary differences which are key factors to interpret correctly these rankings., Comment: Robinson-Garcia, N., Jim\'enez-Contreras, E. (2017). Analyzing the disciplinary focus of universities: Can rankings be a one-size-fits-all? In: Downing, K., F.A. Ganotice (eds). World University Rankings and the Future of Higher Education. IGI Global, pp. 161-185. doi:10.4018/978-1-5225-0819-9.ch009

Social media based indicators or altmetrics have been under scrutiny for the last seven years. Their promise as alternative metrics for measuring scholarly impact is still far from becoming a reality. Up to now, most studies have focused on the understanding of the nature and relation of altmetric indicators with citation data. Few papers have analysed research profiles based on altmetric data. Most of these have related to researcher profiles and the expansion of these tools among researchers. This paper aims at exploring the coverage of the Altmetric.com database and its potential use in order to show universities' research profiles in relationship with other databases. We analyse a sample of four different Spanish universities.First, we observe a low coverage of altmetric indicators with only 36 percent of all documents retrieved from the Web of Science having an 'altmetric' score. Second, we observe that for the four universities analysed, the area of Science shows higher 'altmetric' scores that the rest of the research areas. Finally, considering the low coverage of altmetric data at the institutional level, it could be interesting for research policy makers to consider the development of guidelines and best practices guides to ensure that researchers disseminate adequately their research findings through social media., Comment: Paper accepted for the STI 2016 Conference held in Valencia, September 14-16

This paper aims at offering an evaluation framework of an Research and Development programme in the Biomedical Sciences. It showcases the Spanish Biomedical Research Networking Centres initiative (CIBER) as an example of the effect of research policy management on performance. For this it focuses on three specific aspects: its role on the national research output in the biomedical sciences, its effect on promoting translational research through internal collaboration between research groups, and the perception of researchers on the programme as defined by their inclusion of their CIBER centres in the address field. Research output derived from this programme represents around 25 per cent of the country's publications in the biomedical fields. After analysing a seven year period, the programme has enhanced collaborations between its members, but they do not seem to be sufficiently strong. With regard to the credit given to the initiative, 54.5 per cent of the publications mentioned this programme in their address, however an increase on the share of papers mention it is observed two years after it was launched. We suggest that by finding the point in which the share of mentions stabilises may be a good strategy to identify the complete fulfilment of these types of Research and Development policies., Comment: Pre-refereed version of paper accepted for publication in Research Evaluation (2016). 15 pages, 5 figures, 6 tables. V2 some typos corrected

We present the results of the Bibliometric Indicators for Publishers project (also known as BiPublishers). This project represents the first attempt to systematically develop bibliometric publisher rankings. The data for this project was derived from the Book Citation Index, and the study time period was 2009-2013. We have developed 42 rankings: 4 for by fields and 38 by disciplines. We display six indicators by publisher divided into three types: output, impact and publisher's profile. The aim is to capture different characteristics of the research performance of publishers. 254 publishers were processed and classified according to publisher type: commercial publishers and university presses. We present the main publishers by fields. Then, we discuss the main challenges presented when developing this type of tools. The BiPublishers ranking is an on-going project which aims to develop and explore new data sources and indicators to better capture and define the research impact of publishers., Comment: Paper accepted for publication in Revista Espa\~nola de Documentaci\'on Cient\'ifica. v2 corrects some mispells and errors

We present an analysis of data citation practices based on the Data Citation Index from Thomson Reuters. This database launched in 2012 aims to link data sets and data studies with citations received from the other citation indexes. The DCI harvests citations to research data from papers indexed in the Web of Science. It relies on the information provided by the data repository as data citation practices are inconsistent or inexistent in many cases. The findings of this study show that data citation practices are far from common in most research fields. Some differences have been reported on the way researchers cite data: while in the areas of Science and Engineering and Technology data sets were the most cited, in Social Sciences and Arts and Humanities data studies play a greater role. A total of 88.1 percent of the records have received no citation, but some repositories show very low uncitedness rates. Although data citation practices are rare in most fields, they have expanded in disciplines such as crystallography and genomics. We conclude by emphasizing the role that the DCI could play in encouraging the consistent, standardized citation of research data; a role that would enhance their value as a means of following the research process from data collection to publication., Comment: Paper accepted for publication in the Journal of the Association for Information Science and Technology. v2 revises style on text, title and abstract

Here we describe the Bibliometric Indicators for Publishers Project, an initiative undertaken by EC3Metrics SL for the analysis and development of indicators based on books and book chapters. Its goal is to study and analyze the publication and citation patterns of books and book chapters considering academic publishers as the unit of analysis. It aims at developing new methodologies and indicators that can better capture and define the research impact of publishers. It is an on-going project in which data sources and indicators are tested. We consider academic publishers as an analogy of journals, focusing on them as the unit of analysis. In this working paper we present the http://bipublishers.es website where all findings derived from the project are displayed. We describe the data retrieval and normalization process and we show the main results. A total 482,470 records have been retrieved and processed, identifying 342 publishers from which 254 have been analyzed. Then six indicators have been calculated for each publisher for four fields and 38 disciplines and displayed., Comment: v.2 link to website and typos corrected

This paper analyzes the effect of the Catalan programme ICREA for the selection of talented researchers on the percentage of highly cited papers of Catalan universities, Comment: This paper was presented as poster at the STI Conference held in Leiden, 3-5 september, 2014

Descriptive analysis on the citation distribution of the Thomson Reuters' Data Citation Index by publication type and four broad areas: Science, Engineering & Technology, Humanities & Arts and Social Sciences., Comment: Presented at the STI Conference held in Leiden, 3-5 september 2014

The peer review system has been traditionally challenged due to its many limitations especially for allocating funding. Bibliometric indicators may well present themselves as a complement. Objective: We analyze the relationship between peers' ratings and bibliometric indicators for Spanish researchers in the 2007 National R&D Plan for 23 research fields. We analyze peers' ratings for 2333 applications. We also gathered principal investigators' research output and impact and studied the differences between accepted and rejected applications. We used the Web of Science database and focused on the 2002-2006 period. First, we analyzed the distribution of granted and rejected proposals considering a given set of bibliometric indicators to test if there are significant differences. Then, we applied a multiple logistic regression analysis to determine if bibliometric indicators can explain by themselves the concession of grant proposals. 63.4% of the applications were funded. Bibliometric indicators for accepted proposals showed a better previous performance than for those rejected; however the correlation between peer review and bibliometric indicators is very heterogeneous among most areas. The logistic regression analysis showed that the main bibliometric indicators that explain the granting of research proposals in most cases are the output (number of published articles) and the number of papers published in journals that belong to the first quartile ranking of the Journal Citations Report. Bibliometric indicators predict the concession of grant proposals at least as well as peer ratings. Social Sciences and Education are the only areas where no relation was found, although this may be due to the limitations of the Web of Science's coverage. These findings encourage the use of bibliometric indicators as a complement to peer review in most of the analyzed areas.

In this paper we review the socalled altmetrics or alternative metrics. This concept raises from the development of new indicators based on Web 2.0, for the evaluation of the research and academic activity. The basic assumption is that variables such as mentions in blogs, number of twits or of researchers bookmarking a research paper for instance, may be legitimate indicators for measuring the use and impact of scientific publications. In this sense, these indicators are currently the focus of the bibliometric community and are being discussed and debated. We describe the main platforms and indicators and we analyze as a sample the Spanish research output in Communication Studies. Comparing traditional indicators such as citations with these new indicators. The results show that the most cited papers are also the ones with a highest impact according to the altmetrics. We conclude pointing out the main shortcomings these metrics present and the role they may play when measuring the research impact through 2.0 platforms.

This study aims to analyse whether library loans statistics can be used as a measure of monograph use and as a selection criteria for inclusion in citation indexes. For this, we conducted an exploratory study based on loan data (1000 most borrowed monographs) from two non-Anglo-Saxon European university libraries (Granada and Vienna) with strong social sciences and humanities components. Loans to scientists only were also analysed at the University of Vienna. Furthermore, citation counts for the 100 most borrowed scientific monographs (SM) and textbooks or manuals (MTB) were retrieved from Web of Science and Google Scholar. The results show considerable similarities in both libraries: the percentage of loans for books in national languages represents almost 96 per cent of the total share and SM accounts only for 10 to 13 per cent. When considering loans to scientists only, the percentage of English books increases to 30 per cent, the percentage of SM loans also increases (approx 80 per cent). Furthermore, we found no significant correlations between loans and citations. Since loan statistics are currently insufficient for measuring the use of monographs, their suggested use as an applicable selection criterion for book citation indexes is not yet feasible. Data improvement and aggregation at different levels is a challenge for modern libraries in order to enable the exploitation of this invaluable information source for scientometric purposes., Comment: Paper accepted in ISSI 2013

This paper presents a first approach to analyzing the factors that determine the citation characteristics of books. For this we use the Thomson Reuters' Book Citation Index, a novel multidisciplinary database launched in 2010 which offers bibliometric data of books. We analyze three possible factors which are considered to affect the citation impact of books: the presence of editors, the inclusion in series and the type of publisher. Also, we focus on highly cited books to see if these factors may affect them as well. We considered as highly cited books, those in the top 5% of the most highly cited ones of the database. We define these three aspects and we present the results for four major scientific areas in order to identify field-based differences (Science, Engineering & Technology, Social Sciences and Arts & Humanities). Finally we conclude observing that differences were noted for edited books and types of publishers. Although books included in series showed higher impact in two areas., Comment: Paper accepted for the ISSI 2013. v2 of this paper published in Scientometrics

Bibliometric mapping and visualization techniques represent one of the main pillars in the field of scientometrics. Traditionally, the main methodologies employed for representing data are Multi-Dimensional Scaling, Principal Component Analysis or Correspondence Analysis. In this paper we aim at presenting a visualization methodology known as Biplot analysis for representing bibliometric and science and technology indicators. A Biplot is a graphical representation of multivariate data, where the elements of a data matrix are represented according to dots and vectors associated with the rows and columns of the matrix. In this paper we explore the possibilities of applying the Biplot analysis in the research policy area. More specifically we will first describe and introduce the reader to this methodology and secondly, we will analyze its strengths and weaknesses through three different study cases: countries, universities and scientific fields. For this, we use a Biplot analysis known as JK-Biplot. Finally we compare the Biplot representation with other multivariate analysis techniques. We conclude that Biplot analysis could be a useful technique in scientometrics when studying multivariate data and an easy-to-read tool for research decision makers., Comment: Accepted for publication in Journal of the American Society for Information Science and Technology