Search

The ZHAW Zurich University of Applied Sciences (ZHAW) has become a member of the Coalition for Advancing Research Assessment (CoARA) at the end 2022. In accordance with the commitments of this membership, the ZHAW has drafted an action plan outlining the conceptualisation, testing and implementation of changes in research assessment practices until the end of 2027. The ZHAW is a university of applied sciences and has chosen focus areas in accordance with the nature of our research activities. In particular, we have identified 3 levels and 4 dimensions that are of particular relevance to us: hence our action plan is running under the headline “addressing three levels and four dimensions”. We have found partners across Switzerland and Europe who are willing to discuss and develop these levels and dimensions with us. We thus have been able to be included in the first wave of CoARA working groups and will potentially participate in further such groups. The levels of research assessment we will consider are: the researcher, organizational units within the university, and the university as a whole. The four dimensions we want to focus our discussion on are: assessment of practice-based / transdisciplinary research and development and its impact, integration of Open Science (OS) contributions into research assessments, the responsible use of quantitative research assessment indicators, and career assessment of researchers. We have identified 11 Milestones we will work on until the end of 2027.

The topic of acknowledgements has produced abundant research since the 1970s, though, as previous studies point out, the value of acknowledgements has not yet been demonstrated and further research is limited by lack of conceptualization. This study focuses on funding acknowledgements (FAs), considering that funding represents an important input in the scientific process. In this context, 183 scientific publications retrieved from Scopus from the 1970s until June 2020 were analyzed, with the aim of systematizing conceptually this body of research and contributing to a theory of acknowledgements. Results are summarized into the following main themes: the meaning of FAs; data sources for acknowledgements; the process of funding; association of funding with productivity, impact, and collaboration; and other aspects affected by funding. The literature reviewed shows that a theory of acknowledgements based on the reward triangle, as in previous studies, is unable to capture the extreme complexity of the scientific activity affecting and being affected by FAs. Funding bodies appear as clear and influential actors in the scientific communication system, making important decisions on the research that is supported, and influencing the type of knowledge produced. Funding agencies hold a responsibility regarding the data that they may collect on their programs, as well as the normalization policies they need to develop so that funded authors can reference with less ambiguity the financial source of their projects. Finally, the need to assess the impact of research funding beyond the scientific community that is, the societal impact, is also addressed., Depto. de Biblioteconomía y Documentación, Fac. de Ciencias de la Documentación, TRUE, pub

The use of bibliometric indicators, and in particular of the Impact Factor of journals in evaluating the scientific production of researchers, has recently been subjected to international review, highlighting the need to reconsider the peer-review activity as a procedure that allows to recognize the different activities connected to the research of scholars. The article discusses some reports published internationally – DORA, The Leiden Manifesto, Report of the European Commission of 2021 – and highlights the significant connection between good evaluation practices and the Hong Kong Manifesto, dedicated to research integrity and the ethics of research evaluation.

This report designs and develops an architecture proposal (framework) for the evaluation of university teaching staff in the Spanish university system. The proposal aims to minimize the malpractices detected, adapt to the standards and recommendations from the scientific literature, and include those transversal aspects necessary to assess the impact of the research in the environment in which the evaluation is carried out, without losing sight of the possible extrapolation of the model to any other evaluation environment (departments, institutes, and research centers, etc.). The proposal is also offered as a holistic, flexible, inclusive, and customizable model, with the aim that it can be discussed by different actors: scientific community, the university community, evaluation agencies, government entities, public, private, and non-profit organizations, union and business associations, research staff, evaluation professionals, professionals from the business world, among many other actors. To achieve the main objective of this report, the following specific goals are established: a) compare existing teacher evaluation systems in Spain and implemented by the different evaluation agencies, both national and regional (Autonomous Communities), in order to obtain a general descriptive radiography of the established scales and the indicators used, with the consequent precautions due to the heterogeneity of the different figures analyzed); b) design the architecture of an evaluation model for university teaching staff, especially oriented towards the identification and classification of the different items to be evaluated and their organization (dimensions) and structure (categories, subcategories, facets), without entering into establishing scales, specific scores or professional figures.

[ES] ANECA ha puesto en marcha en diciembre de 2023 un cambio en el modelo de evaluación científica siguiendo los criterios de DORA y de COARA. La evaluación científica gira ahora alrededor de conceptos como el de evaluación cualitativa y bibliometría narrativa. Se precisa, para la solicitud de sexenios de 2024, poner en contexto las métricas y aportar otra serie de datos no basados en citas, buscando el impacto de las investigaciones fuera del mundo académico. Por ello, se sugiere un itinerario para recorrer las fuentes necesarias en la solicitud de sexenios, teniendo en cuenta que ahora, todas las fuentes son ilustrativas, ninguna es normativa, y todas las dimensiones son importantes pero ninguna obligatoria., [ES] ANECA has launched in December 2023 a change in the scientific evaluation model following the DORA and COARA criteria. Scientific evaluation now revolves around concepts such as qualitative evaluation and narrative bibliometrics. It is necessary, for the 2024 sexenios application, to put the metrics in context and provide another series of data not based on citations, seeking the impact of research outside the academic world. For this reason, we suggest an itinerary to go through the necessary sources in the application for six-year periods, bearing in mind that now, all the sources are illustrative, none are normative, and all the dimensions are important but none are compulsory.

This dataset includes the raw data used to carry out a study related to the analysis of the DORA Declaration on Twitter. The dataset includes the tweets collected from the Twitter Academic API (comprising three collections: tweets published by DORA, tweets mentioning DORA, and tweets including a DORA-related hashtag), supplementary material (including extra tables and figures), and the script used to collect data from Twitch API.

[ES] ANECA ha puesto en marcha en diciembre de 2023 un cambio en el modelo de evaluación científica siguiendo los criterios de DORA y de COARA. La evaluación científica gira ahora alrededor de conceptos como el de evaluación cualitativa y bibliometría narrativa. Se precisa, para la solicitud de sexenios de 2024, poner en contexto las métricas y aportar otra serie de datos no basados en citas, buscando el impacto de las investigaciones fuera del mundo académico. Por ello, se sugiere un itinerario para recorrer las fuentes necesarias en la solicitud de sexenios, teniendo en cuenta que ahora, todas las fuentes son ilustrativas, ninguna es normativa, y todas las dimensiones son importantes pero ninguna obligatoria., [ES] ANECA has launched in December 2023 a change in the scientific evaluation model following the DORA and COARA criteria. Scientific evaluation now revolves around concepts such as qualitative evaluation and narrative bibliometrics. It is necessary, for the 2024 sexenios application, to put the metrics in context and provide another series of data not based on citations, seeking the impact of research outside the academic world. For this reason, we suggest an itinerary to go through the necessary sources in the application for six-year periods, bearing in mind that now, all the sources are illustrative, none are normative, and all the dimensions are important but none are compulsory.

[ES] ANECA ha puesto en marcha en diciembre de 2023 un cambio en el modelo de evaluación científica siguiendo los criterios de DORA y de COARA. La evaluación científica gira ahora alrededor de conceptos como el de evaluación cualitativa y bibliometría narrativa. Se precisa, para la solicitud de sexenios de 2024, poner en contexto las métricas y aportar otra serie de datos no basados en citas, buscando el impacto de las investigaciones fuera del mundo académico. Por ello, se sugiere un itinerario para recorrer las fuentes necesarias en la solicitud de sexenios, teniendo en cuenta que ahora, todas las fuentes son ilustrativas, ninguna es normativa, y todas las dimensiones son importantes pero ninguna obligatoria., [ES] ANECA has launched in December 2023 a change in the scientific evaluation model following the DORA and COARA criteria. Scientific evaluation now revolves around concepts such as qualitative evaluation and narrative bibliometrics. It is necessary, for the 2024 sexenios application, to put the metrics in context and provide another series of data not based on citations, seeking the impact of research outside the academic world. For this reason, we suggest an itinerary to go through the necessary sources in the application for six-year periods, bearing in mind that now, all the sources are illustrative, none are normative, and all the dimensions are important but none are compulsory.

This dataset includes the raw data used to carry out a study related to the analysis of the DORA Declaration on Twitter. The dataset includes the tweets collected from the Twitter Academic API (comprising three collections: tweets published by DORA, tweets mentioning DORA, and tweets including a DORA-related hashtag), supplementary material (including extra tables and figures), and the script used to collect data from Twitch API.

A model of a rich and diverse complex CRIS that can be compliant with CoARA requirements. Although designed for the spanish Research Council (CSIC) it can be customised for the needs of other organizations. Presenbted at the "euroCRIS Strategic Membership Meeting 2023" that took place in Pamplona, Spain (Nov 21-23, 2023)

Authorship is associated with scientific capital and prestige, and corresponding authorship is used in evaluation as a proxy for scientific status. However, there are no empirical analyses on the validity of the corresponding authorship metadata in bibliometric databases. This paper aims at looking at differences on the corresponding authorship metadata in Web of Science (WoS) and Scopus, to investigate how the relationship between author position and corresponding authors varies by discipline and country and analyzing changes in the position of corresponding authors over time. We find that both, WoS and Scopus, have accuracy issues when it comes to assigning corresponding authorship. Although the number of documents with a reprint author has increased over time in both databases, however, WoS indexed more of those papers than Scopus, and there are significant differences between the two databases in terms of who the corresponding author is. Although metadata is not complete in WoS, corresponding authors are normally first authors with a declining trend over time, favoring middle and last authors, especially in Medical, Natural & Engineering fields. These results reinforce the importance of considering how databases operationalize and index concepts like corresponding authors, being this particularly important when are used in research assessment.

This paper focuses on systematic errors and propose how these can be measured and reported when informing and offering bibliometric data for policy purposes. We analysed differences in the calculation of impact indicators, when different 23 different classification schemes derived from Clarivate’s InCites suite are used, specifically on five indicators: the Category Normalized Citation Indicator, the total number of citations, the H-index, the 1% of most cited articles, and the 10% of most cited articles. Findings show that citation counts are quite stable, with a difference of 13%. However, proportional indicators, such as top 1% and top 10% most cited articles, tend to vary more between universities, although on average they are lower than in the previous case. The results of this study can be used in estimating the error level in the research assessment of countries, institutions, and individuals.

This report designs and develops an architecture proposal (framework) for the evaluation of university teaching staff in the Spanish university system. The proposal aims to minimize the malpractices detected, adapt to the standards and recommendations from the scientific literature, and include those transversal aspects necessary to assess the impact of the research in the environment in which the evaluation is carried out, without losing sight of the possible extrapolation of the model to any other evaluation environment (departments, institutes, and research centers, etc.). The proposal is also offered as a holistic, flexible, inclusive, and customizable model, with the aim that it can be discussed by different actors: scientific community, the university community, evaluation agencies, government entities, public, private, and non-profit organizations, union and business associations, research staff, evaluation professionals, professionals from the business world, among many other actors. To achieve the main objective of this report, the following specific goals are established: a) compare existing teacher evaluation systems in Spain and implemented by the different evaluation agencies, both national and regional (Autonomous Communities), in order to obtain a general descriptive radiography of the established scales and the indicators used, with the consequent precautions due to the heterogeneity of the different figures analyzed); b) design the architecture of an evaluation model for university teaching staff, especially oriented towards the identification and classification of the different items to be evaluated and their organization (dimensions) and structure (categories, subcategories, facets), without entering into establishing scales, specific scores or professional figures.

The tensions generated in the scientific assessment expose flaws in the mechanisms used for this purpose, especially for the social sciences and humanities, which insist on being different from traditional research methodologies. Including the concept research-creation in the Colombian National System of Categorization of Researchers (SNCI by its Spanish acronym), is a milestone in recognizing these differences. However, its impact on the practice of humanities researchers is scarcely known. Through a qualitative study, this article seeks to improve understanding of the perceptions in the Colombian research landscape since the research-creation appeared in the measurement system. The descriptive analysis of the opinions of researchers in this area allows us to recognize that, although it made visible an important research production, it perpetuates practices that are harmful to research, and corroborates the need for a profound reorientation of how knowledge production is observed., Fricciones generadas por la evaluación científica exponen fallas en los mecanismos usados para tal fin, especialmente para las ciencias sociales y humanas, que insisten en diferenciarse de los procesos tradicionales de investigación. El concepto investigación-creación, que aparece en el modelo de medición de investigadores colombiano, se configura como un hito en el reconocimiento de esas diferencias. Sin embargo, es escaso lo que se conoce sobre su impacto en la práctica de los investigadores de las humanidades. A través de un estudio cualitativo, este artículo busca mejorar la comprensión sobre las percepciones que se configuraron en el panorama investigativo colombiano, desde el surgimiento de la investigación-creación en el sistema de medición. El análisis descriptivo de opiniones de investigadores de esta área permite reconocer que, aunque hizo visible una producción investigativa importante, perpetúa prácticas nocivas para la investigación y corrobora la necesidad de reorientar profundamente cómo se observa la producción de conocimiento.

“Those who can make you believe absurdities can make you commit atrocities” is a famous aphorism commonly attributed to Voltaire. The choice to publish under an alias for François-Marie Arouet, reveals that there was some sort of awareness of the relationship between power, knowledge, truth and people’s tyrannical tendencies on his part. What our dear enlightened friend could most likely not foresee, is the enormity of the scale of how big such atrocities can get under the precept of good intentions. For example, both the reign of terror following the French Revolution and the world wars gathered their emotional energy precisely due to their promise of being a manifestation for the greater good. Such dogmatic moral justification of collective benefit, then rationalize and normalize otherwise inexcusable individual transgressions. One of the fads of Anglophone Higher Education over the recent decades has been the introduction of research impact. Both in terms of quality evaluation of research and as a goal for Higher Education in general. For example, the UK national research evaluation defines impact as a ”change or benefit to the economy, society, culture, public policy or services, health, the environment or quality of life, beyond academia”. However, since such evaluation constitute part of the wider performance-based research-funding ecosystem; it is highly unlikely that anyone would ever intentionally submit something they themselves regard as a negative. Hence, implicit in this evaluation of research impact is an appeal to the public good. The article systematizes conceptually the distinction between the categories of greater and public good as socially constructed within the context of research in particular and wider society in general. Specifically, we apply the theoretical lens of interaction ritual chains to discuss the confluence of purpose, emotional excitement, and social manipulation in the generation of (research) impact. Conceptually, we distinguish

This report designs and develops an architecture proposal (framework) for the evaluation of university teaching staff in the Spanish university system. The proposal aims to minimize the malpractices detected, adapt to the standards and recommendations from the scientific literature, and include those transversal aspects necessary to assess the impact of the research in the environment in which the evaluation is carried out, without losing sight of the possible extrapolation of the model to any other evaluation environment (departments, institutes, and research centers, etc.). The proposal is also offered as a holistic, flexible, inclusive, and customizable model, with the aim that it can be discussed by different actors: scientific community, the university community, evaluation agencies, government entities, public, private, and non-profit organizations, union and business associations, research staff, evaluation professionals, professionals from the business world, among many other actors. To achieve the main objective of this report, the following specific goals are established: a) compare existing teacher evaluation systems in Spain and implemented by the different evaluation agencies, both national and regional (Autonomous Communities), in order to obtain a general descriptive radiography of the established scales and the indicators used, with the consequent precautions due to the heterogeneity of the different figures analyzed); b) design the architecture of an evaluation model for university teaching staff, especially oriented towards the identification and classification of the different items to be evaluated and their organization (dimensions) and structure (categories, subcategories, facets), without entering into establishing scales, specific scores or professional figures.

[ES] La reforma de la evaluación de la investigación emprendida por muchas de las mayores instituciones científicas europeas responde a problemas profundos y requiere cambios que van mucho más allá de la incorporación de nuevos indicadores y la adaptación de las prácticas actuales., [EN] The reform of research evaluation undertaken by many of the largest European scientific institutions responds to deep-seated problems and requires changes that go far beyond the incorporation of new indicators and the adaptation of current practices.

[ES] La evaluación de la ciencia es necesaria para favorecer la investigación de calidad y optimizar las inversiones en investigación. La revisión por pares o peer review es el principal sistema de evaluación de la calidad de la ciencia, llevado a cabo por expertos en las materias a evaluar. La participación de otros actores, sociales o políticos, adquiere especial interés para valorar la relevancia social de la investigación. Los indicadores bibliométricos, aplicados con conocimiento de sus limitaciones, pueden constituir un apoyo importante para los expertos porque aportan objetividad y transparencia. Asistimos hoy a un movimiento de reforma del sistema de evaluación tradicional, propiciado por el uso abusivo y reduccionista de los indicadores cuantitativos en el pasado, y por la necesidad de instaurar aproximaciones más holísticas, aplicar las métricas de forma responsable y apoyar el desarrollo de una ciencia más abierta y colaborativa., [EN] The evaluation of science is necessary to promote high quality research and optimize research investments. Peer review is the main system for evaluating the quality of science, carried out by experts in the subjects to be assessed. The participation of other actors, whether social or political, is of particular interest for assessing the social relevance of research. Bibliometric indicators, applied with knowledge of their limitations, can be an important support for experts because they provide objectivity and transparency. Today we are witnessing a movement to reform the traditional evaluation system, fostered by the abusive and reductionist use of quantitative indicators in the past, and by the need to establish more holistic approaches, to apply metrics responsibly and to support the development of a more open and collaborative science.

The scope of this master thesis is the scientific publishing in Social Science at Lund University. By combining bibliometric studies and qualitative interviews, this thesis investigates how scholars in Social Sciences at Lund University respond to performance- based evaluation. The aim is to explore how research practices, publication patterns and disciplinary norms are affected by the increased role of bibliometric measurement in research evaluation, but also how the individual scholars use bibliometric measurement to evaluate candidates for allocation of research funds and in academic appointments. The thesis is based on the theories of disciplinary differences developed by Richard Whitley and on the theories of the evaluation society by Peter Dahler-Larsen. These theories suggest a framework based on organizational and sociological perspectives that offers ways to explain differences in publication patterns in relation to disciplinary norms in the scientific fields and how they react to evaluation. The study is based on two empirical studies of the faculty of Social Science, the faculty of Economics and the faculty of Law. A bibliometric survey of the researcher's publication patterns has been conducted, and publications that are registered in the local publication database (LUCRIS) have been analyzed. The data was combined with an empirical evaluation of qualitative interviews with senior scientists both in their role of scientist and in their role as evaluators at the Swedish Research Council and at the teacher proposal board. Results from the bibliometric study depicts a substantial increase in publication with scientific journal articles, monographs and book chapters being the most important publishing forms. Furthermore, it demonstrates a steady increase in the proportion of international journal publications. Many respondents remarked in the interviews that publication patterns are changing towards more international publishing and increased article publish

The evaluation of science is a subject that has generated debates and tensions in the scientific and academic community. The dominant model for the evaluation of scientific activity presents a series of criticisms and limitations, with special emphasis on the field of Social Sciences. In order to propose solutions to this phenomenon, the research formulates as a general objective to develop an alternative model of scientific production evaluation for the Social Sciences field, from the perspective of technoscience as a social phenomenon. The research raises methodological triangulation, by integrating qualitative and quantitative analysis approaches, which include methods and techniques such as bibliographic review, application of traditional metrics and alternative science measurement, as well as the application of interviews, surveys and the Delphi method. An analysis of the state of the matter was carried out, which made it possible to identify the questions, criticisms and limitations of the science evaluation processes, which contributed to establishing a balance in relation to the contributions, discussions, inputs, referential frameworks, conceptions, approaches and perspectives. The proposal is based on the techno-scientific changes introduced in the knowledge production that allow a complementation between the knowledge production and society. From this perspective, the quality and relevance of the scientific research results, epistemological diversity and the active participation of the academic and social community are promoted., La evaluación de la ciencia es un tema que ha generado debates y tensiones en la comunidad científica y académica. El modelo dominante de evaluación de la actividad científica presenta una serie de críticas y limitaciones, con especial énfasis en el ámbito de las Ciencias Sociales. Para proponer soluciones a este fenómeno, la investigación formula como objetivo general desarrollar un modelo alternativo de evaluación de la producción científica para el campo de las Ciencias Sociales, desde la perspectiva de la tecnociencia como fenómeno social. La investigación plantea la triangulación metodológica, mediante la integración de enfoques de análisis cualitativos y cuantitativos, que incluyen métodos y técnicas como la revisión bibliográfica, la aplicación de métricas tradicionales y alternativas de medición de la ciencia, así como la aplicación de entrevistas, encuestas y el método Delphi. Se realizó un análisis del estado de la cuestión, que permitió identificar los cuestionamientos, críticas y limitaciones de los procesos de evaluación de la ciencia, lo que contribuyó a establecer un balance en relación a los aportes, discusiones, insumos, marcos referenciales, concepciones, enfoques y perspectivas. La propuesta se basa en los cambios tecno-científicos introducidos en la producción de conocimiento que permiten una complementación entre la producción de conocimiento y la sociedad. Desde esta perspectiva, se promueve la calidad y relevancia de los resultados de la investigación científica, la diversidad epistemológica y la participación activa de la comunidad académica y social., La evaluación de la ciencia es un tema que ha generado debates y tensiones en la comunidad científica y académica. El modelo dominante de evaluación de la actividad científica presenta una serie de críticas y limitaciones, con especial énfasis en el ámbito de las Ciencias Sociales. Para proponer soluciones a este fenómeno, la investigación formula como objetivo general desarrollar un modelo alternativo de evaluación de la producción científica para el campo de las Ciencias Sociales, desde la perspectiva de la tecnociencia como fenómeno social. La investigación plantea la triangulación metodológica, mediante la integración de enfoques de análisis cualitativos y cuantitativos, que incluyen métodos y técnicas como la revisión bibliográfica, la aplicación de métricas tradicionales y alternativas de medición de la ciencia, así como la aplicación de entrevistas, encuestas y el método Delphi. Se realizó un análisis del estado de la cuestión, que permitió identificar los cuestionamientos, críticas y limitaciones de los procesos de evaluación de la ciencia, lo que contribuyó a establecer un balance en relación a los aportes, discusiones, insumos, marcos referenciales, concepciones, enfoques y perspectivas. La propuesta se basa en los cambios tecno-científicos introducidos en la producción de conocimiento que permiten una complementación entre la producción de conocimiento y la sociedad. Desde esta perspectiva, se promueve la calidad y relevancia de los resultados de la investigación científica, la diversidad epistemológica y la participación activa de la comunidad académica y social.

The evaluation of science is a subject that has generated debates and tensions in the scientific and academic community. The dominant model for the evaluation of scientific activity presents a series of criticisms and limitations, with special emphasis on the field of Social Sciences. In order to propose solutions to this phenomenon, the research formulates as a general objective to develop an alternative model of scientific production evaluation for the Social Sciences field, from the perspective of technoscience as a social phenomenon. The research raises methodological triangulation, by integrating qualitative and quantitative analysis approaches, which include methods and techniques such as bibliographic review, application of traditional metrics and alternative science measurement, as well as the application of interviews, surveys and the Delphi method. An analysis of the state of the matter was carried out, which made it possible to identify the questions, criticisms and limitations of the science evaluation processes, which contributed to establishing a balance in relation to the contributions, discussions, inputs, referential frameworks, conceptions, approaches and perspectives. The proposal is based on the techno-scientific changes introduced in the knowledge production that allow a complementation between the knowledge production and society. From this perspective, the quality and relevance of the scientific research results, epistemological diversity and the active participation of the academic and social community are promoted., La evaluación de la ciencia es un tema que ha generado debates y tensiones en la comunidad científica y académica. El modelo dominante de evaluación de la actividad científica presenta una serie de críticas y limitaciones, con especial énfasis en el ámbito de las Ciencias Sociales. Para proponer soluciones a este fenómeno, la investigación formula como objetivo general desarrollar un modelo alternativo de evaluación de la producción científica para el campo de las Ciencias Sociales, desde la perspectiva de la tecnociencia como fenómeno social. La investigación plantea la triangulación metodológica, mediante la integración de enfoques de análisis cualitativos y cuantitativos, que incluyen métodos y técnicas como la revisión bibliográfica, la aplicación de métricas tradicionales y alternativas de medición de la ciencia, así como la aplicación de entrevistas, encuestas y el método Delphi. Se realizó un análisis del estado de la cuestión, que permitió identificar los cuestionamientos, críticas y limitaciones de los procesos de evaluación de la ciencia, lo que contribuyó a establecer un balance en relación a los aportes, discusiones, insumos, marcos referenciales, concepciones, enfoques y perspectivas. La propuesta se basa en los cambios tecno-científicos introducidos en la producción de conocimiento que permiten una complementación entre la producción de conocimiento y la sociedad. Desde esta perspectiva, se promueve la calidad y relevancia de los resultados de la investigación científica, la diversidad epistemológica y la participación activa de la comunidad académica y social., La evaluación de la ciencia es un tema que ha generado debates y tensiones en la comunidad científica y académica. El modelo dominante de evaluación de la actividad científica presenta una serie de críticas y limitaciones, con especial énfasis en el ámbito de las Ciencias Sociales. Para proponer soluciones a este fenómeno, la investigación formula como objetivo general desarrollar un modelo alternativo de evaluación de la producción científica para el campo de las Ciencias Sociales, desde la perspectiva de la tecnociencia como fenómeno social. La investigación plantea la triangulación metodológica, mediante la integración de enfoques de análisis cualitativos y cuantitativos, que incluyen métodos y técnicas como la revisión bibliográfica, la aplicación de métricas tradicionales y alternativas de medición de la ciencia, así como la aplicación de entrevistas, encuestas y el método Delphi. Se realizó un análisis del estado de la cuestión, que permitió identificar los cuestionamientos, críticas y limitaciones de los procesos de evaluación de la ciencia, lo que contribuyó a establecer un balance en relación a los aportes, discusiones, insumos, marcos referenciales, concepciones, enfoques y perspectivas. La propuesta se basa en los cambios tecno-científicos introducidos en la producción de conocimiento que permiten una complementación entre la producción de conocimiento y la sociedad. Desde esta perspectiva, se promueve la calidad y relevancia de los resultados de la investigación científica, la diversidad epistemológica y la participación activa de la comunidad académica y social.

This paper analyses the scientifc activity related to open science in Spain and its infuence on public policy from a bibliometric perspective. For this purpose, Spanish centres’ projects and publications on open science from 2010 to 2020 are studied. Subsequently, policy documents using papers related to open science are analysed to study their infuence on policymaking. A total of 142 projects and 1491 publications are analysed, 15% of which are mentioned in policy documents.The publications cited in policy documents display high proportions of international collaboration, open access publication and publication in frst-quartile journals. The fndings underline governments’ leading role in the implementation of open science policies and the funding of open science research. The same government agencies that promote and fund open science research are shown to use that research in their institutional reports, a process known as knowledge fow feedback. Other non-academic actors are also observed to make use of the knowledge produced by open science research, showing how the open science movement has crossed the boundaries of academia.

[EN] Research evaluation has been a constant activity in the production of scientific knowledge. A significant portion of its formalization has taken place in recent decades. Bbibliometric and quantitative productivity indicators have been the most widely employed tools, but there are now significant criticisms of some of the procedures followed. Guided by these criticisms, we present in this report a diagnosis of the state of recognition of research activity in the field of social anthropology in Spain. We refer to this recognition in the form of “sexenios de investigación” (six-year research periods) conducted by the National Commission for Research Activity Evaluation (dependent on the National Agency for Quality Assessment and Accreditation, an autonomous body of the Spanish government). Based on a comparative study of the pitfalls of these metrics-based evaluation procedures across different areas, we identify some of the main obstacles hindering a higher success rate in the field of social anthropology. Finally, we propose specific measures for improvement, [ES] La evaluación de la investigación ha sido una actividad constante de la producción de conocimiento científico. Buena aparte de su formalización se ha realizado en las últimas décadas. El uso de indicadores bibliométricos y cuantitativos de productividad han sido las herramientas más utilizadas, pero en la actualidad existen ya importantes críticas a una parte de los procedimientos seguidos. Guiados por estas críticas, presentamos en este informe un diagnóstico sobre el estado del reconocimiento de la actividad investigadora en el área de Antropología social en España. Nos referimos al reconocimiento en forma de “sexenios de investigación” que realiza la Comisión Nacional Evaluadora de la Actividad Investigadora (dependiente de la Agencia Nacional de Evaluación de la Calidad y Acreditación, Organismo Autónomo de la Administración General del Estado español). Partiendo de un estudio comparado por áreas sobre las trampas de estos procedimientos de evaluación basado en métricas, se identifican algunos de los obstáculos principales que impiden lograr una mayor tasa de éxito en el área de Antropología social. Finalmente, se sugieren medidas concretas de mejora.

Objective. The scientific performance of Peruvian universities was examined and characterized by aggregating information from their institutional repositories. Design/Methodology/Approach. A protocol was followed that included the following stages: (1) analysis and optimization of the DSpace database model; (2) data recovery and validation with the OAI-PMH Protocol; (3) obtaining an index of resources available in the repositories; (4) metadata extraction and processing; and (5) extraction and inclusion of keywords in the database. Results/Discussion. A diverse use of keywords was identified. 69,1% of them appeared only once. A significant gap was evident in producing theses between public and private universities. Conclusions. The discrepancy between the research areas of Peruvian universities and regional needs highlights the importance of greater alignment between academic production and the demands of society in each region; this underlines the need to adapt research programs to address specific problems and contribute more effectively to regional development., Objetivo: Este estudio generamos información significativa sobre el desempeño científico de las universidades peruanas mediante la agregación de sus repositorios institucionales, centrándose en el análisis de palabras clave. El objetivo principal fue evaluar y comprender la producción académica y científica de estas instituciones. Metodología: Para lograr este propósito, se siguió un protocolo que involucró varias etapas: análisis y optimización del modelo de base de datos de DSPACE, recuperación y validación de datos utilizando el Protocolo OAI-PMH, obtención de un índice de recursos disponibles en los repositorios, extracción y procesamiento de metadatos, y la extracción de palabras clave para su inclusión en la base de datos. Resultados y discusión: Existe una notable diversidad de palabras clave utilizadas, pues 69.1% aparecen solo una vez, también identificamos una brecha en la producción de tesis entre universidades públicas y privadas. Además, se resalta la discrepancia entre los problemas de la sociedad y las áreas de investigación estudiadas en las universidades, enfatizando la importancia de alinear mejor la producción académica con las demandas regionales. Conclusiones: Es necesario mejorar la gestión de los repositorios institucionales en las universidades peruanas debido a marcada diversidad en el uso de palabras clave, la brecha en la producción de tesis entre universidades públicas y privadas y la desalineación entre las áreas de investigación y las demandas sociales. Esto implica invertir en formación y capacitación de personal para asegurar que los repositorios cumplan su función fundamental de promover la investigación académica, científica y social en beneficio de la sociedad peruana.

[EN] There is a strong movement for radical reform of the scientific evaluation processes in accordance with the principles of the open science initiative. To support the new system, one or several open platforms specifically designed for evaluation purposes is badly needed. A brief critical evaluation of some of the services currently used or available is carried out, and it is concluded that, although they provide important content that can be reused, the services alone are not enough nor completely suitable for rigorous, reliable and transparent evaluation. A proposal for a new tool is introduced with a brief analysis of its feasibility, along with a description of the needs and characteristics it must comply with., [ES] Ante las peticiones de reforma de los procesos de evaluación científica en el marco del desarrollo de la llamada Open Science, se propone la creación de una o varias plataformas abiertas específicamente diseñadas para tal fin. Se realiza una breve evaluación crítica de algunos de los servicios actualmente utilizados o disponibles y se concluye que, aunque proporcionan contenidos importantes que pueden ser reutilizados, por si solos no son adecuados para una evaluación rigurosa, fidedigna y transparente. Algunas características que dicha nueva herramienta debe cumplir son descritas y comentada su viabilidad.

Gender differences in research funding exist, but bias evidence is elusive and findings are contradictory. Bias has multiple dimensions, but in evaluation processes, bias would be the outcome of the reviewers’ assessment. Evidence in observational approaches is often based either on outcome distributions or on modeling bias as the residual. Causal claims are usually mixed with simple statistical associations. In this paper we use an experimental design to measure the effects of a cause: the effect of the gender of the principal investigator (PI) on the score of a research funding application (treatment). We embedded a hypothetical research application description in a field experiment. The subjects were the reviewers selected by a funding agency, and the experiment was implemented simultaneously with the funding call’s peer review assessment. We manipulated the application item that described the gender of the PI, with two designations: female PI and male PI. Treatment was randomly allocated with block assignment, and the response rate was 100% of the population, avoiding problems of biased estimates in pooled data. Contrary to some research, we find no evidence that male or female PIs received significantly different scores, nor any evidence of same-gender preferences of reviewers regarding the applicants’ gender.

This study looks at assessment of PhD theses from two perspectives: criteria in use in assessment reports at a science faculty and norms of science. Fifty assessment reports were analysed inductively, resulting in thirteen categories that examiners consider when assessing a thesis. These categories were compared with norms of science as described in the sociology of science. The study shows a high congruence between the two perspectives, but also new categories worthy of further discussion and research. Relevance of the thesis research and quality by proxy (that publication is an indicator of quality) stand out very clearly in this study compared with earlier assessment research. These two categories are both relatively new categories in assessment research and indicate that the classical norms of science are changing with an increasing influence of post-academic norms in academia.

A multi-criteria decision-making system based on stakeholder evaluation is performed to investigate the nexus between research impact and sustainability performance in the agro-food sector, in Spain. This study attempts to go a step further beyond the scientific assessment of research by examining its societal contribution. The empirical application is built upon the ELECTRE III methodology. Combining Evaluation theory and Stakeholder theory, the analysis facilitates the assessment of research impact with the inclusion of stakeholders’ knowledge. Four research programmes are selected from different agro-food industries representing the case studies addressed in this study. Each stakeholder performs an evaluation of the research programmes based on indicators and sub-indicators of sustainability performance. The findings reveal ranking matrices of research impact and demonstrate its implicit contribution to the Sustainable Development Goals. This study provides insights to policy-makers and practitioners and sheds light on how research evaluation accentuates the transition to sustainable agro-food sector., The authors gratefully acknowledge the Editor and the anonymous reviewers for their constructive suggestions and feedback. We would like to thank the Institute of Agri-food Research and Technology (IRTA) for its support providing key information to carry out this study, Postprint (author's final draft)

In light of research complexity and the challenges of rationing limited funding resources, university administrators increasingly use bibliometrics for research evaluation. This effort is essential and worthwhile for better understanding of the university’s research strengths, capabilities, and opportunities. Bibliometric analysis is also a comprehensive tool to identify patterns and trends in research domains. To get a general view of research landscape at HKUST, the Library’s Research Support Services team conducted a small-scale bibliometric study which aimed to find out (i) the popular research topics and (ii) top collaborators of HKUST. This report was intended for anyone with a general interest in bibliometrics within the context of research management.

This year the new 2022 version of Clarivate’s impact factor profoundly changed the setup for some journals. Compli­cated by the pandemic, the introduction of early access into the tally of articles lead to the question of whether impact factors can still be considered the most important parameter for traditional bibliometric evaluations.

This year the new 2022 version of Clarivate’s impact factor profoundly changed the setup for some journals. Compli­cated by the pandemic, the introduction of early access into the tally of articles lead to the question of whether impact factors can still be considered the most important parameter for traditional bibliometric evaluations.

Policymakers are increasingly demanding that their investments in research and knowledge production should yield returns. The interest of evaluating the societal impact of research has grown rapidly as research funders request evidence that their investments lead to public benefits. This thesis is a study of how the Research Council of Norway has operationalized such demands into new evaluation practices revolving around the concept of Impact. The material to this study consists of one evaluation report that introduced Impact into Norwegian research evaluation, the council’s planning procedures in grant applications in addition to four expert interviews. Informed by practice-oriented document analysis, it highlights the transformative capacity of documents and emphasises the ways in which evaluation functions as tools to make visible phenomena, implement policies and transform researchers and their projects into alignment with Impact demands through processes of translation. Informed by actor-network analysis, I explore how evaluation stages and enacts realities into being., Interessa for å evaluere samfunnseffektane til forsking har vakse med at avgjerdstakarar krev bevis for at investeringar i forsking fører til nytte for samfunnet. I denne masteroppgåva undersøker eg korleis Noregs forskingsråd har operasjonalisert slike krav om til nye evalueringspraksisar orientert mot førestillinga "impact". Ved bruk av praksisorientert dokumentanalyse behandlar eg evaluering som eit optisk verktøy som gjer fenomen synlege, men også som eit politisk verktøy som overtyder evalueringsobjektet om validiteten til det som blir evaluert. Videre undersøker eg korleis Forskingsrådet planlegg for "impact" gjennom å redusere prosjektforslag til logiske forhold mellom predefinerte element som interessent, verknad og effekt. Ved hjelp av aktør-nettverksteori skildrar eg desse som omsettingsprosessar. Slik syner eg at meklarrolla Noregs forskingsråd har mellom styresmakta og forskingssektoren ikkje berre gir utslag i dei tematiske programma deira, men korleis Forskingsrådet ved bruk av evaluering formidlar statlege krav til og utøver kraft også mot den basisfinansierte forskingssektoren. Til slutt diskuterer eg evaluering i lys av vitskapssosiologiske innsikt som tilseier at røyndomar er ein konsekvens av og ikkje ei årsak til vitskaplege sanningar.

Higher education institutions (HEIs) have been undertaking considerable efforts to embed sustainability into their system; however, there have been limited studies on the incorporation of sustainability into research. In parallel, it has been found that women have an important role in contributing to sustainability since they are more engaged and have a more holistic perspective on sustainability than men. These two phenomena have resulted in limited studies about the contribution of women to sustainability research (i.e., in scientific publishing). A bibliometric analysis of more than 39,000 documents (with 147,090 authorships) was done to fill this gap, focusing on sustainability peer-review publications in Europe between 2015 and 2020. The results show that women's presence in sustainability research has been increasing during the last years; however, there are still few female authorship publications, even in fields that have been traditionally women-oriented. In addition, their publications have been less recognized by the scientific community. The results also show substantial gender differences in terms of author leadership, where female senior researchers are more likely to mentor female junior researchers than men. Female researchers tend to collaborate nationally, but they could improve their international collaboration since this can improve their research and impact. More support should be provided to female researchers to help foster women's sustainability engagement and holistic perspectives, reduce existing negative feedback loops, and increase positive ones. Achieving gender equality is sine qua non in achieving sustainable societies.

Higher education institutions (HEIs) have been undertaking considerable efforts to embed sustainability into their system; however, there have been limited studies on the incorporation of sustainability into research. In parallel, it has been found that women have an important role in contributing to sustainability since they are more engaged and have a more holistic perspective on sustainability than men. These two phenomena have resulted in limited studies about the contribution of women to sustainability research (i.e., in scientific publishing). A bibliometric analysis of more than 39,000 documents (with 147,090 authorships) was done to fill this gap, focusing on sustainability peer-review publications in Europe between 2015 and 2020. The results show that women's presence in sustainability research has been increasing during the last years; however, there are still few female authorship publications, even in fields that have been traditionally women-oriented. In addition, their publications have been less recognized by the scientific community. The results also show substantial gender differences in terms of author leadership, where female senior researchers are more likely to mentor female junior researchers than men. Female researchers tend to collaborate nationally, but they could improve their international collaboration since this can improve their research and impact. More support should be provided to female researchers to help foster women's sustainability engagement and holistic perspectives, reduce existing negative feedback loops, and increase positive ones. Achieving gender equality is sine qua non in achieving sustainable societies.

Conferences are an essential tool for scientific communication. In disciplines such as Computer Science the majority of original research results are published in conference proceedings. In this study, we have analyzed the role of conference proceedings in various disciplines and propose an alternative approach to research evaluation based on conference proceedings sources indexed in Scopus and Scimago Journal Rank (SJR). This allows one to categorize conference proceedings in quartiles Q1–Q4 by analogy with SJR journal quartiles. Out of 171 conference proceedings sources analyzed, 38 conference proceedings in Engineering (45% of the list) and 23 in Computer Science (32% of the list) have an SJR level corresponding to the first quartile journals in these areas, which emphasizes the exceptional importance of conferences in these disciplines The comparison of this bibliometric-driven ranking with the expert-driven CORE ranking in Computer Science showed a 62% overlap, as well as a significant average rank correlation of the category distribution. © 2022, Springer Nature Switzerland AG.

This year the new 2022 version of Clarivate’s impact factor profoundly changed the setup for some journals. Compli­cated by the pandemic, the introduction of early access into the tally of articles lead to the question of whether impact factors can still be considered the most important parameter for traditional bibliometric evaluations.

This year the new 2022 version of Clarivate’s impact factor profoundly changed the setup for some journals. Compli­cated by the pandemic, the introduction of early access into the tally of articles lead to the question of whether impact factors can still be considered the most important parameter for traditional bibliometric evaluations.

Feedback provided by peer reviewers plays a pivotal role in any journal peer-review model. Peer-review feedback helps authors reconsider their manuscripts in a new light and improve their work before it is published. While there is a wealth of knowledge and empirical evidence focusing on effective feedback practices in educational settings, there is a dearth of research on journal peer-review feedback, especially in some academic disciplines including the social sciences and education. To better understand ‘good’ and ‘bad’ peer-review feedback practices across academic disciplines, we conducted a systematic literature review, informed by grounded theory, that aimed to identify the feedback features and factors that exert an impact on quality of peer-review feedback. Findings from 20 publications indicate a list of good and bad features of peer-review feedback pertaining to content, language, tone, structure and timeliness. We also identified a number of internal and external factors that influence how peer reviewers provide feedback such as academics’ expertise, language skills, motivation and seniority, as well as external influences such as anonymity in peer-review, and interactions between editors and peer reviewers. Implications for researching and practising peer-review are discussed., QC 20230911

The societal impact of research and education, as well as collaborations with industry and society at large occupy a central position in the strategies of many higher education institutions in the Netherlands and across the globe. The current report provides the result of a survey across Dutch universities performed by the members of the Task Force Societal Impact (TFSI). The survey aimed at creating a better understanding of the current approaches of Dutch universities to societal impact evaluation, sharing their considerations, and in identifying relevant initiatives or best practices at the national level.

The evaluation of scientific activity has been the object of study by numerous researchers. In Cuba, there are insufficiencies in the forms of monitoring and evaluation of scientific results. Systematizing existing indicators and methodologies to measure scientific-technological activity at the international level, as well as those used in Cuba, is necessary in a context where the use and abuse of indicators proliferate without taking into account elements of judgment that place them in the evaluative context in which they will be used. It is essential to have system indicators that systematically evaluate this activity and temper with the new currents. Considering the context of the application of the indicators, this type of evaluation should be highlighted three fundamental actors in the development of Cuban science and technology: universities, research centers, and high-technology companies. Currently, standardized indicators and reliable and effective policies for evaluating results and impact from the institutional level have not been established sustainably and sustainably. It has been possible to demonstrate that it is possible to use bibliometrics to evaluate scientific activity. However, constructing a methodological solution and a tool applicable to different institutional typologies is still pending, and its results contribute to decision-making., La evaluación de la actividad científica ha sido objeto de estudio de numerosos investigadores. En Cuba existen insuficiencias en las formas de seguimiento y evaluación de los resultados científicos. Sistema-tizar los indicadores y metodologías existentes para medir la actividad científico-tecnológica a nivel internacional, así como los utilizados en Cuba, se hace necesario en un contexto donde prolifera el uso y abuso de indicadores sin tener en cuenta elementos de juicio que los sitúen en el contexto evaluativo en el que se van a usar. Se considera imprescindible contar con sistemas de indicadores que evalúen de forma sistematizada y atemperada con las nuevas corrientes esta actividad. Teniendo en cuenta el contexto de aplicación de los indicadores, habría que remarcar este tipo de evaluación en tres actores fundamentales en el desarrollo de la ciencia y la técnica cubanas: las universidades, los centros de investigación y las empresas de alta tecnología. En la actualidad, no se han establecido de forma sostenida y estable indicadores normalizados y políticas confiables y eficaces para la evaluación de resultados e impacto desde el nivel institucional. Se ha logrado demostrar que es posible utilizar la bibliometría para evaluar la actividad científica; pero sigue pendiente la construcción de una solución metodológica y una herramienta aplicable a diferentes tipologías institucionales y que sus resultados contribuyan a la toma de decisiones.

Objective. The study aims to analyze indicators of production, collaboration, visibility, impact, and innovation of Cuban universities through Scival during the period 2012-2021. Design/Methodology/Approach. The study design is non-experimental (cross-sectional) and quantitative (descriptive). Eleven Cuban universities were identified in Scival. The indicators were delimited in a time window of 10 years (2012-2021). We analized the results based on 21 indicators covering scientific output, thematic categories, collaboration, visibility, and scientific and patent impact. Results/Discussion. 59.7% of the scientific output is produced by the University of Havana (UH) and the Central University of Las Villas (UCLV). There is a high specialization in engineering and computational sciences. Basic sciences such as mathematics, chemistry and physics are highlighted. 62.1% of the scientific output has been cited. The works’ distribution by journals’ quartiles revealed that 47.3% of the production had been published in 50% of the top Scopus journals during the last ten years. Low performance in innovation and social impact through patents was evidenced. Only UH and UCLV are highlighted. Conclusions. Scival is highly useful for conducting more effective institutional scientific production studies and establishing comparisons of multiple indicators. The biases offered by the tool are minimized, thus offering reliable information for decision-making. The study revealed essential data on Cuban universities' scientific output in Scopus in the last ten years. This is useful for science managers, decision-makers, researchers, and publishers. Originality/Value. The study is novel given that there is a lack of antecedents of Cuban studies using Scival to analyze the universities’ performance. An update on the status of scientific output, visibility and impact is provided., Objetivo. El objetivo del estudio es analizar indicadores de producción, colaboración, visibilidad, impacto e innovación de las universidades cubanas mediante Scival durante el periodo 2012-2021. Diseño/Metodología/Enfoque. El diseño del estudio es no experimental (transeccional) y cuantitativo (descriptivo). Se identificaron las universidades cubanas en Scival para una muestra conformada por 11 universidades. Se delimitaron los indicadores en una ventana temporal de 10 años (2012-2021). Se realizó el análisis de los resultados de las 11 universidades a partir de 21 indicadores que abarcaron la producción científica, categorías temáticas, colaboración, visibilidad e impacto científico y de patentes. Resultados/Discusión. El 59.7% del volumen de producción científica de las universidades cubanas lo aportan la Universidad de la Habana (UH) y la Universidad Central de Las Villas (UCLV). Existe una elevada especialización en las ingenierías y las ciencias computacionales fundamentalmente. Además, se destacan las ciencias básicas como la matemática, química y física. El 62.1% de los trabajos publicados por las universidades fueron citados. La distribución de trabajos por los cuartiles de revistas reveló que el 47.3% de la producción científica de estos últimos 10 años se publicó en el 50% de las revistas de mayor visibilidad de Scopus. El análisis de la actividad relacionada con la innovación e impacto social a través de las patentes mostró un comportamiento bajo y solo se destacan la UH y UCLV con respecto a estos indicadores. Conclusiones. Scival como recurso de información posibilita realizar estudios más efectivos de la producción científica institucional y establecer comparaciones de múltiples indicadores. Al ser una herramienta avanzada de análisis, los sesgos introducidos se minimizan, ofreciendo así información confiable para la toma de decisiones. El estudio reveló datos importantes del rendimiento de la producción científica de las universidades cubanas visible

[EN]: The reform of evaluation proposed at European level in the recent Agreement on Reforming Research Assessment repre-sents a window of opportunity to reform the Spanish evaluation system, at a moment when the new Organic Law of the University System (LOSU) is being debated. This agreement establishes the primacy of peer review based on qualitative criteria in order to be able to recognize the diversity of contributions and research careers, and advocates to stop the inappropriate use of journal-based metrics. How can evaluation in Spain, heavily based on these metrics, be reformed to align with these principles? In this letter we propose that it is not possible to move towards the principles of the European agreement without making structural changes in the governance of evaluation, with a return of autonomy to the universities. We advocate that individual evaluations by agencies outside the university should be limited, and that, instead, institutional evaluation should be encouraged. We collect a proposal for ‘institutional accreditation’ as a public control mechanism for the hiring and promotion of professors at the university., [ES]: La reforma de la evaluación de la investigación propuesta a nivel europeo en el reciente Agreement on Reforming Research Assessment representa una ventana de oportunidad para reformar el sistema español, en un momento en que se debate la nueva Ley Orgánica del Sistema Universitario (LOSU). Este acuerdo europeo establece la primacía de la evaluación por pares basada en criterios cualitativos con el fin de poder reconocer la diversidad de contribuciones y carreras investigadoras, y aboga por el abandono del uso inapropiado de las métricas basadas en revistas. ¿Cómo puede reformarse la evaluación en España, muy basada en estas métricas, para alinearse con estos principios? En esta carta proponemos que no es posible transitar hacia los principios del acuerdo europeo sin realizar cambios estructurales en la gobernanza de la evaluación, con una devolución de autonomía a las universidades. Abogamos que habría que limitar las evaluaciones individuales por agencias externas a la universidad y fomentar, en cambio, la evaluación institucional. Recogemos una propuesta de ‘acreditación institucional’ como mecanismo de control público a los procesos de contratación y promoción de profesores en la universidad.

In this paper we seek to realise the potential that Spaapen and van Drooge’s productive interactions concept offers, but which we argue has been lost through its operationalisation as a process of ‘counting interactions’. Productive interactions arise through moments of contact between two very different systems (the societal and the scientific), and each system values societal impact in very different ways. Finding mutual value in that interaction is important, and we argue that value in both arises when network arrangements shift, as academic disciplines solve urgent scientific problems and as societies improve living conditions. Productive interactions approach assumes the value-frameworks of the wider networks within which particular knowledge sets become actionable. However, our constructive critique highlights the omission of the wider elements of science and social systems within which productive interactions takes place (and whose dynamics ultimately determine the final scientific and societal impact of that research). Indeed, research evaluation to date has not considered the consequences of the productive interactions in terms of these changing relationships. To contribute to this lacuna, we propose a model that conceptualises a meso-level system comprising interactions between actors within two subsystems, highlighting the importance of coupling between researchers and users, valuation signals given to particular productive interactions from researcher and societal communities and the way these signals in turn embed useful knowledge practices. We apply it to a set of examples of productive interactions in the field of social sciences and humanities (SSH) gathered in the framework of a European project.

How to judge the quality of a piece of research is a question that permeates academic work. A clear conception of quality is the backbone of the design and conduct of research, as well as the basis for supervision, manuscript review, and examination papers. Qualitative methodologies justify their usefulness through the development of more sophisticated interpretations of specific study objects than can be achieved within other methodologies. However, it is difficult to arrive at a decision concerning the quality of a piece of research because there are many aspects or facets to consider. Academics develop their competence for reaching decisions through experience, coupled with taking part in ways of reasoning about quality in qualitative research. Within this reasoning process there are a number of quality facets that should be considered. Each facet is grounded in various ways in different research traditions, and each has a foundational meaning but must also be assessed in relation to limitations. From the perspective of the text as a whole, one must consider whether it shows an awareness of the consequences of the research assumptions and other aspects of internal consistency. Ethical considerations are yet another facet. From a narrower perspective, the quality of the interpretation of empirical data must be able to capture rich meaning and present it in a structured way so that the interpretation can be clearly discerned. Another facet is how the interpretation contributes to existing knowledge about the issue or phenomenon that was studied. A classic question concerns the relation between an interpretation and the empirical basis, its credibility or validity. Here there are various facets to consider: Is the interpretation well anchored in data, and how do specific data fit when they are put together? Another facet is whether the interpretation opens up new ways of understanding an issue or process and whether more sophisticated action can result from embracin

Higher education institutions (HEIs) have been undertaking considerable efforts to embed sustainability into their system; however, there have been limited studies on the incorporation of sustainability into research. In parallel, it has been found that women have an important role in contributing to sustainability since they are more engaged and have a more holistic perspective on sustainability than men. These two phenomena have resulted in limited studies about the contribution of women to sustainability research (i.e., in scientific publishing). A bibliometric analysis of more than 39,000 documents (with 147,090 authorships) was done to fill this gap, focusing on sustainability peer-review publications in Europe between 2015 and 2020. The results show that women's presence in sustainability research has been increasing during the last years; however, there are still few female authorship publications, even in fields that have been traditionally women-oriented. In addition, their publications have been less recognized by the scientific community. The results also show substantial gender differences in terms of author leadership, where female senior researchers are more likely to mentor female junior researchers than men. Female researchers tend to collaborate nationally, but they could improve their international collaboration since this can improve their research and impact. More support should be provided to female researchers to help foster women's sustainability engagement and holistic perspectives, reduce existing negative feedback loops, and increase positive ones. Achieving gender equality is sine qua non in achieving sustainable societies.

Higher education institutions (HEIs) have been undertaking considerable efforts to embed sustainability into their system; however, there have been limited studies on the incorporation of sustainability into research. In parallel, it has been found that women have an important role in contributing to sustainability since they are more engaged and have a more holistic perspective on sustainability than men. These two phenomena have resulted in limited studies about the contribution of women to sustainability research (i.e., in scientific publishing). A bibliometric analysis of more than 39,000 documents (with 147,090 authorships) was done to fill this gap, focusing on sustainability peer-review publications in Europe between 2015 and 2020. The results show that women's presence in sustainability research has been increasing during the last years; however, there are still few female authorship publications, even in fields that have been traditionally women-oriented. In addition, their publications have been less recognized by the scientific community. The results also show substantial gender differences in terms of author leadership, where female senior researchers are more likely to mentor female junior researchers than men. Female researchers tend to collaborate nationally, but they could improve their international collaboration since this can improve their research and impact. More support should be provided to female researchers to help foster women's sustainability engagement and holistic perspectives, reduce existing negative feedback loops, and increase positive ones. Achieving gender equality is sine qua non in achieving sustainable societies.

Higher education institutions (HEIs) have been undertaking considerable efforts to embed sustainability into their system; however, there have been limited studies on the incorporation of sustainability into research. In parallel, it has been found that women have an important role in contributing to sustainability since they are more engaged and have a more holistic perspective on sustainability than men. These two phenomena have resulted in limited studies about the contribution of women to sustainability research (i.e., in scientific publishing). A bibliometric analysis of more than 39,000 documents (with 147,090 authorships) was done to fill this gap, focusing on sustainability peer-review publications in Europe between 2015 and 2020. The results show that women's presence in sustainability research has been increasing during the last years; however, there are still few female authorship publications, even in fields that have been traditionally women-oriented. In addition, their publications have been less recognized by the scientific community. The results also show substantial gender differences in terms of author leadership, where female senior researchers are more likely to mentor female junior researchers than men. Female researchers tend to collaborate nationally, but they could improve their international collaboration since this can improve their research and impact. More support should be provided to female researchers to help foster women's sustainability engagement and holistic perspectives, reduce existing negative feedback loops, and increase positive ones. Achieving gender equality is sine qua non in achieving sustainable societies.

We study the effects of an article featured on the cover of the journal Nature on citations to all articles published by its authors. Based on 30 years of bibliometric data, we find that cover articles are cited significantly more than non-cover articles, with this difference being long-lasting. However, when considering all articles (past and future) by Nature authors, we find that the publication of a cover article causes citations to previous articles by its authors to decline relative to citations to articles by non-cover authors.

In light of research complexity and the challenges of rationing limited funding resources, university administrators increasingly use bibliometrics for research evaluation. This effort is essential and worthwhile for better understanding of the university’s research strengths, capabilities, and opportunities. Bibliometric analysis is also a comprehensive tool to identify patterns and trends in research domains. To get a general view of research landscape at HKUST, the Library’s Research Support Services team conducted a small-scale bibliometric study which aimed to find out (i) the popular research topics and (ii) top collaborators of HKUST. This report was intended for anyone with a general interest in bibliometrics within the context of research management.

Policymakers are increasingly demanding that their investments in research and knowledge production should yield returns. The interest of evaluating the societal impact of research has grown rapidly as research funders request evidence that their investments lead to public benefits. This thesis is a study of how the Research Council of Norway has operationalized such demands into new evaluation practices revolving around the concept of Impact. The material to this study consists of one evaluation report that introduced Impact into Norwegian research evaluation, the council’s planning procedures in grant applications in addition to four expert interviews. Informed by practice-oriented document analysis, it highlights the transformative capacity of documents and emphasises the ways in which evaluation functions as tools to make visible phenomena, implement policies and transform researchers and their projects into alignment with Impact demands through processes of translation. Informed by actor-network analysis, I explore how evaluation stages and enacts realities into being., Interessa for å evaluere samfunnseffektane til forsking har vakse med at avgjerdstakarar krev bevis for at investeringar i forsking fører til nytte for samfunnet. I denne masteroppgåva undersøker eg korleis Noregs forskingsråd har operasjonalisert slike krav om til nye evalueringspraksisar orientert mot førestillinga "impact". Ved bruk av praksisorientert dokumentanalyse behandlar eg evaluering som eit optisk verktøy som gjer fenomen synlege, men også som eit politisk verktøy som overtyder evalueringsobjektet om validiteten til det som blir evaluert. Videre undersøker eg korleis Forskingsrådet planlegg for "impact" gjennom å redusere prosjektforslag til logiske forhold mellom predefinerte element som interessent, verknad og effekt. Ved hjelp av aktør-nettverksteori skildrar eg desse som omsettingsprosessar. Slik syner eg at meklarrolla Noregs forskingsråd har mellom styresmakta og forskingssektoren ikkje berre gir utslag i dei tematiske programma deira, men korleis Forskingsrådet ved bruk av evaluering formidlar statlege krav til og utøver kraft også mot den basisfinansierte forskingssektoren. Til slutt diskuterer eg evaluering i lys av vitskapssosiologiske innsikt som tilseier at røyndomar er ein konsekvens av og ikkje ei årsak til vitskaplege sanningar.