Search

The patent-intensive industries play a leading and exemplary role in innovation driven, it is crucial to research evaluation methods to draw their levels and difference of innovation efficiency. This article proposes a new definition method for the patent-intensive industry with the ratio of inventions in force and research and development (RD constructing evaluation index system by comprehensive analysis of previous researches and objective judgment; extracting evaluation data. It evaluates the innovation efficiency of patent-intensive industries, demonstrates the corresponding characteristics: 1) the overall level of innovation efficiency is not high; 2) the RD and 3) the industry with better overall innovation efficiency has better transformation capability. It also provides the targeted industrial policy suggestions by dividing the patent-intensive industries into four categories. The research result shows that the proposed method is suitable for the innovation efficiency evaluation for the patent-intensive industries, and its research framework could be applied to the evaluation of other industries.

Este artículo está sujeto a una licencia CC BY 4.0, This paper analyses the scientifc activity related to open science in Spain and its infuence on public policy from a bibliometric perspective. For this purpose, Spanish centres’ projects and publications on open science from 2010 to 2020 are studied. Subsequently, policy documents using papers related to open science are analysed to study their infuence on policymaking. A total of 142 projects and 1491 publications are analysed, 15% of which are mentioned in policy documents.The publications cited in policy documents display high proportions of international collaboration, open access publication and publication in frst-quartile journals. The fndings underline governments’ leading role in the implementation of open science policies and the funding of open science research. The same government agencies that promote and fund open science research are shown to use that research in their institutional reports, a process known as knowledge fow feedback. Other non-academic actors are also observed to make use of the knowledge produced by open science research, showing how the open science movement has crossed the boundaries of academia., This study was funded under an FPI contract PRE2020-092917 of the Spanish Ministry of Science and Innovation from the DOSSUET project (Diagnosis of open science in the Spanish university system and mechanisms for its transformation and improvement) grant number PID2019-104052RB-C21.

Bu makale, akademik davranışlarda sistematik ölçmenin oynadığı rolü ele almaktadır. Girişimcilik kültürünün akademide ortaya çıkışı, nicel performans ölçümünü gerektiren bireysel liyakat vurgusu yoluyla rekabetçi atmosferleri teşvik etmiştir. Bilgi üretimi akademisyenlerin temel misyonlarından biri olduğu için, bu çalışma, ölçüm araçları, kurumsal yönetim, finansman organları ve devlet politikasını içeren çevresel faktörlerden etkilenen bilgi üretimi biçimleri üzerindeki etkileri ifade etmeyi amaçlamaktadır. Tayvan'daki 41 akademisyenle yapılan mülakatlar vasıtasıyla onların akademik uygulamalara ilişkin anlatılarını keşfe çıkan bu makale, bilgi üretiminin politik atmosfer tarafından nasıl şekillendiğini ampirik olarak analiz etmektedir. Bu değiştirici davranış kalıpları; araştırma konularının seçimini, araştırma gündemi tasarımını, hibe başvuruları ve yayınları için strateji oluşturmayı, araştırmanın epistemik özelliklerinde hassas değişiklikleri içerir. Çalışma buradan hareketle, neoliberal çağda araştırmacılar, bilimsel topluluklar ve devlet arasındaki güç ilişkilerini karakterize etmektedir., This paper considers the role played by systematic measuring in academic conducts. The introduction of entrepreneurial culture into academia fosters competitive atmospheres through its emphasis on personal merit, which entails quantitative performance measurement. As knowledge production is one of scholars’ core missions, this study aims to articulate impacts on ways of knowledge production influenced by environmental factors, comprising measuring instruments, institutional management, funding bodies and state policy. By interviewing 41 scholars in Taiwan and exploring their accounts of academic practices, this paper empirically analyses how knowledge production has been shaped by policy environments. These modifying behaviour patterns include a selection of research topics, a design for research agenda, strategies for grant applications and publications, engendering delicate changes in the epistemic properties of research. From this, it characterizes power relations between researchers, scientific communities and the state in the neoliberal age.

Opening Keynote. Force 11 Annual Conference 2023. 18 April. Online. https://force11.org/force2023/ Title: When institutions fail communities get busy. Putting open scholarly data in the hands of people who care. Abstract The Curtin Open Knowledge Initiative is a project that was born of frustration. We wanted to help senior leaders in our university to look beyond league tables and citation counts as indicators of the ‘quality’ and ‘value’ of research, and to start new conversations about why openness and diversity matter. By combining critical humanities perspectives with cloud based computing and data science expertise, we have been able to engage with publicly available data relating to global research communication at scale. As we have discovered, community investments in open metadata infrastructures have led to a phase-change in what open data can now deliver. Questions about the impacts of making research open; the openness of research associated with particular topics, institutions, and regions; as well as the diversity of communities making and using expert knowledge can all be explored through entirely open data sets. But in spite of the availability of open data sets, and the need for research evaluation tools that can support diversity and local context, commercial data continues to dominate the research evaluation space. In this talk, I will introduce the work that the Curtin Open Knowledge Initiative is doing with open data; and ask what we can learn from other activist communities about strategies that work in the face of commercial monopolies and institutional limitations.&nbsp

Opening Keynote. Force 11 Annual Conference 2023. 18 April. Online. https://force11.org/force2023/ Title: When institutions fail communities get busy. Putting open scholarly data in the hands of people who care. Abstract The Curtin Open Knowledge Initiative is a project that was born of frustration. We wanted to help senior leaders in our university to look beyond league tables and citation counts as indicators of the ‘quality’ and ‘value’ of research, and to start new conversations about why openness and diversity matter. By combining critical humanities perspectives with cloud based computing and data science expertise, we have been able to engage with publicly available data relating to global research communication at scale. As we have discovered, community investments in open metadata infrastructures have led to a phase-change in what open data can now deliver. Questions about the impacts of making research open; the openness of research associated with particular topics, institutions, and regions; as well as the diversity of communities making and using expert knowledge can all be explored through entirely open data sets. But in spite of the availability of open data sets, and the need for research evaluation tools that can support diversity and local context, commercial data continues to dominate the research evaluation space. In this talk, I will introduce the work that the Curtin Open Knowledge Initiative is doing with open data; and ask what we can learn from other activist communities about strategies that work in the face of commercial monopolies and institutional limitations., PDF version is slides only. .pptx version is slides plus notes. .mov version is the full, unedited, recording of the talk.

Transferring scientific knowledge to non-academic audiences is an essential aspect of the open science agenda, which calls for scholars to pursue a popularization of their research. Accordingly, purposefully introducing scientific insights to the public at large is almost univocally deemed commendable. Indeed, in today’s models of research evaluation, the objects and activities considered are being extended beyond peer-reviewed journal articles to include non-scholarly popular communication. Although altmetrics offer one instrumental way to count some interactions with lay audiences, their reliance on social media makes them susceptible to manipulation, and mostly reflect circulation among niche audiences. In comparison, attention from non-scholarly media like newspapers and magazines seems a more relevant pathway to effectuate societal impact, due to its recognition in qualitative assessment tools and its broad, societal reach. Based on a case study of social scientists’ attention by newspapers and magazinesin Flanders (northern Dutch-speaking region of Belgium) in 2019, this paper highlights that frequent participation in the public debate is reserved for high-status researchers only. Results show highly skewed media appearance patterns in both career position and gender, as eight male professors accounted for almost half of all 2019 media attention for social scientists. Because media attention is highly subject-dependent moreover, certain disciplines and fields offer easier pathways to popularization in media than others. Both the open science agenda and research assessment models value presence of researchers in popular media, adding written press attention to existing evaluation assessments however would disproportionately disadvantage early career researchers and exacerbate existing inequalities in academia.

This study, based on bibliometrics (related to scientometrics) and science mapping, along with some sociological analytical insights, assesses contexts of the concept 'rural' in articles written by authors affiliated to JCEA (Journal of Central European Agriculture) member countries: Bulgaria, Czech Republic, Croatia, Hungary, Poland, Romania, Serbia, Slovakia, Slovenia (with addition of Austria because of geographical proximity and historical ties), using citation databases Scopus and Web of Science (WoS), and visualization software VOSviewer. Most papers are published outside the scope of agricultural production studies, clustered strongly in the fields of medicine, physical environment, social sciences, and ecological interactions. A small proportion of the reference to “rural” in agricultural science seems self-assumed in the general association of agricultural production with rural space. The papers mainly refer to geographical area, space or territory where interactions between specific social and natural processes take place. Topics tackling rural space as a societal form and arrangements, e.g. sustainable rural development and transformation, are the most recent. National and regional journals show strong recent presence but papers are also published in principal international journals. Co-author networking (countries / institutions / affiliations) indicates strong regional co-operation which is strongest in the immediate neighborhood. Co-authorship based on spatial proximity can be partly explained by less ambitious internationalization of research activities. Although publication patterns in both databases show similar traits, some database effect can be observed, also due to the inclusion of ESCI (Emerging Sources Citation Index) in 2015., Raziskava, temelječa na bibliometriji (sorodno scientometriji) in znanstvenem kartiranju, skupaj z nekaterimi sociološko-analitičnimi vpogledi preverja kontekst pojma 'podeželje' v člankih avtorjev, povezanih z državami članicami JCEA (Journal of Central European Agriculture): Bolgarija, Češka, Hrvaška, Madžarska, Poljska, Romunija, Srbija, Slovaška, Slovenija (in Avstrija zaradi zemljepisne bližine in zgodovinskih povezav), z uporabo citatnih baz podatkov Scopus in Web of Science (WoS) ter programske opreme za vizualizacijo VOSviewer. Večina prispevkov je objavljenih zunaj študij kmetijske proizvodnje, najbolj pa so prisotna področja medicine, fizičnega okolja, družboslovja in ekoloških interakcij. Majhen delež omembe pojma rural v kmetijski znanosti se zdi privzet ob splošni asociaciji kmetijske proizvodnje s podeželskim prostorom. Prispevki se nanašajo predvsem na geografsko območje, prostor ali ozemlje, kjer potekajo medsebojni vplivi med specifičnimi družbenimi in naravnimi procesi. Obravnave, ki se lotevajo podeželskega prostora kot družbene forme in ureditve, npr. trajnostni razvoj in preoblikovanje podeželja, so najnovejše. Nacionalne in regionalne revije kažejo močno prisotnost objavljanja v zadnjem času, dela pa so objavljena tudi v uglednih mednarodnih revijah. Vendar pa soavtorstvo (držav / institucij / naslovov) kaže na zelo močno regionalno sodelovanje, ki je najmočnejše v neposredni soseščini. Soavtorstvo, ki temelji na neposredni prostorski bližini, je mogoče delno razložiti z manj ambiciozno internacionalizacijo raziskovalnih dejavnosti. Čeprav vzorci objavljanja kažejo podobne lastnosti v obeh bazah podatkov, je mogoče opaziti učinek baze podatkov, tudi zaradi nedavne vključitve WoS ESCI (Emerging Sources Citation Index).

Presentation at the ESO workshop Peer Review Under Review in Garching, Germany on February 7, 2023.

Purpose – Several frameworks have been developed to map and document scientific societal interaction and impact, each reflecting the specific forms of impact and interaction that characterize different academic fields. The ReAct taxonomy was developed to register data about “productive interactions” and provide an overview of research activities within the social sciences and humanities (SSH). The purpose of the present research is to examine whether the SSH-oriented taxonomy is relevant to the Science, Technology, Engineering and Mathematics (STEM) disciplines when clarifying societal interactions and impact, and whether the taxonomy adds value to the traditional STEM impact indicators such as citation scores and H-index. Design/methodology/approach – The research question was investigated through qualitative interviews with nine STEM researchers. During the interviews, the ReAct taxonomy and visual research profiles based on the ReAct categories were used to encourage and ensure in-depth discussions. The visual research profiles were based on publicly available material on the research activities of the interviewees. Findings – The study provided an insight into how STEM researchers assessed the importance of mapping societal interactions as a background for describing research impact, including which indicators are useful for expressing societal relevance and impact. With regard to the differences between STEM and SSH, the study identified a high degree of cohesion and uniformity in the importance of indicators. Differences were more closely related to the purpose of mapping and impact assessment than between scientific fields. The importance of amalgamation and synergy between academic and societal activities was also emphasised and clarified.Practical implications – The findings highlight the importance of mapping societal activities and impact, and that societal indicators should be seen as inspiring guidelines depending on purpose and use. A significant contribution is the identification of both uniformity and diversity between the main fields of SSH and STEM, as well as the connection between the choice of indicators and the purpose of mapping, e.g., for impact measurement, profiling, or career development.Originality/value – The work sheds light on STEM researchers’ views on research mapping, visualisation, and impact assessment, including similarities and differences between STEM and SSH research. Purpose: Several frameworks have been developed to map and document scientific societal interaction and impact, each reflecting the specific forms of impact and interaction that characterize different academic fields. The ReAct taxonomy was developed to register data about “productive interactions” and provide an overview of research activities within the social sciences and humanities (SSH). The purpose of the present research is to examine whether the SSH-oriented taxonomy is relevant to the science, technology, engineering and mathematics (STEM) disciplines when clarifying societal interactions and impact, and whether the taxonomy adds value to the traditional STEM impact indicators such as citation scores and H-index. Design/methodology/approach: The research question was investigated through qualitative interviews with nine STEM researchers. During the interviews, the ReAct taxonomy and visual research profiles based on the ReAct categories were used to encourage and ensure in-depth discussions. The visual research profiles were based on publicly available material on the research activities of the interviewees. Findings: The study provided an insight into how STEM researchers assessed the importance of mapping societal interactions as a background for describing research impact, including which indicators are useful for expressing societal relevance and impact. With regard to the differences between STEM and SSH, the study identified a high degree of cohesion and uniformity in the importance of indicators. Differences were more closely related to the purpose of mapping and impact assessment than between scientific fields. The importance of amalgamation and synergy between academic and societal activities was also emphasised and clarified. Practical implications: The findings highlight the importance of mapping societal activities and impact, and that societal indicators should be seen as inspiring guidelines depending on purpose and use. A significant contribution is the identification of both uniformity and diversity between the main fields of SSH and STEM, as well as the connection between the choice of indicators and the purpose of mapping, e.g. for impact measurement, profiling, or career development. Originality/value: The work sheds light on STEM researchers' views on research mapping, visualisation and impact assessment, including similarities and differences between STEM and SSH research.

The scientific community considers readership analysis of academic artifacts to be a significant endeavor. The reference manager’s readership count is a momentous indication for early research evaluation. In response, this study demonstrates the characteristics of Mendeley readership for EPS articles from twelve narrow disciplines and compares them with citations. The bibliographic and citation data have been collected from Scopus and the corresponding readers’ data from Mendeley. The Spearman correlation was performed among citations and readers for all unique articles for all investigated disciplines. Further, we also looked at the relationships between articles with non-zero readers, as well as articles satisfied by percentile ranking of the top 75 per cent, 50 per cent, and 25 per cen treaders. The result indicates large correlations among citations and readers (avg. 0.669) for all investigated disciplines. If we analysed only non-zero readers, as well as a percentile ranking of articles, the correlation results show a decreasing trend. Around 98.57 per cent of articles have at least one reader in Mendeley and AS (97.53 %) discipline has registered the highest one. The CES discipline had registered the largest MRS of 32.25 and MCS of 12.75. Most of the readers come from post-doctoral students and Ph.D. students. The correlation results indicate that the readership statistics should be used as an impact indicator for EPS discipline.

Background The interest in and demand for healthcare innovation has heightened amid the COVID-19 pandemic. Organizations are challenged to balance the goals of daily operations with innovation to stay relevant and compete in the marketplace. Innovation is critical for not only the success and sustainability of organizations, but the well-being of the faculty, staff, and clients they serve. Purpose In this article, we present an overview of several Nursing Innovation Centers in the United States as well as examples of colleges without formal innovation centers but who are addressing innovation in their programs. Methods We examined the subjective experience of nursing innovation in seven colleges of nursing using semi-structured intervieweds and thematic analysis. FIndings We discuss four themes for creating an innovation center or innovation focus and six themes important for sustainability and impact. In addition, we provide a working model for these themes and provide lessons learned along with trends and recommendations for the future. Discussion This information provides guidance and a framework for academic and practice organizations aspiring to create opportunities for innovation to flourish in their institutions. We also encourage leadership to critically evaluate and address biases in faculty hiring, research evaluation, publication practices, educational opportunities and mentoring to overcome the diversity innovation paradox.

Background: Evaluations are a useful tool to learn more about the effectiveness of public measures. In the era of evidence-based policymaking, recent research suggests that quality is an important determinant of the utilisation of evaluations. Despite this claim, hardly any empirical study has investigated whether the quality of an evaluation – measured by a meta-evaluation – influences its perceived utilisation by decision makers.Aims and objectives: This article asks how the quality of an evaluation study is related to its perceived use, and investigates the relationship between the quality of an evaluation, assessed through a meta-evaluation, and how the evaluation is perceived and accepted by the parties concerned.Methods: The basis for the empirical analyses were 34 external evaluations, conducted from 2006 to 2014, of upper secondary schools in the canton of Zurich, as well as a standardised survey conducted among 307 representatives of these schools (teachers, administrators, members of quality development teams, and the heads of school oversight commissions).Findings: We conclude that the quality of the evaluation, as assessed in a meta-evaluation, is not particularly associated with the perception of evaluation quality and the perceived use of the evaluation. The perceived quality, however, is related to the perceived impact of an evaluation.Discussion and conclusion: These findings are relevant for evaluation research and practice, since they show that the quality of an evaluation and evaluation use do not necessarily go hand in hand. Key messages Evaluators have to be aware that a systematically assessed quality of an evaluation does not go hand in hand with the perceived quality of that evaluation; Evaluators often focus on the instrumental form of evaluation use, but they should not ignore other forms of use and maybe try to maximise these utilisation forms in the design of their evaluation; Evaluators should be more active in advising stakeholders when it comes to evaluation use, for example, through policy narratives; Evaluators should carefully think about the measurement of evaluation quality and evaluation effects in research on evaluation.

Our study examines the discursive construction of academic subjectivity within the context of the current reforms of academia, driven by the forces of economization and metricization. We focus in p...

This article comparatively analyzes two manifestos in the field of quantitative science evaluation, the Altmetrics Manifesto (AM) and the Leiden Manifesto (LM). It employs perspectives from the Sociology of (E-) Valuation to make sense of highly visible critiques that organize the current discourse. Four motifs can be reconstructed from the manifestos’ valuation strategies. The AM criticizes theconfinednessof established evaluation practices and pledges for anexpansionof quantitative research evaluation. The LM denounces theproliferationof ill-applied research metrics and calls for anenclosureof metric research assessment. It can be shown that these motifs are organized diametrically: The two manifestos represent opposed positions in a critical discourse on (e-) valuative metrics. They manifest quantitative science evaluation as a contested field.

Social media is one of the major channels for knowledge diffusion, including scientific discovery. As a consequence, it has been under discussion whether altmetrics can be used as an alternative indicator for research evaluation, complementing the traditional indicators such as citations. The study contributes to the discussion by exploring the factors relevant to the attention that publications receive on social media. Based on 20,310 academic papers published by the Royal Society in the past ten years, the study found that posting accounts are the major force for boosting the amount of attention received on Twitter. Timely posting and accessibility also help to increase the attention. The findings suggest that publications’ social popularity can be easily influenced by non-paper factors. The study receives financial support from the National Natural Science Foundation of China (NSFC) (No. 72004118), and ISTIC-CLARIVATE Analytics Joint Laboratory for Scientometrics.

Background The development of virtual reality (VR) technology has the potential to provide enormous opportunities for food and consumer behavior research to further explore current research methodologies and establish new ones. VR is different from the environment we experience in daily life. Therefore, the extent to which VR can be applied in food consumer behavior research needs to be assessed on the basis of the evidence provided by the studies that have employed VR. Approach To better understand VR technology and its potential applications and limitations in food and consumer behavior research, we conducted a systematic review of the literature on VR in food and consumer behavior research. The applications of VR addressed in the articles were extracted. The validity of VR for food and consumer behavior research was analyzed. Key findings VR has been used to build complex and realistic contextual environments for conducting food sensory evaluation research. VR was also used as a substitute (e.g., a VR buffet and supermarket) to real life settings for investigating consumers' purchasing behavior and food choices. Virtual food and food-related cues were used to induce psychological and physiological responses in consumers. The results supported the validity of VR as a tool for investigating consumers’ behavior toward food., Trends in Food Science & Technology, 116, ISSN:0924-2244, ISSN:0968-0020

Efficiency analysis is an important step in the environmental performance evaluation research, especially in the context of the implementation of the Environmental Protection Tax Law in China. In t...

[EN] The purpose of this note is to describe the new citation context classification feature provided by Web of Science, in which citations received by publications are classified into five categories (Background, Basis, Support, Differ, and Discuss). To exemplify the functionality, two case studies have been carried out: one involving a journal (Profesional de la información; 1,604 publications) and the other, an author (Loet Leydesdorff; 341 publications). Both cases reflect the still low coverage of classified citations, which currently limits the use of this functionality. Finally, some of the questions that arise with the use of these context metrics (precision, comprehension, simplificatio, [EN] El objetivo de esta nota es describir la nueva funcionalidad de clasificación de citas por contexto proporcionada por Web of Science (WoS), en la que las citas recibidas por un trabajo se clasifican en cinco categorías (Background, Basis, Support, Differ, Discuss). Con el fin de testear las prestaciones de la funcionalidad se han llevado a cabo dos casos de estudio. Por un lado, una revista (Profesional de la información; 1,604 publicaciones) y, por otro lado, un autor (Loet Leydesdorff; 341 publicaciones). Ambos casos reflejan la todavía baja cobertura de citas clasificadas (inferior al 6%) que limitan el uso de esta funcionalidad actualmente. Finalmente se discuten algunos de los interrogantes que se abren con el uso de estas métricas de contexto, tales como la precisión, comprensión, simplificación, representatividad, comparabilidad, idioma, agregación, uso evaluativo, efectos y extrapolación.

With the world descending into chaos of 21st century namely ecological and environmental challenges, there is a need for force to work among us and to lift us, so as to create order out of those above mentioned chaos. Resources are depleting, they need to be saved, the environment needs to be protected. Several countries have started facing earth overshoot day almost every year now. To overcome these issues, different organizations are working on it and one of them in India is GRIHA. Also govt. of India has made it mandatory for some government projects to be GRIHA certified, and they are providing some incentives for that. Some PSU buildings also have made it compulsory to be GRIHA certified. There are different variants of GRIHA and one of them is, “GRIHA for large development”. Whenever large projects are designed, executed etc., they leave great impact on build environment, hence affecting ecology, the environment, so they must be well assessed with regards to building performance, since educational campuses are such large projects. In this paper, a qualitative evaluation research method is used. This paper studies the GRIHA’s variant for large development (mainly university campuses as they are like a small city within a country) and analyses the green features of the university campuses that are GRIHA certified.

In higher education, student evaluation surveys are an important tool for the assessment and improvement of teaching. Despite decades of investigation, research has yet to resolve disagreement abou...

To solve the problem of regional water system evaluation, this paper proposes a system efficiency evaluation method based on the game cross-efficiency model and conducts an empirical analysis. First, autopoiesis is introduced as the theoretical basis. The characteristics of the authigenic system are combined with a regional water system, and the connotation and characteristics of the regional water system are defined. Second, based on the competitive relationship between regional water systems, the existing game crossover efficiency model is improved. A crossover efficiency model of other games is proposed to evaluate the efficiency of regional water systems. Then, the Pearl River Delta urban agglomeration is selected as the research object. The effects of four systematic evaluation methods based on the DEA method are compared horizontally to find the optimal system efficiency evaluation method. Finally, the characteristics of the regional water system in the Pearl River Delta are systematically analysed through the evaluation results, and the present situation of the regional water system is fully explained.

Monitoring, evaluation, and research activities generate important data, but they often fail to change policies or programs. In addition, local program staff and partners often feel disconnected from these activities, which undermines their ownership of data and results. To bridge the gaps between monitoring, evaluation, and research and to give them a common purpose through locally owned, actionable learning, we developed an integrated Monitoring, Evaluation, Research, Learning, and Adapting (MERLA) framework. Implemented and refined in over 20 countries, the framework integrates data-gathering methodologies with collaborative techniques for translating evidence into knowledge that influences program practice and government policy. In this publication, we describe the MERLA framework and present two case studies demonstrating how its application enables learning and adaptation led by in-country teams.

Traditionally, statistical power was viewed as relevant to research planning but not evaluation of completed research. However, following discussions of high false finding rates (FFRs) associated with low statistical power, the assumed level of statistical power has become a key criterion for research acceptability. Yet, the links between power and false findings are not as straightforward as described. Assumptions underlying FFR calculations do not reflect research realities in personality and social psychology. Even granting the assumptions, the FFR calculations identify important limitations to any general influences of statistical power. Limits for statistical power in inflating false findings can also be illustrated through the use of FFR calculations to (a) update beliefs about the null or alternative hypothesis and (b) assess the relative support for the null versus alternative hypothesis when evaluating a set of studies. Taken together, statistical power should be de-emphasized in comparison to current uses in research evaluation.

The important role of policy makers is to develop and improve program implementation. The purpose of the study in general is to evaluate the supporting components of the implementation of online learning which include learning facilities at home and parents' attention in learning, the input component of students' initial abilities, and the process components of activeness following online learning in relation to increasing the competence of SMK students in Malang and Batu. The research design uses evaluation research. The research sample involved 240 students of the multimedia expertise program at SMKN 4, 5, 11 Malang City and SMKN 3 Batu. The data analysis technique used in this research was SEM (Structural Equation Modeling) and LISREL. The results of this study are, 1) learning facilities at home have a direct effect on students' initial abilities, student activity. 2) parents' attention in learning has a direct effect on students' initial ability and student competence, and has an indirect effect on student learning activeness and competence.

PurposeThis study applies D3.js and social network analysis (SNA) to examine the impact of collaboration patterns, research productivity patterns and publication patterns on the Ministry of Education (MOE) evaluation policies across all Management Information Systems (MIS) departments in Taiwan.Design/methodology/approachThis study first retrieved data from the Ministry of Science and Technology of Taiwan (MOST) website from 1982 to 2015, the Journal Citation Reports (JCR) website, the Web of Science (WOS) website and Google Scholar. Then it applied power-law degree distribution, cumulative distribution function, weighted contribution score, exponential weighted moving average and network centrality score to visualize the MIS collaborations and research patterns.FindingsThe analysis concluded that most MIS professors focused primarily on SCIE-/SSCI-/TSSCI-/core indexed journals after 2005. Professors from public universities were drawn to collaboration and publishing in high-quality-based journals, while professors from private universities focused more on quantity-based publications. Female professors, by contrast, have a slightly higher single-authorship publication rate in SCIE-/SSCI-/TSSCI-indexed journals than do male professors. Meanwhile, professors in northern Taiwan emphasized quantity-based journal publications, while a focus on quality was more typical in the south. Furthermore, National Cheng Kung University has the most single-authorship or intrauniversity publications in SCIE-/SSCI-/TSSCI-/core journals, and National Sun Yat-Sen University published more SSCI-indexed articles than SCIE-indexed articles. All of these findings show that there is an explicit relation between MOE evaluation policies and MIS faculty members' collaboration/publication strategies.Originality/valueThe above findings explain how MOE evaluation policies affected MIS faculty members' collaboration and publication strategies in Taiwan, and the authors hope that such findings can constitute a resource for understanding and characterizing networking with MIS departments in Taiwan.

This article reflects on ethical issues that arose during the course of two different evaluation projects that used photovoice method to engage with marginalized populations. The evaluations serve as case studies for a critical discussion about potential barriers that researchers may face when employing photovoice method while trying to balance the principles of community-based participatory research with the requirements of Institutional Review Boards. We reflect on ethical dilemmas related to the meaning of photography within the cultural context of participants’ lives, the compensation of participants as collaborators, and the representation and dissemination of participant photos. We conclude by examining how researchers may approach ethical requirements without compromising the important collaborative relationships central to photovoice method. We additionally call on researchers to engage with ethics review committees to create a new “participant–researcher” category with its own set of protocols that recognizes the nuanced role members of disenfranchised communities play in the research process.

This paper is an overview of bibliometrics, a subfield of library and information science. It briefly explains what bibliometrics is and why it is important in research evaluation and impact analysis. It summarizes the latest development and trends over the past decade. Three major trends are identified and discussed. They are alternative metrics, responsible use of bibliometrics and responsible research evaluation movement, and application of artificial intelligence (AI) and machine learning in bibliometrics practice.

This study seeks to determine the professional competence of social studies teachers and how individual background factors affect professional competence. This research is a type of evaluation research, using the Benchmarking evaluation model, in which the results obtained will be compared with the professional competency standards of teachers prevailing in Indonesia. The study population was all social studies teachers at MTs in Mranggen District, amounting to 31 people. The data analysis technique used is quantitative analysis. The data analysis step of this research is descriptive data analysis and regression test. The results showed that 32.2% of the teachers had professional competence in the very good category, 25.8% in the good category, and 34.4% in the sufficient category. The regression results show that there is an effect of individual teacher factors on the formation of the professional competence of social studies teachers at MTs in Mranggen District. Teaching experience has the greatest influence in the formation of teacher professional competence, namely 44.63%, then the work environment is 31.16%, educational background is 25.80% and training experience is 0.35%.

From a purely epistemological point of view, evaluating and predicting the future success of new research projects is often considered very difficult. Is it possible to forecast important findings and breakthrough in science, and if not, then what is the point trying to do it anyway? Still, that is what funding agencies all over the world expect their reviewers to do, but a number of previous studies has shown that this form of evaluation of innovation, promise and future impact are a fundamentally uncertain and arbitrary practice. This is the context that I will discuss in the present essay, and I will claim that there is a deeply irrational element embedded in today’s heavy reliance on experts to screen, rank and select among the increasing numbers of good research projects, because they can, in principal, never discern the true potential behind the written proposals. Hence, I think it is motivated to see grant peer review as an ‘oracle of science’. My overall focus will be on the limits of competitive funding and also that the writing and reviewing of proposals is a waste of researchers’ precious time. And I will propose that we really need to develop new ways of thinking about how we organize research and distribute opportunities within academia.

Distance learning during the pandemic is a new strategy in the world of education. The purpose of this research is to measure and assess the success rate of distance learning implementation at Madarasah Aliyah in Malang district. This type of research is an evaluation research. The subjects of this study amounted to 10 people consisting of teachers and school principals from three Aliyah madrasas in the Malang area. The sampling technique used purposive sampling. Methods of data collection using interviews and observation. The data analysis technique uses the CIPP model with stages, namely preparation, implementation, completion and reporting. This study concludes the results in the form of four important points. First, not all schools can carry out the distance learning process optimally. Second, teachers do not have uniform competence and can determine suitable learning strategies. Third, the evaluation of distance learning outcomes has not been able to fully describe students' abilities. Fourth, the obstacles faced in the form of teacher readiness and student learning motivation. This study still has many limitations, both in terms of research procedures, themes raised or the type of research taken. Subsequent research conducted will enrich similar research themes.

It is difficult to spot failures in port machinery and equipment, and maintaining such systems is even more complex. Maintenance such modifications in a reasonable time is a tough challenge since each change might have an endless number of test cases run. It's critical to have a risk assessment of the impact of such maintenance fixes. In the software engineering community, there has been a considerable amount of study on failure prediction. Regrettably, there is little evidence of their application in day-to-day software development in port machinery and equipment. In this paper, we propose an unsupervised machine learning (k-means clustering) method for categorising cranes for maintenance and use a machine learning pipeline to solve the classification of crane failure data. The crane's maintenance decision data demonstrates the method's effectiveness. It was demonstrated that the Linear Support Vector Machine could give a superior classification accuracy of crane maintenance prediction with a 100 percent accuracy in train set and 94.5 percent accuracy in test set.

This is a short volume, but it is rich in arguments and bibliography. The author is a winner of the Young Feltrinelli Prize in Moral Sciences, a prestigious prize conferred on scholars under 40 yea...

Evaluating open and distance learning journals will be helpful to the discipline and the global community at large, as research evaluation metrics are useful for measuring research productivity and...

China has launched a series of talent-recruitment policies in the last years, in order to attract back Chinese nationals who stayed abroad. Yet, little is known about the effect of such policies. This paper examines whether researchers recruited in one of the Chinese flagship talent-recruitment policies—the ‘Young Thousand Talents’ policy (Y1000T)—had, in the following years after recruitment, better research performance. We compare these recipients against other Chinese nationals who got PhDs in equally prestigious non-Chinese universities but continued to work abroad (mostly in the USA). Results of difference-in-differences regressions show that returning to China has an effect of positioning returnees both at the bottom and at the very summit of the distribution of quality of publications. Nevertheless, some Y1000T researchers seem to have prioritized the quantity of outputs, arguably to the detriment of quality. This is probably due to certain research evaluation criteria in place until recent times.

Data tables showing mean and median publishing rates in 170 disciplines, over various timeframes, and across professorial ranks. Data are derived from Academic Analytics LLC's commercial data warehouse of scholarly research at American Ph.D. granting universities (database version AAD2019-1740). Publishing data cover journal articles, conference proceedings, books, and chapters in edited volumes. Tables are available in PDF and Excel formats.

The study of national research assessment exercises serves to evaluate the effectiveness of policies versus their objectives and to improve the formulation of future initiatives. The aim of the current study is to verify whether the introduction of the first performance-based research funding in Italy, based on the 2004–10 VQR assessment, achieved the intended objective of inducing greater international collaboration on the part of researchers. For this, we apply a bibliometric approach based on the observation of coauthorships in Italian and worldwide scientific publications over a 14-year period, beginning in the target years of the VQR assessment. Through an Interrupted Time Series Analysis, we compare the expected and observed patterns of international coauthorship for Italy and the rest of the world. Although the rate of internationalization of Italian research is seen to be increasing, whether this is a consequence of the VQR incentives, or rather part of a global phenomenon of recourse to international collaboration in response to the increasingly complex scientific challenges, it is open to interpretation.

Suomalaisissa vankiloissa on tehty systemaattista ohjelmatoimintaa noin 20 vuoden ajan. Ohjelmatoiminnan tärkeimpänä tavoitteena on edistää rikoksetonta elämäntapaa, ja siten vähentää vapautumisen jälkeistä uusintarikollisuutta. Uusintarikollisuusvaikutusten todentaminen edellyttää vaikutustutkimuksiin perustuvaa tutkimusnäyttöä, jota on vähitellen alkanut kertymään myös Suomesta. Tässä artikkelissa käydään läpi yksittäisten ohjelmien uusintarikollisuusvaikutuksia koskevien kotimaisten arviointitutkimusten keskeiset tulokset, sekä pohditaan ohjelmatoiminnan vaikutusten arvioinnin mahdollisuuksia tilastollisten tutkimusmenetelmien, tutkimusaineistojen ja ohjelmatoiminnan volyymin näkökulmista. Ohjelmatoiminnan vaikutuksista uusintarikollisuuteen Suomessa ei ole toistaiseksi saatu selvää näyttöä. Vaikka ohjelmien vaikutusten tutkimiseen tarvittavat aineistot ovatkin parantuneet 2000-luvulla selvästi, uskottavaa arviointia vaikeuttavat edelleen hyvien tutkimusasetelmien puute sekä yksittäisten ohjelmien pienet osallistujamäärät. Vaikutusarviointien menetelmällisessä laadussa on parantamisen varaa. Koska yksittäisen ohjelman vaikutus uusintarikollisuuteen on saatavilla olevan tutkimustiedon valossa luultavasti aika maltillinen, kovin pienillä tutkimusaineistoilla vaikutuksia ei kannata jatkossa yrittää tilastollisesti tutkia. Pienet vaikutukset eivät kuitenkaan tarkoita sitä, että ohjelmat eivät voisi olla kannattavia ja kustannustehokkaita. Merkittävä osa nykyisestä ohjelmatoiminnasta on osallistujamääriltään niin pienimuotoista, ettei niiden toimivuudesta tai toimimattomuudesta ole helppo saada luotettavaa tietoa. Tämä tilanne haastaa tutkimustietoon perustuvan ohjelmatoiminnan kehittämistä. Sasu Tyni and Mikko Aaltonen: Evaluation research on rehabilitation programs in prison. Rehabilitation programs have been used systematically in Finnish prisons now for two decades. The main aim of these programs is to promote desistance and reduce recidivism after release. Several Finnish evaluations of prison programs have been published during the recent years. In this article, we start by reviewing the key results of these studies and consider the possibilities and limitations of quasi-experimental program evaluation in the light of available register data and the scale of program uptake. So far none of the Finnish evaluations have shown evidence of programs reducing recidivism. Even though the datasets needed for evaluation have improved clearly in the 21st century, credible evaluation of programs’ causal effects is still hampered by lack of strong research designs and low participant rates in most programs. The methodological quality of evaluations should be improved. Given that the available evidence suggests that true effect sizes of programs are likely to be relatively small, large samples are needed to attain enough statistical power to detect effects or their absence. At the same time, even if true effect sizes are small, the programs can still be cost-effective. The low numbers of participants in most prison programs present a challenge to evidence-based program development. Keywords: prison rehabilitation – recidivism – program evaluation – statistical power – causal inference

Being an economist has become a quite fascinating profession. Alongside teaching, the economist’s job is to produce research and then translate it into policy guidance for governments and other pub...

Background and aim: This study reports the outcomes from an international study that implemented a set of person-centred nursing key performance indicators across a range of services provided to sick children. It aims to explore the factors that influenced the successful implementation of those indicators, and assess the impact of the evidence generated on person-centred practice across the children’s services. Methods: Twelve organisations across Australia and Europe participated, with data collected in 2015 and 2016 and analysed using thematic analysis. Semi-structured interviews and focus groups were conducted with staff, including executive sponsors, managers, facilitators and clinical nursing teams. The COREQ guidelines were considered in the conduct and reporting of this study. Results: The results describe the impact of implementing the key performance indicators from the perspective of nurses operating at different levels within organisations. The following six themes became apparent: credibility of the key performance indicators; uncovering the whole picture; embracing nursing; engaging in the process; connecting with others; and achieving healthful outcomes. Conclusion: The findings from this study confirm an emerging relationship between the nature of evidence generated by the person-centred nursing key performance indicators and its use in practice. Also highlighted was the importance of facilitation in the process of data collection and in understanding the data and the context in which it is used. Implications for practice: This study will be of interest to clinical nurses, educators, managers and facilitators of cultural change. The data generated through the implementation of the person-centred nursing key performance indicators evidences the global contribution of nursing across services provided to children and their families Engaging staff at all levels in an organisation enhances implementation of evidence that is meaningful for nursing practice The triangulated evidence generated by the key performance indicators and accompanying measurement tools offers feedback that can drive improvements in nursing practice

Zusammenfassung Hintergrund Alkohol ist nicht nur offline, sondern inzwischen auch online fast allgegenwärtig. Ziel der Arbeit Die vorliegende Arbeit verfolgt das Ziel, den Status quo der Alkoholprävention auf führenden Social-Media-Plattformen im deutschsprachigen Raum zu beschreiben. Material und Methoden Dazu wird zunächst der internationale Forschungsstand zur Alkoholkommunikation in sozialen Medien in einem narrativen Review zusammengefasst. Im Zuge einer Social-Media-Analyse wird dann untersucht, welche deutschsprachigen alkoholbezogenen Kanäle auf Plattformen wie Facebook und Instagram große Reichweiten haben. Anschließend werden mittels Inhaltsanalyse N = 470 Beiträge und N = 3015 Nutzerkommentare von dem reichweitenstärksten Social-Media-Kanal der Alkoholprävention untersucht. Schließlich wird durch eine Onlinebefragung von N = 1150 Jugendlichen (16–20 Jahre) deren alkoholbezogene Social-Media-Nutzung erkundet. Ergebnisse Laut Forschungsstand findet in sozialen Medien vor allem eine glorifizierende Kommunikation zu Alkohol statt. Auf führenden Social-Media-Plattformen sind die reichweitenstärksten deutschsprachigen alkoholbezogenen Kanäle dem Marketing und Humor gewidmet, Prävention ist deutlich weniger verbreitet. Der bislang reichweitenstärkste Kanal der Alkoholprävention ist die Facebook-Seite der Jugendkampagne „Alkohol? Kenn dein Limit.“ der Bundeszentrale für gesundheitliche Aufklärung (BZgA), die mit Videos und Umfragen die höchsten Interaktionsraten erzielt. Hier äußern sich junge Social-Media-Nutzer alkoholkritisch (11 % der Kommentare), aber oft weiterhin alkoholbefürwortend (21 %). Rund die Hälfte der befragten Jugendlichen hat sich schon an alkoholbezogener Onlinekommunikation beteiligt. Diskussion Die Alkoholprävention steht vor der Aufgabe, der in sozialen Medien dominierenden Alkoholverherrlichung sensibilisierende Botschaften entgegenzusetzen.

This study was conducted to analyze the students’ mistake in writing recount text. The subject of this research was the eighth-grade students of Mts Al-Amin Ngetos, Nganjuk which was consisted of 38 students. The researcher needs to analyze the students’ mistakes in writing recount text because writing is one of the four crucial skills to master English. In learning English at school students also learn how to write a text and organize their idea into a proper text. In getting the data, the researcher applied class evaluation research using purposive sampling technique which the sample was taken from 5 students’ recount text of the eight grade students of MTs Al-Amin Ngetos, Nganjuk. The researcher analyzed the students’ recount text based on the writing aspect which should be considered by the students in writing a recount text they were content, organization, grammar, vocabulary, and mechanics. The results showed that there were some mistakes made by the students, mostly their mistakes were inappropriate used of grammar and mechanics. So, the researcher provided suggestions for the teacher to help the students to know the writing aspects and learn how to apply them in writing text.

The h-index is an indicator of the scientific impact of an academic publishing career. Its hybrid publishing/citation nature and inherent bias against younger researchers, women, people in low resourced countries, and those not prioritizing publishing arguably give it little value for most formal and informal research evaluations. Nevertheless, it is well-known by academics, used in some promotion decisions, and is prominent in bibliometric databases, such as Google Scholar. In the context of this apparent conflict, it is important to understand researchers’ attitudes towards the h-index. This article used public tweets in English to analyse how scholars discuss the h-index in public: is it mentioned, are tweets about it positive or negative, and has interest decreased since its shortcomings were exposed? The January 2021 Twitter Academic Research initiative was harnessed to download all English tweets mentioning the h-index from the 2006 start of Twitter until the end of 2020. The results showed a constantly increasing number of tweets. Whilst the most popular tweets unapologetically used the h-index as an indicator of research performance, 28.5% of tweets were critical of its simplistic nature and others joked about it (8%). The results suggest that interest in the h-index is still increasing online despite scientists willing to evaluate the h-index in public tending to be critical. Nevertheless, in limited situations it may be effective at succinctly conveying the message that a researcher has had a successful publishing career.

National research evaluation systems that use metrics for the assessment of academic institutions are usually regarded as exemplifying the same neoliberal model of governance that, with minor varia...

Over the past two decades, several methods have been developed to evaluate the societal impact of research. Compared to the practical development of the field, the conceptual development is relatively weak. This review article contributes to the latter by elucidating the theoretical aspects of the dominant methods for evaluating societal impact of research, in particular, their presuppositions about the relationship between scientific and societal value of research. We analyse 10 approaches to the assessment of the societal impact of research from a constructivist perspective. The methods represent different understandings of knowledge exchange, which can be understood in terms of linear, cyclical, and co-production models. In addition, the evaluation methods use a variety of concepts for the societal value of research, which suggest different relationships with scientific value. While some methods rely on a clear and explicit distinction between the two types of value, other methods, in particular Evaluative Inquiry, ASIRPA, Contribution Mapping, Public Value Mapping, and SIAMPI, consider the mechanisms for producing societal value integral to the research process. We conclude that evaluation methods must balance between demarcating societal value as a separate performance indicator for practical purposes and doing justice to the (constructivist) science studies’ findings about the integration of scientific and societal value of research. Our analytic comparison of assessment methods can assist research evaluators in the conscious and responsible selection of an approach that fits with the object under evaluation. As evaluation actively shapes knowledge production, it is important not to use oversimplified concepts of societal value.

Se presentan los resultados de una investigación multicéntrica de evaluación de los Comités de Ética de Investigación (CEI) en Panamá, la cual busca mecanismos que ayuden a fortalecer su capacidad para evaluar, desde un enfoque de bioética y derechos humanos, los protocolos de los ensayos clínicos financiados por la industria farmacéutica o por los centros extranjeros de investigación, así como monitorear la implementación de los protocolos aprobados. Se revisó documentación suficiente y se hicieron entrevistas a miembros de cuatro CEI. A partir de ahí, se concluye que en Panamá los CEI respetan las normas internacionales de funcionamiento y de revisión de protocolos, y que los problemas encontrados guardan relación con su experiencia, el contexto social y la evolución de los intereses de las compañías farmacéuticas.

Chronic kidney disease (CKD) is a major health concern in Australia, with a prevalence of 9% among non-Indigenous adult Australians and 18% among Aboriginal and Torres Strait Islander people. CKD is a risk factor for cardiovascular disease, kidney failure and other complications.The aim of this article is to outline recent advances in CKD pathophysiology and management, focusing on strategies for slowing disease progression and preventing cardiovascular and other complications.All patients with CKD benefit from a healthy lifestyle including regular physical activity, avoiding smoking and obesity, and limiting sodium intake. Nephrotoxic medications and other risk factors for acute kidney injury should be avoided. Treatment of the underlying pathology is important, along with control of hypertension, reduction in albuminuria (if present) and treatment of dyslipidaemia. Patients with renal anaemia, mineral and bone disorder, or advanced disease should normally be cared for in partnership with a nephrologist.