Search

With the growing complexity of societal and scientific problems, research centers have emerged to facilitate the conduct of research beyond disciplinary and institutional boundaries. While they have become firmly established in the global university landscape, research centers raise some critical questions for research evaluation. Existing evaluation approaches designed to assess universities, departments, projects, or individual researchers fail to capture some of the core characteristics of research centers and their participants, including the diversity of the involved researchers, at what point in time they join and leave the research center, or the intensity of their participation. In addressing these aspects, this article introduces an advanced approach for the ex post evaluation of research centers. It builds on a quasi-experimental within-group design, bibliometric analyses, and multilevel statistics to assess average and individual causal effects of research center affiliation on participants along three dimensions of research performance. The evaluation approach is tested with archival data from a center in the field of sustainability science. Against a widely held belief, we find that participation in research centers entails no disadvantages for researchers as far as their research performance is concerned. However, individual trajectories varied strongly.

Assessing societal impacts of research is more difficult than assessing advances in knowledge. Methods to evaluate research impact on policy processes and outcomes are especially underdeveloped, and are needed to optimize the influence of research on policy for addressing complex issues such as chronic diseases. Contribution analysis (CA), a theory-based approach to evaluation, holds promise under these conditions of complexity. Yet applications of CA for this purpose are limited, and methods are needed to strengthen contribution claims and ensure CA is practical to implement. This article reports the experience of a public health research center in Canada that applied CA to evaluate the impacts of its research on policy changes. The main goal was to experiment with methods that were relevant to CA objectives, sufficiently rigorous for making credible claims, and feasible. Methods were ‘good enough’ if they achieved all three attributes. Three cases on government policy in tobacco control were examined: creation of smoke-free multiunit dwellings, creation of smoke-free outdoor spaces, and regulation of flavored tobacco products. Getting to ‘good enough’ required careful selection of nested theories of change; strategic use of social science theories, as well as quantitative and qualitative data from diverse sources; and complementary methods to assemble and analyze evidence for testing the nested theories of change. Some methods reinforced existing good practice standards for CA, and others were adaptations or extensions of them. Our experience may inform efforts to influence policy with research, evaluate research impacts on policy using CA, and apply CA more broadly.

Interdisciplinary research centers are typically viewed as a vehicle for creating opportunities in science where the intricacy of the research problem calls for persistent collaboration across multiple disciplines. This case study analyzed the effects of an interdisciplinary research center on the publication and collaboration behaviors of faculty affiliated with the center. The study also sought to determine through faculty interviews what factors contributed to these effects for participants whose publication and collaboration behaviors were most changed after affiliation. Results of the study indicate that affiliation with the center has a significant positive effect on participant collaboration activities, and a moderate positive effect on publication activities (i.e. publishing in new fields). Factors contributing to success cited by interviewees included organized leadership, a positive atmosphere, breaking into sub-groups, and the ability to collaborate with researchers with whom they would not have interacted outside of the center. This case study may be useful in providing a framework for early evaluation of the effects of interdisciplinary research centers on affiliated participants.

Science is increasingly organized in large-scale science centers. Expectations are that center-based research should yield qualitatively different research than single-investigator-produced research, but demonstrating the particular effects of a center is challenging. The notion of 'centerness' is that science centers can yield findings that reflect conceptual collisions across disciplinary or geographic boundaries between a center's scientific members. Collaborative research commonly is assessed using co-authorships of scientific publications. Co-authorship measures can be conveniently and unobtrusively operationalized, yet they can yield indicators (especially for a small center) showing little change in collaboration in a center-context, while richer findings of collaboration are evidenced in observing other aspects of center members' behavior. This article puts forth the use of social network analysis and bibliographic coupling as measures of 'centerness' and presents their characteristics in the context of an National Science Foundation (NSF) science center. Bibliographic coupling is characterized as a measure of centerness over the time period of the establishment of an NSF-funded science center. We also examine changes in bibliographic coupling indicators alongside changes in co-authorship networks and a social network survey of center participants. Bibliographic coupling is shown to make a distinctive contribution to measurement of centerness, demonstrating increasing coupling from the pre-center and since-center periods. This increase in centerness as represented in bibliographic coupling does not dramatically change when self-citations are removed. Overall, results show this to be a useful and consistent measure of centerness and knowledge coalescence within the research center environment. Copyright The Author 2013. Published by Oxford University Press. All rights reserved. For Permissions, please email: journals.permissions@oup.com, Oxford University Press.

The presence of research centers on university campuses has expanded dramatically over the past two decades. The research center is a prominent feature of the contemporary academic landscape with some institutions hosting dozens or even more than one hundred. They are important not only due to their sheer number but also because they reflect deeper undercurrents on what is happening in universities. Research centers are entities that arrange human and material resources for research in specific ways that contrast with the rest of their academic environment. At the same time, there is diversity among centers depending on the rationale for their existence, the circumstances and profiles of their members, and the specific academic contexts of HBCUs, EPSCoR, and Majority universities. In this article, we explore the correspondence between the role of centers and their function in context, on the one hand, and the patterns of human, social and material resources that emerge on the other. It highlights dimensions of value and impact of a research policy instrument, namely, the research center, when a broader framework of assessment that considers its effect on the overall academic system is used. We do so with a mixed methods approach that combines a set of qualitative case studies of centers and the results of a survey of academic researchers both affiliated and not affiliated with research centers in each academic context. With a triangulation design, the results of the two methods are analyzed with a convergent approach. We find that research centers are transitional entities with different local impacts and that some differences in the profiles of participating and non-participating researchers seems to be emerging. Copyright The Author 2012. Published by Oxford University Press. All rights reserved. For Permissions, please email: journals.permissions@oup.com, Oxford University Press.

While traditional grants remain central in US federal support of academic scientists and engineers, the role of multidisciplinary NSF Centers is growing. Little is known about how funding through these Centers affects scientific output or (as is an NSF aim) increases academic collaboration with industry. This paper tests the use of CVs to examine how Center funding affects researchers' publication rates and their obtaining industry grants. Copyright , Beech Tree Publishing.

Ex-ante, interim and outcome evaluation approaches used at the program, center and project-level of analysis for four separate co-operative research center initiatives are examined. Center initiatives have received much evaluative scrutiny since 1980, particularly interim and outcome assessments. The modified peer-review technique has most use. The strengths and weaknesses of novel approaches and their applicability to more traditional research funding schemes are discussed. Copyright , Beech Tree Publishing.

Most agricultural research organizations strive to address societal challenges and contribute to positive societal impacts. Fulfilling this ambition involves embedding a culture of impact in organizational culture, which, in our view, entails three main elements: understanding the role of the research community in contributing to impacts over the long term within the systems in which it operates; equipping researchers to support positive change; and implementing strategies that allow the culture of impact to percolate at various levels of the organization. To build just such a culture, in the past 8 years, Cirad, the French Agricultural Research Center for International Development, embarked on a transformational process, from which we draw key lessons. Building a culture of impact requires fostering transdisciplinary dialogue on the multiple roles of researchers, on their contribution to societal impacts, and on the relevance of this reflection. This involves adapting from pre-existing visions, interactions, and practices. Formalization in the organization's strategy and the action of leading change agents foster its institutionalization. Strengthening capacity to build shared visions of change and collective processes in research design, implementation, and evaluation while respecting the diversity of profiles and approaches in the organization favors appropriation. This requires adequate funding at the project, institutional and funding bodies level, and targeted communication to ensure buy-in by internal and external change agents. We argue that a culture of impact is a reflective culture and long-term dynamics that aims to overcome the dichotomy between research and development and bring agricultural research closer to societal needs.