Search

Background Hospital websites are important sources for patients to access health information. Aim The purpose of this study was to develop the quality evaluation model for hospital websites. Methods The quantitative study was conducted through the modified Delphi method in 2014-2015. The population of the study includes 10 experts were chosen by targeting non-randomized method. A questionnaire was prepared based on the prototype that was developed by research papers and related models. The validity of the questionnaires was confirmed by face validity and CVI and CVR estimation. Reliability was obtained by split-half method (α = 0.8). Experts' opinions were collected through interview. Then, their frequency and percentage were determined. Items with options completely agree and agree over 75% was approved, items below 50% were removed, and items 50%-75% were removed after three interviews repetitions. Results Most of the experts agreed about the pleasant and harmonious colors, the readable and consistent fonts (100%). The least frequency was allocated to correct grammar and words, support for multilingual and rapidly changing of displaying pages in the website with a frequency of 2 (20%). Conclusion The minimum qualitative criteria for a website are usability, efficiency, user friendly, service, reliability and interaction.

Background: Contributing to health informatics research means using conceptual models that are integrative and explain the research in terms of the two broad domains of health science and information science. However, it can be hard for novice health informatics researchers to find exemplars and guidelines in working with integrative conceptual models. Objectives: The aim of this paper is to support the use of integrative conceptual models in research on information and communication technologies in the health sector, and to encourage discussion of these conceptual models in scholarly forums. Methods: A two-part method was used to summarize and structure ideas about how to work effectively with conceptual models in health informatics research that included (1) a selective review and summary of the literature of conceptual models; and (2) the construction of a step-by-step approach to developing a conceptual model. Results: The seven-step methodology for developing conceptual models in health informatics research explained in this paper involves (1) acknowledging the limitations of health science and information science conceptual models; (2) giving a rationale for one’s choice of integrative conceptual model; (3) explicating a conceptual model verbally and graphically; (4) seeking feedback about the conceptual model from stakeholders in both the health science and information science domains; (5) aligning a conceptual model with an appropriate research plan; (6) adapting a conceptual model in response to new knowledge over time; and (7) disseminating conceptual models in scholarly and scientific forums. Conclusions: Making explicit the conceptual model that underpins a health informatics research project can contribute to increasing the number of well-formed and strongly grounded health informatics research projects. This explication has distinct benefits for researchers in training, research teams, and researchers and practitioners in information, health, and other disciplines.

BackgroundThe network structure of an organization influences how well or poorly an organization communicates and manages its resources. In the Millennium Villages Project site in Bonsaaso, Ghana, a mobile phone closed user group has been introduced for use by the Bonsaaso Millennium Villages Project Health Team and other key individuals. No assessment on the benefits or barriers of the use of the closed user group had been carried out. ObjectiveThe purpose of this research was to make the case for the use of social network analysis methods to be applied in health systems research—specifically related to mobile health. MethodsThis study used mobile phone voice records of, conducted interviews with, and reviewed call journals kept by a mobile phone closed user group consisting of the Bonsaaso Millennium Villages Project Health Team. Social network analysis methodology complemented by a qualitative component was used. Monthly voice data of the closed user group from Airtel Bharti Ghana were analyzed using UCINET and visual depictions of the network were created using NetDraw. Interviews and call journals kept by informants were analyzed using NVivo. ResultsThe methodology was successful in helping identify effective organizational structure. Members of the Health Management Team were the more central players in the network, rather than the Community Health Nurses (who might have been expected to be central). ConclusionsSocial network analysis methodology can be used to determine the most productive structure for an organization or team, identify gaps in communication, identify key actors with greatest influence, and more. In conclusion, this methodology can be a useful analytical tool, especially in the context of mobile health, health services, and operational and managerial research.

Modern organizations need effective ways to assess cybersecurity risk. Successful cyber attacks can result in data breaches, which may inflict significant loss of money, time, and public trust. Small businesses and non-profit organizations have limited resources to invest in cybersecurity controls and often do not have the in-house expertise to assess their risk. Cyber threat actors also vary in sophistication, motivation, and effectiveness. This paper builds on the previous work of Lerums et al., who presented an AnyLogic model for simulating aspects of a cyber attack and the efficacy of controls in a generic enterprise network. This paper argues that their model is an effective quantitative means of measuring the probability of success of a threat actor and implements two primary changes to increase the model's accuracy. First, the authors modified the model's inputs, allowing users to select threat actors based on the organization's specific threat model. Threat actor effectiveness is evaluated based on publicly available breach data (in addition to security control efficacy), resulting in further refined attack success probabilities. Second, all three elements - threat effectiveness, control efficacy, and model variance - are computed and evaluated at each node to increase the estimation fidelity in place of pooled variance calculations. Visualization graphs, multiple simulation runs (up to 1 million), attack path customization, and code efficiency changes are also implemented. The result is a simulation tool that provides valuable insight to decision-makers and practitioners about where to most efficiently invest resources in their computing environment to increase cybersecurity posture. AttackSimulation and its source code are freely available on GitHub.

Despite the critical importance of Computer Assisted Language Learning (CALL) Teacher Education (CTE) programmes, L2 practitioners often report that there are not many such onsite/online programmes, and the ones that exist may not directly meet their specific needs. Moreover, there seems to be a lack of systematic evaluation studies of such courses. As a result, problems in such courses cannot be diagnosed, and they are left unresolved. This paper presents a study undertaken to evaluate an online Master’s of Arts (MA) in CALL programme in the Republic of Cyprus. The investigation was based on a conceptual multidimensional e-learning evaluation model rated by the students. Data were collected from 25 graduate students via an online anonymous survey. This focussed on the participants’ perceptions of the value of the following aspects: (1) their engagement; (2) the course and its organisation, teaching mode, and materials; (3) course strengths; (4) course aspects most helpful to learning; and (5) course aspects which were obstacles to learning. This paper discusses the findings and offers some first recommendations for further improvement to the MA in CALL programme.

La presente investigación pretende establecer la importancia que juegan los diferentes indicadores de naturaleza metodológico-analítica incluidos en los informes de investigación presentados en los artículos científicos. Así pues, nuestro interés se centra en determinar cuáles de dichos indicadores son los mejores predictores para establecer un perfil de artículo científico con un mayor o menor grado de concreción. Para este fin, seleccionamos todos los artículos científicos publicados en las siete revistas españolas del ámbito de la investigación educativa indexadas en la base de datos Journal Citation Report (JCR) durante el año 2016. Exactamente seleccionamos un total de 479 trabajos publicados durante el trienio 2014-2016. Cada una de las revistas obtuvo una puntuación general dependiendo de la presencia vs ausencia de dichos indicadores metodológicos. Tras la implementación de la técnica de Árboles de Decisión, donde se ha tomado como variable criterio dicha variable transformada en una nueva variable ficticia agrupada en tres niveles y como predictores los diez indicadores metodológico-analíticos hemos comprobado que, efectivamente, algunos de estos indicadores juegan un papel más importante que otros en la concreción de los diseños de investigación planteados en los trabajos objeto de investigación. De esta forma, apreciamos como son los indicadores relativos a los criterios de rigor metodológico (CA1); tipo de muestreo-s considerado-s (MU3); y la metodología de abordaje del estudio (ME) los que han resultado determinantes para el establecimiento de diseños metodológico-analítico más completos., This research aims to establish the importance of the different indicators of a methodologicalanalytical nature included in the research reports presented in scientific articles. Thus, our interest is focused on determining which of these indicators are the best predictors for establishing a scientific article profile with a higher or lower degree of concreteness. For this purpose, we selected all the scientific articles published in the seven Spanish journals within the field of educational research indexed in the Journal Citation Report (JCR) database during 2016. More precisely, we selected a total of 479 papers published during the three-year period 2014-2016. Each journal obtained an overall score depending on the presence vs absence of such methodological indicators. Following the implementation of the Chi-squared Automatic Interaction Detection technique, where this variable, transformed into a new fictitious variable grouped into three levels, has been taken as a variable criterion, and, as predictors, the ten methodological-analytical indicators, it has been found that, in fact, some methodological-analytical indicators play a more important role than others in the concreteness of the research designs raised in the research work. In this way, we are able to determine how these indicators related to the quality criteria are (QUC); the type of sampling-s considered (ST3); and the study methodology (ME), which have been decisive for the establishment of more complete methodological-analytical designs.

As teachers become increasingly positioned as change agents in educational and social transformation, they need continuing professional development (CPD) to support their transition to this new role. Recent innovations in the objectives and processes of CPD programs have responded to this need, but less work has focused on program evaluation. This paper presents the development and application of an evaluation framework within a CPD program in sustainability education. The results demonstrate the value of the framework for investigating how the intentions and expectations of participants and instructors shape experiences and outcomes of CPD programs.

This paper looks at measures for cultural participation and engagement, in relation to UK cultural policy and cultural programming for the London 2012 Olympics. It considers the meanings and interpretation of these terms specifically in relation to the evaluation framework for the WE PLAY programme in the North West of England, an initiative funded by Legacy Trust UK and part of the London 2012 Cultural Olympiad.The paper reviews methodologies for understanding arts participation and engagement used within evaluation and evidence-based policy making. It argues that in spite of the dissonance between arts and sports within Olympics programmes (Inglis 2008) and claims of its deleterious impact on arts funding, particular within the regions, London 2012 has engendered creative programming which strategically deploys the Cultural Olympiad to satisfy local cultural policy objectives and which demands a sophisticated understanding of participation in order to articulate its role within these agendas.

This paper highlights the importance of research in the present volatile external environment in order to improve value to clients. It is focused at Building Services Engineers in Ireland.

Despite recent studies confirming task shifting is both safe and effective, its implementation has proven difficult in practice. So too in the Netherlands, where legal barriers enforcing strict professional boundaries have historically limited task shifting. In recent years, Dutch policymakers have experimented with temporary expanded scopes of practice (ESP) for several professional groups, with the aim to facilitate task shifting in order to increase the overall effectiveness and efficiency of health care. The Clinical Technologist (CT), is an emerging new professional group that has received such a temporary ESP pending an evaluation. This paper reports the qualitative findings of the implementation process of providing CTs with an temporary ESP. Data collection consisted of 69 semi-structured interviews, 3 focus group interviews and 9 participant observations, conducted between September 2015 and October 2017. Analysis was conducted through an 'editing analysis style' whereby data were categorized using the conceptual framework of Grol & Wensing's implementation model. The study suggests that social features are of great importance when implementing task shifting. In situations with few social barriers, organizational and administrative barriers seem to be less dominant, thereby expediting the overall implementation process. Consequently, we recommend that policymakers should prioritize social features over organizational features when implementing task shifting. (C) 2019 Elsevier B.V. All rights reserved.

Good reasons exist for combating stress at work. It is a burden for individual employees and their families and costly to companies and society. Moreover preventing stress at work is a sign of good corporate citizenship as it respects modern legislation that stimulates the provision of a good quality of working life. In order to recommend good interventions, we need to know which interventions work. Outcome evaluation research is, therefore, important but not enough. We not only need to know “what works”, but also “when, how, and why” this may be the case. This means that we need to combine classical effect evaluation research with process evaluation. Combining these two approaches fits in a young scientific discipline: implementation research. Implementation refers to “the way a program is put into practice and delivered to participants” (1). Implementation research crosses traditional disciplinary boundaries, with contributions being made from both occupational and public health domains but also from psychological theories on motivation, behavioral change, and social influence. In a recent commentary, Durlak (1) has highlighted major findings in implementation research. He concludes: “We now know that it is not evidence-based programs that are effective, but it is well-implemented evidence-based programs that are effective” (p1124). He also explains: “If we do not assess implementation, we do not know if a program has been put to an adequate test. It may fail not because the intervention lacks value, but because the intervention was not implemented at a sufficiently high level to produce its effects” (p1124). Effect-only evaluation data may thus mask intervention effects that are sensitive to variations in intervention processes (2). This all means that checking effects is not enough to determine whether a program works or not. To answer this question properly, we need to open up the black box because we also must know how it was implemented. Ten years earlier – in an occupational health intervention context – Tage Kristensen made comparable arguments (3). Kristensen explained the difference between implementation and theory failure by describing the case of a patient (employee) and a pill (stress intervention): “It does not help that the pill has effect if the patient does not take it, and it does not help that the patient takes the pill if it has no effect” (p207). In both cases, process evaluation is needed to find out what actually happened and make the right decisions to proceed. In the first case (pill not taken), the implementation failed and it is therefore not possible to determine if the program works. The underlying theory still may be right. Had the implementation not been studied, the erroneous conclusion might have been that “the theory was wrong”. In this first case, the next step should hence be to work on better implementation. In the second case (the patient took the pill), there were no problems with implementation, but there was no effect. This means that the theory behind the intervention seems to be wrong and needs to be reconsidered. Process evaluation is thus needed to evaluate the generalizability of an intervention (to answer questions such as “under which circumstances will an intervention work?” and “which were the factors that hindered or stimulated the change?”), so that it can be implemented successfully in a variety of settings (2, 4). Studying implementation and process factors is both complicated and challenging. One of the problems is that there are many factors that may codetermine whether a certain intervention does or does not bring about the desired change in health or health behavior. To overcome the problem of long ad hoc lists and enumerations, more systematic taxonomies are needed. Such conceptual frameworks have yet been suggested, for example by Steckler and Linnan (5), Saunders and colleagues (6), and Nielsen & Randall (2). The first two examples merely focus on intervention delivery and participation. They distinguish between fidelity, dose delivered, dose received (exposure and satisfaction), reach, recruitment procedures and context aspects. The broader framework of Nielsen & Randall makes a distinction between (i) the intervention design and implementation, (ii) the context, and (iii) participants’ mental models, such as readiness for change. These latter authors have developed a checklist in order to systematically check these three clusters. Checkpoints in the first cluster are: Who initiated the intervention and for what purpose?; Did the intervention activities target the problems of the workplace?; Did it reach the target group?; Who were the drivers of change?; and What kind of information was provided to participants during the study? The main checkpoint in the context cluster is “which hindering and facilitating factors in the context influenced intervention outcomes?”, whereas the third cluster checks the role of participants’ mental models, such as resistance. Kompier made an earlier comparable attempt (7), arguing that high quality intervention research needs to study three clusters of questions, ie, with respect to the content of the interventions, the context of the study and its interventions, and the design of the study at hand. Also in the organizational health intervention domain, another promising “context, process and outcome evaluation model” has recently been published (8). When it comes to interventions to reduce occupational stress, there certainly is a need for the development, refinement, and utilization of such taxonomies. Various reviews of organizational level intervention studies have been published since 2000 (for example 9–13). The focus of such studies is on the methodological quality of the included studies, which of course is a good thing, and on the effects of interventions (outcome evaluation), but not so much on the intervention itself and how it is implemented. This means that the link between process evaluation and outcome has not yet been systematically addressed (14). Therefore it is hard to understand what actually happened in these intervention studies, and as a consequence is it hard to answer questions such as “How, when and why interventions have or have no effects” (see also 15). In search of better answers to these questions process evaluation is a conditio sine qua non. In this issue of the Scandinavian Journal of Work, Environment and Health, Havermans et al (16) present an interesting overview of process variables in organizational stress management interventions. In a thorough search process, they selected 44 articles that each report on process evaluations. Process variables were defined as “any measure included in the evaluation study that is hypothesized to be associated with the process of implementation of the stress management intervention” (p2). Each article was assessed with a clear template that, as proposed by Nielsen & Randall (2), classified process variables into three major categories: context, intervention, and mental models. Each of these three clusters was itself divided in a number of sub-clusters. The context cluster was made up of context, and barriers and facilitators. The intervention cluster constituted initiation, intervention activities, implementation, and implementation strategy. The mental model comprised readiness for change, perceptions, and changes in mental models. All in all, in 44 articles, 47 process factors were identified. In the majority of these articles, these process factors were collected at one point in time. Process factors were mostly measured at the level of the individual employee and post intervention. Together these authors provide a broad overview of the wealth of factors that may influence the outcomes of an intervention. Interestingly half of the 44 articles did not refer to the process evaluation literature. This suggests that, until now, process factors often are investigated in a merely explorative manner. Furthermore, as Havermans et al point out, many studies do not seem to include a program theory for the intervention under study, ie, explaining how the authors assume the intervention to work and which proximate, intermediate and distal changes (8) they expected. Without predictions stated in a program theory, however, it remains unclear when a certain implementation process is good enough or if intermediate goals were reached. This makes it more difficult to integrate process and effects evaluation. Also, more than half of the studies had a participatory format, which means cooperation of different stakeholders in the assessment, targeting and prevention of work stress. This is an interesting observation that raises the question why employee participation may be essential. We believe that there are several assets to such an approach. First, it may support an accurate problem identification and analysis, and may optimize the fit of the intervention to the organizational culture and context. This simply acknowledges the fact that employees “have agency” and are experts as to their own work situation. Employee participation may also mean improved communication and less resistance to change. It may increase responsibility for dealing with the problems identified and increase commitment to change strategies. And, finally, it may constitute an intervention in its own right as it reflects the concrete enactment of job control (17). The formulation of a program theory and process evaluation, for example through the repeated measurement of commitment and quality of communication, may contribute to the further testing of such assumptions. So far organizational stress interventions reveal mixed results and this makes it difficult to find the best ways for reducing stress at work and to convince businesses to engage in these activities. Outcome research is essential, but only measuring effects without taking the implementation into account may lead to erroneous conclusions. We thereby risk discarding potentially powerful interventions concepts without having tested them properly. The “does it work?” question is a simplification. More informative questions are “under which circumstances will this intervention work?” and “what are hindering and stimulating factors?”. In organizational stress intervention research, process evaluation is thus needed as “an add-on” to – and not substitute for – effect evaluation. The Havermans et al paper can be interpreted as a plea for the integration of process evaluation in all studies of organizational stress interventions, in a way that enables the relation of process aspects to outcomes. We believe that future research may aim at a further standardization of a taxonomy that is rich but not overly detailed. It seems that the Nielsen & Randall classification scheme provides a good starting point. We also believe that such organizational stress management intervention research will falsify an “implementation research hypothesis” formulated by Mick Jagger and Keith Richards in 1965. They assumed that “It’s the singer, not the song”, whereas we argue that it is both the intervention (the song) and the implementation (the singer) that make a difference when it comes to organizational stress management interventions.

Background Evaluating clinical ethics support services (CESS) has been hailed as important research task. At the same time, there is considerable debate about how to evaluate CESS appropriately. The criticism, which has been aired, refers to normative as well as empirical aspects of evaluating CESS. Main body In this paper, we argue that a first necessary step for progress is to better understand the intervention(s) in CESS. Tools of complex intervention research methodology may provide relevant means in this respect. In a first step, we introduce principles of “complex intervention research” and show how CESS fulfil the criteria of “complex interventions”. In a second step, we develop a generic “conceptual framework” for “ethics consultation on request” as standard for many forms of ethics consultation in clinical ethics practice. We apply this conceptual framework to the model of “bioethics mediation” to make explicit the specific structural and procedural elements of this form of ethics consultation on request. In a final step we conduct a comparative analysis of two different types of CESS, which have been subject to evaluation research: “proactive ethics consultation” and “moral case deliberation” and discuss implications for evaluating both types of CESS. Conclusion To make explicit different premises of implemented CESS interventions by means of conceptual frameworks can inform the search for sound empirical evaluation of CESS. In addition, such work provides a starting point for further reflection about what it means to offer “good” CESS.

This paper examines the intellectual structure of evaluation by means of citation analysis. By using various article attributes and citation counts in Google Scholar and (Social) Science Citation Index Web of Science, we analyze all articles published in Evaluation and Program Planning from 2000 until 2012. We identify and discuss the characteristics and development of the field as reflected in the history of those citations.

Existing literature indicates that physical attractiveness positively affects variables such as income, perceived employee quality and performance evaluations. Similarly, in the academic arena, studies indicate instructors who are better looking receive better teaching evaluations from their students. Previous analysis of the website RateMyProfessors.com confirms this, indicating that instructors who are viewed by students as “hot” receive higher “quality” ratings than those who are “not.” However, psychology literature indicates that perceptions of attractiveness are influenced by positive illusions, a property whereby individuals with higher quality relationships view each other more positively than objective observers. This paper uses data from Rate My Professors to investigate the existence of positive illusions in the instructor-student relationship. It finds that positive illusions exist, suggesting that existing literature overestimates the premium associated with physical attractiveness. Furthermore, the source of these illusions varies significantly between male and female instructors with important implications for the role of gender in workplace evaluations, hiring, promotion, and tenure. Accessed 4,387 times on https://pareonline.net from February 16, 2015 to December 31, 2019. For downloads from January 1, 2020 forward, please click on the PlumX Metrics link to the right.

[spa] La Evaluación Participativa (EP) se utiliza con frecuencia para evaluar los planes y acciones comunitarias. Pero, ¿cómo se diseña y se lleva a cabo un proceso de EP?, ¿qué elementos metodológicos diferencian la EP de otras prácticas evaluativas?, ¿qué tipo de herramientas e instrumentos se utilizan? Este artículo intenta responder a estos interrogantes; para ello, se revisa la bibliografía y guías más recientes sobre metodología de la EP. A continuación se describen y se aportan las reflexiones metodológicas sobre el proceso de EP realizado en tres planes comunitarios. Se analizan los aspectos metodológicos más relevantes observados durante la entrada a la comunidad (1), el análisis del contexto (2), la formación del grupo motor (3), la aplicación de técnicas y dinámicas participativas para evaluar las acciones comunitarias y su multiplicación (4), y la evaluación y cierre (5) de los tres procesos de EP. Los resultados permiten identificar aportaciones metodológicas relevantes para la implementación de futuros procesos de Evaluación Participativa en entornos comunitarios, como: los agentes clave para el proceso de entrada en la comunidad, los ejes a tener en cuenta para fomentar la participación, o el papel del equipo de evaluadores., [eng] Participatory evaluation (PE) is frequently used to assess community plans and actions. But how is a PE process designed and carried out? Which methodological elements differentiate PE from other assessment practices? And what kind of tools and instruments are used? This article attempts to answer these questions, though a review of the most recent literature and guidebooks on the EP methodology. Some methodological reflections on the EP process conducted in three community plans are given and described. In this paper we analyze the most relevant methodological aspects observed during entry into the community (1), context analysis (2), the formation of the steering group (3), the application of participatory techniques and dynamics to evaluate community actions, their multiplication (4), and the evaluation and closure of the three processes of EP (5). Results allow identifying relevant methodological contributions to the implementation of future processes of Participatory Evaluation in community settings, as the key stakeholders to the process of entry into the community, some elements to consider in order to encourage participation, and the role of the professional evaluators team.

El objetivo de este estudio es analizar la opinión del profesorado universitario en Contabilidad sobre el modelo universitario actual de acreditación nacional y la valoración de los distintos criterios establecidos para la promoción universitaria. Los resultados obtenidos en una encuesta ponen de manifiesto un cierto descontento de la comunidad académica del área de Contabilidad en lo relativo a la ponderación de artículos JCR incluidos en listados ISI, considerada excesiva. A su vez existe una creencia general acerca de que el actual sistema de acreditación promueve actitudes individualistas y no valora adecuadamente las actividades docentes, aunque existen diferencias de opinión en función del género, tamaño de universidad, edad, categoría profesional y posesión o no de tramos de investigación. The objective of this paper is to analyze the opinions of accounting professors about The National Accreditation System, as well as about the valuation of the criteria. The survey results show that the accounting community is not satisfied with the excessive weight given to ISI journals. In addition, there is a general belief that the current system promotes individualistic attitudes and does not consider adequately teaching activities, even though opinions vary according to gender, university size, age, professional category and research intensity.

La orientación de los estudiantes universitarios a lo largo de su carrera es una necesidad generalmente sentida y admitida. En este artículo presentamos los resultados obtenidos en la evaluación del diseño del programa informatizado de orientación académica MagallanesAtando Cabos dirigido principalmente a los estudiantes de nuevo ingreso en la universidad. El proceso de investigación evaluativa sobre dicho programa se inicia con la evaluación de su diseño, que persigue determinar la adecuación de esta herramienta de orientación al contexto y a las necesidades de los destinatarios del programa. En primera instancia aludiremos a los antecedentes que han dado lugar al desarrollo del programa objeto de estudio, para pasar posteriormente a explicar el diseño de evaluación desarrollado, los resultados más destacables y las conclusiones acerca de la calidad intrínseca del mismo. University students’ career guidance is all over accepted as one of the more important students’ academic needs. This paper deals with an evaluation research on an academic guidance program: Magallanes-Atando Cabos; the program has been designed to meet the freshmen academic needs in university settings. First we present the research process followed to determine program design quality, or its fitness to quality standards defined by scientific literature for programs in this particular guidance field. Rationale, research design, data collected, outcomes and implications for the intended quality program design are the several contributions we display in the present work.

University students’ career guidance is all over accepted as one of the more important students’ academic needs. This paper deals with an evaluation research on an academic guidance program: Magallanes-Atando Cabos; the program has been designed to meet the freshmen academic needs in university settings. First we present the research process followed to determine program design quality, or its fitness to quality standards defined by scientific literature for programs in this particular guidance field. Rationale, research design, data collected, outcomes and implications for the intended quality program design are the several contributions we display in the present work. La orientación de los estudiantes universitarios a lo largo de su carrera es una necesidad generalmente sentida y admitida. En este artículo presentamos los resultados obtenidos en la evaluación del diseño del programa informatizado de orientación académica Magallanes-Atando Cabos dirigido principalmente a los estudiantes de nuevo ingreso en la universidad. El proceso de investigación evaluativa sobre dicho programa se inicia con la evaluación de su diseño, que persigue determinar la adecuación de esta herramienta de orientación al contexto y a las necesidades de los destinatarios del programa. En primera instancia aludiremos a los antecedentes que han dado lugar al desarrollo del programa objeto de estudio, para pasar posteriormente a explicar el diseño de evaluación desarrollado, los resultados más destacables y las conclusiones acerca de la calidad intrínseca del mismo.

Das Interesse und der Bedarf an Evaluationen boomt. Entgegen dem allgemeinen Arbeitsmarkttrend nimmt die Zahl von Stellenanzeigen für Experten mit Evaluationskenntnissen zu. Auch die externe Vergabe von Evaluationen wächst kontinuierlich. Das Thema Evaluation erlebt nicht nur im Rahmen der Diskussion um "New Public Management" und "Neue Steuerungsmodelle" eine deutliche Aufwertung, sondern auch im Kontext der Diskussion um Qualitätsmanagement und Qualitätsentwicklung. Maßnahmen sind dadurch zu legitimieren, dass sie Wirkungen erzielen, Verbesserungen herbeiführen und nachhaltige Entwicklungsprozesse einleiten. Evaluationen werden deshalb erstens nachgefragt, um Programme und Interventionsprozesse zu begleiten, Informationen zu sammeln und Bewertungen vorzunehmen, damit im Sinne einer effektiven Zielerreichung frühzeitig angemessene Steuerungsentscheidungen getroffen werden können ("formative Evaluation"). Und zweitens, um den Output, die Wirksamkeit und Nachhaltigkeit von Programmen und Interventionsmaßnahmen zu dokumentieren ("summative Evaluation"). Deshalb ist zu beobachten, dass staatliche wie nicht-staatliche Institutionen zunehmend auf das Instrument der Evaluation zurückgreifen. Der bisher institutionell und fachlich stark zersplitterte Evaluationsbereich beginnt sich als Folge dieser erheblich gewachsenen quantitativen Nachfrage und qualitativen Ansprüche zu professionalisieren. Wie zuvor bereits in anderen europäischen Ländern hat sich im Jahr 1997 auch in Deutschland eine Gesellschaft für Evaluation e.V. (DeGEval) als Fachverband sowohl für Evaluatoren wie auch für Auftraggeber und Nutzer von Evaluationen gegründet. Im Jahr 2002 wurde die erste deutschsprachige "Zeitschrift für Evaluation" (ZfEv) gegründet, die sich zum Ziel gesetzt hat, zu einem Brennpunkt für Evaluation zu werden, in dem Fachkenntnisse interdisziplinär gebündelt, ein wechselseitiger Transfer zwischen Wissenschaft und Praxis unterstützt und theoretische Konzeptionen und methodische Evaluationsverfahren vermittelt und diskutiert werden. Zur weiteren Professionalisierung der Evaluation will auch das in Mai 2002 mit finanzieller Unterstützung der saarländischen Landesregierung und der Universität des Saarlandes gegründete Centrum für Evaluation (CEval) einen Beitrag leisten. Das CEval setzt die bisher am Lehrstuhl für Soziologie durchgeführte Evaluationsforschung in einem anderen organisatorischen Rahmen fort. Dieser Beitrag beschreibt Aufgaben und Programmatik des CEval als integriertes Lehr- und Forschungsinstitut im Bereich Evaluation. In this research paper it is demonstrated that evaluation has become professionalized in Germany in the last 10 years. Besides the foundation of a German Evaluation society in 1997 and the publishing of an evaluation journal, the Centre for Evaluation (CEval) has been established in 2002 at the University in Saarbrücken, supported by the federal government of the Saarland.

This article responds to the misconceptions that continue to characterize the delivery of knowledge content in architectural courses. Based on reviewing the literature on pedagogy the paper explores the value and benefits of introducing evaluation research as a mechanism for critical inquiry and knowledge construction in theory courses in architecture and urbanism. A framework is developed and employed to demonstrate the way in which this type of learning can be incorporated. The development and implementation of a series of in-class and off campus exercises in two different contexts reveal that structured actions and experiences help students to be in control over their learning while invigorating their understanding of the body of knowledge delivered in a typical lecture format. It firmly believed that this would offer students multiple learning opportunities while fostering their capabilities to shift from passive listeners to active learners and from knowledge consumers to knowledge producers.

In this article, the author explores the issues associated with the practical signifi cance of evaluation research in the policy making environment of Latvia. Sporadic, selective and rare use of the results of evaluation research is viewed as the central issue of the paper. The objective of this article is to reveal the connection between the action strategy of the groups involved in the implementation of evaluation research and manifestations of practical signifi cance of evaluation. Implementation, utilisation and practical importance of evaluation research are examined not only in relation to rational action strategies of the groups, but also in relation to the understanding of the concept of practical signifi cance of evaluation research. The theoretical framework of the article is based on the conceptual understanding of J. Habermas’ purposive rationality and communicative rationality, as well as C. H. Weiss’ understanding of practical utilisation of evaluation (the instrumental and enlightened models). The practical importance of evaluation research as well as action strategies of implementing evaluation research are studied empirically in the form of qualitative interviews. The author concludes that the rational action strategy (typical of the Latvian policy making environment) of the groups involved in evaluation is not readily compatible with the evaluation approach that focuses on practical utilisation. The lack of communicative rationality in the action strategy of the interacting and involved groups substantially diminishes the role of evaluation research in the identifi cation and implementation of the most eff ective models of solving social issues in policy making in Latvia., Studies of Transition States and Societies, Vol 3, No 3 (2011)