Search

This paper provides an introduction to issues surrounding the participation rights of young people in research and the implications of their growing involvement in research as well as providing a discourse on the ethical implications related to consent. The unique contribution of this paper is that it considers children's rights in respect to the increasing opportunities for young people to take part in evaluation research. The aim of this paper, therefore, is to acknowledge the growing involvement for young people in research and the implications of ensuring that their rights of participation are respected. Secondly, we will consider the children's rights legislation and our obligations as researchers to implement this. Finally, we will explore consent as an issue in its own right as well as the practicalities of accessing participants. This paper will postulate that any research about young people should involve and prioritize at all stages of the research process; including participation in decision-making. We conclude by identifying five key principles, which we believe can help to facilitate the fulfilment of post-primary pupils' ability to consent to participate in trials and evaluative research.

Describes a study that seeks to confirm several key factors in computer-based versus paper-based assessment. Based on earlier research, the factors considered in this study of undergraduates include content familiarity; computer familiarity; competitiveness; and gender. Reports results of analysis of variance that showed the computer-based test group outperformed the paper-based test group. (Author/LRW)

This study examined the psychometric equivalence of paper-and-pencil and Internet formats of key questionnaires used in couple research. Self-report questionnaires assessing interpersonal constructs (relationship satisfaction, communication/conflict management, partner support, emotional intimacy) and intrapersonal constructs (individual traits, psychological symptoms, contextual influences) were administered to young adults in committed dating relationships. The same measures were administered twice via paper-and-pencil and/or Internet methods over a 2-week period. Method order was counterbalanced among participants, and temporal stability was controlled. Intrapersonal and interpersonal measures generally remained reliable when administered online and demonstrated quantitative and qualitative equivalence across methods. The implications of online administration of questionnaires are discussed, and specific recommendations are made for researchers who wish to transition to online data collection. (Contains 4 tables.)

This study investigates the measurement equivalence of a five-factor measure of personality across two groups applying for jobs, who completed the same questionnaire using either a paper-and-pencil (n = 429) or a web online answer format (n = 651). The data were collected using the Big Five Questionnaire-2 (BFQ-2; which is a measure of the Five Factor Model) of personality traits. Multiple-group confirmatory factor analysis was used to test for the equivalence of factor covariance and mean structures of the BFQ-2. Findings suggested that the Five Factor Model scales have the same measurement unit and origin across applicants using different administration modes. However, latent means were slightly higher for applicants who responded in a web and unproctored condition than for applicants who completed a paper-and-pencil version of the same test. Practical implications for personality assessment are discussed. (Contains 1 table and 2 notes.)

Ruscio, Seaman, D'Oriano, Stremlo, and Mahalchik (this issue) evaluate 22 bibliometric indicators, including conventional measures, like the number of publications, the "h" index, and many "h" index variants. To assess the quality of the indicators, their well-justified criteria encompass conceptual, empirical, and practical issues: ease of understanding, accuracy of calculation, effects on incentives, influence of extreme scores, and validity. Although they found that some indicators are more attractive for research evaluation purposes than others, the correlation between most of the indicators is (very) high. A generally high correlation between the indicators points out a redundant contribution each to each other. Calculating a comprehensive metaanalysis, Bornmann and colleagues (Bornmann, Mutz, Hug, & Daniel, 2011) came to a very similar result: Only a few "h" index variants make a nonredundant contribution to the "h" index; most of the variants correlate very highly with the "h" index. They presented the first metaanalysis of studies that computed correlations between the "h" index and variants of the "h" index that have been proposed and discussed in the literature. Thus, the study was not about the added value of the "h" index or "h" index variants to other bibliometric indices, e.g., total number of citations. They tested whether there is a generally high (or low) correlation between the "h" index and the variants. The metaanalysis included a total of 135 different bivariate correlation coefficients between the "h" index and an "h" index variant (such as the "g" index). In total, they analyzed 37 different "h" variants. Their results show, with an overall mean value between 0.8 and 0.9, there is a high correlation between the "h" index and the "h" index variants. According to Navon (2009), "high correlations indicate that despite the differences in how the metrics are calculated, there is too much redundancy in the information they yield." Even if the "actual" correlations might be higher than those claimed in the original papers introducing the new "h"-type indices due to systematic sampling selection effects, a mean correlation coefficient of between 0.8 and 0.9 is still high and might not justify the development of more and more "h" index variants. However, not all "h" index variants have a high correlation with the "h" index. Their results indicate that some "h" index variants have been developed that have a relatively low correlation with the "h" index, and it can be assumed that they can make a nonredundant contribution to the "h" index. These variants are mainly the MII (Sypsa & Hatzakis, 2009) and the "m" index (Bornmann, Mutz, & Daniel, 2008). For evaluative purposes, then, the "h" index could be combined with these "h" index variants to better depict research performance bibliometrically. (Contains 1 table.)

This article describes rates of missing item responses in personal digital assistant (PDA) assessments as compared to paper assessments. Data come from the evaluation of a classroom-based leisure, life skills, and sexuality education program delivered to high school students in Cape Town, South Africa. Analyses show that the paper assessments had much higher rates of missingness than PDA assessments. This association is moderated by item order. Certain analyses also suggest that paper assessments have higher rates of missingness for items pertaining to participants' sexual behavior. Implications of these results for evaluation research will be discussed. (Contains 5 tables, 1 figure, and 2 notes.)

Many colleges have either begun or are contemplating using the web-based survey to gather student feedback on faculty teaching. Unlike the paper-based method, the web-based approach gives every student in the class the opportunity to provide feedback. Hence, the populations that participate in the web-based and paper-based methods may be quite different, and so may be the feedback. This paper compares the results of student feedback gathered through paper-based and web-based approaches. The results provide information to faculty and administrators on any differences they may expect as they make the transition from a paper-based to a web-based survey of faculty teaching.

Web-based surveys have become more prevalent in areas such as evaluation, research, and marketing research to name a few. The proliferation of these online surveys raises the question, how do response rates compare with traditional surveys and at what cost? This research explored response rates and costs for Web-based surveys, paper surveys, and mixed-mode surveys. The participants included evaluators from the American Evaluation Association (AEA). Results included that mixed-mode, while more expensive, had higher response rates. (Contains 3 tables and 1 figure.)

The authors have been using components of team-based learning (TBL) in two undergraduate courses at the University of Texas for several years: an educational psychology survey course--Cognition, Human Learning and Motivation--and Introduction to Statistics. In this chapter, they describe how they used technology in classes of fifty to seventy students to improve the implementation of three key TBL activities: readiness assurance tests, reporting complex team assignments, and providing feedback on peer evaluations.

A survey of business faculty showed that they preferred the traditional paper method of teacher evaluation to an online version, mainly because they believed it would produce a higher and more accurate response than the online method. Found no characteristics that correlated to attitudes. (EV)

Objective: To assess differences between traditional paper bag ordering and online ordering from primary school canteens in terms of menu, usage and lunch order characteristics.Design: A cross-sectional study.Setting: New South Wales (NSW) primary schools that offered both paper bag and online canteen ordering.Participants: Students (aged 5-12 years) with a lunch order on the day of the observation.Results: Across the six school canteens, 59-90 % of all available items were listed on both the online and paper menus, with no significant differences in the nutritional quality ('Everyday'/'Occasional') or nutritional content (kJ/saturated fat/sugar/sodium) of menu items. In total, 387 student lunch orders were placed, containing 776 menu items. Most orders (68 %) were placed online. There were no significant differences between order modality in the quantity of items ordered or the cost of orders, or the nutritional quality of orders based on the classification system of the NSW Healthy School Canteen Strategy ('Everyday'/'Occasional'). However, nutritional analysis revealed that paper bag orders contained 222 fewer kJ than online orders (P = 0·001), 0·65 g less saturated fat (P = 0·04) and 4·7 g less sugar (P < 0·001).Conclusions: Online canteens are commonly used to order canteen lunches for primary school children. This is the first study to investigate differences between traditional paper bag ordering and online ordering in this setting. Given the rapid increase in the use of online ordering systems in schools and other food settings and their potential to deliver public health nutrition interventions, additional research is warranted to further investigate differences in ordering modalities. [ABSTRACT FROM AUTHOR]

This article presents the author's address at the 2007 "Journal of Applied Quantitative Methods" ("JAQM") prize awarding festivity. The festivity was included in the opening of the 4th International Conference on Applied Statistics, November 22, 2008, Bucharest, Romania. In the address, the author reflects on three theses that question the gnoseological and operational efficiency of quantitative methods in the social domain. The first refers to symbolical analysis, where the connections between different qualitative categories are transposed into equation systems, including different types of inequalities or existence conditions. The second refers to empirical research. The third thesis upon which the author reflects refers to the skepticism concerning the predictive capacity of quantitative analysis. Here, the author cites several causal factors contributing to the recent global crisis in the context of these theories. (Contains 1 note.)

There is general consensus that teaching and learning centres have a positive impact on promoting and supporting a culture that values teaching and learning, yet there is limited evidence on how centres evaluate their work. For this purpose, a survey was developed and administered to the directors of 88 Canadian centres; 46 of which completed the survey. In this paper, we share centres' approaches to evaluation that emerged from the survey findings and the artifact analysis. By raising awareness of evaluation approaches, we are better positioned to identify further opportunities to develop a more systematic and coherent approach to evaluation.

The medical field is facing a physician-scientist shortage, threatening future medical research and development. Medical institutions can contribute to developing physician-scientists by stimulating students' involvement in research. In this study, a medical undergraduate teaching module to develop research skills and encourage interest in research was held in Chongqing, China. Undergraduate medical students at Chongqing Medical University completed research skills training. Before and after the training, an online, anonymous, self-reported questionnaire was administered. The self-reported questionnaire investigated students' demographic characteristics, students' perception of attitudes toward conducting research, research skills (e.g., identifying and utilizing information, critical appraisal of literature, paper writing, and understanding of research), and feedback on scientific research training. The module was implemented with 25 students from July 2019 to October 2019, and 96.0% (N = 24) of participants responded to the questionnaire both prior to and after the training. In the evaluation of research skills, results showed that use of appropriate tools for research (Z = -3.340, p < 0.01), students' ability to undertake a focused literature search (Z = -3.40, p < 0.01), identifying and utilizing information (Z = -3.34, p < 0.01), and paper-writing skills (Z = -3.49, p < 0.01) were significantly improved after the undergraduates participated in the scientific research training. A qualitative analysis of the feedback showed that students found that the training helped to enhance their knowledge, improve their study scores, and motivate them to conduct research in the future. Early scientific research ability training strengthened the research skills of medical undergraduates and motivated them to pursue research.

Transparency in reporting is essential to scientific progress. No report should be considered complete without a full account of uncertainties, including those due to natural variation and measurement and model error and those incurred by handling problematic data, such as outliers. We randomly selected 132 papers published in 2019 from a list of 100 scientific journals to characterize the current state of uncertainty reporting in ecosystem studies. Each paper was evaluated for the extent to which it reported measures of uncertainty in any of four topic areas common to ecosystem studies: vegetation, soils, precipitation, and surface water. We found that most papers reported a minority of the uncertainty sources we deemed relevant. Papers on surface water reported the highest fraction of uncertainty sources (averaging 47% ± 5%), followed by soils (45% ± 4%), vegetation (32% ± 4%), and precipitation (21% ± 8%). A greater fraction of relevant uncertainty sources were reported when the topics were the primary focus of the paper (44% ± 3%) than when they were not (32% ± 4%). Sampling error—the uncertainty in replicate measurements—was the source most commonly reported in studies of vegetation (84%), soil mass (56%), and surface water (76% of papers). The source of measurement error most often reported was chemical analysis, with 41% of papers on surface water and 75% of papers on precipitation reporting this source, if applicable. In contrast, only 1 of 12 papers reporting chemistry of vegetation provided information on analytical uncertainty. Fewer papers reported what methods were used for handling missing or unusable data and observations below detection limits, but it was difficult to judge whether these sources were relevant if they were not mentioned. Finally, we found that a minority of the papers made all (21%) or some (an additional 21%) of their data available in online repositories, after correcting for a failure rate of 13% of the links. Clearly, there is room for improving the completeness and transparency of scientific reporting in ecosystem studies. [ABSTRACT FROM AUTHOR]

While several test concordance tables have been published, the research underpinning such tables has rarely been examined in detail. This study aimed to survey the publically available studies or documentation underpinning the test concordance tables of the providers of four major international language tests, all accepted by the Australian Department of Home Affairs for Australian visa purposes. To evaluate the concordance studies, we first identified the good practice principles in concordance research through a review of both the relevant literature and leading professional standards in the field of educational measurement and language assessment. Next, we reviewed the concordance studies against the identified good practice principles. Our findings revealed that the information supplied by test providers varied, with some making the full research papers available, whereas others providing little information about their underpinning research. None of the concordance studies fulfilled all the good practice principles. Based on the findings of this study, we offer recommendations for future concordance research in the field of language testing as well as suggestions for practice.

The General Certificate of Education (GCE) A* grade was first awarded to students in 2010. It was introduced to assist higher education institutions in differentiating between the highest performing students and to promote and reward greater stretch and challenge. This paper, based on a synthesis of key policy documents, an analysis of quantitative data and a review of existing research, provides an analysis of the GCE A* grade. The paper begins by introducing the historical and political context that led to the introduction of the A* grade, drawing together policy documents and research carried out by the awarding bodies. The paper then presents an analysis of 2011 A-level data, considering the characteristics of students who achieve the grade and addressing concerns around equality and fairness. A further line of evidence concerns public confidence in the new grade, especially in relation to whether it is fulfilling its purposes. The paper concludes by identifying a number of policy lessons for the future of the GCE A* grade. Forthcoming reforms to the A-level will necessitate a reappraisal of the way A* is calculated: this paper will be a useful source of evidence with which to consider the issues.

A symposium was organized at the BELMAS/SCRELM Research Conference in July 2004 to explore the issues arising from the experiences of those contracted by the National College for School Leadership to evaluate its programmes. This article presents four contributions that drew on a wide range of evaluation experience and a response from a representative of the NCSL to the issues being raised. These are: (1) "Control, partnership and empowerment" by Tony Bush; (2) "Matching methods to purpose" by Tim Simkins; (3) "Making evaluation count" by Nigel Wright and Derek Colquhoun; and (4) "Assessing impact--why evaluation is hard" by Peter Earley. In a response from the NCSL, Colin Conner identified three major themes from the papers. The first theme relates to the kind of knowledge that NCSL commissioned evaluations generate. The second theme relates to the issue of ownership and control of evaluations. A third theme draws together a number of issues, in particular the finance and time scales available for evaluations and the extent to which it is possible to explore issues in depth. A summary of these major themes is presented.

This paper provides an introduction to this symposium issue on participatory action research in geographic teaching, learning and research. It introduces the themes of the symposium and contributions from the participating authors, and also offers additional discussion of the attendant benefits and challenges of using participatory action research within a pedagogic context. The authors begin by discussing the recent growth of participatory pedagogies in higher education, including service learning, research-based teaching and community-based research. They contend that participatory action research may be incorporated in all of these learner-centred and community-engaged pedagogies with beneficial outcomes for teaching, student learning and academic research; however, they advocate for its incorporation into research-oriented and research-based teaching strategies where possible. They also discuss some of the unique political, ethical and logistical challenges involved, and argue that an engagement with participatory action research in our teaching requires more than attention to methods, but, rather, a rethinking of theory-practice, teacher-student, and university-community relationships. They conclude with a brief discussion of the symposium and the authors' contributions. (Contains 1 table and 3 notes.)

Peer review is an umbrella term that refers to a class of selection and oversight practices, including the familiar mechanisms of the review of proposals submitted for funding, of manuscripts for scholarly publications, and of personnel qualifications and portfolios for selection and promotion. Peer review has long been a cornerstone of modern scientific method premised on the assumption that those within a discipline are best suited to assess the work of others within that field. As such, it is also frequently employed to evaluate proposals submitted for professional meetings such as the annual conference of the American Evaluation Association (AEA). This paper presents a blind peer review method developed by AEA's Graduate Student & New Evaluators (GS&NE) Topical Interest Group (TIG) in an effort to construct an impartial and reliable process in proposal selection. Implications for conference review processes, AEA, and the field of evaluation in general are discussed. (Contains 2 tables and 1 endnote.)

Background/Context: In the last several decades, a college education, in particular the bachelor's degree, has become the key to higher earnings, and overall, to a middle-class lifestyle in the United States. In an increasingly globalized economy, privileging information and communication technologies, it is more than likely that this emphasis on higher education in the American labor market will continue in the future. While we know much about the increasing link between a college education and social mobility, writ large, we know considerably less about who actually makes the transition to college, how this occurs, and why. Despite the strides made, large numbers of individuals are not making the transition to college. Among those who are, there are important differences in the kinds of postsecondary institutions they are attending, and in completion rates, with attendant implications for social mobility. Purpose/Objective/Research Question/Focus of Study: This article has two purposes: (1) to synthesize the key facets of our knowledge base of how to expand the college pipeline and relatedly, to outline particular areas that have been under-examined across the disciplines; (2) to provide future directions that will allow us to better address this important inquiry along substantive and methodological lines. Research Design: This article draws on disciplinary papers commissioned by the Social Science Research Council's Transitions to College Committee. The discussion of preparation is framed around the contributions from the fields of anthropology, and political science; access vis-a-vis history, sociology, and demography; paying for college vis-a-vis economics; and completion vis-a-vis the field of higher education. Conclusions/Recommendations: The article calls for an overall integrative model, e.g., K-16 and interdisciplinary, improved coverage in large-scale datasets, and a more refined mixed methods approach to attend to notable gaps in our understandings of the transition to college. Across disciplines, a key substantive gap is variation along the lines of race, ethnicity, immigrant status, gender, and age across domains, which needs to be illuminated.

Impact assessment research has developed theory-based approaches to trace the societal impact of scientific research. Impact assessment typically starts from the perspective of a research investment, organization, or project. Research users, non-academic actors involved in knowledge production, translation, and application, are well represented in many of these approaches. Researcher users are usually positioned as contributors to research, recipients of research outputs, or beneficiaries of research-driven outcomes. This paper argues that impact assessment would benefit from a more comprehensive understanding and analysis of research valorization processes from the user perspective. The first half of the paper reviews key impact assessment literature to identify how research users are positioned and portrayed in relation to valorization processes. In the second half of the paper, we use the results of this review to propose a set of principles to guide a systematic approach to constructing user perspectives on research impact. We suggest four concepts for operationalization of this approach. The paper concludes that the addition of a more comprehensive research user perspective on research valorization would complement and enhance existing impact assessment approaches.

Background: Research on evaluation use focuses on putting evaluation recommendations into practice. Prior theoretical research proposes varied frameworks for understanding the use (or lack) of program evaluation results. Purpose: Our purpose is to create and test a single, integrated framework for understanding evaluation use. This article relies on prior theoretical research regarding categories of utilization, typologies of recommendations, and factors affecting utilization to frame an empirical study of evaluation use that then tests the integrated theory. Setting: The empirical part of the article draws on post-evaluation interviews with sixteen agencies that have engaged in evaluation research. Subjects: The agencies are mostly local non-profits, but the sample also includes a state agency, a city agency, and two university-community partnerships. All agencies had undergone a program evaluation between 2003 and 2006. Intervention: Having participated in an evaluation is the main "intervention" of interest in this article, in which we consider the relationship between evaluation use theory and empirical evidence on the topic. Research Design: A qualitative approach, our research design involved examining each of the sixteen agencies within two years of their having been evaluated. Data Collection and Analysis: Data collection included structured in-person interviews with at least one key informant in each agency. In addition, a short, closed-ended survey was administered to research participants. Interview data were analyzed using content analysis of themes and grouping agencies according to their evaluation's outcomes (favorable or not) and experiences. Survey data were analyzed with simple descriptive statistics and similarly involved a subgroup analysis, according to agencies' reported use of evaluation. Findings: Most evaluation use is conceptual, and few agencies studied actually implemented specific evaluation recommendations. Agencies perceived recommendations as changes to rules and structure, which theory and prior research suggest suppresses use. An important human factor that influenced evaluation use was minimal post-evaluation interaction with evaluators. Some long-term influence of the evaluation is evident, but only as conceptual and not as instrumental. In fact, very little instrumental use existed in this empirical assessment. Conclusions: Evidence suggests that evaluation use in practice aligns with theory, specifically emphasizing the conceptual use dimension. The proposed integrated model of prior evaluation use theories may hold value for future theoretical and empirical work. (Contains 1 figure, 3 tables and 2 footnotes.)

The article is concerned with the research approaches and nature of methodologies and epistemologies used by practitioner-researchers in a range of occupational areas. The authors use the experiences of candidates on a generic Doctorate in Professional Studies who undertake a module on research and development for professional practice. They also bring their experience as tutors and programme leaders and the results from focus groups with their peers in other universities to this current and topical area of the higher education curriculum. Part of the module asks candidates to write an assignment in which they investigate the methodologies and epistemologies used as a practitioner-researcher. A close analysis of ten of these assignments reveals that the methodologies selected for practice-based projects may be as a result of the candidates' personal, professional and disciplinary backgrounds as well as a time imperative and desire for brevity that arises from an immediacy in the situation. The authors also conclude that a range of factors were important to the practitioner-researchers and that these have important epistemological implications. The social-science-oriented approaches introduced on the programme offer tried and tested ways to research, but more practice-based approaches to practitioner-research could be developed further. (Contains 2 figures.)

Background/Context: Prior research on the challenges of college going and retention among adolescents today, particularly low-income, minority, and first-generation college-going youth, provide the context for this article. Purpose/Objective/Research Question/Focus of Study: This article sets the stage for the special issue articles by framing the issue's purpose and content. The purpose is to synthesize existing research and knowledge about the adolescent college transition from multiple disciplines and fields, in order to develop a more robust foundation for further research and policy development. The article also describes the knowledge development process used by the Social Science Research Council and its advisory committee to support field and discipline based literature reviews and to synthesize their implications over an 18 month period. Findings/Results: The article provides a summary of the articles presented in the special issue and a summary of the committee's recommendations, as framed in a separate publication, Questions That Matter. It explains that the six discipline and field-based essays (history, demography, anthropology, sociology, economics, and higher education) presented in the special issue reflect a discussion of the organization of research in each field and characterizes the contributions of each field to our understanding of transitions. Together these are a rich collection of essays that map the state of the respective fields by identifying key topics and the research questions posed. Conclusions/Recommendations: The authors conclude by drawing attention to two primary points: key elements of the challenge to improvements in research and persisting conceptual issues that challenge research, policy and practice.

This paper examines the quality evaluations of the Greek Universities highlighting those which offer a full-fledged study program of primary education. There are eight principles-criteria according to which each university is evaluated. For each principle, scores may range from 1 (noncompliance with the principle) to 4 (fully compliance). I present and compare results of the Greek university evaluations completed by the Hellenic Quality Assurance and Accreditation Agency (HQA). Surprisingly, no university has been graded as non-compliant in any of the eight principles-criteria. These evaluations are performed by an alleged "external" and "independent" committee. For all practical purposes, they are, nevertheless, based on subjective and biased opinions of academics affiliated with international universities with links to Greece and its universities. Independent and evidence-based evaluations paint a different picture. Universities which get perfect scores in the quality evaluation processes perform badly in the pertinent international ranking systems. As a case study, their primary education departments have lower research performance with high variability between faculty members and departments. Given that one criterion of evaluating quality is research, then not all Greek universities can be evaluated as highly performing research institutions, either in absolute or relevant terms. This criterion of quality is not satisfied by the primary education departments of Greek universities. According to this evidence, using the HQA as an agency to assess and accredit quality is useless. It should be abolished. A new system should be based on objective criteria such as independent teaching evaluations and research performance. These do not require any committee to evaluate performance and can be constructed on evidence-based policy. The latter relies solely on rigorously established objective facts.

The quality of a research study application sends a distinct signal to the institutional review board (IRB) about the skills, capacities, preparation, communication, experience, and resources of its authors. However, efforts to research and define IRB application quality have been insufficient. Inattention to the quality of an IRB application is consequential because the application precedes IRB review, and perceptions of quality between the two may be interrelated and interdependent. Without a clear understanding of quality, IRBs do not know how to define quality and researchers do not know how to achieve quality. This position has not been systematically studied to date, and future research could provide much-needed empirical validation. This paper lays the conceptual groundwork for future investigation into what constitutes quality in an IRB application. It includes a landscape review of multidisciplinary research on quality, as well as a discussion of quality frameworks analogous to research with human participants that exist in the published literature. It also examines the background and significance of federal research regulations, regulatory burdens, researchers' regulatory literacy, and the roles and responsibilities of IRB professionals within this ecosystem.

This paper is concerned with the process of selecting between the increasingly popular bi-factor model and the second-order factor model in measurement research. It is indicated that in certain settings widely used in empirical studies, the second-order model is nested in the bi-factor model and obtained from the latter after imposing appropriate parameter constraints. These restrictions can be directly tested within the framework of the latent variable modeling methodology employing widely circulated software. The outlined model selection procedure provides a readily applied means of choosing between the two models of growing interest to measurement scholars, and is illustrated using numerical data.

Although encompassing a variety of research approaches, qualitative research in music education shares the assumption that reality is socially constructed; it takes this construction to be based on the specific perspective of the individual human; and it considers epistemology and ontology to be different fields of study. The posthuman theory of agential realism, on the other hand, argues that the world is becoming through intra-actions; it decentres the individual humanist subject; and it studies onto-epistemology -- practices of knowing-in-being. Considering these fundamental differences, the quality criteria of qualitative research are not applicable to posthuman music education research. Nevertheless, posthuman research is concerned with the ethics of research and how to response-ably and ethically take part in the world's becoming. Thus, the emerging field of posthuman music education research must develop other ways of evaluating research beyond the quality criteria found in qualitative methodologies. This paper argues that diffraction -- both as a physical and musical phenomenon and as a philosophical concept -- might be a fruitful approach. By reading quality criteria diffractively through the agential realist concept of response-ability, it poses critical and creative questions, moving us towards evaluating the quality of posthuman music education research on its own terms.

Background: Previous research has found that a "web-push" approach to data collection, which involves contacting people by mail to request an Internet survey response while withholding a paper response option until later in the contact process, consistently achieves lower response rates than a "paper-only" approach, whereby all respondents are contacted and requested to respond by mail.Method: An experiment was designed, as part of the Millennium Cohort Family Study, to compare response rates, sample representativeness, and cost between a web-push and a paper-only approach; each approach comprised 3 stages of mail contacts. The invited sample (n = 4,935) consisted of spouses married to U.S. Service members, who had been serving in the military between 2 and 5 years as of October, 2011.Results: The web-push methodology produced a significantly higher response rate, 32.8% compared to 27.8%. Each of the 3 stages of postal contact significantly contributed to response for both treatments with 87.1% of the web-push responses received over the Internet. The per-respondent cost of the paper-only treatment was almost 40% higher than the web-push treatment group. Analyses revealed no meaningfully significant differences between treatment groups in representation.Conclusion: These results provide evidence that a web-push methodology is more effective and less expensive than a paper-only approach among young military spouses, perhaps due to their heavy reliance on the internet, and we suggest that this approach may be more effective with the general population as they become more uniformly internet savvy. [ABSTRACT FROM AUTHOR]

Purpose: The purpose of this study is to systematically review the practice of reverse mentoring and draw a timeline of the research over the past two decades. Considering the novelty of this intervention, this paper proposed an agenda for future research on this burgeoning topic. Design/methodology/approach: By adopting narrative literature review and Gregory and Denniss' (2018) four-step process, this paper reviewed 54 studies grounded in conceptual, literature review and empirical research published between 1998 and 2020. Findings: The articles included in the literature review on reverse mentoring research were summarized according to journal publications, research methodologies, contextual settings, theoretical framework, purpose and outcomes. Reverse mentoring studies are dominantly published in educational journals using primarily qualitative and conceptual approaches to explore both academic and business contexts within the USA and Europe. Theories frequently used to frame and examine the need of reverse mentoring included social exchange theory and leader-member exchange theory. The fundamental purpose of reverse mentoring research is to transfer knowledge and to bridge the technology divide between intergenerational groups. Reverse mentoring has been used to promote inclusivity between multiple generations in relation to gender, ethnicity and culture. Originality/value: As per the knowledge, this is the first-ever comprehensive English summary of reverse mentoring research done in the past two decades. Findings from this research can be used to better understand reverse mentoring research trends and directions.

We study the role of gender in the evaluation of economic research using submissions to four leading journals. We find that referee gender has no effect on the relative assessment of female- versus male-authored papers, suggesting that any differential biases of male referees are negligible. To determine whether referees as a whole impose different standards for female authors, we compare citations for female- and male-authored papers, holding constant referee evaluations and other characteristics. We find that female-authored papers receive about 25% more citations than observably similar male-authored papers. Editors largely follow the referees, resulting in a 1.7 percentage point lower probability of a revise and resubmit verdict for papers with female authors relative to a citation-maximizing benchmark. In their desk rejection decisions, editors treat female authors more favorably, though they still impose a higher bar than would be implied by citation maximization. We find no differences in the informativeness of female versus male referees or in the weight that editors place on the recommendations of female versus male referees. We also find no differences in editorial delays for female- versus male-authored papers. [ABSTRACT FROM AUTHOR]

The purpose of this research is to make a meta-evaluation of the program evaluation studies on teacher training programs in Turkey. Meta-evaluation is the process of revealing the deficiencies and errors of the research as the last stage of the program evaluation process. In this context, the steps of the meta-evaluation processes were followed methodically. In the in this research, 9 program evaluation studies conducted between 2010 and 2020 about teacher training programs, using any program evaluation approach and model, were meta-evaluated. The sampling method of the research is criterion sampling.. Research data was collected using the "Program Evaluation Standards Checklist" developed by the researchers. During the development of the relevant data collection tool, the "Program Evaluation Standards" created by the Joint Committee on Standards for Educational Evaluation (JCSEE) were benefited. The studies included in the research were examined by 6 program experts who formed the meta-evaluation team. Each expert evaluated 3 studies using the checklist. Research data was analyzed using descriptive analysis method. Research findings indicate that the examined program evaluation studies meet the program evaluation standards by 55.67%. From this point, some suggestions that are believed to contribute to future program evaluation and meta-evaluation studies on teacher training were presented. [This study was an oral presentation in "International Pegem Conference on Education", Diyarbakir, Türkiye, 16-18 September 2020.]

Background: When a journal receives a duplicate publication, the ability to identify the submitted work as previously published, and reject it, is an assay to publication ethics best practices. The aim of this study was to evaluate how three different types of journals, namely open access (OA) journals, subscription-based journals, and presumed predatory journals, responded to receiving a previously published manuscript for review.Methods: We performed a quasi-experimental study in which we submitted a previously published article to a random sample of 602 biomedical journals, roughly 200 journals from each journal type sampled: OA journals, subscription-based journals, and presumed predatory journals. Three hundred and three journals received a Word version in manuscript format, while 299 journals received the formatted publisher's PDF version of the published article. We then recorded responses to the submission received after approximately 1 month. Responses were reviewed, extracted, and coded in duplicate. Our primary outcome was the rate of rejection of the two types of submitted articles (PDF vs Word) within our three journal types.Results: We received correspondence back from 308 (51.1%) journals within our study timeline (32 days); (N = 46 predatory journals, N = 127 OA journals, N = 135 subscription-based journals). Of the journals that responded, 153 received the Word version of the paper, while 155 received the PDF version. Four journals (1.3%) accepted our paper, 291 (94.5%) journals rejected the paper, and 13 (4.2%) requested a revision. A chi-square test looking at journal type, and submission type, was significant (χ2 (4) = 23.50, p < 0.001). All four responses to accept our article came from presumed predatory journals, 3 of which received the Word format and 1 that received the PDF format. Less than half of journals that rejected our submissions did so because they identified ethical issues such as plagiarism with the manuscript (133 (45.7%)).Conclusion: Few journals accepted our submitted paper. However, our findings suggest that all three types of journals may not have adequate safeguards in place to recognize and act on plagiarism or duplicate submissions. [ABSTRACT FROM AUTHOR]

Background: The essence of humanism in medicine and health care is relationships-caring relationships between clinicians and patients. While raising concerns regarding professional-patient boundaries has positively contributed to our understanding and prevention of potentially harmful boundary violations, there is controversy about which types of relationships, caring acts, and practices are acceptable versus cross boundary lines.Objective: To examine primary care physicians' practices and attitudes regarding acts that have been questioned as potentially "inappropriate" or "unethical" crossing of professional-patient boundaries.Design: Surveys conducted via in-person polling or electronic and mailed paper submissions from April 2016 to July 2017. We calculated descriptive statistics and examined associations with practices and attitudes using logistic regression.Participants: Random sample of all US primary care physicians who treat adult patients; convenience sample of attendees at medicine grand rounds presentations.Main Measures: Outcomes were self-reported practices and attitudes related to giving patients rides home, paying for patients' medication, helping patients find jobs, employing patients, going to dinner with patients, and providing care to personal friends.Key Results: Among 1563 total respondents, 34% had given a ride home, 34% had paid for medications, 15% helped patients find a job, 7% had employed a patient, 10% had dinner with patients, and 59% provided care to personal friends. A majority disapproved of dinner with a patient (75%) but approved of or were neutral on all other scenarios (61-90%).Conclusions: The medical profession is quite divided on questions related to drawing lines about appropriate boundaries. Contrary to official and widespread proscriptions against such practices (with exception of dinner dates), many have actually engaged in such practices and the majority found them acceptable. [ABSTRACT FROM AUTHOR]

Although robust evidence has been accumulating over the last decades supporting the effectiveness of career counseling in various populations, yet there are scarce conceptual or empirical studies investigating the impact of distinct approaches, such as traditional-"psychometric," narrative, or a mixed (integrative) one. The primary aim of this paper is to make a contribution in the field, providing a synthetic overview of contemporary research findings regarding the effectiveness of narrative and mixed career counseling approaches with a special focus on university students. The recent trend of utilizing mixed approaches in career assessment and intervention is also emphasized in the article, suggesting they may have a great potential in helping young adults advance meaning-making about career and life design. Finally, a discussion of both benefits and challenges relating to narrative and mixed interventions along with recommendations for extending research in the area of assessing career interventions' effectiveness for university students is provided.

Cysticercosis is an infestation of Cysticercus cellulosae. When it occurs in the brain, chronic neurological complications can ensue, most commonly seizures. Neurocysticercosis is usually diagnosed by neuroimaging, a technique not available in most endemic countries. Hence immunological tests are valuable for diagnosis and epidemiological surveys. We evaluated the suitability of paper for storing blood and cerebrospinal fluid (CSF) until subsequent testing by enzyme-linked immunosorbent assay (ELISA), by testing whole blood samples on filter paper from 305 patients and CSF samples from 117 patients stored on ordinary white typing paper and on filter paper. Optimal preservation of biological samples is achieved when whole blood is stored on filter paper, CSF on white paper, and when samples are frozen within 1 week after collection. Our results could improve diagnostic capabilities and facilitate epidemiological surveys in endemic countries where immunodiagnostic tests cannot be rapidly performed because of inadequate laboratory infrastructure. [ABSTRACT FROM AUTHOR]

Purpose: This study compared a tablet PC questionnaire with a paper method for reliability and patient preferences in the acquisition of patient-reported outcomes (PROs) for patients treated with radiotherapy. By comparing the two modes of PRO administration, we aimed to evaluate the adequacy of using tablet PC questionnaires in future clinical use.Methods: Patients were randomized in a crossover study design using two different methods for PRO entry. A group of 89 patients answered a paper questionnaire followed by the tablet PC version, whereas 89 patients in another group completed the tablet PC questionnaire followed by the paper version. Surveys were performed four times per patient throughout the course of the radiotherapy. The Korean versions of the M.D. Anderson Symptom Inventory (MDASI-K) and the Brief Fatigue Inventory (BFI-K) were used. The primary endpoint of our current study was an assessment of patient preference for the survey method. The proportions of patients preferring each mode of questionnaire were evaluated.Results: The proportion of patients who preferred the tablet PC version, paper form, or who had no preference was 52.2, 22.1, and 25.7 %, respectively. More than half of the patients preferred the tablet PC to the paper version in all four surveys. Age, gender, educational status, prior experience of using a tablet PC, and the order of paper to tablet PC administration did not impact patient preferences. Inter-class correlation coefficients (ICCs) between the modes were 0.92 for MDASI-K and 0.94 for BFI-K and ranged from 0.91 to 0.96 on both instruments during the four surveys.Conclusions: A tablet PC-based PRO is an acceptable and reliable method compared with paper-based data collection for Korean patients receiving radiotherapy. [ABSTRACT FROM AUTHOR]

This paper evaluates how technical assistive devices for manual materials handling were analysed in the literature in a warehousing context. Works that discuss the economic and/or human factors impact of assistive devices on the warehousing system or the people employed therein were identified in a systematic literature review. Building on a conceptual framework proposed in this paper, our evaluation of the literature shows which types of assistive devices were analysed in the past, and from which perspective these devices were examined. Some works studied the devices exclusively from an operator well-being or an efficiency perspective, while several works analysed the devices' performance in terms of both dimensions. Several works contained in our literature sample highlighted trade-offs between both ergonomic and economic measures and, within the first category, between alternative ergonomic indicators, which shows that assistive devices have to be evaluated carefully in light of their intended application. The paper further identifies research gaps and emphasises the need to understand the interactions between human- and system-related variables that can be supported by assistive devices in designing effective manual materials handling systems. [ABSTRACT FROM AUTHOR]

The issue of connecting research evidence with practice referred to as evidence-informed practice (EIP) has received considerable critical attention in the field of education. However, assessment literature has failed to fully acknowledge the significance of this issue. This conceptual paper concentrates on the process of informing classroom assessment practices by policy-makers, researchers, and teachers. It draws on the important literature on EIP in education and its scarce literature in assessment to highlight a possible productive relationship between assessment research and practice. The paper aims to propose a set of principles as to address the tension between AfL research and practice by defining the roles of each party in the process based on cultural and contextual issues. Accordingly, a cyclical process of collaborative research conduction is proposed. This process provides policy-makers, researchers and practitioners with important implications for practicing AfL research evidence.

Background/objectives: The growing interest of medical community about sarcopenia resulted in the production of several clinical practice guidelines (CPGs), with an unavoidable variability in terms of the overall quality of those publications. Our aim is to evaluate the quality of CPGs on sarcopenia using the AGREE II instrument.Subjects/methods: We performed an online literature search for sarcopenia CPGs using different databases. Four independent reviewers evaluated the quality of CPGs using the AGREE II instrument. To classify the quality of each guideline, we defined specific thresholds of final score: high-quality if five or more domains scored >60%; average-quality if three or four domains scored >60%; low-quality if ≤2 domains scored >60%.Results: Our literature search yielded 315 articles, and after applying exclusion criteria our final analysis included 19 CPGs. The overall quality of CPGs was remarkable, as 13/19 (68.4%) were considered of "high-quality" CPGs, with more than four domains reached a score higher than 60%. "Scope and Purpose" and "Clarity of Presentations" had the best domain results (78.4% and 73.8%, respectively), while the two domains with the lowest scores were "Rigor of Development" and "Applicability" (61.5% and 58.7%, respectively). Interobserver variability ranged between moderate (0.624) and fair (0.275).Conclusions: Our study showed that the overall quality of CPGs about sarcopenia was noteworthy, as more than two-third of paper obtained a "high-quality" score. The domain "applicability" had the lowest score, suggesting that emphasis should be put on possible strategies for helping other doctors to implement guideline recommendations in clinical practice. [ABSTRACT FROM AUTHOR]

Purpose: The aim of the present study is to comparatively evaluate the anti-inflammatory and antiapoptotic effects of bone marrow and adipose-derived mesenchymal stem cells (MSCs) applied subconjunctivally after alkaline corneal burn. Methods: Thirty-two rats were divided into 4 groups and included in the study (n = 8). While no intervention was made in the control group, a chemical burn was created by applying 4 μL of NaOH soaked in 6 mm filter paper to the right eye of each subject in the other groups under general anesthesia. While only subconjunctival 0.1 mL phosphate-buffered saline (PBS) was injected to in the group 1, 2 × 106 adipose or bone marrow-derived MSC in 0.1 mL PBS was applied subconjunctivally to the subjects in the remaining groups (Group 2 and 3, respectively). Tissue samples were collected for histological analysis on the third day after the burn. Tissue samples were evaluated light microscopically and immunohistochemically stained for interleukin-1 beta (IL-1β), tumor necrosis factor alpha (TNF-α), caspase-3 (Cas-3), and CD68. Results: The IL-1β and TNF-α staining scores and the number of CD68- and Cas-3-positive stained cells were significantly lower in the groups given bone marrow and adipose-derived MSC compared to the alkaline burn group (P < 0.0001, for all parameters). Epithelial IL-1β and TNF-α staining scores were significantly lower in the bone marrow-derived MSC group compared to the adipose-derived MSC group (P < 0.0001, for all parameters). Conclusions: The presented study shows that both bone-marrow and adipose-derived MSCs support wound healing in the corneal tissue and strongly suppress the inflammation occured in the tissue. [ABSTRACT FROM AUTHOR]

We study gender differences in the evaluation of submissions to economics conferences. Using data on more than 9,000 submissions from the Annual Congress of the European Economic Association (2015–17), the Annual Meeting of the Spanish Economic Association (2012–17) and the Spring Meeting of Young Economists (2018), we find that all‐female‐authored papers are 3.3% points (p.p.), or 6.8%, less likely to be accepted than all‐male‐authored papers. The estimated gap ranges from 5.4 p.p. (95% CI: 2.5 p.p., 8.3 p.p.) to 2.9 p.p. (0 p.p., 5.8 p.p.). This gap is present after controlling for number of authors of the paper; field; referee fixed effects; cites of the paper; authors' previous publication record, affiliations, and experience; and connections between the authors of a given paper and the referees that evaluate it. We provide evidence suggesting that the gap is driven by stereotypes against female authors: it is entirely driven by male referees, only exists for lesser‐known authors, and seems larger in more masculine fields, especially in finance. [ABSTRACT FROM AUTHOR]

Describes the development of a tool, FACT (Formative Automated Computer Testing), to formatively assess information technology skills of college students in the United Kingdom. Topics include word processing competency; tests designed by tutors and delivered via a network; and results of an evaluation that showed students preferred automated testing as compared to traditional written tests. (Author/LRW)

Although there are tools to assess student's readiness in an "online learning context," little is known about the "psychometric" properties of the tools used or not. A systematic review of 5107 published and unpublished papers identified in a literature search on student online readiness assessment tools between 1990 and 2010 was conducted. The objective of this paper was to identify via a systematic review different tools allowing to assess the level of student's preparation in an online learning environment and which were published or not in scientific journals, and determine which of these tools have been validated. The results of the systematic review show that a standard tool does not exist, and that only ten instruments have been developed and published over the past 20 years to assess student's readiness. In addition, few tools published demonstrated good psychometric qualities, and many unpublished tools, considered as homemade tools, were internally developed in the universities by a team of professors without regard to their psychometric quality. Also, it appears that the tools that were published in scientific journals are rarely used by universities that offer online courses. Generally, the universities prefer to develop their own instrument that fits their online programs.

Background: Patients with disabilities often require healthcare accommodations in order to access high-quality, equitable healthcare services. While attention has been paid to accommodation needs in specific disability populations, limited research to date has explored healthcare accommodations that cross-cut diverse disability populations.Objective: To identify a deeper understanding regarding accommodations in healthcare settings that could apply across disability populations and promote equitable healthcare.Design: We conducted qualitative focus groups with patients with disabilities and caregivers to understand their experiences and preferences for healthcare accommodations.Participants: We recruited patients and caregivers across all major disability categories to participate in focus groups. Participants were recruited through advocacy organizations and healthcare settings in Southeastern Minnesota.Approach: A total of eight focus groups were conducted with 56 participants. Participants described their healthcare experiences and desires for healthcare accommodations. The multidisciplinary research team recorded, transcribed verbatim, and coded all focus groups. The team thematically coded transcripts using content analysis within and across focus groups to identify major themes.Key Results: Patients identified four challenges and corresponding steps healthcare team could take to promote equitable care: (1) consistent documentation of disabilities and needed accommodations in the medical record; (2) allowance for accommodations to the environment, including adapting physical space, physical structures, and scheduling and rooming processes; (3) provide accommodations for administrative tasks, such as completing paper or electronic forms; and (4) adapt communication during interactions, such as speaking slower or using terms that patients can easily understand.Conclusion: These identified themes represent specific opportunities for healthcare teams to effectively provide accessible care to patients with disabilities. Many of the accommodations require minimal financial investment, but did require behavioral changes by the healthcare team to ensure equitable healthcare. [ABSTRACT FROM AUTHOR]