12,656 results
Search Results
2. New Visions in Action: National Assessment Summit Papers
- Author
-
National K-12 Foreign Language Resource Center and Rosenbusch, Marcia Harmon
- Abstract
The National Assessment Summit was designed to bring together users and producers of assessments and engage them in a discussion of unmet assessment needs and untapped assessment capacities. The meeting was truly a summit in that national foreign language organizations and associations were asked to nominate representatives to bring their organization's perspective to the forum and also to take back to their organization ideas for action. Specifically, the National Assessment Summit aimed to identify a set of priorities for the next two years (2005-07). At a follow-up National Assessment Summit in 2007 in Portland, Oregon, under the sponsorship of the Center for Applied Second Language Studies (CASLS) at the University of Oregon and the Center for Advanced Research in Language Acquisition (CARLA) at the University of Minnesota, these organizations will gather to report on progress made and to consider an agenda for the following two years (2007-09). The National Assessment Summit identified seven priorities for the coming two years. Each priority is addressed in a section of this series of papers and is authored by one of the participants. These priorities are: (1) Articulation; (2) Assessment Literacy; (3) Suite of Assessments; (4) Test Database; (5) Models and Anchors; (6) Research; and (7) Advocacy. Taken as a whole, these papers describe state-of-the-art language assessment and articulate a two-year agenda for improving assessment and educational practice. This publication includes: (1) Introduction (Carl Falsgraf); (2) Achieving Articulation through Assessment (Jacque Bott Van Houten); (3) Assessment Literacy (Peggy Boyles); (4) Suite of Assessments (Ann Tollefson); (5) Foreign Language Assessment Database (Margaret E. Malone and David W. MacGregor); (6) Models and Anchors (Elvira Swender); (7) Assessments Research (Ursula Lentz); (8) Promoting Language Advocacy Through Assessment (Martha G. Abbott); and (9) Conclusion (Paul Sandrock). Individual papers contain references and footnotes.
- Published
- 2006
3. An Analysis of Self-Reported Graduates. Technical Paper
- Author
-
National Centre for Vocational Education Research and Braysher, Ben
- Abstract
The annual Student Outcomes Survey collects information on the outcomes of two groups of students--those that have completed a qualification (graduates) and those that have completed only part of a course and then left the vocational education and training (VET) system (module completers). At the time of selecting the survey sample, insufficient information is available to identify "actual" module completers. Instead, a sample of potential module completers is drawn that includes students still in training and students who self-report that they have completed the qualification; that is, self-reported graduates. For many years these students have been counted as graduates in survey outputs. This paper examines whether: (1) self-reported graduates were eligible for the qualifications they claimed (in fact, around two-thirds were not eligible); and (2) it is possible to predict a self-reported graduate's eligibility for the claimed qualification using their personal and training characteristics. The report recommends that self-reported graduates should be assigned to the graduate or module completer category using a predictive model. This model incorporates information on the type of training provider, field of education, level of qualification, whether an apprentice or trainee, enrolment type, and method of answering the survey. The current practice of treating self-reported graduates biases the survey results. The predictive model will be used for 2012 Student Outcomes Survey reporting. The methodology will change estimates from previous surveys substantially, so these will be back cast to 2005 using the method described. Appended are: (1) Characteristics of self-reported graduates; (2) Variables fitted and their status in the final logistic model; (3) Regression results; (4) Comparison of self-reported graduates' eligibility for the claimed qualification; and (5) Comparison of survey outputs and modelled estimates. (Contains 16 tables and 1 footnote.)
- Published
- 2012
4. Mapping Adult Literacy Performance. Background Paper
- Author
-
National Centre for Vocational Education Research, Circelli, Michelle, Curtis, David, and Perkins, Kate
- Abstract
Language, literacy and numeracy are necessary for greater workforce participation, productivity and social inclusion. Being able to measure the level of proficiency in these skills, and any changes in the level of skills, is important for getting a sense of how well language, literacy and numeracy programs are working. Two measurement tools used in Australia are the Adult Literacy and Life Skills (ALLS) survey and the Australian Core Skills Framework (ACSF). This paper outlines a National Centre for Vocational Education Research study which will investigate whether it is possible to map the performance levels of the ALLS to the performance levels of the ACSF in order to gain more frequent information than is currently available about the literacy and numeracy progression of adult learners against national goals. Appended are: (1) About the ALLS survey and the ACSF; and (2) Delphi method and descriptions of ALLS items. (Contains 1 table and 9 footnotes.)
- Published
- 2011
5. Empirical Issues in the Design of Group-Randomized Studies to Measure the Effects of Interventions for Children. MDRC Working Papers on Research Methodology
- Author
-
MDRC, Bloom, Howard, Zhu, Pei, and Jacob, Robin
- Abstract
This paper provides practical guidance for researchers who are designing studies that randomize groups to measure the impacts of interventions on children. To do so, the paper: (1) provides new empirical information about the values of parameters that influence the precision of impact estimates (intra-class correlations and R-squares); (2) examines the implications of planning group-randomized studies for three-level hierarchical situations, using empirical information obtained by estimating two-level hierarchical models (which under many conditions appears to not be problematic); and (3) assesses the implications of the uncertainty that exists when the design of group-randomized studies is based on estimates of intra-class correlations. Data for the paper come from two studies: the Chicago Literacy Initiative: Making Better Early Readers study (CLIMBERs) and the School Break-fast Pilot Project (SBPP). The analysis sample from CLIMBERs comprised 430 4-year old children from 47 preschool classrooms in 23 Chicago public schools. The analysis sample from the SBPP study comprised 1,151 third-graders from 233 classrooms in 111 schools in six school districts. Four appendixes are included: (1) Description of Outcome Measures; (2) Definition of the Multiplier M; (3) Complete Set of Results for Three-Level vs. Two-Level Model Comparisons: Nonstandardized Unconditional Variance Components; and (4) Proofs of the Relationship between Three-Level Models and Two-Level Models in Terms of Precision. (Contains 20 footnotes, 1 figure and 14 tables.) [This document is also sponsored by Judith Gueron Fund for methodological Innovation in Social Policy Research at MDRC, and the Jerry Lee foundation.]
- Published
- 2008
6. Designing and Testing Evaluation Tools to Inform Foundation Education Reforms and Build Grantee Capacity. WCER Working Paper No. 2010-1
- Author
-
Wisconsin Center for Education Research, Kimball, Steven M., Lander, Rachel, and Thorn, Christopher A.
- Abstract
Beginning in 2002, The Chicago Community Trust embarked on an ambitious grant- making strategy to improve education outcomes primarily in the city of Chicago and Cook County, Illinois. Known as The Education Initiative, this effort focused on three priority grant areas: literacy, professional development, and alternative models of schools. In June, 2005 (the 3rd year of the initiative), the Trust awarded a contract to researchers at the University of Wisconsin-Madison, Wisconsin Center for Education Research (WCER) to assess the impact of the Education Initiative. During the 3rd year of the evaluation, the program officers and evaluators were especially interested in pilot testing evaluation guidelines that were developed as part of the overall evaluation of the Education Initiative. These guidelines and related evaluation reporting tools were intended to help grantees tailor projects to the foundations priorities, and help the foundation assess the impact of their diverse grant activities. The authors report on the design and pilot testing of the evaluation guidelines and reporting tools developed to support the education program of the Trust. The next section describes the impetus for developing these tools and the evaluation findings that informed their design. In the fourth section, the authors briefly describe the pilot study design and the participants. They also report findings from the pilot study. In the final section, they describe implications for use of the guidelines and tools by the Trust and other foundations. Appendices include: (1) Draft CCT Evaluation Guidelines; (2) Draft Evaluation Plan Template; and (3) Logic Model Example Adapted from BSF Grantee Model. (Contains 1 table, 7 footnotes, and 2 figures.) [This paper was written with Katie Herrem.]
- Published
- 2010
7. Institutional versus Academic Discipline Measures of Student Experience: A Matter of Relative Validity. A Student Experience in the Research University (SERU) Project Research Paper. Research & Occasional Paper Series: CSHE.8.07
- Author
-
University of California, Berkeley, Center for Studies in Higher Education and Chatman, Steve
- Abstract
The University of California's census survey of undergraduates, UCUES [University of California Undergraduate Experience Survey], presents an opportunity to measure both disciplinary and institutional differences in students' academic experience. Results from nearly 60,000 responses (38% response rate) from the 2006 administration found greater variance among majors within an institution than between equivalent majors across institutions. Cluster analysis techniques were employed to establish disciplinary patterns, with traditional distinctions between hard and soft sciences generally supported. Reporting practices called into question range from institutional comparisons that ignore academic program mix and discipline to campus performance comparisons that do not recognize pedagogical differences by academic major. More specifically, these results suggest that calls for comparable institutional performance measures, as proposed by the Spellings Commission, must take into consideration disciplinary differences in instruction. (Contains 1 table and 9 figures.) [A version of this paper was presented at the SERU Project Symposium, "Assessing the Undergraduate Experience in the Postmodern University" (Berkeley, CA, April 25, 2007).]
- Published
- 2007
8. Evaluating and Interpreting Research Syntheses in Adult Learning and Literacy. NCSALL Occasional Paper
- Author
-
National Center for the Study of Adult Learning and Literacy, Boston, MA., Cooper, Harris, Cooper, Harris, and National Center for the Study of Adult Learning and Literacy, Boston, MA.
- Abstract
This occasional paper introduces the methods of research synthesis and meta-analysis to researchers and consumers of research in the field of adult learning and literacy. To begin, the first section of the paper defines key terms and offers a brief history of how the methodologies developed. The second section provides a conceptualization of research synthesis that views it no differently from other research endeavors in the social sciences. Then, the tasks of research synthesis are presented in more detail within the context of a hypothetical example drawn from the literature on adult learning and literacy. (Contains 1 figure and 5 tables.)
- Published
- 2007
9. Civil Rights Activists in the Information Age: The Development of Math Literacy Workers. CIRCLE Working Paper 50
- Author
-
Watts, Roderick J. and Guessous, Omar
- Abstract
Roderick Watts and Omar Guessous of Georgia State University conducted exploratory research to investigate the link between math and civic engagement. Their research is based on an evaluation of the Young People's Project (YPP), a national program that recruits, trains, and deploys high school and college Math Literacy Workers for mentoring middle and elementary school students. The purpose of this study is to examine whether the Young Peoples Project, as a community-based math and social justice organization for youth, contributes to the math, academic, sociopolitical, ethnoracial identity, and positive youth development of participants. Five specific research questions addressed by this study were: (RQ #1) Program effect: Do YPP participants show more favorable outcomes than the two comparison groups on outcome indicators of interest? (RQ #2) Dosage effect: Do veterans show more favorable outcomes than novices on the outcome indicators of interest? (RQ #3) Program x dosage interaction: Is the dosage effect stronger among YPP participants than among the School Controls? In other words, is there an interaction effect between program membership and level of seniority? (RQ #4) Math--SPD link: Is there a positive relationship between math confidence and indicators of sociopolitical development? (RQ #5) Math--SPD x Program: Is this link moderated by group membership, such that YPP participants demonstrate a stronger positive relationship between math confidence and indicators of sociopolitical development than the School Controls (SC)? Participant in this cross-sectional study were Chicago-area residents in grades 8 and above who were administered a paper-and-pencil survey in a group format at the site of each YPP and After School Matters (ASM) program. YPP participants were encouraged to recruit a peer who did not participate in YPP to serve as matched comparisons. Additionally, ASM participants from three Chicago public high schools were recruited for participation in this study, to serve as an additional comparison group (N=73). There were a total of 196 participants. Each variable of interest was assessed using one or more self-report survey instruments that were answered on a numerical Likert scale. These included measures of social analysis, sense of urgency, societal involvement, cultural and ethnoracial identity. academic and math self-concept, and positive youth development. The five research questions were examined using Univariate Analysis of Covariance and Hierarchical Multiple Regression. Among the outcomes was a favorable link between YPP participation and aspirations for mathematics education, and another was evidence of sociopolitical development. The third was an exploration of other youth development outcomes of interest to the program and the researchers. Sense of Agency was positively related to math self confidence and negatively related to math self-doubt. Similarly, commitment to societal involvement activities was negatively related to math self-doubt. And in the only instance were the ethnoracial variables came into play, cultural pride reinforcement was found to positively correlate to math self confidence. The relationship between sociopolitical development and math confidence was consistently stronger among YPP participants compared to School Controls. This suggests that YPP contributed to a link between math knowledge or reasoning and to both sociopolitical agency and commitment; that is, YPP experiences are associated with the idea that "math is power." (Contains 14 tables and 9 figures.) [This Working Paper was produced by CIRCLE (The Center for Information and Research on Civic Learning and Engagement). For CIRCLE Working Paper 49, see ED494074.]
- Published
- 2006
10. Integrating Formative and Summative Assessment: Progress toward a Seamless System? OECD Education Working Papers, No. 58
- Author
-
Organisation for Economic Cooperation and Development and Looney, Janet W.
- Abstract
A long-held ambition for many educators and assessment experts has been to integrate summative and formative assessments so that data from external assessments used for system monitoring may also be used to shape teaching and learning in classrooms. In turn, classroom-based assessments may provide valuable data for decision makers at school and system levels. Currently there are important technical barriers to this kind of seamless integration. Nevertheless there are a number of promising developments in the field. Ongoing research and development aims at improving testing and measurement technologies, as well strengthening classroom-based formative assessment practices. Improved integration of formative and summative assessment will require investments in new testing technologies, teacher training and professional development, and further research and development.
- Published
- 2011
- Full Text
- View/download PDF
11. New Estimates of Design Parameters for Clustered Randomization Studies: Findings from North Carolina and Florida. Working Paper 43
- Author
-
Urban Institute, National Center for Analysis of Longitudinal Data in Education Research (CALDER), Xu, Zeyu, and Nichols, Austin
- Abstract
The gold standard in making causal inference on program effects is a randomized trial. Most randomization designs in education randomize classrooms or schools rather than individual students. Such "clustered randomization" designs have one principal drawback: They tend to have limited statistical power or precision. This study aims to provide empirical information needed to design adequately powered studies that randomize schools using data from Florida and North Carolina. The authors assess how different covariates contribute to improving the statistical power of a randomization design and examine differences between math and reading tests; differences between test types (curriculum-referenced tests versus norm-referenced tests); and differences between elementary school and secondary school, to see if the test subject, test type, or grade level makes a large difference in the crucial design parameters. Finally they assess bias in 2-level models that ignore the clustering of students in classrooms. (Contains 5 figures, 46 tables and 2 footnotes.)
- Published
- 2010
12. Policy Effectiveness of Interim Assessments in Providence Public Schools. WCER Working Paper No. 2008-10
- Author
-
Wisconsin Center for Education Research, Clune, William H., and White, Paula A.
- Abstract
Many urban districts have adopted interim assessments in recent years as a supplement to annual testing. Possible purposes for such tests include monitoring and assisting student progress, aligning the curriculum, and practicing for state exams, but little is known about how effectively real systems advance these purposes. The Providence Public School District (PPSD) implemented a system of quarterly assessments at every grade starting in 2004 and discontinued them in 2007. During the implementation period, authors interviewed district officials about what they were trying to accomplish, and teachers and other school personnel about their experiences with the assessments and their use of the results. Evidence of effectiveness as well as limitations were found. Uncertainty about the magnitude of positive impacts raised questions about the net value of the assessments given their considerable costs to districts, teachers, and students. The authors conclude that interim assessments are costly, and costs increase if the desired effects on student achievement require significant investments in professional development of teachers. Sound policy, therefore, should be carefully designed and evaluated. Two appendixes are included: (1) Interview Protocol; and (2) Findings from Interviews and Survey. (Contains 3 tables.)
- Published
- 2008
13. Should the Curriculum Be Set by State Fiat? An Empirical Test Using Economics Courses in High School. Occasional Paper.
- Author
-
Columbia Univ., New York, NY. National Center for the Study of Privatization in Education. and Belfield, Clive R.
- Abstract
This paper estimates the effects of state-imposed curriculum mandates on the test scores of public-school students who took the SAT in 2001. By 1998, 14 states had mandates that high-school students should take an economics course. For these states, the proportion of public-school students taking high-school economics was around twice that of states without mandates. The mandate may be interpreted as a regulation on input use in the education sector, potentially impairing the efficiency of schools. It is hypothesized that where there is a mandate, test scores should be lower. The merit of the mandate on scores can be assessed empirically. Using microlevel data on a large sample of students across the United States, this inquiry tests whether students' academic performances are higher or lower in states where there are mandates. This test offers a direct evaluation of the efficiency of the mandate. The mandate reduces test scores by as much as 0.25 standard deviations for those students who would not otherwise have enrolled. This paper sets out the theoretical explanation from the imposition of a mandate, describes the dataset, sets out the estimation method, and reports the results. (Contains 6 tables and 24 references.) (Author/WFA)
- Published
- 2002
14. University Grade Inflation through Twenty Years: An Analytical and Empirical Investigation. ASHE Annual Meeting Paper.
- Author
-
Wood, Anda L., Ridley, Dennis R., and Summerville, Richard M.
- Abstract
This paper presents a new method to test for grade inflation. Defining grade inflation as the decreasing value of grades in the coin of student achievement, the study avoids the assumption that a rise in mean grade is a necessary condition of grade inflation. The study, which was conducted at a single university, also sought to determine whether grade inflation could be demonstrated if mean grades have leveled off. To measure general academic abilities, Graduate Record Examination (GRE) data was used. Data was grouped into three time spans (1976-80, 1984-88, and 1992-96). To examine the influence of students' majors, data were also separated into two broad groups, "humanistic" and "scientific," because literature had shown roughly that the former revealed more grade inflation than the latter. Findings suggested grade inflation relative to GRE total and verbal scores, but no evidence of grade inflation when GRE quantitative or GRE analytical scores were used, and it is postulated that this is a result of declining academic standards in the sense that verbal reasoning ability associated with grade point average is lower than it used to be. Differences were also found between disciplines, with some evidence of grade inflation found for science majors. (Contains 17 references.) (RH)
- Published
- 1999
15. Interaction as a Construct of Oral Proficiency. Paper I.
- Author
-
Pollard, John D. E.
- Abstract
This paper describes efforts to devise a more effective method of assessing oral proficiency in English using a computer-based testing and computer-based test result management system. This new test would be designed to directly operationalize the second language constructs it measured, and would be developed with reference to the best practices revealed by recent research in the field. Scores are automatically sent to databases at the completion and evaluation of each task. Student performance is then used to drive an adaptive algorithm, determining the difficulty of successive tasks. This computerized testing would aid in assigning skills or constructs to individual tasks, apportion the skills required for each task, and identify the features of interaction elicited in individual tasks. The study results revealed not only which tasks elicited most assessor/interlocutor interaction, but also that a great deal of the interaction took place outside the task boundaries. There has been a substantial move towards acknowledging the importance of interaction in oral proficiency and, therefore, oral proficiency testing featuring discourse and conversation analysis. The research detailed in this paper indicates that greater interaction does seem to be associated with greater oral proficiency, yet there are nagging issues of test validity still to be addressed in future research. (Contains 47 references.) (KFT)
- Published
- 1998
16. Alternative Designs for Evaluating Workplace Literacy Programs. Conference Proceedings and Commissioned Papers at the 'Design Guidance for Evaluating Workplace Literacy Programs' Work Group Conference (Washington, D.C. April 13, 1993).
- Author
-
Research Triangle Inst., Research Triangle Park, NC.
- Abstract
This document contains the five papers presented at a meeting at which key issues in evaluating workplace literacy programs were discussed. In "Key Components of Workplace Liteacy Projects and Definitions of Project 'Modules,'" Judith A. Alamprese describes the context for evaluating the National Extension Program, components of workplace literacy programs, and validation of program components. Design alternatives for a national program evaluation and design options for an effective practices study are explored by Jorie W. Philippi in "Measuring Workplace Literacy Program Effectiveness: Evaluation Purposes and Methodologies." In "Advantages and Disadvantages of Longitudinal Designs for Evaluating Workplace Literacy Programs," Elisabeth Hayes examines the following: the short-term nature of literacy gains, random assignment versus other methods, availability of comparison/control groups, and practical feasibility of alternative designs and procedures. The last two papers, "Articulation and Measurement of Participant Outcomes in Workplace Literacy Programs" (Larry Mikulecky) and "Articulation and Measurement of Program Outcomes" (Anthony R. Sarmiento), focus on the relationships between curriculum/instructional design and various participant/program outcomes and assess the availability and appropriateness of data and various data collection methods. Most papers include substantial bibliographies. Appended are the work group conference agenda, biographies of the presenters, and a list of work group participants. (MN)
- Published
- 1994
17. Estimating the Home Schooled Population. Working Paper.
- Author
-
Office of Educational Research and Improvement (ED), Washington, DC. and Lines, Patricia M.
- Abstract
Curriculum suppliers, state departments of education, and home school leaders, are the sources used to estimate that between 248,500 and 353,500 school-aged children (K-12) were educated at home in the 1990-91 school year. This paper explains how this estimate was derived. It provides information on process, possible bias, alternate survey collection of data, and why the three methods were chosen. In the estimates reported, most adjustments to actual data were based on a survey of a sample of the membership of the Home School Legal Defense Association. In deriving figures for the first method, a survey of 16 suppliers of curricular packages yielded an estimate of between 309,000 and 353,500 K-12 children who are educated at home. For the second method, 37 state departments of education reported 82,061 children for whom parents have filed papers with state or local officials, representing a fraction of the total. For the third method, information from 25 home schooling leaders in 17 states yielded a national estimate of between 248,500 and 304,000. A brief analysis of recent trends with a conclusion is given. Appended are home school data collected by states, information from associations, and a summary of selected research reports from two state departments of education. (RR)
- Published
- 1991
18. Quality Assurance and Transparency Tools. ENQA Workshop Report 15
- Author
-
European Association for Quality Assurance in Higher Education (ENQA), Costes, Nathalie, Hopbach, Achim, Kekalainen, Helka, van IJperen, Robin, and Walsh, Padraig
- Abstract
The European Association for Quality Assurance in Higher Education (ENQA) realises that there is a steadily growing interest among students and all stakeholders of Higher Education and the public at large in accessing detailed and reliable information on the quality of individual study programmes, faculties and higher education institutions. ENQA is aware that external quality assurance activities in higher education set up by quality assurance agencies are producing only part of the information. Besides quality assurance agencies that enshrine their activities in the ESG, there is a growing number of providers of information about higher education. The proliferation of ranking schemes and other transparency tools is one aspect of this increasing need for information about higher education institutions and their activities. The purpose of this seminar was to discuss expectations from higher education institutions and other stakeholders concerning information on quality of higher education, and the contribution of quality assurance and transparency tools to meet these expectations. The outcome of the discussions is a position paper on information and transparency tools, which can be found in Part 2 of this report. This paper is divided into two parts: (1) Key messages; and (2) ENQA position paper on transparency tools. Programme of the Seminar is appended. (Contains 2 footnotes.)
- Published
- 2010
19. The Relative Effectiveness of Private and Public Schools: Evidence from Two Developing Countries. Living Standards Measurement Study Working Paper No. 60.
- Author
-
World Bank, Washington, DC., Jimenez, Emmanuel, and Cox, Donald
- Abstract
Fiscal constraints have limited education budgets in developing countries. Increases necessary to expand access to highly subsidized and publicly provided education have not been forthcoming. Another concern is that education is being inadequately and inefficiently provided by the public sector. This study is an inquiry into the nature of the achievement differential between public and private schools in Colombia and Tanzania. It questions the extent to which the differential in academic achievement can be attributed to student characteristics rather than school inputs, attempts to estimate the quality differences between the public and private secondary schools in these two countries, as measured by student performance on the achievement tests (ACATESTs). Estimated sample-selection effects suggest that Colombian students sort themselves by type of institution, public or private; but Tanzanian students are selected by a hierarchical mechanism, with the lowest-scoring students entering private institutions. These effects are consistent with the different institutional frameworks for educational choice in these two countries. For each country, however, private schools offer an achievement advantage. By standardizing for differences in student and school attributes, private school students have higher achievement test scores. (Author/NL)
- Published
- 1989
20. Investigating differences between traditional (paper bag) ordering and online ordering from primary school canteens: a cross-sectional study comparing menu, usage and lunch order characteristics.
- Author
-
Leonard, Alecia, Delaney, Tessa, Seward, Kirsty, Zoetemeyer, Rachel, Lamont, Hannah, Sutherland, Rachel, Reilly, Kathryn, Lecathelinais, Christophe, and Wyse, Rebecca
- Subjects
PAPER bags ,PRIMARY schools ,SCHOOL children ,CROSS-sectional method ,LUNCHEONS ,PACKAGED foods ,RESEARCH ,FERRANS & Powers Quality of Life Index ,RESEARCH methodology ,MEDICAL cooperation ,EVALUATION research ,COMPARATIVE studies ,SCHOOLS ,FOOD service - Abstract
Objective: To assess differences between traditional paper bag ordering and online ordering from primary school canteens in terms of menu, usage and lunch order characteristics.Design: A cross-sectional study.Setting: New South Wales (NSW) primary schools that offered both paper bag and online canteen ordering.Participants: Students (aged 5-12 years) with a lunch order on the day of the observation.Results: Across the six school canteens, 59-90 % of all available items were listed on both the online and paper menus, with no significant differences in the nutritional quality ('Everyday'/'Occasional') or nutritional content (kJ/saturated fat/sugar/sodium) of menu items. In total, 387 student lunch orders were placed, containing 776 menu items. Most orders (68 %) were placed online. There were no significant differences between order modality in the quantity of items ordered or the cost of orders, or the nutritional quality of orders based on the classification system of the NSW Healthy School Canteen Strategy ('Everyday'/'Occasional'). However, nutritional analysis revealed that paper bag orders contained 222 fewer kJ than online orders (P = 0·001), 0·65 g less saturated fat (P = 0·04) and 4·7 g less sugar (P < 0·001).Conclusions: Online canteens are commonly used to order canteen lunches for primary school children. This is the first study to investigate differences between traditional paper bag ordering and online ordering in this setting. Given the rapid increase in the use of online ordering systems in schools and other food settings and their potential to deliver public health nutrition interventions, additional research is warranted to further investigate differences in ordering modalities. [ABSTRACT FROM AUTHOR]- Published
- 2021
- Full Text
- View/download PDF
21. What Does the Research Say about Students with Disabilities and Service Learning?
- Author
-
Frankson, Delsue and Nevin, Ann
- Abstract
The purpose of this paper is to summarize the research on service learning pedagogy into the realm of professional practice for students with disabilities. At this point in the development of a research base for service learning, an emerging need for studies is reported. Limited research exists which studies the impact of service learning activities with respect to the retention of students with disabilities, particularly those who are also African American. The authors suggest that one role that university faculty can take is to create, implement, and evaluate service-learning programs in urban communities that incorporate students, adults with disabilities, the local public school k-12, and teacher educators and researchers from nearby colleges and universities. (Contains 2 tables and 2 footnotes.)
- Published
- 2007
22. Some Epistemological Considerations Concerning Quantitative Analysis
- Author
-
Dobrescu, Emilian
- Abstract
This article presents the author's address at the 2007 "Journal of Applied Quantitative Methods" ("JAQM") prize awarding festivity. The festivity was included in the opening of the 4th International Conference on Applied Statistics, November 22, 2008, Bucharest, Romania. In the address, the author reflects on three theses that question the gnoseological and operational efficiency of quantitative methods in the social domain. The first refers to symbolical analysis, where the connections between different qualitative categories are transposed into equation systems, including different types of inequalities or existence conditions. The second refers to empirical research. The third thesis upon which the author reflects refers to the skepticism concerning the predictive capacity of quantitative analysis. Here, the author cites several causal factors contributing to the recent global crisis in the context of these theories. (Contains 1 note.)
- Published
- 2008
23. Digital SAT® Research Summary
- Author
-
College Board
- Abstract
Over the past several years, content experts, psychometricians, and researchers have been hard at work developing, refining, and studying the digital SAT. The work is grounded in foundational best practices and advances in measurement and assessment design, with fairness for students informing all of the work done. This paper shares learnings from key research studies that have informed the design and development of the digital SAT and the current understanding of how well the assessment is working for its intended uses.
- Published
- 2023
24. The current state of uncertainty reporting in ecosystem studies: a systematic evaluation of peer‐reviewed literature.
- Author
-
Yanai, Ruth D., Mann, Thomas A., Hong, Sunghoon D., Pu, Ge, and Zukswert, Jenna M.
- Subjects
LITERARY criticism ,ECOSYSTEMS ,ERRORS-in-variables models ,SOIL sampling ,EVALUATION research ,UNCERTAINTY ,SAMPLING errors - Abstract
Transparency in reporting is essential to scientific progress. No report should be considered complete without a full account of uncertainties, including those due to natural variation and measurement and model error and those incurred by handling problematic data, such as outliers. We randomly selected 132 papers published in 2019 from a list of 100 scientific journals to characterize the current state of uncertainty reporting in ecosystem studies. Each paper was evaluated for the extent to which it reported measures of uncertainty in any of four topic areas common to ecosystem studies: vegetation, soils, precipitation, and surface water. We found that most papers reported a minority of the uncertainty sources we deemed relevant. Papers on surface water reported the highest fraction of uncertainty sources (averaging 47% ± 5%), followed by soils (45% ± 4%), vegetation (32% ± 4%), and precipitation (21% ± 8%). A greater fraction of relevant uncertainty sources were reported when the topics were the primary focus of the paper (44% ± 3%) than when they were not (32% ± 4%). Sampling error—the uncertainty in replicate measurements—was the source most commonly reported in studies of vegetation (84%), soil mass (56%), and surface water (76% of papers). The source of measurement error most often reported was chemical analysis, with 41% of papers on surface water and 75% of papers on precipitation reporting this source, if applicable. In contrast, only 1 of 12 papers reporting chemistry of vegetation provided information on analytical uncertainty. Fewer papers reported what methods were used for handling missing or unusable data and observations below detection limits, but it was difficult to judge whether these sources were relevant if they were not mentioned. Finally, we found that a minority of the papers made all (21%) or some (an additional 21%) of their data available in online repositories, after correcting for a failure rate of 13% of the links. Clearly, there is room for improving the completeness and transparency of scientific reporting in ecosystem studies. [ABSTRACT FROM AUTHOR]
- Published
- 2021
- Full Text
- View/download PDF
25. A Meta-Evaluation Research on Teacher Training Programs in Türkiye
- Author
-
Akinci, Muhamm and Köse, Erdogan
- Abstract
The purpose of this research is to make a meta-evaluation of the program evaluation studies on teacher training programs in Turkey. Meta-evaluation is the process of revealing the deficiencies and errors of the research as the last stage of the program evaluation process. In this context, the steps of the meta-evaluation processes were followed methodically. In the in this research, 9 program evaluation studies conducted between 2010 and 2020 about teacher training programs, using any program evaluation approach and model, were meta-evaluated. The sampling method of the research is criterion sampling.. Research data was collected using the "Program Evaluation Standards Checklist" developed by the researchers. During the development of the relevant data collection tool, the "Program Evaluation Standards" created by the Joint Committee on Standards for Educational Evaluation (JCSEE) were benefited. The studies included in the research were examined by 6 program experts who formed the meta-evaluation team. Each expert evaluated 3 studies using the checklist. Research data was analyzed using descriptive analysis method. Research findings indicate that the examined program evaluation studies meet the program evaluation standards by 55.67%. From this point, some suggestions that are believed to contribute to future program evaluation and meta-evaluation studies on teacher training were presented. [This study was an oral presentation in "International Pegem Conference on Education", Diyarbakir, Türkiye, 16-18 September 2020.]
- Published
- 2022
26. Peer Review of Submissions to the Annual American Evaluation Association Conference by the Graduate Student & New Evaluators Topical Interest Group
- Author
-
Schroter, Daniela C., Coryn, Chris L. S., and Montrosse, Bianca E.
- Abstract
Peer review is an umbrella term that refers to a class of selection and oversight practices, including the familiar mechanisms of the review of proposals submitted for funding, of manuscripts for scholarly publications, and of personnel qualifications and portfolios for selection and promotion. Peer review has long been a cornerstone of modern scientific method premised on the assumption that those within a discipline are best suited to assess the work of others within that field. As such, it is also frequently employed to evaluate proposals submitted for professional meetings such as the annual conference of the American Evaluation Association (AEA). This paper presents a blind peer review method developed by AEA's Graduate Student & New Evaluators (GS&NE) Topical Interest Group (TIG) in an effort to construct an impartial and reliable process in proposal selection. Implications for conference review processes, AEA, and the field of evaluation in general are discussed. (Contains 2 tables and 1 endnote.)
- Published
- 2008
27. The Evaluation and the Accreditation Process of Greek HEIs with an Emphasis on Primary Education Departments
- Author
-
Papanikos, Gregory T.
- Abstract
This paper examines the quality evaluations of the Greek Universities highlighting those which offer a full-fledged study program of primary education. There are eight principles-criteria according to which each university is evaluated. For each principle, scores may range from 1 (noncompliance with the principle) to 4 (fully compliance). I present and compare results of the Greek university evaluations completed by the Hellenic Quality Assurance and Accreditation Agency (HQA). Surprisingly, no university has been graded as non-compliant in any of the eight principles-criteria. These evaluations are performed by an alleged "external" and "independent" committee. For all practical purposes, they are, nevertheless, based on subjective and biased opinions of academics affiliated with international universities with links to Greece and its universities. Independent and evidence-based evaluations paint a different picture. Universities which get perfect scores in the quality evaluation processes perform badly in the pertinent international ranking systems. As a case study, their primary education departments have lower research performance with high variability between faculty members and departments. Given that one criterion of evaluating quality is research, then not all Greek universities can be evaluated as highly performing research institutions, either in absolute or relevant terms. This criterion of quality is not satisfied by the primary education departments of Greek universities. According to this evidence, using the HQA as an agency to assess and accredit quality is useless. It should be abolished. A new system should be based on objective criteria such as independent teaching evaluations and research performance. These do not require any committee to evaluate performance and can be constructed on evidence-based policy. The latter relies solely on rigorously established objective facts.
- Published
- 2023
28. Defining Institutional Review Board Application Quality: Critical Research Gaps and Future Opportunities
- Author
-
Kimberley Serpico
- Abstract
The quality of a research study application sends a distinct signal to the institutional review board (IRB) about the skills, capacities, preparation, communication, experience, and resources of its authors. However, efforts to research and define IRB application quality have been insufficient. Inattention to the quality of an IRB application is consequential because the application precedes IRB review, and perceptions of quality between the two may be interrelated and interdependent. Without a clear understanding of quality, IRBs do not know how to define quality and researchers do not know how to achieve quality. This position has not been systematically studied to date, and future research could provide much-needed empirical validation. This paper lays the conceptual groundwork for future investigation into what constitutes quality in an IRB application. It includes a landscape review of multidisciplinary research on quality, as well as a discussion of quality frameworks analogous to research with human participants that exist in the published literature. It also examines the background and significance of federal research regulations, regulatory burdens, researchers' regulatory literacy, and the roles and responsibilities of IRB professionals within this ecosystem.
- Published
- 2024
- Full Text
- View/download PDF
29. An Evaluation Use Framework and Empirical Assessment
- Author
-
Peck, Laura R. and Gorzalski, Lindsey M.
- Abstract
Background: Research on evaluation use focuses on putting evaluation recommendations into practice. Prior theoretical research proposes varied frameworks for understanding the use (or lack) of program evaluation results. Purpose: Our purpose is to create and test a single, integrated framework for understanding evaluation use. This article relies on prior theoretical research regarding categories of utilization, typologies of recommendations, and factors affecting utilization to frame an empirical study of evaluation use that then tests the integrated theory. Setting: The empirical part of the article draws on post-evaluation interviews with sixteen agencies that have engaged in evaluation research. Subjects: The agencies are mostly local non-profits, but the sample also includes a state agency, a city agency, and two university-community partnerships. All agencies had undergone a program evaluation between 2003 and 2006. Intervention: Having participated in an evaluation is the main "intervention" of interest in this article, in which we consider the relationship between evaluation use theory and empirical evidence on the topic. Research Design: A qualitative approach, our research design involved examining each of the sixteen agencies within two years of their having been evaluated. Data Collection and Analysis: Data collection included structured in-person interviews with at least one key informant in each agency. In addition, a short, closed-ended survey was administered to research participants. Interview data were analyzed using content analysis of themes and grouping agencies according to their evaluation's outcomes (favorable or not) and experiences. Survey data were analyzed with simple descriptive statistics and similarly involved a subgroup analysis, according to agencies' reported use of evaluation. Findings: Most evaluation use is conceptual, and few agencies studied actually implemented specific evaluation recommendations. Agencies perceived recommendations as changes to rules and structure, which theory and prior research suggest suppresses use. An important human factor that influenced evaluation use was minimal post-evaluation interaction with evaluators. Some long-term influence of the evaluation is evident, but only as conceptual and not as instrumental. In fact, very little instrumental use existed in this empirical assessment. Conclusions: Evidence suggests that evaluation use in practice aligns with theory, specifically emphasizing the conceptual use dimension. The proposed integrated model of prior evaluation use theories may hold value for future theoretical and empirical work. (Contains 1 figure, 3 tables and 2 footnotes.)
- Published
- 2009
30. An experimental comparison of web-push vs. paper-only survey procedures for conducting an in-depth health survey of military spouses.
- Author
-
McMaster, Hope Seib, LeardMann, Cynthia A., Speigle, Steven, Dillman, Don A., and Millennium Cohort Family Study Team
- Subjects
MILITARY spouses ,INTERNET surveys ,RESEARCH methodology ,EPIDEMIOLOGY ,ACQUISITION of data ,COMPARATIVE studies ,INDUSTRIES ,INTERNET ,LONGITUDINAL method ,MEDICAL cooperation ,POSTAL service ,RESEARCH ,MILITARY personnel ,SPOUSES ,SURVEYS ,EVALUATION research - Abstract
Background: Previous research has found that a "web-push" approach to data collection, which involves contacting people by mail to request an Internet survey response while withholding a paper response option until later in the contact process, consistently achieves lower response rates than a "paper-only" approach, whereby all respondents are contacted and requested to respond by mail.Method: An experiment was designed, as part of the Millennium Cohort Family Study, to compare response rates, sample representativeness, and cost between a web-push and a paper-only approach; each approach comprised 3 stages of mail contacts. The invited sample (n = 4,935) consisted of spouses married to U.S. Service members, who had been serving in the military between 2 and 5 years as of October, 2011.Results: The web-push methodology produced a significantly higher response rate, 32.8% compared to 27.8%. Each of the 3 stages of postal contact significantly contributed to response for both treatments with 87.1% of the web-push responses received over the Internet. The per-respondent cost of the paper-only treatment was almost 40% higher than the web-push treatment group. Analyses revealed no meaningfully significant differences between treatment groups in representation.Conclusion: These results provide evidence that a web-push methodology is more effective and less expensive than a paper-only approach among young military spouses, perhaps due to their heavy reliance on the internet, and we suggest that this approach may be more effective with the general population as they become more uniformly internet savvy. [ABSTRACT FROM AUTHOR]- Published
- 2017
- Full Text
- View/download PDF
31. Using Focus Groups To Explore Student Opinion.
- Author
-
Franklin, Kathy Kramer and Knight, W. Hal
- Abstract
This paper discusses the use of focus groups to investigate criteria used by college students to determine their satisfaction with higher education, and presents the results of a pilot study. It reviews research on the use of focus groups, discusses the development of a focus group guide for the current study, and examines problems associated with recruiting students to participate in focus groups, such as incentives for volunteers and no-shows at group sessions. The paper discusses the role of the rapporteur and the scribe in group sessions, as well as the actual conduct of the sessions. The pilot study found that the overall undergraduate student criterion for determining satisfaction or dissatisfaction with their college experience at a regional university included: (1) finding pride and inner satisfaction with accomplishments; (2) a flexible curriculum; (3) a university with status and prestige; (4) a university degree that opens the door to career opportunities; (5) the encouragement of student involvement; (6) a caring faculty; (7) an opportunity for independence; (8) a student-oriented university administration; and (9) a university experience that provides an opportunity for growth and development. An appendix contains a copy of the focus group guide. (Contains 21 references.) (MDM)
- Published
- 1995
32. Validity Research on Teacher Evaluation Systems Based on the Framework for Teaching
- Author
-
Milanowski, Anthony T.
- Abstract
After decades of disinterest, evaluation of the performance of elementary and secondary teachers in the United States has become an important educational policy issue. As U.S. states and districts have tried to upgrade their evaluation processes, one of the models that has been increasingly used is the Framework for Teaching. This paper summarizes validity evidence pertaining to several different implementations of the Framework. It is based primarily on reviewing the published and unpublished studies that have looked at the relationship between teacher evaluation ratings made using systems based on the Framework and value-added measures of teacher effectiveness. The research results summarized suggest that teacher ratings made using evaluation systems based on or related to the Framework have a correlation with value-added estimates of teacher effectiveness, in the 0.2 to 0.3 range. Measurement error in both the value-added estimates and evaluation ratings likely bias these correlations downward. Though fragmentary, the evidence reviewed suggests that Framework-based evaluation ratings can show substantial inter-rater agreement and, if multiple observers and multiple occasions of observation are used, the ratings can be reliable. However, a score based on a single observation by one observer is likely to have low reliability. The paper also discusses ways to extend and improve validity research on teacher evaluation systems. "Inter-rater Agreement Details for Cincinnati" is appended to this document. The contributions to this research of current and former colleagues Steven Kimball, Herbert Heneman, Allan Odden, H. Alix Gallagher and Bradford White, are gratefully acknowledged. (Contains 6 tables, 2 figures, and 6 footnotes.)
- Published
- 2011
33. Engagement, Inclusion, Knowledge Sharing, and Talent Development: Is Reverse Mentoring a Panacea to All? Findings from Literature Review
- Author
-
Chaudhuri, Sanghamitra, Park, Sunyoung, and Johnson, Karen R.
- Abstract
Purpose: The purpose of this study is to systematically review the practice of reverse mentoring and draw a timeline of the research over the past two decades. Considering the novelty of this intervention, this paper proposed an agenda for future research on this burgeoning topic. Design/methodology/approach: By adopting narrative literature review and Gregory and Denniss' (2018) four-step process, this paper reviewed 54 studies grounded in conceptual, literature review and empirical research published between 1998 and 2020. Findings: The articles included in the literature review on reverse mentoring research were summarized according to journal publications, research methodologies, contextual settings, theoretical framework, purpose and outcomes. Reverse mentoring studies are dominantly published in educational journals using primarily qualitative and conceptual approaches to explore both academic and business contexts within the USA and Europe. Theories frequently used to frame and examine the need of reverse mentoring included social exchange theory and leader-member exchange theory. The fundamental purpose of reverse mentoring research is to transfer knowledge and to bridge the technology divide between intergenerational groups. Reverse mentoring has been used to promote inclusivity between multiple generations in relation to gender, ethnicity and culture. Originality/value: As per the knowledge, this is the first-ever comprehensive English summary of reverse mentoring research done in the past two decades. Findings from this research can be used to better understand reverse mentoring research trends and directions.
- Published
- 2022
- Full Text
- View/download PDF
34. Identification of Student- and Teacher-Level Variables in Modeling Variation of Mathematics Achievement Data
- Author
-
Tarr, James E., Ross, Daniel J., McNaught, Melissa D., Chavez, Oscar, Grouws, Douglas A., Reys, Robert E., Sears, Ruthmae, and Taylan, R. Didem
- Abstract
The Comparing Options in Secondary Mathematics: Investigating Curriculum (COSMIC) project is a longitudinal study of student learning from two types of mathematics curricula: integrated and subject-specific. Previous large-scale research studies such as the National Assessment of Educational Progress (NAEP) indicate that numerous variables are associated with mathematics performance; however, consideration of too many variables can result in biased models that inflate the amount of variation explained. This paper addresses key issues in the design of studies of curricular effectiveness including data collection, data reduction, and coherence in modeling student achievement in Year 1. At the student-level, demographic and prior achievement data was collected for 2621 secondary school students in five states. These scores were transformed onto NAEP scales to yield a COSMIC Prior Achievement (CPA) score that provided a control for prior achievement across the entire sample. At the teacher-level, data related to teaching experience, professional development, teacher beliefs, and curriculum implementation was collected. Student learning was measured using the Iowa Test of Educational Development: Mathematics (Level 15) and two project-developed assessments (Test A and Test B) that embodied curricular validity (National Research Council, 2004). Item Response Theory (IRT) scaled student scores were adjusted for CPA and subsequently aggregated by teacher. Seven factors were extracted from 24 teacher-level variables through principle components analysis, four related to curriculum implementation and three to teacher characteristics. Curriculum Type, Percent Free/Reduced Lunch (%FRL), and Lesson Development variables were examined in relation to student outcomes. Full and partial correlations procedures were used to identify key variables associated with student mathematics learning. Significant correlations included Curriculum Type with student scores on Test A and Test B; %FRL correlated (negatively) with scores on Test A and ITED-15; Technology & Collaboration with scores on Test B and ITED-15; Opportunity-to-Learn (OTL) with performance on all three outcome measures; and Knowledge of Standards with scores on Test A and Test B. When controlling for %FRL, Curriculum Type was significantly correlated with all three dependent measures. A similar pattern emerged when controlling for OTL, suggesting a relationship between OTL and %FRL. In contrast, controlling for Technology & Collaboration weakened the correlations between Curriculum Type and all three assessments. Recommendations related to key issues in data collection, data reduction, and coherence that are prerequisite to the productive modeling of student achievement data in comparative curriculum studies are provided. (Contains 13 tables, 5 figures, and 4 footnotes.) [This paper is based on research conducted as part of the Comparing Options in Secondary Mathematics: Investigating Curriculum (COSMIC) project, a research study supported by the National Science Foundation.]
- Published
- 2010
35. On Evaluating the Impact of an Innovative Educational Project.
- Author
-
Huberty, Carl J. and Klein, Gerald A.
- Abstract
The thesis of this paper is that, in evaluating most innovative educational projects that are conducted in schools, tenets of formal experimental design and associated inferential data analysis methods should be given limited emphasis. The basis of this thesis lies in the problems and difficulties that undermine the design and implementation of the typical project. Sections of the paper include discussions on design problems and difficulties, data collection, data analysis and reporting, statistical proportions as an evaluative measure, and graphs. These are followed by a presentation of the considerations that may be made in an evaluation report in the face of the problems and difficulties. (Contains 14 references.) (RR)
- Published
- 1992
36. Research into Sexism in Language Testing & Its Implications to Language Testing in China
- Author
-
Tao, Baiqiang
- Abstract
This paper reviews foreign and domestic sexism research and practice in language testing and reveals that China lags behind in this sociolinguistics perspective in both theoretical study and practice. The paper indicates that sexism is represented in the listening comprehension section in National Matriculation English Test (NMET) after a case study into the listening comprehension part in NMET 2000-2005. It is hoped this tentative research will trigger more studies, ameliorate language testing practice in the perspective of gender equality and reduce DIF [differential item functioning] (from which validity will suffer) in especially large scale language testing programs. (Contains 1 endnote.)
- Published
- 2007
37. Examining the Strengths and Limitations of Ethnographic Research: An Evaluation of Two Studies in Distinctive Educational Contexts
- Author
-
Howard, Natalie-Jane
- Abstract
Ethnography offers a holistic approach to qualitative researchers in educational contexts and appeals to scholars who wish seek to reveal rich narratives through their immersion in specific domains. This review paper examines the mobilization of the ethnographic research approach reported in studies from two distinctive learning contexts: an elementary school and a vocational college. Employing the specific evaluative criteria of Punch (2005), the desk-based study draws on existing literature to document the strengths and limitations of ethnographic method and reportage to reveal edifying insights to novice and experienced qualitative researchers who may be contemplating an ethnographic study in the future. The review reveals how extensive ethnography lends itself well to presenting thick descriptions in rich narratives to demonstrate high veracity. In contrast, this research approach may be limited in its verisimilitude, especially if ethnographers abridge their methodological and analytical descriptions and fail to acknowledge reactivity.
- Published
- 2021
38. Towards a Comprehensive Method for Evaluating Social Studies Curriculum Materials: With Examples from the Elementary School Curriculum.
- Author
-
Bisland, Beverly Milner
- Abstract
The evaluation methodology delineated in this paper is a work in progress. The purpose is to develop a method for evaluating social studies curriculum materials that encompasses the materials' comprehensibility and the perspective offered. The paper draws from past evaluations that focused on either comprehensibility or perspective but have not typically encompassed both types of evaluation. It offers examples of studies for comprehensibility such as those done by I. L. Beck and M. G. McKeown (1991), Bonnie B. Armbruster and Thomas H. Anderson (1984), Jere Brophy (1990) and Jane J. White (1988). It offers examples of studies for perspective as well, such as those by Jean Anyon (1978, 1983), Ray Allen Billington (1966), Joel Taxel (1984), Linda S. Levstik (1990), and others. As a general overview, the evaluation methodology determines whether the curriculum materials are age appropriate, educationally significant, and offers the perspective of groups of people important to the topic. Noting that a content goal is necessary for each topic being evaluated, the paper states that this content goal serves as a model for age appropriate and educationally significant information on the particular topic. It explains that, for the first use of the evaluation, three topics from the original textbook series for elementary school social studies by Hanna and Rugg: (1) pre-industrial Pacific Island societies, specifically Samoa and New Guinea; (2) Dutch colonial New York; and (3) the beginnings of industrialization in the 18th century English textile industry. It notes that the aim is to use this evaluation in the future with other materials, modern and historical fiction and non-fiction tradebooks, videos and computer resources as well as textbooks and materials at grade levels other than elementary. (Contains 22 references.) (BT)
- Published
- 2002
39. Are Referees and Editors in Economics Gender Neutral?
- Author
-
Card, David, DellaVigna, Stefano, Funk, Patricia, and Iriberri, Nagore
- Subjects
GENDER role ,GENDER ,ECONOMICS ,ECONOMIC research ,EVALUATION research - Abstract
We study the role of gender in the evaluation of economic research using submissions to four leading journals. We find that referee gender has no effect on the relative assessment of female- versus male-authored papers, suggesting that any differential biases of male referees are negligible. To determine whether referees as a whole impose different standards for female authors, we compare citations for female- and male-authored papers, holding constant referee evaluations and other characteristics. We find that female-authored papers receive about 25% more citations than observably similar male-authored papers. Editors largely follow the referees, resulting in a 1.7 percentage point lower probability of a revise and resubmit verdict for papers with female authors relative to a citation-maximizing benchmark. In their desk rejection decisions, editors treat female authors more favorably, though they still impose a higher bar than would be implied by citation maximization. We find no differences in the informativeness of female versus male referees or in the weight that editors place on the recommendations of female versus male referees. We also find no differences in editorial delays for female- versus male-authored papers. [ABSTRACT FROM AUTHOR]
- Published
- 2020
- Full Text
- View/download PDF
40. Discourses of Assessment--Discourses of Mathematics.
- Author
-
Morgan, Candia
- Abstract
This paper examines the discourses that dominate thinking about assessment in mathematics education; that is, to analyze the sets of constructs, assumptions, and values that underpin research, curriculum development, and teacher education with regard to assessment. Dominant discourses within mathematics education are analyzed and the social functions that assessment fulfills within the classroom and in the broader society are discussed. (Contains 34 references.) (KHR)
- Published
- 2000
41. Measuring Father Involvement in the Early Head Start Evaluation: A Multidimensional Conceptualization.
- Author
-
Mathematica Policy Research, Princeton, NJ., Cabrera, Natasha J., Tamis-LeMonda, Catherine S., Lamb, Michael E., and Boller, Kimberly
- Abstract
Early Head Start (EHS) is a comprehensive, two-generation program that includes intensive services that begin before the child is born and concentrate on enhancing the child's development and supporting the family during the critical first 3 years of a child's life. This paper discusses approaches to measuring father involvement in their children's lives and examines specific measurement methods used in the Father Studies of the Early Head Start Evaluation Project. In addition, the paper highlights lessons from the field that have emerged as father involvement is measured in the ongoing EHS project. Father involvement is described as a multidimensional, continually evolving concept. Time use measures of father involvement are described, and various models of involvement are presented. The paper notes limitations of existing instruments, including the questionable validity of self-report, the often interchangeable use of generic fathering versus child-specific fathers, and the limited generalization of findings from middle-class, European American groups to other groups. The paper describes the EHS program, summarizing the 17 EHS program sites participating in the national evaluation and local research and representing a diversity of locations, populations, culture, ethnicity, and urban-rural settings. Four strands of father studies planned are detailed: (1) interviews with fathers of 24- and 36-month-olds; (2) study of mothers and fathers of newborns; (3) study of strategies used by programs to engage fathers and father figures; and (4) local research studies focused on fatherhood issues significant to local populations. Measurement instruments are then detailed, including questionnaires and videotaped father-child interactions that will yield both qualitative and quantitative data. Finally, the paper explores some of the challenges in assessing father involvement among low-income men, including developing coding systems, obtaining reliability in coding father-child interactions, operationalizing specific constructs, retaining participants, and studying social fathers. The paper concludes by pointing out advances that have been made in the measurement of father involvement. (Contains 23 references.) (KB)
- Published
- 1999
42. System-Wide Assessment in the Arts: A Developing Model.
- Author
-
Australian Inst. of Art Education, Melbourne. and Church, Terrence
- Abstract
This paper discusses the development and implementation of the system-wide assessment of arts learning in primary and secondary schools that took place in Western Australia in September 1996. The paper describes the background to the assessment, the design brief, and the assessment package as a whole, focusing on the visual arts tests. It discusses issues that arose during the development of the test and outcomes highlighted in the data analysis. (BT)
- Published
- 1999
43. No Comparison: Distance Education Finds a New Use for 'No Significant Difference.'
- Author
-
Lockee, Barbara B., Burton, John K., and Cross, Lawrence H.
- Abstract
This paper details the origins of the media comparison study methodology, its current use as an evaluation instrument in distance education, and recommendations for more stringent discrimination between research and evaluation in the field of distance learning. The following topics are discussed: (1) the history of research in instructional technology from 1968 to the present; (2) recent research in distance learning, including the difference between evaluation and research, and weaknesses in many instructional technology studies (e.g., failing to link the study to a robust theory, poor literature review, weak treatment implementation, measurement flaws, inadequate sample size, and poor analyses) that bias the research toward not finding a statistically significant difference; and (3) evaluation versus research in distance education, focusing on appropriate uses of media comparisons for distance program evaluation, as well as alternative methods and exemplary models. (Contains 55 references.) (MES)
- Published
- 1999
44. Techniques for Evaluating Distance Learning Events.
- Author
-
Wisher, Robert A. and Curnow, Christina K.
- Abstract
This paper describes techniques for creating a simplified form for evaluating the effectiveness of a distance learning event (i.e., a training or educational program occurring within 1 day) for which the development of comprehensive measures is often not feasible because the brevity of the event precludes the justification for a lengthy evaluation. The paper focuses on distance learning events for the National Guard Bureau. Background is provided on evaluation measures, including student reaction, learning, behavioral criteria, and results criteria. Development of the compressed evaluation form--one page of five assessment categories (course, technology, instructor, demographics, and motivation) with a total of 22 items--is discussed. Testing of the compressed evaluation form is described. Eight distance learning events were sampled, representing the primary distance learning events available on a nation-wide basis during the evaluation time frame. Training was delivered through a multi-point, one-way video, two-way audio connection through satellite links. A total of 1,306 soldiers participated in the training; the mean rate of returned evaluations was 74%. Two tables present data on student ratings of events and cross tabulation of amount learned by previous training. It is concluded that a wealth of information can be obtained from a simple, one-page evaluation form. (DLS)
- Published
- 1998
45. Educational Rankings of Higher Education: Fact or Fiction?
- Author
-
Hattendorf, Lynn C.
- Abstract
Since educational statistics, which are relatively easy to obtain, can only attempt to measure "quality," this paper asks how quality in higher education is assessed and how educational rankings, which are defined as benchmarks or attempts to measure, contribute to this process. The paper notes that while attempts to rank institutions of higher education date to the late nineteenth century, two seminal studies published in the 1980s and 1990s generated immense interest in the academic community and prompted much analysis. Since then there has been a proliferation of annual publications by the media, but the paper cautions that many are not published on a timely basis nor are their data comparable. Some discussion is devoted to the four major ranking methodologies--reputational rankings, citation analysis, faculty productivity, and statistical rankings--and five articles, which are considered classic studies, are noted as are efforts at international rankings. The paper then offers eight requirements for perfect educational rankings, including standardized reporting methods, peer review, multidimensional techniques, and focus on interdisciplinary programs and emerging disciplines. The paper concludes that while statistics-based rankings are based on hard facts, consumers need to be aware of the shortcomings of opinion-based rankings. (Contains 34 reference notes.)(CH)
- Published
- 1996
46. Measuring Reading Comprehension with the Lexile Framework.
- Author
-
Stenner, A. Jackson
- Abstract
This paper shows how the concept of general objectivity can be used to improve behavioral science measurement, particularly as it applies to the Lexile Framework, a tool for objectively measuring reading comprehension. It begins with a dialogue between a physicist and a psychometrician that details some of the differences between physical science and behavioral science measurement. Building on these distinctions, a definition of measurement is offered that describes what goes on in the physical sciences and represents an attainable ideal of what should go on in the behavioral sciences. This definition of measurement is formalized in an equation that turns out to be the Rasch model, with the important difference that indicant calibrations are obtained via theory, not data. Through the use of theory-based calibrations, a generally objective estimation of the measure parameter in the Rasch model is achieved. The paper then examines the differences between local objectivity obtained with the Rasch model and general objectivity obtained with a theory-enhanced version of that model. Next, it reports on a 10-year study of reading comprehension measurement that implemented the concept of general objectivity through the development of the Lexile Framework. Finally, it summarizes several of the benefits of objective measurement and general objectivity as they might be realized in the measurement of constructs other than reading comprehension. Contains 52 references, and 4 tables and a figure of data. (EF)
- Published
- 1996
47. A Review and Critique of the Tell-Me-A-Story (TEMAS) Apperceptive Test.
- Author
-
Gesek, Glenn J.
- Abstract
The Tell-Me-A-Story Apperceptive Test (TEMAS) is an individual apperceptive test, similar to the Thematic Apperception Test (TAT) of personality assessment. The TEMAS is intended for use with children and adolescents ages 5-18 years old, and is marketed as a personality assessment tool for minority, primarily Hispanic and African-American, and non-minority girls and boys. This paper contains a "Practical Evaluation" section, which: (1) describes the physical composition of a TEMAS; (2) discusses possible confounding factors intrinsic to the administration and interpretation of the TEMAS; and (3) describes how TEMAS stories are scored. The paper's "Technical Evaluation" section examines a sample of 642 children and discusses TEMAS' internal consistency, test-retest reliability, interrater reliability, content validity, construct validity, criterion validity, and concurrent validity as obtained from the data collected. The conclusion is drawn that the TEMAS is an excellent attempt at balancing out the disparity in multi-cultural assessment, while its primary value currently seems to be as a clinical instrument. Further studies of the African-American population's experiences with this instrument need to be conducted, and work needs to be done in general to ascertain whether the TEMAS will be suitable for suburban minority youth. (TS)
- Published
- 1996
48. Stress testing journals: a quasi-experimental study of rejection rates of a previously published paper.
- Author
-
Cobey, Kelly D., Rice, Danielle B., Lalu, Manoj M., Abramowitz, Daniel, Ahmadzai, Nadera, Cunningham, Heather, Ayala, Ana Patricia, Raffoul, Hana, Khan, Faizan, Shamseer, Larissa, and Moher, David
- Subjects
PREDATORY publishing ,PUBLISHED articles ,CHI-squared test ,ACQUISITION of manuscripts ,EXPERIMENTAL design ,RESEARCH ,MASS media ,RESEARCH methodology ,EVALUATION research ,MEDICAL cooperation ,COMPARATIVE studies ,NEWSLETTERS - Abstract
Background: When a journal receives a duplicate publication, the ability to identify the submitted work as previously published, and reject it, is an assay to publication ethics best practices. The aim of this study was to evaluate how three different types of journals, namely open access (OA) journals, subscription-based journals, and presumed predatory journals, responded to receiving a previously published manuscript for review.Methods: We performed a quasi-experimental study in which we submitted a previously published article to a random sample of 602 biomedical journals, roughly 200 journals from each journal type sampled: OA journals, subscription-based journals, and presumed predatory journals. Three hundred and three journals received a Word version in manuscript format, while 299 journals received the formatted publisher's PDF version of the published article. We then recorded responses to the submission received after approximately 1 month. Responses were reviewed, extracted, and coded in duplicate. Our primary outcome was the rate of rejection of the two types of submitted articles (PDF vs Word) within our three journal types.Results: We received correspondence back from 308 (51.1%) journals within our study timeline (32 days); (N = 46 predatory journals, N = 127 OA journals, N = 135 subscription-based journals). Of the journals that responded, 153 received the Word version of the paper, while 155 received the PDF version. Four journals (1.3%) accepted our paper, 291 (94.5%) journals rejected the paper, and 13 (4.2%) requested a revision. A chi-square test looking at journal type, and submission type, was significant (χ2 (4) = 23.50, p < 0.001). All four responses to accept our article came from presumed predatory journals, 3 of which received the Word format and 1 that received the PDF format. Less than half of journals that rejected our submissions did so because they identified ethical issues such as plagiarism with the manuscript (133 (45.7%)).Conclusion: Few journals accepted our submitted paper. However, our findings suggest that all three types of journals may not have adequate safeguards in place to recognize and act on plagiarism or duplicate submissions. [ABSTRACT FROM AUTHOR]- Published
- 2020
- Full Text
- View/download PDF
49. Program Evaluation Models for Science Education: A Synthesis of Literature Sources.
- Author
-
Altschuld, James W. and Kumar, David D.
- Abstract
A literature search of the ERIC system was conducted with regard to papers focused on evaluation models in science education. The purposes were to identify models, their common and unique features, and to examine the guidance they could provide for evaluating science education programs. The search yielded a small number of entries pertinent to the areas of interest which were subsequently classified into three categories--micro (formative through the completion and summative evaluation of programs and products) and macro (contextual) levels and the interface between them. The categories were discussed along with some of the more general evaluation literature dealing with evaluation models. Conclusions were then drawn about why only a limited literature base existed, general models versus a more "situation" imbedded approach to understanding evaluation, the value of model type thinking, and implications for the future needs of evaluation in science education. (Author)
- Published
- 1994
50. Innovative Educational Restructuring for America 2000: Time To Bury Political Bureaucracies and Begin Systematic Assessment, Profiling & Technological Improvement of School Organizations.
- Author
-
Packard, Richard D. and Dereshiwsky, Mary I.
- Abstract
Despite current interest with the concept of the "New American School" model discussed in "America 2000," school systems continue to approach educational reform and restructuring by tinkering with key organizational components in isolation. The total school organization requires assessment and profiling to determine which key components are drags on efficient and effective increases in the goal of student achievement. Arguments, fighting, interest-group power-plays, and politics do not accomplish discovery and application of already-known ways to improve organizational effectiveness. The path to positive restructuring and reform is through use of the latest available technology and qualitative/quantitative evaluation research procedures. To accomplish organizational assessment from the ground up requires the innovative, validated, and holistic model described in this paper. The process, procedures, and outcomes involved with readiness assessment, profiling, and improvement planning correspond to exhibits A through F and include: (1) adopting an assessment model; (2) identifying key organizational factors; (3) organizational profiling; (4) reporting results of organizational research; (5) improvement planning; and (6) prioritizing factors. Every component and function of educational organizations must be held accountable for the goals of student learning and development. Exhibits A through F are included. (17 references) (MLH)
- Published
- 1992
Discovery Service for Jio Institute Digital Library
For full access to our library's resources, please sign in.