2,676 results on '"Outcome measures"'
Search Results
2. Impact Evaluation of Flashlight360 in Mountain View Public Schools: Year 2
- Author
-
Johns Hopkins University, Center for Research and Reform in Education (CRRE), Michael A. Cook, and Steven M. Ross
- Abstract
This study examined the effectiveness of Flashlight360 by continuing a retrospective, mixed-methods quasi-experimental design of ELLs in Grades 1-12 during the 2023-24 school year in a large western state school district. Outcome measures included composite, speaking, and writing achievement gains on the WIDA ACCESS assessment administered to students in schools that implemented Flashlight360, relative to those of students in schools that did not implement the program. Impact analyses showed significant positive impacts of Flashlight360 on WIDA ACCESS composite score and proficiency levels, both for two-year and one-year student participants. Additional analyses showed significant positive impacts of Flashlight360 on WIDA ACCESS Speaking and Writing subscale scores for one-year students, as well as directionally positive impacts for two-year students. Digital usage variables were not significantly associated with WIDA ACCESS scores or proficiency levels.
- Published
- 2024
3. Technology-Focused Multitasking Self-Efficacy and Performance: Whether You Think You Can or Think You Can't -- You Can't
- Author
-
Peter E. Doolittle, Krista P. Wojdak, C. Edward Watson, Dawn N. Adams, and Gina Mariano
- Abstract
Multitasking has been demonstrated to negatively impact performance across a wide range of tasks, including in the classroom, yet students continue to multitask. This study examined the relationship between college students' perceptions and performance of technology-based multitasking. Technology-based multitasking and self-efficacy data were collected and analyzed from 265 undergraduate students. Students engaged in a technology-based multitasking perceptions survey, a video + survey multitasking task or a video-only non-multitasking task, and a technology-based self-efficacy survey. An analysis of student perceptions indicated that students understood that different tasks required different levels of mental effort to complete successfully and that multitasking across high-mental effort tasks required greater effort than multitasking across low-mental effort tasks. In addition, students in the video + survey multitasking group significantly underperformed students in the video-only non-multitasking group. Finally, the relationship between technology-based multitasking and self-efficacy was addressed in a correlational analysis between student technology-based multitasking scores and technology-based self-efficacy scores, yielding no significant relationship. The study findings indicate that most students have an understanding and awareness of multitasking, but ultimately, whether they believed they could multitask or not, multitasking significantly impeded performance.
- Published
- 2024
4. Some College, No Credential: A 2024 Snapshot for the Nation and the States. Fifth in the Series 'Some College, No Credential'
- Author
-
National Student Clearinghouse Research Center, B. Berg, J. Causey, J. Cohen, M. Ibrahim, M. Holsapple, and D. Shapiro
- Abstract
The fall 2023 and spring 2024 undergraduate enrollment increases, marking the first growth since the COVID-19 pandemic, show signs of a post-pandemic turnaround for higher education. However, a significant share of current undergraduates will eventually disengage from college before earning a degree or other credential. They will join tens of millions of other adult Americans who are Some College, No Credential (SCNC). The SCNC population has been consistently rising over time. Re-engaging those who stop out remains a persistent challenge and a priority for the forty states that have set ambitious postsecondary attainment goals. This report aims to provide timely insights into the SCNC population, offering state leaders and policymakers accurate data on its current status, along with tracking progress and outcome measures for SCNC students. The first section of this report describes who makes up the SCNC population and how it has changed since the last report. In this section, the authors pay particular attention to Recent Stopouts, who joined the SCNC population after being stopped out between January 2021 and July 2022. In the second section, the authors report on SCNC re-enrollment in the 2022-23 academic year as well as first-year credential earning for re-enrollees. The authors also provide new updates on continued enrollment and second-year credential earning for SCNC re-enrollees in the 2021-22 academic year, whom were first reported on last year.
- Published
- 2024
5. Pandemic Impact and Recovery in Hawaii: A Descriptive Case Study
- Author
-
National Center for the Improvement of Educational Assessment, Inc. (NCIEA), Damian Betebenner, and Jeri Thompson
- Abstract
This study explores the effects of COVID-19 on academic growth and achievement in Hawaii schools, and strategies that school leaders used in the recovery from the pandemic. The calculations in this report can support states as they investigate the impact of COVID on students and schools and their recovery from the pandemic. The Center for Assessment examined COVID's impact on student test scores, with some schools experiencing high impact and others a far lower impact. The Center also studied rates of student learning, and characterized some schools as "high recovery" because students were on track to catch up to where they would have been had the pandemic not occurred. Other studied schools were categorized as "low recovery" because students were significantly off-track to catch up to where they would have been pre-COVID. Using a case study design, the authors interviewed school leaders at more than a dozen Hawaii schools to gain insight into programs, strategies and structures that influenced a school's degree of recovery. They did not find any correlation between those supports and whether schools later experienced high or low recovery. A fuller understanding of schools' recovery from COVID's impacts would require a deeper understanding of the relationships among leaders and staff, educators, students and the community, and the leadership qualities that supported high recovery where it occurred.
- Published
- 2024
6. Culturally Responsive Student Outcome Measures
- Author
-
Region 19 Comprehensive Center (R19CC) and Melly Wilson
- Abstract
Culturally Responsive Education (CRE) refers to a comprehensive ecosystem of "pedagogy, curriculum, theories, attitudes, practices and instructional materials that center students' culture, identities and contexts throughout education systems" (Peoples, 2019). In the context of these simultaneous and complementary efforts, culturally responsive measures help ensure that what students are taught and assessed on align to learning outcomes in step with their local contexts and daily experiences, and which treat these experiences as valid and powerful sources of knowledge. This brief from the Region 19 Comprehensive Center provides an overview of CRE and provides examples of culturally responsive measures.
- Published
- 2024
7. Reconsidering the Direct vs. Indirect Evidence Dichotomy
- Author
-
Kevin Whiteacre
- Abstract
High quality student outcome measures provide the foundation for effective learning assessment. These outcomes can be organized into different categories, such as skills or habits of mind, and evidence types, such as direct or indirect. Prevailing models of assessment, however, focus only on the distinction between direct and indirect evidence without incorporating the outcome categories. Such a binary model of evidence can be limiting, privileges measures of skills and knowledge over habits of mind and values, and risks excluding validated psychometric tools as a source of measurement. An argument is made for a new 2x2 outcome matrix which incorporates outcome category and evidence type as a possible alternative to the dominant direct vs. indirect dichotomy.
- Published
- 2024
8. Transforming Assessments of Clinician Knowledge: A Randomized Controlled Trial Comparing Traditional Standardized and Longitudinal Assessment Modalities
- Author
-
Shahid A. Choudhry, Timothy J. Muckle, Christopher J. Gill, Rajat Chadha, Magnus Urosev, Matt Ferris, and John C. Preston
- Abstract
The National Board of Certification and Recertification for Nurse Anesthetists (NBCRNA) conducted a one-year research study comparing performance on the traditional continued professional certification assessment, administered at a test center or online with remote proctoring, to a longitudinal assessment that required answering quarterly questions online on demand. A randomized controlled trial of 1,000 certified registered nurse anesthetists (500 randomly assigned to the traditional assessment group and longitudinal assessment group) aimed to 1) compare assessment performance between groups, 2) compare perceptions and user experience between groups; and 3) describe participant feedback about usability of the longitudinal assessment platform. The mean scaled score for the traditional assessment group exceeded that of the longitudinal assessment group when scoring the first responses; however, upon scoring the longitudinal assessment group's most recent responses on repeat questions previously answered incorrectly, the mean scaled score was higher than the traditional assessment group. Both groups were satisfied with their experience, with slightly higher feedback ratings for the longitudinal assessment group who also found the platform easy to use and navigate. Overall results suggest the longitudinal assessment is a feasible, acceptable, and usable format to assess specialized knowledge for continued healthcare professional certification.
- Published
- 2024
9. Measures of Economic Disadvantage Explain Outcomes Differently across Geographies
- Author
-
Robin Clausen
- Abstract
Alternative poverty measures have been proposed in response to the emerging insufficiencies of the National School Lunch Program (NSLP) eligibility data. The analysis presented here involves seven poverty measures. Using outcome measures as a yardstick, we can assess how poverty measures explain these outcomes and note variations between geographical locales (assessing predictive validity). An analysis of 2019 data from Montana revealed that no poverty measure emerges as consistently meeting or exceeding the results found with the NSLP on the state level. Results are mixed based on locale (size) and distance from an urban centre, and within school communities.
- Published
- 2024
10. Use of an Education and Coaching Intervention to Promote Occupational Balance in Graduate Occupational Therapy Students
- Author
-
Linda M. Rini and Ingrid Provident
- Abstract
This small-scale pilot study was developed to examine the perceived occupational balance of graduate occupational therapy students and implement an education and coaching intervention to promote occupational balance for those who have difficulty achieving and maintaining it. Although literature provides many examples of interventions to facilitate stress relief in graduate students, this is the first to address the promotion of occupational balance in this effort, which has been identified as a student-reported need. A sequential explanatory mixed-methods approach was used to obtain quantitative and qualitative data on occupational therapy students' satisfaction with their balance of activity in all areas of occupation. The impact of that balance on stress levels and occupational satisfaction was examined utilizing an investigator developed scaled survey and The Canadian Occupational Performance Measure. Participants (N=22) were MSOT and OTD students from all academic years of their program. A six-week group education program was implemented. One additional 30-minute individual coaching session was held virtually with each student and the primary investigator to target individual goals. Survey data collected before the start of the educational series and again at its conclusion indicate an aggregate decrease in students' perceived stress (-0.85); an increase in perceived occupational performance (mean +1.26); and an increase in perceived occupational satisfaction (mean +2.10). The desired outcome of the intervention was to provide students with strategies to increase lifestyle balance to help manage some of the stresses of graduate academia, and ultimately increase students' health and wellness to promote retention and successful completion of graduate education.
- Published
- 2024
11. Exploring Missing Learning Phenomenon in Preschool Settings during COVID-19 Pandemic: Teachers' Perspectives
- Author
-
Nurul Hijja Mazlan, Wardatul Hayat Adnan, Suffian Hadi Ayub, and Mohd Zaidi Zeki
- Abstract
Purpose--This study aims to explore the missing learning phenomenon due to school closures during the COVID-19 pandemic in preschool settings from teachers' perspectives in Selangor, Malaysia. Related studies have shown that missing learning phenomena from different settings leads to delayed literacy development and raises the risk of dropping out among children. There is a lack of in-depth documentation on the missing learning phenomenon among preschoolers exploring the socioecology of preschool settings in a local context. Methodology--This research employed a case study approach. In-depth interviews were conducted with six preschool teachers from Selangor, Malaysia, using an inductive approach for a holistic understanding of the missing learning phenomenon during the COVID-19 pandemic. The data were analyzed using Computer Assisted Qualitative Data Analysis Software (CAQDAS), NVivo version 12 software, which mapped the findings to understand further the ecosystem behind the preschoolers' absence of learning during the COVID-19 pandemic. Findings--Three major themes were revealed from the data: the missing learning demographic, socioecological factors, and mitigation plans. Several sub-themes (the compensative learning solutions, the demographic of impacted people, the delayed progress, teachers' readiness to facilitate online learning, engagement towards online learning, revised syllabus, personalized learning, and parental involvement) are classified under the respective major themes. Several main categories also emerged from the themes: lack of experiences and technology-pedagogical knowledge, unavailable online modules, low readiness to utilize online learning, unsupportive family climate, and accessibility. Learning intervention and online engagement with pupils during the pandemic are crucial in ensuring a successful continuous education. Significance--There is also a need for comprehensive remote instructional modules that consider socioecological actors -- parents and home climate; in the future and the enhanced instructional technology competency among preschool teachers.
- Published
- 2024
12. Literary Discussions in the Modern Classroom: Online and In-Person Implementation Strategies
- Author
-
Albert Carter, DeSuan Dixon, and Xia Li
- Abstract
Literary discussions are widely utilized in education, yet their profound impact on students' comprehension requires in-depth exploration. This paper delves into the influence of literary discussions on student comprehension and examines how these discussions can enhance students' understanding and analytical skills. Through literature reviews and case analyses, this study investigates teaching strategies, student engagement levels, classroom ambiance, and other factors involved in literary discussions, revealing the potential mechanisms through which literary discussions influence student comprehension. The paper explores variations in the application of literary discussions across different types of literary works (novels, poetry, drama, etc.) and their relationship with students' creative thinking. Finally, the paper provides recommendations to assist educators in effectively utilizing literary discussions to enhance students' comprehension skills. Students must have the opportunity to discuss literature in ways that provoke deep thought and meaning from each work they encounter. Therefore, English Language Arts teachers and instructors must be creative in finding unique ways in which they encourage their students to experience and respond to literature. This paper discusses a variety of strategies to implement in virtual and in-person classrooms, such as Socratic seminar, Teacher-led as a facilitator, Teacher-led as a promoter of diverse perspectives, Using student-generated discussion questions whole class, Using student-generated discussion questions in small groups, Using Close-ended questions, Using Open-ended questions, Inquiry-based, Making real-world connections to the literature, Literature Circles. These literary discussion strategies can be used for both fiction and nonfiction literature.
- Published
- 2024
13. Measuring Two Constructs of Afterschool Activity Participation: Breadth and Intensity
- Author
-
Jose R. Palma, Martin Van Boekel, and Ashley S. Hufnagle
- Abstract
The benefits of afterschool activity participation for youth development are well-documented. An interesting question dominating this field is whether there is a threshold at which point participating in too many activities (breadth) and spending too much time in those activities (intensity) is negatively associated with desirable outcomes. Using 9th grade student data (N=115,731) from three administrations of a state-wide school survey, we explore whether students' breadth and intensity of afterschool participation is associated with GPA and perceived family and community support. Findings corroborate prior research in demonstrating the association between breadth and intensity. Importantly, we extend the discussion, with three important observations. First, a linear model is insufficient for modeling these complex associations with outcomes. Second, there is a threshold at which too much participation has a negative impact in these outcomes. Third, variations in activities, time windows and indices have small or no influence in the association with outcomes.
- Published
- 2024
14. Micro-Analyses Reveal Increased Parent-Child Positive Affect in Children with Poorer Adaptive Functioning Receiving the ESDM
- Author
-
Yana Sinai-Gavrilov, Tali Gev, Ilanit Gordon, Irit Mor-Snir, Giacomo Vivanti, and Ofer Golan
- Abstract
Synchronous positive affect (SPA) is a key element of parent-child interaction quality which is related to favorable developmental outcomes. Children with autism spectrum disorder (ASD) and their parents tend to show less SPA compared to other populations. The current study explored changes in SPA made by parents and their children with ASD following the Preschool-Based Early Start Denver Model (PB-ESDM) intervention. Thirty children receiving PB-ESDM and 23 receiving treatment-as-usual (TAU) were assessed pre- and post- intervention using microanalysis of video-recorded parent-child interactions, in which SPA was quantified. Results showed a significant increase in SPA among children receiving PB-ESDM who had lower pre-treatment adaptive functioning. These findings suggest that SPA may serve as a sensitive treatment outcome measure for children with poorer adaptive functioning, who often struggle to show significant changes on standardized measures. The study's modest sample and non-randomized design are noted as limitations.
- Published
- 2024
- Full Text
- View/download PDF
15. Do Social Programs Help Some Beneficiaries More than Others? Evaluating the Potential for Comparison Group Designs to Yield Low-Bias Estimates of Differential Impact
- Author
-
Andrew P. Jaciw
- Abstract
In the current socio-political climate, there is an extra urgency to evaluate whether program impacts are distributed fairly across important student groups in education. Both experimental and quasi-experimental designs (QEDs) can contribute to answering this question. This work demonstrates that QEDs that compare outcomes across higher-level implementation units, such as schools, are especially well-suited to contributing evidence on differential program effects across student groups. Such designs, by differencing away site-level (macro) effects, on average produce estimates of the differential impact that are closer to experimental benchmark results than are estimates of average impact based on the same design. This work argues for the importance of routine evaluation of moderated impacts, describes the differencing procedure, and empirically tests the methodology with seven impact evaluations in education. The hope is to encourage broader use of this design type to more-efficiently develop the evidence base for differential program effects, particularly for underserved students.
- Published
- 2024
- Full Text
- View/download PDF
16. Implementation Drivers of Data-Based Instruction for Students with Intensive Learning Needs: A Systematic Review
- Author
-
Seohyeon Choi, Emma Shanahan, Bess Casey-Wilke, Jechun An, and LeAnne Johnson
- Abstract
Despite decades of research efforts, data-based instruction (DBI) for students with intensive intervention needs are not being widely used in practice as anticipated, and many educators have difficulties in implementing it. This systematic review aimed to examine what kinds of implementation drivers and strategies have been used to support educators implementing DBI and what kinds of implementation outcomes researchers have measured. Eighteen studies were synthesized using the Implementation Drivers framework and Implementation Outcomes taxonomy and were quality appraised. We found that the majority of studies primarily used competency drivers to increase teachers' DBI expertise, while a limited number of studies focused on organizational and leadership drivers. Acceptability and fidelity were frequently assessed as implementation outcomes. We discussed the implications of the findings, including the need for researchers to incorporate implementation drivers and outcomes at diverse levels to best support educators' implementation of DBI, as well as the limitations of this review, such as the limited generalizability of the findings.
- Published
- 2024
- Full Text
- View/download PDF
17. Parent's Use of Naturalistic Instruction upon Receiving Technology-Enhanced Performance-Based Feedback
- Author
-
Christan Grygas Coogle, Emil Majetich, Sloan O. Storie, and Clarissa Bunch Wade
- Abstract
We used a multiple baseline across strategies design to determine the effect of technology-enhanced performance-based feedback on parent's use of preventive strategies to address children's challenging behaviors for two parent--child dyads. A father, mother, and each of their young children participated. We also measured associated child outcomes and parent's perceptions of the technology-enhanced performance-based feedback intervention. We observed a functional relation between our intervention and parent practice. Although variable, desired child behaviors increased, and challenging behaviors decreased. Finally, parents indicated that they perceived the intervention to be socially valid.
- Published
- 2024
- Full Text
- View/download PDF
18. Using Bayesian Meta-Analysis to Explore the Components of Early Literacy Interventions. Appendices. WWC 2023-008
- Author
-
National Center for Education Evaluation and Regional Assistance (NCEE) (ED/IES), What Works Clearinghouse (WWC) and Mathematica
- Abstract
The appendices accompany the full report "Using Bayesian Meta-Analysis to Explore the Components of Early Literacy Interventions. WWC 2023-008," (ED630495), which pilots a new taxonomy developed by early literacy experts and intervention developers as part of a larger effort to develop standard nomenclature for the components of literacy interventions. The What Works Clearinghouse (WWC) uses Bayesian meta-analysis--a statistical method to systematically summarize evidence across multiple studies--to estimate the associations between intervention components and intervention impacts. Twenty-nine studies of 25 early literacy interventions that were previously reviewed by the WWC and met the WWC's rigorous research standards were included in the analysis. The following apprendices are presented: (1) Components of Early Literacy Interventions; (2) Data from the What Works Clearinghouse's Database of Reviewed Studies; (3) The Bayesian Meta-Analytic Model; (4) Additional Results; and (5) Component Coding Protocol.
- Published
- 2023
19. 2021-2023 Interagency Autism Coordinating Committee Strategic Plan for Autism Research, Services, and Policy
- Author
-
US Department of Health and Human Services, Interagency Autism Coordinating Committee and National Institute of Mental Health (NIMH) (DHHS/NIH), Office of National Autism Coordination (ONAC)
- Abstract
The Interagency Autism Coordinating Committee (IACC) is a federal advisory committee that advises the Secretary of Health and Human Services (HHS) on issues related to autism spectrum disorder (ASD). It was established by the "Children's Health Act of 2000" (Public Law 106-310), reconstituted under the Combating Autism Act of 2006 (CAA; Public Law 109-416), and was most recently renewed under the "Autism Collaboration, Accountability, Research, Education, and Support (CARES) Act of 2019" (Public Law 116-60). One of the statutory responsibilities of the IACC under the CAA and subsequent authorizations is the development of a strategic plan for autism, to be updated annually. The "Autism CARES Act of 2019" requires that the strategic plan address the "conduct of, and support for, autism spectrum disorder research, including as practicable for services and supports." With each update of its "Strategic Plan," the IACC re-evaluates the needs of the autism community and the best ways to achieve progress. The "2021-2023 IACC Strategic Plan for Autism Research, Services, and Policy" summarizes current understanding of autism-relevant topics and addresses current gaps and opportunities in autism research, services and supports, and policy. As in previous years, the "IACC Strategic Plan" is organized around seven general topic areas that are represented in the Plan as community-focused Questions. Each question is assigned a chapter in the "Strategic Plan" that provides an Aspirational Goal, or long-term vision for the question; a description of the state of the field; the needs and opportunities in research, services, and policy; and three broad Objectives. In this edition of the "IACC Strategic Plan," the Objectives from the "2016-2017 IACC Strategic Plan" have been updated and renamed as "Recommendations." The 24 total updated Recommendations in this "Strategic Plan," including the new equity Recommendation and the Budget Recommendation, address critical gaps and potential opportunities for advancement that were identified by the IACC. The IACC's goal during the development of this "Plan" is to present a collective voice detailing the current status and future goals of autism research, services, and policy. [For the "Interagency Autism Coordinating Committee Strategic Plan for Autism Spectrum Disorder, 2018-2019 Update," see ED608308.]
- Published
- 2023
20. Beyond Test Scores: Broader Academic Consequences of the COVID-19 Pandemic on American Students. Report from a Consensus Panel
- Author
-
Arizona State University (ASU), Center on Reinventing Public Education (CRPE), Morgan Polikof, Isabel Clay, and Daniel Silver
- Abstract
Recent state and national achievement exam results, as well as academic progress reports, have underscored how the COVID-19 pandemic and related school closures had a large, negative impact on students' reading and math development. While a great deal is known about the test-based academic impacts of the pandemic such as test scores that provide valuable evidence, there are a range of non-test measures like attendance, engagement in school, student retention, course failure rates, degree completion, and enrollment in subsequent levels of education that also yield important insights about students' educational well-being. These measures matter in their own right and in how they affect longer-term outcomes like employment, earnings, and adult well-being. This consensus panel report reviewed the best available evidence to understand how the pandemic affected non-tested academic areas, and it offers recommendations for shaping a policy response.
- Published
- 2023
21. First Year Implementation of Exact Path Leads to Sizable Growth in NWEA MAP Reading Scores
- Author
-
Weiling Li, Aaron Butler, Catherine Oberle, Anabil Munshi, and Amy J. Dray
- Abstract
Edmentum offers a personalized learning platform called Exact Path. This quasi-experimental study, designed to meet ESSA Tier 2 evidence and What Works Clearinghouse standards with reservations, aimed to assess the efficacy of Exact Path in a district from the Midwestern United States. The goal was to provide specific recommendations to educators within the district and inform the broader community of policymakers and practitioners about the potential benefits of personalized learning for enhancing student academic achievement. The study found that the use of Exact Path was positively related to Reading achievement in NWEA MAP tests, after controlling for students' prior test scores and their socioeconomic status. These findings suggest that Exact Path could be an effective tool for improving student success in this district and potentially other similar contexts. Therefore, these results may have important implications for educators, policymakers, and researchers interested in improving student outcomes through personalized learning.
- Published
- 2023
22. Proficiency with Number Concepts and Operations: Replicating the Efficacy of a First-Grade Mathematics Intervention
- Author
-
Christian T. Doabler, Ben Clarke, Jessica E. Turtura, Marah Sutherland, Jenna A. Gersib, Taylor Lesner, Madison Cook, Georgia L. Kimmel, Keith Smolkowski, and Derek Kosty
- Abstract
Conceptual replications are part and parcel of education science. Methodologically rigorous conceptual replication studies permit researchers to test and strengthen the generalizability of a study's initial findings. The current conceptual replication sought to replicate the efficacy of a small-group, first-grade mathematics intervention with 240 first-grade students with mathematics difficulties in a new geographical region. Participating students were randomized into one of three conditions: (a) 2:1 mathematics intervention group, (b) 5:1 mathematics intervention group, or (c) business-as-usual instruction. Relative to the original study, findings from the replication varied. When comparing the treatment groups to the control, results suggested positive effects on all outcome measures, including a follow-up assessment administered one year later. However, differences between the two treatment groups based on group size were not found in the mathematics outcome measures. Both groups also received commensurate levels of observed instructional interactions. Implications for unpacking contextual differences between original research and their replications as well as using future research to explore the quantity and quality of instructional interactions as ways to explain variation in findings of group size are discussed.
- Published
- 2024
- Full Text
- View/download PDF
23. Expectations and Experiences of a Dance Programme for Autistic Children: A Qualitative Study of Parents, Teachers and Therapists
- Author
-
Ali Golding, Zoe Ambrose, Joanne Lara, Christina Malamateniou, and Dido Green
- Abstract
This study explores the expectations of dance therapists/practitioners and parents and teachers of autistic children engaging in a developmental dance programme. Information gathered will support development of an evaluation tool aligned with the International Classification of Functioning Disability and Health (ICF) Core Sets for autism spectrum disorders (ASD). A qualitative study included a convenience cohort of teachers (n = 6), parents (n = 2) of children with ASD and therapists (n = 3). Three role specific focus groups were undertaken considering potential benefits and challenges of the programme. Content and thematic analysis was undertaken using NVivo12. Findings reflected four positive themes relating to behaviour, skills, social interaction and environmental supports. Therapists, teachers and parents focused differently on stereotypical and restricted behaviours, environmental supports and habits and routines respectively. These themes also emerged as challenges (to implement/achieve); with parents identifying more emotional and behavioural restrictions. A fourth challenge theme of transferability of skills emerged from teachers and therapists. Items mapped against 28 ICF Core Sets (across the lifespan) and six to ICF categories, with creativity and imagination mismatched. Findings highlight need for a specific outcome measure for dance and/or movement programmes for autistic individuals that captures meaningful functions across ICF domains for differing stakeholders.
- Published
- 2024
- Full Text
- View/download PDF
24. School-Based Organizational Skills Training for Students in Grades 3 to 5: A Cluster Randomized Trial
- Author
-
Jenelle Nissley-Tsiopinis, Thomas J. Power, Phylicia F. Fleming, Katie L. Tremont, Bridget Poznanski, Shannon Ryan, Jaclyn Cacia, Theresa Egan, Cristin Montalbano, Alex Holdaway, Ami Patel, Richard Gallagher, Howard Abikoff, A. Russell Localio, and Jennifer A. Mautone
- Abstract
Objective: Research has demonstrated the effectiveness of interventions to reduce organizational skills deficits and homework problems, including the clinic-based Organizational Skills Training (OST-C) program (Abikoff et al., 2013). In this study, OST-C was adapted for schools as a small-group (Tier 2) intervention delivered by school partners (OST-T2). Method: The study was conducted in 22 schools serving students from diverse backgrounds. Students (n = 186; 122 male) in grades 3-5, ages 8-12 (M = 9.7 years; SD = 0.88) with organizational skills deficits referred by teachers were enrolled. Schools were randomly assigned to OST-T2 or treatment-as-usual with waitlist (TAU/WL). OST-T2 consisted of sixteen 35-minute child sessions, two caregiver, and two teacher consultations. Outcomes were evaluated with longitudinal mixed effects modeling at post-treatment, 5-month and 12-month follow-up using caregiver and teacher reports of organizational skills, homework, and academic performance. Results: OST-T2 resulted in reductions in organizational skills deficits on caregiver and teacher report (ps < 0.001) at post-treatment and 5-month follow-up (effect sizes [ES], Cohen's d = 0.96,1.20). Findings also revealed a reduction in caregiver-reported homework problems at post-treatment and 5-month follow-up (ps < 0.001, ES = 0.60, 0.72), and an improvement in teacher-rated homework at post-treatment (p = 0.007, ES = 0.64). Effects were attenuated at 12-month follow-up. The effects of OST-T2 on academic measures were not significant. Conclusions: Findings provide evidence for the immediate and short-term effectiveness of OST-T2 delivered by school professionals. [This paper will be published in the "Journal of Consulting and Clinical Psychology."]
- Published
- 2024
25. Pilot Randomized Trial of a Caregiver-Mediated Naturalistic Developmental Behavioral Intervention in Part C Early Intervention
- Author
-
Melanie Pellecchia, Brooke Ingersoll, Steven C. Marcus, Keiran Rump, Ming Xie, Jeannette Newman, Lisa Zeigler, Samantha Crabbe, Diondra Straiton, Elena Carranco Chávez, and David S. Mandell
- Abstract
Few studies examine the effectiveness of community-based early interventions in unselected samples of autistic children, especially those from minoritized backgrounds. These types of studies require attention to strategies for recruitment, retention, data collection, and support for community providers beyond those used in university-based trials. We conducted a pilot trial of Project ImPACT, a promising caregiver-mediated naturalistic developmental behavioral intervention, in partnership with the early intervention system in a poor, large city. We assessed recruitment and retention strategies, training protocol, and measurement battery. We recruited representative provider and family samples, and had good measure completion among retained participants. Retention varied by study arm, providers had relatively poor fidelity to the intervention despite substantial support, and our measures did not appear sensitive to change. Lessons learned include the need for (a) a ramp-up training period prior to starting the study, (b) intensive implementation supports, and (c) additional strategies for family retention.
- Published
- 2024
- Full Text
- View/download PDF
26. Re-examining the Relation between Social Validity and Treatment Integrity in Ci3T Models
- Author
-
Katie Scarlett Lane Pelton, Kathleen Lynne Lane, Wendy Peia Oakes, Mark Matthew Buckman, Nathan Allen Lane, Grant E. Allen, D. Betsy McCoach, David James Royer, and Eric Alan Common
- Abstract
Educators across the United States have designed and implemented Comprehensive, Integrated, Three-tiered (Ci3T) models to meet K-12 students' academic, behavioral, and social and emotional well-being needs. As part of implementation efforts, educators collect and use social validity and treatment integrity data to capture faculty and staff views of the plan's goals, procedures, and outcomes and the degree to which the plan is implemented as designed (e.g., procedures for teaching, reinforcing, and monitoring). In this study, we re-examined the relation between social validity and treatment integrity utilizing hierarchical linear modeling with extant data from a research partnership across 27 schools in five midwestern districts. Findings suggested an educator's fall and spring social validity score on the Primary Intervention Rating Scale (PIRS) predicted their treatment integrity scores on the Ci3T Treatment Integrity: Teacher Self-Report (CI3T TI: TSR) in the same timepoint. Schoolwide average fall PIRS scores also statistically significantly predicted spring Ci3T TI: TSR scores. Results suggested schoolwide context is important for sustained implementation of Tier 1 procedures during the first year. Findings demonstrate the complex nature of implementing a schoolwide plan, involving each individual's behavior while also relying on others to facilitate implementation. We discuss limitations and future directions.
- Published
- 2024
- Full Text
- View/download PDF
27. The Effectiveness and Impact of Post-Observation Feedback Sessions in an In-Service Training Program
- Author
-
Derya Ilgin Yasar and Nurdan Gürbüz
- Abstract
An essential component of any training activity is observations, and in almost any teacher education or professional development context, observations are followed by feedback sessions aimed at improving teaching practices. Nevertheless, whether these sessions accomplish this aim is an issue of concern as it is highly related to how effective it is perceived to be. This study aims to explore how effective post-observation feedback sessions are perceived by the observed instructors, what factors make feedback effective, and whether feedback has any impact on their practices. Data were collected qualitatively by conducting semi-structured interviews with the instructors and Professional Development Unit (PDU) members. The results showed that the feedback was considered effective by the instructors, and it impacted their teaching practices positively, while helping them with their adaptation to their new institution. The results also shed light on the factors that made feedback effective from the instructors' and the PDU members' perspectives.
- Published
- 2023
28. Learning Support Zones: Former Students' Experience and Perceived Impact on Home and Work Environment
- Author
-
Cassar, Elise and Abela, Angela
- Abstract
This study explored how past students who attended the Learning Support Zone (LSZ) in Maltese secondary schools perceived their attendance at the LSZ and its impact on their home and work environments. In-depth interviews were conducted with eleven adults who attended the LSZ when they were in secondary school. Thematic analysis was used to elicit a set of themes. The findings highlight the multiple struggles that students with social, emotional and behavioural difficulties (SEBD) face. The narratives highlight the positive impact of the LSZ and its staff, but the families of these students were hardly involved. Social skills, independent living skills, and emotional literacy taught at the LSZ are believed to have helped these youths enhance their motivation and perceived self-worth and deal with adversities more effectively both at home and at work. The key recommendations of the study are the re-evaluation of inclusive policies within schools, consideration of the perspective of the students, better understanding, and monitoring of students with SEBD within schools, and more active involvement of all relevant stakeholders, especially families.
- Published
- 2023
29. Study Review Protocol. Version 5.0
- Author
-
National Center for Education Evaluation and Regional Assistance (NCEE) (ED/IES), What Works Clearinghouse (WWC)
- Abstract
The What Works Clearinghouse (WWC) Study Review Protocol accompanies the "WWC Procedures and Standards Handbook, Version 5.0," and guides reviews of studies by the WWC. The WWC uses this protocol to review all studies, including those cited as evidence for U.S. Department of Education grant competitions, studies that were funded by the Department, and studies identified for systematic reviews of evidence based on a search of the research literature in a particular topic area. As articulated in the "Handbook," when the Study Review Protocol is used to review studies for systematic reviews, an accompanying topic area synthesis protocol will provide criteria for the literature search; guidance on how to identify and prioritize relevant studies for review and inclusion in evidence synthesis products; and guidance on intervention, sample, and outcome eligibility criteria for the synthesis. [For "What Works Clearinghouse: Procedures and Standards Handbook, Version 5.0. WWC 2022008," see ED621928.]
- Published
- 2023
30. Economic Evaluation of Early Interventions for Autistic Children: A Scoping Review
- Author
-
Katherine Pye, Hannah Jackson, Teresa Iacono, and Alan Shiell
- Abstract
Many autistic children access some form of early intervention, but little is known about the value for money of different programs. We completed a scoping review of full economic evaluations of early interventions for autistic children and/or their families. We identified nine studies and reviewed their methods and quality. Most studies involved behavioral interventions. Two were trial-based, and the others used various modelling methods. Clinical measures were often used to infer dependency levels and quality-adjusted life-years. No family-based or negative outcomes were included. Authors acknowledged uncertain treatment effects. We conclude that economic evaluations in this field are sparse, methods vary, and quality is sometimes poor. Economic research is needed alongside longer-term clinical trials, and outcome measurement in this population requires further exploration.
- Published
- 2024
- Full Text
- View/download PDF
31. Can a Writing Intervention Using Mainstream Assistive Technology Software Compensate for Dysgraphia and Support Reading Comprehension for People with Aphasia?
- Author
-
Becky Moss, Jane Marshall, Celia Woolf, and Katerina Hilari
- Abstract
Background: Stroke profoundly affects quality of life (QOL), including loss of employment, reduced social activity, shrinking social networks and low mood. Dysgraphia (impaired writing) is a common symptom of aphasia yet is rarely targeted in rehabilitation. Recent technological advances might challenge this, since much communication is now conducted digitally through writing. The rehabilitation of writing may therefore help to address the wider consequences of stroke and aphasia. Aims: Can assistive technology (AT) training for people with dysgraphia: (1) improve written output, and are gains achieved only with AT? (2) improve reading comprehension scores, and are gains achieved only with AT? and (3) affect social participation, mood or QOL. Methods and Procedures: Design: A mixed-methods, repeated measures, small group study design was adopted (qualitative outcomes will be reported elsewhere). Participants: Recruited from community settings, for example, Stroke Association communication support groups. Inclusion criteria: over 18 years old, aphasia due to stroke, acquired dysgraphia, writing more impaired than speech, fluent English prior to stroke, access to computer and Internet. Exclusion criteria: currently receiving speech and language therapy, significant cognitive impairment, neuromuscular/motor-speech impairments/structural abnormalities, developmental dyslexia, uncorrected visual/auditory impairments. Procedures: Screening and diagnostic assessments at time T1 (first baseline). Outcome measures at T1; repeated at T2 (second baseline), T3 (end of intervention), T4 (3-month follow up). Social participation assessment and cognitive monitoring at T2, T3, T4. Intervention: Seven-ten hours individual therapy weekly and additional email support. Participants were trained to operate Dragon NaturallySpeaking (speech to text package) and ClaroRead (read writing aloud). Outcome measures were administered on pen and paper (control) and on computer, with AT enabled only at T3, T4. Outcomes and Results: Computer narrative writing was significantly improved by AT training (Friedman's X[superscript 2] (3) = 8.27, p = 0.041), indicating a compensatory effect of AT. Though reading comprehension significantly improved in the computer condition (Friedman's X[superscript 2] (3) = 21.07, p = 0.001), gains could not be attributed to the AT. Gains were achieved only when measures were administered on the keyboard, with AT enabled. Thus, a compensatory rather than remediatory effect was suggested. Social network size significantly increased; there were no significant changes in mood/QOL. Individual success rates varied. Conclusion and Implications: The customisable AT training was acceptable to participants and resulted in significantly improved narrative writing. Compensatory AT interventions are a useful adjunct to remediatory writing interventions and may particularly support functional writing.
- Published
- 2024
- Full Text
- View/download PDF
32. EPIC: Educator Performance Incentive and Career Pathways. Teacher and School Leader Program. Final Evaluation and Impact Study Report
- Author
-
The Evaluation Group (TEG), Christy Derrick, Catherine Snyder, Kristin LaRoche, Stephanie Marshall, and Kathy Dowell
- Abstract
Union County Public Schools (UCPS) received a U.S. Department of Education Teacher and School Leader Incentive Program grant in 2017, supporting the EPIC program aimed at transforming education in 13 low-performing, high-poverty schools. The EPIC initiative had two primary goals: redesigning the Human Capital Management System and Performance-Based Compensation System to attract, retain, and sustain effective educators, and enhancing teacher and school leader effectiveness to positively impact student academic achievement. UCPS implemented a Performance-Based Compensation System (PBCS) that provided monetary incentives to individual teachers and school leaders based on attendance, growth targets, and evaluations. The EPIC impact study evaluated the program's effects on student outcomes using a short-interrupted time series design with comparison (SIT-C), comparing standardized achievement scores in math and reading for Grades 5 and 8, and high school levels, with matched comparison schools. The study spanned five years, tracking proficiency scores before and after EPIC implementation. Results at the elementary level showed that neither the treatment nor comparison groups achieved the projected mean percent proficient in post-intervention years for math. However, the treatment group exceeded the mean in reading, unlike the comparison group. In middle schools, the treatment group surpassed the projected mean percent proficient in both math and reading, while the comparison group did not. At the high school level, neither group achieved the projected mean percent proficient in math post-intervention, but the treatment group exceeded it in English compared to the comparison group. Teacher outcomes indicated improvements in retention and diversity across EPIC schools. Seven of the 13 schools saw a substantial increase in teacher retention, and the percentage of teachers of color increased by 10 points over five years. Teacher attendance showed improvement, with a decrease in average days absent. UCPS's EPIC program demonstrated positive outcomes in teacher retention, diversity, and attendance, with varying impacts on student proficiency across grade levels. The study emphasized the need for cautious interpretation of data due to external factors like the COVID pandemic. [The report was submitted to Union County Public Schools.]
- Published
- 2024
33. Trajectory Research in Children with an Autism Diagnosis: A Scoping Review
- Author
-
Stephen J. Gentles, Elise C. Ng-Cordell, Michelle C. Hunsche, Alana J. McVey, E. Dimitra Bednar, Michael G. DeGroote, Yun-Ju Chen, Eric Duku, Connor M. Kerns, Laura Banfield, Peter Szatmari, and Stelios Georgiades
- Abstract
Researchers increasingly employ longitudinal trajectory methods to understand developmental pathways of people on the autism spectrum across the lifespan. By assessing developmental or health-related outcome domains at three or more timepoints, trajectory studies can characterize their shape and varying rates of change over time. The purpose of this scoping review was to identify and summarize the published breadth of research that uses a trajectory study design to examine development in children (to age 18 years) diagnosed with autism. Using a systematic search and screening procedure, 103 studies were included. This review summarizes methodological characteristics across studies including the varying statistical approaches used. A series of figures maps where published research is available across 10 outcome domains and the ages over which children have been followed. Evidence gaps, informed by the perspectives of the autistic and caregiver stakeholders that were engaged in this review, are discussed. We recommend that future trajectory research addresses the absence of studies from low- and middle-income countries, considers longitudinal assessment of outcome domains that caregivers and autistic people consider meaningful, and plans follow-up periods with assessment timepoints that cover the gaps in ages where more outcome-specific data are needed.
- Published
- 2024
- Full Text
- View/download PDF
34. 2023-2024 Florida Adult Education Assessment Technical Assistance Paper
- Author
-
Florida Department of Education, Division of Career and Adult Education and Kevin O’Farrell
- Abstract
This technical assistance paper provides policy and guidance to individuals with test administration responsibilities in adult education programs. The Florida assessment policies and guidelines presented in this technical assistance paper are appropriate for state and federal reporting. Therefore, guidance and procedures regarding the selection and use of appropriate student assessment are included. The following important information for adult education programs is provided: (1) Definition of key terms and acronyms; (2) Selection of appropriate assessments by student and program type; (3) Appropriate student placement into program and instructional level; (4) Verification of student learning gains, EFL, and/or program completion; (5) Accommodation for students with disabilities and other special needs; (6) Assessment procedures for Distance Education; and (7) Training for all staff who administer the standardized assessments.
- Published
- 2023
35. The Impact of Service-Learning on Occupational Therapy Doctoral Students
- Author
-
Valdes, Kristin, Rider, John, Leach, Christen, and Manalang, Katie Capistran
- Abstract
The aim of this study was to determine how participation in a community service project impacts entry-level OT doctoral (OTD) students at different time points in their education. This study employed a mixed-method, prospective cohort design. The Community Service Attitudes Scale (CSAS) is an instrument to measure college students' attitudes about community service. Open-ended questions were asked after participation in the experience in addition to the CSAS to gather a deeper reflection of the S-L experience. A total of 62 OTD students participated in the study. There was no statistically significant difference between the CSAS scores between the OTD first-year and third-year students. Students reported that they gained knowledge regarding occupational justice and community-based interventions, how to establish a therapeutic relationship, and the impact of occupation-based interventions when asked what they learned from the experience. Through the participation in a S-L experience, the students gained knowledge regarding occupational justice and community-based interventions, how to establish a therapeutic relationship, and an awareness of the impact of occupation-based interventions on clients. S-L enables OT educators to provide opportunities for students to gain critical skills that will translate into clinical practice.
- Published
- 2023
36. Effects of EMI-CLIL on Secondary-Level Students' English Learning: A Multilevel Meta-Analysis
- Author
-
Lee, Jang Ho, Lee, Hansol, and Lo, Yuen Yi
- Abstract
This meta-analysis synthesized the effects of the English medium instruction and content and language integrated learning (EMI-CLIL) approach on secondary-level students' English learning. The dataset included 44 samples (N = 7,434) from 38 primary studies. The results revealed EMI-CLIL's overall effectiveness for the development of English competence compared to the mainstream condition in the short term (d = 0.73, SE = 0.06, 95% CI [0.61, 0.86]) and longer term (d = 1.01, SE = 0.06, 95% CI [0.88, 1.15]). Additionally, we found that EMI-CLIL's overall effectiveness was influenced by several moderator variables. Its effectiveness was significantly: (1) higher for learners whose first language (L1) was linguistically related to English; (2) lower for primary studies which confirmed the homogeneity of the EMI-CLIL and comparison groups; (3) lower when studies targeted the productive (rather than receptive or overall) dimension of English learning; and (4) higher when outcome measures focused on vocabulary. Implications for pedagogy and future research are discussed.
- Published
- 2023
37. Longitudinal Study of Georgia's Pre-K Program. Final Report: Pre-K through 4th Grade
- Author
-
University of North Carolina at Chapel Hill, FPG Child Development Institute, Soliday Hong, S., Zadrozny, S., Walker, J., Love, E. N. G., Osborne, J. D., Owen, J. L., and Peinser-Feinberg, E.
- Abstract
The purpose of this evaluation study was to examine children's longitudinal academic and social outcomes associated with attendance in Georgia's Pre-K Program and to examine the quality of the classrooms attended. This report covers pre-k to 4th grade outcomes collected between the 2013-2014 and 2018-2019 school years as well as a sub-study comparing 3rd and 4th grade outcomes of children who attended Georgia's Pre-K Program with children who did not attend any pre-k program. The primary evaluation questions include: (1) What are the longitudinal outcomes through 4th grade for children who attended Georgia's Pre-K Program?; (2) What factors predict better longitudinal outcomes for children?; (3) What is the quality of children's instructional experiences from pre-k through 4th grade?; and (4) Are there differential long-term outcomes for children with and without Georgia's Pre-K experience? [For the summary report, see ED630838.]
- Published
- 2023
38. 'I Have a Few Questions': Reframing Assessment Practice as Asking and Answering Questions That Matter
- Author
-
Finney, Sara J.
- Abstract
Student affairs educators are asked fundamental questions about programming and its effectiveness. Stakeholders (e.g., students, parents, accreditors) ask what programming (e.g., activities, strategies, curriculum) is offered, "why" it is offered, and "who" benefits in terms of learning and development (e.g., Carpenter, 2001; US Department of Education, 2006). Given these questions are typical and expected, I illustrate how outcomes assessment can be represented as a process of answering common and pertinent questions that matter in higher education. In turn, the assessment process is presented as a valued activity to student affairs educators, not something novel or an add-on. Moreover, a question-answering approach has been shown to be less controlling than direct appeals (Walton & Wilson, 2018), prompting subsequent task engagement (Wood et al., 2016). Therefore, processing assessment-related questions should prompt engagement in outcomes assessment.
- Published
- 2023
39. An AI Generated Test of Pragmatic Competence and Connected Speech
- Author
-
Stefan O'Grady
- Abstract
Language testing is witnessing increasing interest in the potential for AI to support test development and validation. To date, published research involving AI in language testing has typically been conducted in the context of high-stakes proficiency tests and the potential for this technology to support local language testing is under-researched. The current study seeks to address this gap by reporting on the piloting of an AI generated language test in the context of a university in the UK. The focus of the paper was selected to explore two key areas in the work of J.D. Brown, namely pragmatics and connected speech. In the study, international students with English as a second language completed an AI generated test of pragmatics created from a transcript of spontaneous interaction, and a test-taking strategy questionnaire. Results demonstrated that the test did not provide a sufficiently reliable measurement of test takers' pragmatic competence, although reliability estimates did improve with the removal of problematic items. Implications for language test development involving AI are discussed.
- Published
- 2023
40. E-Learning Performance Assessment Model Proposal for E-Learning Academies
- Author
-
Yahya Yilmaz and Mutlu Tahsin Üstündag
- Abstract
Corporates provide e-learning systems to their employees to improve their knowledge and competences needed in job descriptions. This study aims to propose a model that measures and analyzes real user data in e-learning system to measure the targeted personal development and learning level of employees of corporates with elearning academies according to different criteria, compare the success level, and evaluate the effect of training on job performance. Employees' training activities, departments, positions, assessment and survey results, and other related data are recorded in the e-learning system and collected from the e-learning system, LMS (Learning Management System), by data mining method. The document also reviews the System Approach, Kirkpatrick's Four Levels of Training Evaluation Model, Balanced Scorecard, KPI (Key Performance Index), and OKR (Objectives and Key Results). In order to make a performance assessment in the model, data collected from e-learning systems is used, and the academy enters its own target data into the model. The results are associated with the corporate's KPI and OKR targets in the model. Model output is visualized for management review. The results declare that the model helps the academy have a holistic perspective for training activities associated with corporate target, a realistic review of effects of training on job performance, and possible opportunities and plans for future development of the trainings. [This article includes an extended summary in Turkish.]
- Published
- 2023
41. In Need of a Research Base: Evidence-Based Reading Interventions for Elementary Students with Overlapping EBD and LD
- Author
-
Burke, Mack D., Boon, Richard T., Bowman-Perrott, Lisa, and Hatton, Heather
- Abstract
Students with or at-risk of emotional/behavioral disorders (EBD) often have reading difficulties and/or undiagnosed learning disabilities (LD). Reading challenges among this group of children and youth often exacerbate associated emotional and/or behavioral problems. This systematic review and quantitative synthesis yielded seven studies focused on improving the reading outcomes of students with or at-risk of EBD at the elementary school level. Summarized are participant and reading intervention characteristics across the seven studies. Effect sizes were calculated for each study, and results are reported for both academic and behavioral outcome measures. Implications for research and practice are discussed.
- Published
- 2023
42. Development and Evaluation of Impact Statements for the Expanded Food and Nutrition Education Program (EFNEP)
- Author
-
Kylie Pybus, Ronald L. Gibbs, Karen Franck, and M. Catalina Aragón
- Abstract
Extension professionals often communicate program outcomes to external stakeholders using impact statements. We developed and evaluated four impact statements for the Expanded Food and Nutrition Education Program (EFNEP). We drafted the statements after conducting literature reviews for core content areas of EFNEP that include diet quality, food resource management, physical activity, and food safety. Subsequently, we evaluated the statements by facilitating expert panels made up of subject matter experts and communication professionals(n=14) from 12 Land-grant Universities. These impact statements aim to support EFNEP and other Extension professionals when communicating program value with key external stakeholders.
- Published
- 2023
43. Improving Noncognitive Constructs for Career Readiness and Success: A Theory of Change for Postsecondary, Workplace, and Research Applications. Research Report. ETS RR-22-04
- Author
-
Williams, Kevin M., Martin-Raugh, Michelle P., and Lentini, Jennifer E.
- Abstract
Researchers and practitioners in postsecondary and workplace settings recognize the value of noncognitive constructs in predicting academic and vocational success but also perceive that many students or employees are lacking in these areas. In turn, there is increased interest in interventions designed to enhance these constructs. We provide an empirically informed theory of change (ToC) that describes the inputs, mechanisms, and outputs of noncognitive construct interventions (NCIs). The components that inform this ToC include specific relevant constructs that are amenable to intervention, intervention content and mechanisms of change, methodological considerations, moderators of program efficacy, recommendations for evaluating NCIs, and suggested outcomes. In turn, NCIs should provide benefits to individuals, institutions, and society at large and also advance our scientific understanding of this important phenomenon.
- Published
- 2022
44. Literacy Footprints Guided Reading System: Effects on Grade 3 English Language Arts Assessment Performance
- Author
-
McREL International, Rentz, Bradley, Merth, Robert, and Dey, Darienne
- Abstract
Pioneer Valley Books (PVB) contracted McREL International to investigate the effects of the third grade Literacy Footprints (LFP) guided reading system on English language arts (ELA) performance on state assessments. LFP is a research-based, comprehensive, small-group literacy tool that facilitates explicit and systematic guided reading instruction to students in grades K-6. McREL researchers conducted a matched pair cluster randomized control trial to investigate LFP's possible effects on student achievement. The trial, conducted during the 2021/22 school year, included a sample of 3,071 students in 29 public elementary schools in a large suburban Florida school district. The district purchased LFP kits for grade 3 teachers in 14 elementary schools (SY2021/22) that were randomly assigned to the treatment group. Treatment teachers also obtained access to the Digital Reader and received virtual training and follow-up support to implement materials with students. Business-as-usual (BAU) instruction occurred at 15 schools (the control group). After the trial was completed, control group schools received the kits in the 2022/23 school year with access to the Digital Reader. Using a design-based estimator that accounts for both school-level clustering and school-matched pairs, the study found a statistically significant difference in the average English language arts standardized tests scores between treatment and control groups. By the end of the study, the treatment group students on average earned a 4.31-point higher scale score than the control group students. Results showed that, despite implementation challenges and limitations, students who received the treatment outperformed their control group peers on the Florida Statewide Assessment in English language arts. However, although statistically significant, the effect size (Hedges' g = 0.19) is small. This study showed promising evidence for the LFP guided reading system. Further research is needed to examine the generalizability of these results across different school districts and contexts and under more favorable conditions (e.g., without frequent COVID disruptions). [Faith Connelly and Paul Burkander contributed to this report. Lisa M. Jones and Sheila A. Arens co-led the project.]
- Published
- 2022
45. Next-Generation Fraction Intervention and the Long-Term Advantage of Interleaved Instruction
- Author
-
Fuchs, Lynn S., Malone, Amelia S., Preacher, Kristopher J., Cho, Eunsoo, Fuchs, Douglas, and Changas, Paul
- Abstract
This study's 1st purpose was to investigate effects of a 4th- and 5th-grade "next-generation" fraction intervention, which included 6 enhancements over a previously validated fraction intervention, designed to address Career- and College-Readiness standards. The next-generation intervention is referred to as "Super Solvers." The study's 2nd purpose was to assess effects of the next-generation fraction intervention at follow-up, 1 year after intervention ended. The 3rd purpose was to isolate the effects of 1 of the 6 intervention enhancements: interleaved fraction calculations instruction. Students with intensive intervention needs were randomized to next-generation fraction intervention ("Super Solvers": SSINT) with blocked calculations instruction (SSINT_B), SSINT with interleaved calculations instruction (SSINT_I), and control. On a mix of proximal and transfer outcomes, SSINT (across) conditions produced strong, significant effects over control at posttest. At follow-up, effect sizes were weaker but remained significant on calculations: g = 1.22. On other measures, follow-up g was 0.39-0.58. The effect of SSINT_I over SSINT_B, although not significant at posttest (g = 0.28), was statistically significant and large at follow-up (g = 0.65), in line with the cognitive science literature showing long-term advantages for interleaved instruction. Results suggest next-generation fraction intervention efficacy for intensive needs students and the importance of interleaved instruction. [This paper will be published in "Exceptional Children."]
- Published
- 2022
46. Part IV: Working with Pell Grant Completion Data. Pell Access and Completion Series
- Author
-
The Institute for College Access & Success (TICAS) and Hillman, Nick
- Abstract
The Pell Grant is targeted to students with the greatest financial need, and many policymakers find the Pell Grant a useful policy lever for promoting access and success for students from lower- and moderate-income backgrounds. There are several examples where policymakers use Pell eligibility to allocate resources and hold colleges accountable for their outcomes; however, basic information about Pell access and completion is not well documented in policy research. To address this problem -- and to provide baseline statistics useful in policy conversations -- this analysis provides information about the data sources used throughout the creation of the series and seeks to highlight potential improvements on data collection in the field. The fourth and final piece of TICAS' series on Pell Grant access and completion, this brief presents an overview of reporting process for the whole series. It reviews the use of College Scorecard and IPEDS Outcome Measures (OM), followed by a review of missing data, and concludes with a comparison of completion rates using both data sources. [For Part I, see ED623192. For Part II, see ED623193. For Part III, see ED623194.]
- Published
- 2022
47. The Utah Gender Presentation Scale for Communication: Development and Validation of a Measure of Voice-Related Gender Incongruence
- Author
-
Brett R. Myers, Jenny L. Pierce, and Pamela Mathy
- Abstract
Purpose: The aim of the present study was to develop and validate a patient-reported outcome measure for gender incongruence specific to voice and communication parameters, including pitch, intonation, resonance, loudness, speech smoothness, speech clarity, word choice, facial expression, gesture, and posture. Method: The Utah Gender Presentation Scale for Communication (U-GPS) includes 10 items, each on a 10-point scale from masculine to feminine. Items were selected based on literature review and patient focus groups. During test administration, respondents provide their current rating and goal rating for each item. The difference between these scores is used to calculate an overall incongruence score. Prospective data from transgender and gender-diverse (n = 155) and cisgender (n = 69) individuals were then used for a multiparametric psychometric evaluation of the measure. Results: Findings demonstrate excellent levels of internal consistency (Cronbach's alpha for current scores: [Alpha] = 0.943; Cronbach's alpha for goal scores: [Alpha] = 0.970), test-retest reliability (intraclass correlation coefficient = 0.905), longitudinal validity (improvement in therapy for trans women: F = 293.0, p < 0.001; nonbinary folx: F = 80.9, p < 0.001), concurrent validity (correlation with the Trans Woman Voice Questionnaire: r = 0.51, p < 0.001), and known-group expectations (differences among five gender groups: F = 82.7, p < 0.001). Conclusion: The U-GPS is a meaningful measure of voice-related gender incongruence, which is clinically relevant for assessing goals in gender-affirming voice and communication therapy for individuals across the gender spectrum.
- Published
- 2024
- Full Text
- View/download PDF
48. Measuring Youth Development: How Out-of-School Time Programs Collect and Use Data
- Author
-
Child Trends, Hannah Lantos, Zakia Redd, Jessica Warren, Michael Bradley, and Sham Habteselasse
- Abstract
Out-of-school-time (OST) programs and their funders rely on sound data to make decisions about everything from professional development and student recruitment to the selection of activities to offer students. Programs operate at a range of times (before and after school, weekends, summer) and in a variety of locations (e.g., schools, community-based organizations, city parks and recreation centers), are run by a variety of entities (e.g., government agencies, private community organizations), and receive funding from a variety of sources (e.g., government, philanthropy)--each of which may be interested in a different set of data and come with its own reporting requirements. This means there is a great deal of variation in the types of data programs collect. In 2019, The Wallace Foundation (Wallace) commissioned Child Trends to conduct a study of the kinds of youth outcomes OST programs are interested in measuring, the tools they use to measure those outcomes, and the challenges they experience in doing so. The study included a literature scan and interviews with leaders and staff members at 28 OST programs. Twelve of the 28 also completed surveys; a separate group of 10 provided information by survey only. The study expands on past research by a) focusing on programs that work in specific content areas (e.g., the arts, civic engagement and social justice, career and workforce development) and b) covering both "quantitative" approaches (i.e., tracking numerical data) and "qualitative" approaches (gathering descriptive information through surveys, interviews, etc.) to data collection. While its findings apply to OST programs in general, the study focused on particular types of programs (i.e., afterschool, summer, online) and particular content areas, as well as programs that serve school-age children and adolescents from marginalized communities, those that support students' social and emotional learning (SEL), those that serve systems-involved youth, and those that focus on promoting equity--for example, by training staff to recognize and overcome personal biases or by recruiting and retaining leaders and staff who reflect the diversity of the participants served.
- Published
- 2024
- Full Text
- View/download PDF
49. The Impact of Formative Assessment on K-12 Learning: A Meta-Analysis
- Author
-
Yuankun Yao, Michelle Amos, Karrie Snider, and Terrell Brown
- Abstract
To facilitate a more definitive understanding of the usefulness of formative assessment, this meta-analysis examined the impact of formative assessment on student academic achievement in the K-12 classroom. The study analyzed 258 effect sizes from 118 primary studies published around the world. By applying a broad lens for defining and understanding formative assessment, the study confirmed the usefulness of formative assessment. The overall effect size was 0.25 (Hedges' g) for all studies included, and 0.22 for studies conducted within the US. The effect size was similar across different types of formative assessments (e.g., self-assessment vs peer assessment) and content domains (e.g., English language, mathematics, science, etc.). The mean effect size was significantly larger for studies conducted outside North America and Western Europe, for formative assessments for high school students, and for studies using the quasi-experimental design. No publication bias was detected for studies conducted in North America and Western Europe.
- Published
- 2024
- Full Text
- View/download PDF
50. The Relationship between Classroom Racial/Ethnic Composition and Teacher Perceptions of Work Conditions
- Author
-
Fabian Barch and Hua-Yu Sebastian Cherng
- Abstract
Purpose: Prior research reveals that teachers have lower job satisfaction when they have more Black students, but this work does not consider how different aspects of work conditions - and the increasing diversity of students beyond a Black/White binary - may matter. This study aims to examine the relationship between teachers' perceptions of work conditions and the racial/ethnic compositions of the classes they teach. Design/methodology/approach: This study leverages data from the Measures of Effective Teaching (MET) study. Analyses consist of both descriptive statistics and multilevel regression modeling. Findings: Findings show that teachers' satisfaction with working conditions varies in relation to the racial/ethnic composition of their students. Increase in the percent of Black, Latinx and Asian American students in a teacher's classroom, was associated with a decrease in satisfaction with community involvement and student behavior. For increase in Latinx and Asian American students, this study finds significant decrease in satisfaction with measures of pedagogical and job support. For measures of school leadership and responsiveness to professional development needs, this study sees no significant relationship, which suggests that perceptions of some working conditions are more strongly tied to classroom demographics than others. Originality/value: This work adds nuance to previous research on teacher job satisfaction by exploring satisfaction with various working conditions and how it varies in relation to classroom racial composition. This study concludes with a discussion of potential explanations for observed differences in teacher satisfaction, as well as potential ways to address these differences.
- Published
- 2024
- Full Text
- View/download PDF
Catalog
Discovery Service for Jio Institute Digital Library
For full access to our library's resources, please sign in.