27,675 results on '"PREDICTIVE validity"'
Search Results
2. Synthesizing Validity and Reliability Evidence for the Draw-A-Scientist Test
- Author
-
Julia Brochey-Taylor and Joseph A. Taylor
- Abstract
The purpose of this synthesis study was to assess the reliability and validity of the Draw-A-Scientist Test (DAST) and its variations across multiple studies, aiming to understand limitations and propose modifications for future application within and beyond the science domain. Given the existence of multiple DAST versions, this study quantified the frequency of validity threats across various DAST variations. Literature review results indicated that despite its widespread use, the DAST and its variations consistently encounter challenges related to construct validity and external validity. Additionally, this synthesis identified literature limitations in testing concurrent validity, predictive validity, and inter-rater reliability when applicable.
- Published
- 2024
3. The Validity and Reliability Study of the Theory of Mind Inventory-2 (TOMI-2) Turkish Version
- Author
-
Canan Keles Ertürk and Kezban Tepeli
- Abstract
This study aims to conduct the Turkish adaptation, validity, and reliability study of the Theory of Mind Inventory-2 (TOMI-2) developed by Hutchins and Prelock (2016) for 3-5-year-old children. The study group consists of 310 mothers with children in the 3-5 age group in Konya city center. Personal Information Form and Theory of Mind Inventory-2 (TOMI-2) were used as data collection tools in the study. After the TOMI-2 was translated into Turkish, the normality assumption was checked with the "Shapiro-Wilk" test. The relationship between two continuous variables was evaluated with the Pearson Correlation Coefficient. Exploratory Factor Analysis, Confirmatory Factor Analysis, Content Validity, Criterion Validity, and Reliability analyses were also used in the study. The findings of the analyses show that the Turkish version of the TOMI-2 is a valid and reliable measurement tool for children aged 3-5, with 60 items in the original form.
- Published
- 2024
4. The Difference between Estimated and Perceived Item Difficulty: An Empirical Study
- Author
-
Ayfer Sayin and Okan Bulut
- Abstract
Test development is a complicated process that demands examining various factors, one of them being writing items of varying difficulty. It is important to use items of a different range of difficulty to ensure that the test results accurately indicate the test-taker's abilities. Therefore, the factors affecting item difficulty should be defined, and item difficulties should be estimated before testing. This study aims to investigate the factors that affect estimated and perceived item difficulty in the High School Entrance Examination in Türkiye and to improve estimation accuracy by giving feedback to the experts. The study started with estimating item difficulty for 40 items belonging to reading comprehension, grammar, and reasoning based on data. Then, the experts' predictions were compared with the estimated item difficulty and feedback was provided to improve the accuracy of their predictions. The study found that some item features (e.g., length and readability) did not affect the estimated difficulty but affected the experts' item difficulty perceptions. Based on these results, the study concludes that providing feedback to experts can improve the factors affecting their item difficulty estimates. So, it can help improve the quality of future tests and provide feedback to experts to improve their ability to estimate item difficulty accurately.
- Published
- 2024
5. Measures of Economic Disadvantage Explain Outcomes Differently across Geographies
- Author
-
Robin Clausen
- Abstract
Alternative poverty measures have been proposed in response to the emerging insufficiencies of the National School Lunch Program (NSLP) eligibility data. The analysis presented here involves seven poverty measures. Using outcome measures as a yardstick, we can assess how poverty measures explain these outcomes and note variations between geographical locales (assessing predictive validity). An analysis of 2019 data from Montana revealed that no poverty measure emerges as consistently meeting or exceeding the results found with the NSLP on the state level. Results are mixed based on locale (size) and distance from an urban centre, and within school communities.
- Published
- 2024
6. Development of a Scoring Key to Evaluate the Creative Story Writing Levels of Secondary School Seventh Grade Students
- Author
-
Ebru Öztürk and Erol Duran
- Abstract
In this study, it was aimed to develop a rubric to evaluate the creative story writing skill levels of seventh grade secondary school students. The research was designed in quantitative research method and survey model. In the research, convenience sampling technique was used and 270 students studying at the seventh grade level of secondary school were studied. In the process of creating the item pool of the research, 11 academicians, 8 teachers and 7 graduate students were consulted. In order to create an item pool, firstly, a literature review was conducted. The prepared item pool was presented to the field experts and the draft scale was applied to the sample group. As a result of the findings obtained, it was concluded that there was a high level of consistency between the evaluations made by different raters. The validity studies of the scale were conducted and it was determined that the content and criterion validity were appropriate. In order to determine the reliability of the scale items, Cronbach's Alpha value was determined. It was determined that the Cronbach's Alpha value of the scale with all sub-dimensions was above 0.7. Cohen's Kappa statistic ([kappa]) was used to ensure internal consistency between raters (p<0.05). As a result of the analyses, it was determined that there was almost perfect agreement in the items in the scale. Finally, Cohen's Kappa coefficient was determined to determine whether there was agreement between the raters, and a significant level of agreement was found between the raters of the scale. In addition, as a result of the results obtained from the expert opinions, it was decided that the rubric should consist of 6 dimensions and 24 items in total, namely "imagination", "originality", "different perspective", "content", "language and expression", "form", and five levels, namely "strongly agree", "agree", "partially agree", "disagree" and "strongly disagree".
- Published
- 2024
7. Predictive and Mediating Effects of Learning Strategies and Styles on Chinese Undergraduate Students' English Achievement
- Author
-
Meihua Liu and Zhangwei Chen
- Abstract
The present large-scale quantitative study investigated the predictive effects of learning strategies and styles on Chinese undergraduate students' English achievement and their mediating effects on each other's relation to English achievement. A total of 439 students from different universities answered the 30-item Perceptual Learning Style Preference Questionnaire and 50-item Strategy Inventory for Language Learning as well as a background questionnaire. Analyses of the data revealed the following major findings: (a) Learning strategies were significantly positively related to learning styles, (b) both learning strategies and styles were significantly positively related to and predicted students' English achievement, and (c) learning strategies significantly mediated the relation between learning styles and English achievement. These findings clearly highlight the importance of learning strategies and styles in second/foreign language learning. Based on these findings, some implications are discussed.
- Published
- 2024
- Full Text
- View/download PDF
8. Measuring Mathematical Skills in Early Childhood: A Systematic Review of the Psychometric Properties of Early Maths Assessments and Screeners
- Author
-
Laura A. Outhwaite, Pirjo Aunio, Jaimie Ka Yu Leung, and Jo Van Herwegen
- Abstract
Successful early mathematical development is vital to children's later education, employment, and wellbeing outcomes. However, established measurement tools are infrequently used to (i) assess children's mathematical skills and (ii) identify children with or at-risk of mathematical learning difficulties. In response, this pre-registered systematic review aimed to provide an overview of measurement tools that have been evaluated for their psychometric properties for measuring the mathematical skills of children aged 0-8 years. The reliability and validity evidence reported for the identified measurement tools were then synthesised, including in relation to common acceptability thresholds. Overall, 41 mathematical assessments and 25 screeners were identified. Our study revealed five main findings. Firstly, most measurement tools were categorised as child-direct measures delivered individually with a trained assessor in a paper-based format. Secondly, the majority of the identified measurement tools have not been evaluated for aspects of reliability and validity most relevant to education measures, and only 15 measurement tools met the common acceptability thresholds for more than two areas of psychometric evidence. Thirdly, only four screeners demonstrated an acceptable ability to distinguish between typically developing children and those with or at-risk of mathematical learning difficulties. Fourthly, only one mathematical assessment and one screener met the common acceptability threshold for predictive validity. Finally, only 11 mathematical assessments and one screener were found to concurrently align with other validated measurement tools. Building on this current evidence and improving measurement quality is vital for raising methodological standards in mathematical learning and development research.
- Published
- 2024
- Full Text
- View/download PDF
9. Noncognitive Factors and Student Long-Run Success: Comparing the Predictive Validity of Observable Academic Behaviors and Social-Emotional Skills. EdWorkingPaper No. 22-657
- Author
-
Annenberg Institute for School Reform at Brown University, Jing Liu, Megan Kuhfeld, and Monica Lee
- Abstract
Noncognitive constructs such as self-efficacy, social awareness, and academic engagement are widely acknowledged as critical components of human capital, but systematic data collection on such skills in school systems is complicated by conceptual ambiguities, measurement challenges and resource constraints. This study addresses this issue by comparing the predictive validity of two most widely used metrics on noncogntive outcomes--observable academic behaviors (e.g., absenteeism, suspensions) and student self-reported social and emotional learning (SEL) skills--for the likelihood of high school graduation and postsecondary attainment. Our findings suggest that conditional on student demographics and achievement, academic behaviors are several-fold more predictive than SEL skills for all long-run outcomes, and adding SEL skills to a model with academic behaviors improves the model's predictive power minimally. In addition, academic behaviors are particularly strong predictors for low-achieving students' long-run outcomes. Part-day absenteeism (as a result of class skipping) is the largest driver behind the strong predictive power of academic behaviors. Developing more nuanced behavioral measures in existing administrative data systems might be a fruitful strategy for schools whose intended goal centers on predicting students' educational attainment.
- Published
- 2023
10. Digital SAT® Pilot Predictive Validity Study -- A Comprehensive Analysis of First-Year College Outcomes
- Author
-
College Board, Westrick, Paul A., Marini, Jessica P., Young, Linda, Ng, Helen, and Shaw, Emily J.
- Abstract
This pilot study examines digital SAT® score relationships with first-year college performance. Results show that digital SAT scores predict college performance as well as paper and pencil SAT scores, and that digital SAT scores meaningfully improve our understanding of a student's readiness for college above high school grade point average (HSGPA) alone. In this study, there was a 22% improvement in the prediction of college performance when the SAT and HSGPA were used together, instead of using the HSGPA alone. For STEM majors, the added SAT value was 38%. Similar results were found when the outcome examined was course credits earned in the first year, a metric for understanding student progress toward degree completion. Findings from this study show that the SAT remains a powerful tool for understanding students' readiness for college, for course placement and academic major field decisions, scholarship and honors program decisions, and identifying students who may need academic support.
- Published
- 2023
11. Predicting Achievement from WISC-V Composites: Do Cognitive-Achievement Relations Vary Based on General Intelligence?
- Author
-
Jacqueline M. Caemmerer, Stephanie Ruth Young, Danika Maddocks, Natalie R. Charamut, and Eunice Blemahdoo
- Abstract
In order to make appropriate educational recommendations, psychologists must understand how cognitive test scores influence specific academic outcomes for students of different ability levels. We used data from the WISC-V and WIAT-III (N = 181) to examine which WISC-V Index scores predicted children's specific and broad academic skills and if cognitive-achievement relations varied by general intelligence. Verbal abilities predicted most academic skills for children of all ability levels, whereas processing speed, working memory, visual processing, and fluid reasoning abilities differentially predicted specific academic skills. Processing speed and working memory demonstrated significant interaction effects with full-scale IQ when predicting youth's essay writing. Findings suggest generalized intelligence may influence the predictive validity of certain cognitive tests, and replication studies in larger samples are encouraged.
- Published
- 2024
- Full Text
- View/download PDF
12. Assessing the Predictive Validity of the Massachusetts Candidate Assessment of Performance
- Author
-
Bingjie Chen, James Cowan, Dan Goldhaber, and Roddy Theobald
- Abstract
We evaluate the predictive validity of the Massachusetts Candidate Assessment of Performance (CAP), a practice-based assessment of teaching skills that is now a requirement for teacher preparation program completion in Massachusetts. We find that candidates' performance on the CAP significantly predicts their in-service summative performance evaluations in their first 2 years in the teaching workforce, but it is not predictive of their value added to student test scores in these years. These findings suggest that the CAP captures aspects of candidates' skills and competencies that are better reflected in their future performance evaluations than by their impacts on student performance.
- Published
- 2024
- Full Text
- View/download PDF
13. An Empirical Comparison of Four Generalized Trust Scales: Test-Retest Reliability, Measurement Invariance, Predictive Validity, and Replicability
- Author
-
Blaine G. Robbins
- Abstract
The Stranger Face Trust scale (SFT) and Imaginary Stranger Trust scale (IST) are two new self-report measures of generalized trust that assess trust in strangers--both real and imaginary--across four trust domains. Prior research has established the reliability and validity of SFT and IST, but a number of measurement validation tests remain. Across three separate studies, I assess the test-retest reliability, measurement invariance, predictive validity, and replicability of SFT and IST, with the misanthropy scale (MST) and generalized social trust scale (GST) serving as benchmarks. First, tests of internal consistency, test-retest reliability, and longitudinal measurement invariance established that all four generalized trust scales were acceptably reliable, with SFT and IST yielding greater overall reliability than MST and GST. Second, tests of multiple group measurement invariance revealed that SFT and IST were equivalent across gender, race, education, and age groups, while MST and GST were non-equivalent across the same sociodemographic groups. Third, an investment game established the predictive validity of SFT and MST, with IST and GST yielding poor predictive validity. Fourth, tests of factor structure and measurement invariance indicated that all four generalized trust scales replicated across samples. The present findings bolster the validity, reliability, and measurement equivalence of SFT and IST, while illustrating the compromised validity and measurement non-equivalence of MST and GST. Implications for the measurement of generalized trust are discussed.
- Published
- 2024
- Full Text
- View/download PDF
14. Additional Evidence of Predictive Validity of SRSS-IE Scores with Elementary Students
- Author
-
Kathleen Lynne Lane, Wendy Peia Oakes, Mark Matthew Buckman, Nathan Allen Lane, Katie Scarlett Lane, Kandace Fleming, Rebecca E. Swinburne Romine, Rebecca L. Sherod, Chi-Ning Chang, and Emily Dawn Cantwell
- Abstract
In this article, we examined predictive validity of the newly defined "Student Risk Screening Scale--Internalizing and Externalizing" (SRSS-IE 9, with nine items) when used by teachers for the first time, with a sample of 24,535 elementary students from 74 schools in four U.S. geographic regions. Results indicated fall externalizing and internalizing latent factors as well as subscale scores (SRSS-E5 and SRSS-I4, respectively) from the SRSS-IE 9 predicted year-end behavioral outcomes (office discipline referrals, suspensions, and nurse visits), academic outcomes (reading and math scores and course failures), as well as referrals to special education for elementary students. SRSS-IE 12 subscale scores for externalizing (SRSS-E7) and internalizing (SRSS-I5) using the original 12-item version subscales were similarly predictive of these outcomes. We discussed educational implications, limitations, and directions for future inquiry.
- Published
- 2024
- Full Text
- View/download PDF
15. Single-Item Principal Stress and Coping Measures: Concurrent and Predictive Validity and Comparisons to Teacher Measures
- Author
-
James Sebastian, David Aguayo, Wenxi Yang, Wendy M. Reinke, and Keith C. Herman
- Abstract
The present study analyzed concurrent and predictive validity of single-item scales for assessing principal stress and coping. We examined concurrent and prospective relations among stress and coping single-items with principal job satisfaction, overall health, perceptions of school safety, and principal leadership self-efficacy. We also compared principals and teachers on their stress and coping levels using the same single-item scales. Consistent with the literature on teacher stress and coping, the correlations of principal coping with different outcomes-job satisfaction, overall health, leadership efficacy, and safety perceptions--were stronger in comparison to the correlations between principal stress and those same outcomes. In regression models with both stress and coping, only principal coping predicted concurrent and future principal job satisfaction and overall health, as well as change in those outcomes. Coping also predicted concurrent but not future perceptions of school safety. Stress and coping did not consistently predict concurrent or future measures of leadership self-efficacy. Last, we found that principals reported even higher levels of stress than the well-documented high levels reported by teachers. We discuss areas for further research and potential use of these measures.
- Published
- 2024
- Full Text
- View/download PDF
16. Maximizing Gateway English Throughput for International Students in the California Community Colleges: Understanding the Predictive Validity of Common ESL Tests
- Author
-
RP Group and Hayward, Craig
- Abstract
The RP Group's Multiple Measures Assessment Project (MMAP) produced this technical report as part of a series on how California's community colleges can ensure more English learners (ELs) successfully complete "gateway" English coursework -- courses that satisfy the English writing requirements for completion of an associate's degree as well as for transfer to a four-year college or university -- in a timely way. The report summarizes findings on how well information about international students' initial curricular pathway (English or ESL), level of placement in the pathway, and ESL placement test score(s) can predict their throughput in gateway English. It specifically focuses on the value of different commonly used ESL placement tests to this process, including the Combined English Language Skills Assessment (CESLA) and multiple types of ACCUPLACER tests. It begins with key findings from this research followed by a description of the research methodology. Then, it offers a detailed analysis of the predictive validity of different ESL placement tests for students' throughput. It concludes with high-level recommendations based on these findings. The report is intended to support the California Community Colleges Chancellor's Office (CCCCO) and other educators and policymakers involved in development education reform in their decision-making about ways to continue supporting the success and completion of English learners in the system.
- Published
- 2023
17. The Validity of Heart Rate Variability (HRV) in Educational Research and a Synthesis of Recommendations
- Author
-
Hyun Jin Kim, Yuyi Park, and Jihyun Lee
- Abstract
This study provides a comprehensive overview of methodological aspects when using heart rate variability (HRV) measures in educational research. Following PRISMA 2020 guidelines, we searched four databases for relevant studies published until March 5, 2024. From the 48 studies reviewed, we extracted data across three analytical categories: (1) area of study interest and participant populations, (2) data collection and analysis methodologies, and (3) the concurrent and predictive validity of HRV measurement for educational research. Study quality was evaluated using QualSyst assessment criteria. Most studies measured stress and enlisted undergraduate students as participants. Data were predominately collected using wearable devices, measuring HRV for durations of less than 30 min, and in varied contexts, including during exams, while learning, and in experiments. The parameters analyzed varied within both time and frequency domains. HRV data had a moderate level of concurrent validity as a measure of stress in an educational context. The concurrent validity of HRV data for measuring attention remains uncertain with insufficient evidence. Limited correlations appeared between stress and performance. The findings, potentials, and limitations of HRV measures are discussed, and synthesized recommendations for educational research using HRV data are provided.
- Published
- 2024
- Full Text
- View/download PDF
18. Does Knowledge of Results Affect Motor Skill Learning and Adaptation in Interception-Like Tasks?
- Author
-
Cláudio Manoel Ferreira Leite, Herbert Ugrinowitsch, and Crislaine Rangel Couto
- Abstract
Knowledge of results (KR), particularly its informational role, has often been regarded as redundant for learning interception-like tasks, such as coincidence-anticipation timing tasks. However, it is possible that the KR's guiding effect might be detrimental to motor adaptation, instead of only redundant, leading to a dependency on KR and steering the sensorimotor system away from relevant information of the task. In this study, we aimed to investigate KR's effect on learning a coincidence-anticipation timing tasks and on the adaptation to unpredictable perturbations. Two groups of participants practiced a coincidence-anticipation timing tasks with or without KR on 1 day and underwent testing the next day for learning (Retention test) and for adaptation to unpredictable perturbations (Exposure phase). Both groups exhibited similar learning results but failed to adapt to the perturbations, contradicting the assumption of negative guidance effects of KR and the positive effects of relying solely on intrinsic information (no KR). These findings suggest that motor adaptation may require specific information during the acquisition process, highlighting for more systematic analyses to understand this phenomenon better. Such insights could have practical implications in contexts like sports and rehabilitation, by providing learners with appropriate information for acquiring adaptive internal representations of tasks.
- Published
- 2024
- Full Text
- View/download PDF
19. University Teacher Credibility as a Strategy to Motivate Students
- Author
-
García, Alfonso Javier, Froment, Facundo A., and Bohórquez, María Rocío
- Abstract
University students' motivation can be affected by several factors, one being their perceptions of teacher behaviour in the classroom. This study aimed to predict university students' state motivation from their perceptions of teacher credibility. The participants were 344 students from the University of Seville. A structural equation model was used with the partial least squares method (PLS-SEM), a technique based on variance, employed to test, and validate the proposed hypotheses. The results reveal, on the one hand, a positive effect of teacher credibility on state motivation and, on the other hand, that teacher credibility has predictive power and predictive relevance for state motivation. Likewise, there is evidence of predictive validity in that teacher credibility can predict values for new cases of state motivation. Strategies are provided for a university faculty to manage their behaviour in the classroom to increase their students' state motivation, highlighting the use of PLS-SEM as a data analysis tool suitable for application in higher education.
- Published
- 2023
20. Construct Validation of the Revised 'Metacognitive Awareness of Reading Strategies Inventory (MARSI-R)' and Its Relation to Learning Effort and Reading Achievement
- Author
-
Alamer, Abdullah and Alsagoafi, Ahmad
- Abstract
Language students apply different strategies to learn a second language (L2), especially when they want to attain proficiency in reading. The aim of the present study was to revisit the validity of the "Metacognitive Awareness of Reading Strategies Inventory" (MARSI-R) among Saudi students using a new statistical method of confirmatory composite analysis (CCA). Past studies modeled MARSI-R as a "common factor" and applied confirmatory factor analysis (CFA) to test its validity. However, studies struggled to provide support for the validity of the MASRI-R with each suggesting different model. Instead, we treat the inventory as a "composite," meaning that the items in MARSI-R form and define the inventory and not the other way around. We use partial least squared structural equation modeling (PLS-SEM) to allow the composite model to be estimated. The results indicated that the constructs of MARSI-R are better operationalized as composites not common factors as supported through CCA exclusively. After confirming the nature of the inventory, we evaluated the extent to which MARSI-R is related to reading proficiency through the mediational mechanism of motivational intensity (i.e., learning effort). Descriptive statistics illustrated that problem-solving strategies are the most used strategy and that females used the strategies more frequently than their male counterparts. Most importantly, the structural model showed that metacognitive reading strategies only exert an indirect effect on reading proficiency, suggesting that the effect of strategies is mediated by motivational intensity (i.e., learning effort). Thus, motivational intensity seems to be mediator in the relationship between metacognitive reading strategies and reading proficiency. Finally, methodological and educational implications are provided.
- Published
- 2023
21. Azerbaijani Adaptation of the Perceived School Experience Scale: Examining Its Impact on Psychological Distress and School Satisfaction
- Author
-
Elnur Rustamov, Ulkar Zalova Nuriyeva, Malak Allahverdiyeva, Tahmasib Abbasov, and Narinj Rustamova
- Abstract
School experience holds significant importance in the realm of educational environments in the 21st century. Thus, the aim of this study was to adapt the Perceived School Experience Scale for application among Azerbaijani adolescents and examine the interconnections between school experience, psychological distress, and school satisfaction. Data were collected from 1095 Azerbaijani adolescents (635 female and 456 male). Throughout the adaptation process, confirmatory factor analysis, assessments of criterion-related validity, and reliability were conducted. Additionally, the associations between school experience, psychological distress, and school satisfaction were explored through mediational analysis. The results from the confirmatory factor analysis demonstrated that the questionnaire encompassed 14 items, featured three school-related protective factors, and displayed an acceptable fit. The scale has been found to have satisfactory reliability coefficients for both its factors and the overall score. Furthermore, the findings established that school experience exhibited positive correlations with school satisfaction while demonstrating negative associations with psychological distress. Finally, the results unveiled that psychological distress played a mediating role in influencing school satisfaction through the pathways of school experience. These findings collectively suggest that the Azerbaijani version of the Perceived School Experience Scale boasts commendable psychometric properties.
- Published
- 2023
22. Reliability and Validity of Methods to Assess Undergraduate Healthcare Student Performance in Pharmacology: Comparison of Open Book versus Time-Limited Closed Book Examinations
- Author
-
David Bell, Vikki O'Neill, and Vivienne Crawford
- Abstract
We compared the influence of open-book extended duration versus closed book time-limited format on reliability and validity of written assessments of pharmacology learning outcomes within our medical and dental courses. Our dental cohort undertake a mid-year test (30xfree-response short answer to a question, SAQ) and end-of-year paper (4xSAQ, 1xessay, 1xcase) in pharmacology. For our first year medical cohort, pharmacology is integrated within a larger course, contributing 20xclinical vignette questions (to select the single best answer (SBA) to each question from a choice of 5 plausible answers) to a mid-year test and 3-5xSAQ to an end-of-year paper. Our experience indicates that SAQ are as reliable as SBA for closed-book time-limited assessments; reliability correlates with number of questions employed. We have found good correlation between mid-year and end-of-year performance (predictive validity), between questions (factorial validity) and between pharmacology and other subjects within the assessment (concurrent validity). Adoption of open-book extended duration assessments resulted in only modest reduction in reliability and validity.
- Published
- 2023
23. Predictive Analytics in Education: Considerations in Predicting versus Explaining College Student Retention
- Author
-
Paterson, Kevin and Guerrero, Adam
- Abstract
Data from a moderately-selective state university in the Midwest is used to cross-examine the most appropriate data analytical techniques for predicting versus explaining college student persistence decisions. The current research provides an overview of the relative benefits of models specializing in prediction versus explanation with particular emphasis on estimation methodologies, model specification by estimation technique, and model diagnostics, including classification tables and measurements of the goodness of fit. The predictive validity of a model of college student retention estimated using logistic regression is compared with that of discriminant analysis estimated using cognitive and non-cognitive predictors of retention. Key contributions to the literature include a unique analysis sample, a unique set of independent variables, and statistical estimation methodologies that build upon traditional frameworks, including machine learning techniques. The currents study ends with a discussion that will allow leaders in higher education and policymakers to make better data-informed decisions surrounding prediction and explanation so that they can proactively intervene with the most appropriate attrition-minimization policies.
- Published
- 2023
24. An Analysis of the Relationship between Secondary Checkpoint and IGCSE Results
- Author
-
Gill, Tim
- Abstract
Secondary Checkpoint assessments are taken by students at the end of the Cambridge Lower Secondary programme (aged 14) in countries around the world. Many students continue with Cambridge after this and take IGCSE exams two years later. Given that there is a high level of coherence between the curricula in the two stages, performance in Secondary Checkpoint should be a good indicator of performance at IGCSE. In this article, I investigate whether there is evidence to support this contention, by calculating correlations between Checkpoint scores and IGCSE grades, across a range of subjects. I also look at whether students in schools offering the Cambridge Lower Secondary programme go on to perform better at IGCSE than schools not offering the programme.
- Published
- 2023
25. Generalizing Predictive Models of Reading Ability in Adaptive Mathematics Software
- Author
-
Husni Almoubayy, Stephen E. Fancsali, and Steve Ritter
- Abstract
Recent research seeks to develop more comprehensive learner models for adaptive learning software. For example, models of reading comprehension built using data from students' use of adaptive instructional software for mathematics have recently been developed. These models aim to deliver experiences that consider factors related to learning beyond performance in the target domain for instruction. We investigate the extent to which generalization is possible for a recently developed predictive model that seeks to infer students' reading comprehension ability (as measured by end-of-year standardized test scores) using an introductory learning experience in Carnegie Learning's MATHia intelligent tutoring system for mathematics. Building on a model learned on data from middle school students in a single school district in a mid-western U.S. state, using that state's end-of-year English Language Arts (ELA) standardized test score as an outcome, we consider data from a school district in a south-eastern U.S. state as well as that state's end-of-year ELA standardized test outcome. Generalization is explored by considering prediction performance when training and testing models on data from each of the individual school districts (and for their respective state's test outcomes) as well as pooling data from both districts together. We conclude with discussion of investigations of some algorithmic fairness characteristics of the learned models. The results suggest that a model trained on data from the smaller of the two school districts considered may achieve greater fairness in its predictions over models trained on data from the other district or both districts, despite broad, overall similarities in some demographic characteristics of the two school districts. This raises interesting questions for future research on generalizing these kinds of models as well as on ensuring algorithmic fairness of resulting models for use in real-world adaptive systems for learning. [This paper was published in: "Proceedings of the 16th International Conference on Educational Data Mining," edited by M. Feng et al., International Educational Data Mining Society, 2023, pp. 207-16.]
- Published
- 2023
26. Assessing the Potential to Teach. Predicting Later Classroom Practice from Assessment Centre Activities: Conclusions from a Four-Year Study. Research Report
- Author
-
Education Development Trust (United Kingdom), Richard Churches, Kate Wastie, Max Jones, and Colin Penfold
- Abstract
Teacher professional development is complex, and despite evidence from randomised controlled trials about key factors influencing this process, there is little research about how abilities prior to initial teacher training (ITT) influence early classroom practice. In England, many schools face teacher recruitment and retention challenges and finding ways to select candidates likely to find the challenges of teaching more aligned to their existing skills could have an important impact. The authors report the final analysis in a four-year study looking at the ability of the Future Teaching Scholars programme assessment centre to predict later practice. Building on previous reports, they present final conclusions about the approach's predictive validity and the standards of teaching achieved by participants at the end of their first term of teaching and conclude with lessons for policymakers.
- Published
- 2023
27. Are Students Who Take the Extended Project Qualification Better Prepared for Higher Education? Research Report
- Author
-
Cambridge University Press & Assessment (United Kingdom) and Gill, Tim
- Abstract
The Extended Project Qualification (EPQ) is available for students in Key Stage 5 (KS5), to be taken alongside other qualifications, such as A levels. It differs from most other academic qualifications at KS5 because it is not examined, but instead involves students undertaking an in-depth project in an area of their choosing. Students are required to plan the project, research and analyse sources of information, write up their analysis, draw conclusions and produce an evaluation of the processes involved. As such, it is promoted by exam boards as providing the skills required for university study or for work. The main purpose of this research was to investigate whether students taking EPQ were better prepared for higher education (HE) than students not taking it. The following questions were addressed: (1) Are EPQ students more likely than non-EPQ students to progress to HE (after accounting for other factors likely to affect this, such as prior attainment)? (2) Are EPQ students less likely than non-EPQ students to drop-out of HE (after accounting for other factors)? and (3) Is taking an EPQ associated with better degree performance (after accounting for other factors)? This investigation is of particular interest now due to the increase in uptake of the qualification in recent years: between 2008 and 2016, the number of students taking it increased from 1,706 to 38,548 (He & Black, 2018). By 2019, this had increased to 45,687 (Gill, 2022). The main source of data for this project was a dataset linking students between their records in the National Pupil Database (NPD) and in the Higher Education Statistics Agency (HESA) database.
- Published
- 2022
28. Predictive Validity of High School Grade Point Average before and after AB 705
- Author
-
RP Group, Fagioli, Loris, and Newell, Mallory
- Abstract
Implementation of AB 705 in fall 2019 required California community colleges to use high school grade point average (HSGPA) as the main measure of placement and maximize the likelihood of students being placed into transfer-level English and math courses. Since that time, research has shown that with the vast increase in access to transfer-level courses, the one-year completion rate of these courses (i.e., throughput) also increased. These completion outcomes exceeded prior throughput rates when students largely began in basic skills coursework. However, as throughput increased for all student groups, course success rates decreased at varying rates across colleges, causing some concern about the validity of high school transcript data as a predictor of success in transfer-level English and math courses. To further explore this issue, The RP Group's Multiple Measures Assessment Project (MMAP) research team examined the validity of HSGPA in predicting course grades in students' first community college course in either English or math when that course was at transfer level. The following brief reviews the methodology and highlights key findings and conclusions. Graphical displays of the results are included in the brief, while the full results of each model are included in the appendix. As this summary demonstrates, strong evidence continues to support HSGPA as a valid measure for community college placement into transfer-level English and math.
- Published
- 2022
29. Modeling the Interplay of EFL Learners' Basic Psychological Needs, Grit and L2 Achievement
- Author
-
Majid Elahi Shirvan and Abdullah Alamer
- Abstract
Given the recent attention to language-domain-specific grit in the field of SLA and the scarcity of research on the antecedents of L2 grit, we proposed a model that links L2 learners' basic psychological needs (BPN) (i.e. autonomy, competence, and relatedness), L2 grit (i.e. perseverance of effort (PE) and consistency of interest (CI)), and L2 achievement (measured by students' scores on English courses). L2 Saudi undergraduate students who study English were invited to participate. The findings indicated that except for the path from competence to achievement and from CI to achievement, all direct paths in the model were significant. More specifically, only PE, but not CI, was positively associated with L2 achievement. Also, all constructs of BPN have a positive influence on PE, but only autonomy has a positive link with CI. The findings illustrated plausible antecedents of L2 grit and showed pathways in which L2 grit operated as mediators in the association between BPN and L2 achievement. Using partial least squares structural equation modelling (PLS-SEM) to examine the mediational model allowed us to predict unseen scores on the outcome; thus, providing support for the out-of-sample predictive power (predictive validity) of our structural model.
- Published
- 2024
- Full Text
- View/download PDF
30. Predictive Relationships between Learning Management System Engagement Measures and Academic Performance in College
- Author
-
Ayad Saknee
- Abstract
Higher education institutes experience lower success rates in online learning environments compared to traditional learning. Students' engagement within the learning management system (LMS) is one of the main factors affecting students' academic performance and retention. This quantitative correlational-predictive study examined if, and to what extent students' engagement frequency (SEF) and students' engagement duration (SED) collectively and/or individually predicted students' academic performance (SAP) in business courses at one community college in Arizona. The theoretical framework for this study included engagement theory and a model of student engagement, prior knowledge, and academic performance. The research questions asked to what extent the number of times a student logged into the LMS and the time student spent logged into the LMS during the course on the learning management system predicted their academic performance in multiple business courses. A multiple linear regression was performed on deidentified archival data from a sample of 80 community college students who had enrolled in multiple business classes in the Fall of 2023. The dataset included the total number of student logins, total time each student spent on the entire course, and student final grade. The results showed a nonsignificant predictive relationship between the two predictors considered together and SAP, R[superscript 2] = 0.067, adj. R[superscript 2] = 0.043, F(2, 77) = 2.762, p = 0.069, Cohen's effect size f[superscript 2] = 0.0718 (midway between small and medium). The SEF was the only individually significant predictor in the model (std. [beta] = 0.258, t =2.280, p = 0.025), which could be enhanced through the course requirements structure and LMS tracking to improve learning. [The dissertation citations contained here are published with the permission of ProQuest LLC. Further reproduction is prohibited without permission. Copies of dissertations may be obtained by Telephone (800) 1-800-521-0600. Web page: http://www.proquest.com/en-US/products/dissertations/individuals.shtml.]
- Published
- 2024
31. How Predictive of Teacher Retention Are Ratings of Applicants from Professional References? CEDR Working Paper No. 03212024-1
- Author
-
University of Washington, Bothell. Center for Education Data & Research (CEDR), Dan Goldhaber, and Cyrus Grout
- Abstract
Turnover in the teacher workforce imposes significant costs to schools, both in terms of student achievement and the time and expense required to recruit and train new staff. This paper examines the potential for structured ratings of teacher applicants, solicited from their professional references, to inform hiring decisions through the selection of teachers who are less likely to turn over. Specifically, we analyze the predictive validity of reference ratings with respect to retention outcomes among subsequently employed applicants. We find that a summative reference ratings measure is modestly predictive of retention in a teacher's school, with a one-standard deviation change associated with a 3.2 percentage point increase in the probability of school retention. When we account for rater fixed effects, we find substantially stronger relationships between reference ratings and retention, with a one-standard deviation change in our summative ratings measure associated with an increase in the probability of school retention of 8.5 percentage points. These findings suggest that raters themselves are a large source of variation in the distribution of reference ratings. So, while we find predictive validity of professional ratings, their potential to inform good hiring decisions depends on, among other things, the ability of hiring managers to account for rater variation when interpreting references' assessments of applicants.
- Published
- 2024
32. A Meta-Analysis of the Relationship between Wonderlic Test Scores and School Success
- Author
-
Chet Robie, Sabah Rashe, Stephen D. Risavy, and Piers Steel
- Abstract
This meta-analysis examined the validity of an alternative to traditional assessments called the Wonderlic which is a brief measure of general mental ability. Our results showed significant, positive correlations between Wonderlic scores and academic performance in general ([r-bar] = 0.26), between Wonderlic scores and undergraduate GPA in particular ([r-bar] = 0.27, [p-bar] = 0.33), and between Wonderlic scores and retention ([r-bar] = 0.09, [p-bar] = 0.12). We also identified several significant moderators of the relationship between Wonderlic scores and relevant outcomes (e.g., test publisher reported coefficients were larger than those reported by other sources). Subgroup differences in test scores were in the same range as other post-secondary admissions assessments (e.g., ACT and SAT scores). Overall, the Wonderlic has similar levels of subgroup differences and is less strongly related to GPA than traditional assessments but still retains useful levels of predictiveness and is a shorter, less expensive assessment that requires less preparation than the ACT or SAT.
- Published
- 2024
- Full Text
- View/download PDF
33. The Parent-Report Childhood Executive Functioning Inventory (CHEXI) in Italian Preschoolers: Factor Structure, Reliability, and Criterion Validity
- Author
-
Mara Morelli, Matilde Brunetti, Antonio Chirumbolo, Pietro Spataro, Fiorenzo Laghi, and Emiddia Longobardi
- Abstract
Executive functions (EFs) are a set of high-level cognitive processes that enable children to perform a goal-directed behaviour. During the preschool years, EFs undergo significant developmental changes. Therefore, it is crucial to have reliable measures that accurately evaluate preschoolers' EFs in preschool-aged children. The present study aimed to investigate the factor structure of the Childhood Executive Functioning Inventory (CHEXI), replicating the expected two-factor model in Italian preschoolers. A total of 210 Italian parents completed an online survey on their children aged between 36 and 72 months (Mage = 54.12; SDage = 11.22; 44.3% girls). A confirmatory factor analysis confirmed the two-factor model of the CHEXI (working memory and inhibition) for Italian preschoolers, also demonstrating the invariance of the model across gender and age. Furthermore, the criterion validity of the CHEXI was established by examining correlations with the Children's Behavior Questionnaire. The present study provides evidence that the CHEXI is a valid and reliable rating measure for assessing EFs abilities of Italian preschoolers.
- Published
- 2024
- Full Text
- View/download PDF
34. Translation and Validation of the Chinese Version of the Problematic Media Use Measure
- Author
-
Juan Li, Jingyao Wang, Bowen Xiao, Yan Li, and Hui Li
- Abstract
"Research Findings": The Problematic Media Use Measure (PMUM) assesses young children's excessive or problematic media use. The current study aims to translate the original English version of PMUM into a Chinese version and validate it with Chinese preschoolers (ages 3 to 6). The instrument was translated, back-translated, pretested, and reviewed by a panel. The Chinese version of PMUM was administered to mothers of preschoolers (N = 1896). The results indicated that the construct validity of the PMUM was supported by the best-fit two-factor model: psychosocial problems and tolerance & withdrawal. Concurrent validity was established by demonstrating that: (1) the PMUM subscales scores were positively related to children's screen time and (2) the PMUM subscales scores were negatively related to parental mediation. The predictive validity of the PMUM subscales was also established: (1) the PMUM subscales scores were negatively correlated with prosocial behaviors, and (2) the PMUM subscales scores were positively correlated with emotional symptoms, hyperactivity/inattention, conduct problems, and peer relationship problems. Internal reliability and test-retest reliability were both satisfactory. "Practice or Policy": The findings support the further development and application of the PMUM to monitor children's media use and also emphasize the need for additional research on problematic media use.
- Published
- 2024
- Full Text
- View/download PDF
35. Screening for Depression: Psychometric Properties of the German Version of the Glasgow Depression Scale for People with a Learning Disability (Individual and Carer Version)
- Author
-
Müller, Katrin, Helmes, Almut, Kleischmann, Annika, Graser, Johannes, and Bengel, Jürgen
- Abstract
Background: Adults with intellectual disabilities should participate in the diagnostics of their mental health. The Glasgow Depression Scale for people with a Learning Disability (GDS-LD) and its Carer Supplement (GDS-CS) were translated into German and tested. Methods: Internal consistency, criterion validity and inter-test reliability were tested in 64 adults with borderline, mild or moderate intellectual disabilities and their carers. Convergent validity was analysed in 57 adults without intellectual disabilities. Results: Internal consistency was good ([alpha] = 0.81) for GDS-LD and acceptable ([alpha] = 0.72) for GDS-CS. The GDS-LD did not differentiate between groups with and without depression. The GDS-CS significantly differentiated between these groups. Convergent validity of the GDS-LD was high. The correlation of GDS-LD and GDS-CS was non-significant. Conclusions: In its present form the German version of the GDS-LD does not meet the psychometric properties to be used in clinical practice. This leads to the broader question, how to measure depression in people with learning disabilities with the knowledge of the fallibility of existing measures when utilised with this clinical cohort. Also, future studies need to investigate the role of self-rating.
- Published
- 2024
- Full Text
- View/download PDF
36. The Predictive Validity of the GRE across Graduate Outcomes: A Meta-Analysis of Trends over Time
- Author
-
David F. Feldon, Kaylee Litson, Brinleigh Cahoon, Zhang Feng, Andrew Walker, and Colby Tofel-Grehl
- Abstract
This meta-analysis assesses the predictive validity of the Graduate Record Examination (GRE) across outcome variables, including grade point average, for graduate students. In addition to aggregate effects, this paper also assessed changes in observed effects over time as related to increasing diversity in the graduate student population and as a function of gender and racial/ethnic composition of study samples. Framed using a lens of critical whiteness, this analysis examined n = 1,659 individual effects across k = 201 studies. Overall, 62.3% of reported effects were nonsignificant (i.e. no predictive value of GRE scores on student outcomes). Further, the magnitude of observed predictive relationships decreased significantly over time. The aggregate mean effect across all studies and outcomes was small, significant, and positive: GRE score predicted 3.24% of variance across measured outcomes, 4% of variance in overall GPA, and 2.56% of variance in first-year graduate GPA. Sample composition effects by race/ethnicity were notable under some conditions, but nonsignificant, with increasing proportions of people of Color within a study sample associated with poorer predictive validity for GPA. Likewise, the magnitude of negative effects where lower GRE scores predicted stronger student outcomes showed increasing trends from 0.16% of variance for all-white samples to 7.3% for samples comprised entirely of people of Color.
- Published
- 2024
- Full Text
- View/download PDF
37. An Evaluation of the Forecasting Accuracy of Two Trend Estimation Methods from Two Computer Adaptive Tests
- Author
-
Ethan R. Van Norman and Emily R. Forcht
- Abstract
This study evaluated the forecasting accuracy of trend estimation methods applied to time-series data from computer adaptive tests (CATs). Data were collected roughly once a month over the course of a school year. We evaluated the forecasting accuracy of two regression-based growth estimation methods (ordinary least squares and Theil-Sen). The precision, or accuracy, of predictions were heavily influenced by how far performance was forecasted into the future (1, 3, or 5 months) and the number of observations available to estimate growth (3, 4, or 5). When performance was forecasted further and growth estimates were based upon fewer observations, predictions were off by as much as two times the average conditional standard error of measurement for a grade level. No combinations of data collections schedule, forecasting length, or estimation method led to consistent bias in predictions. Educators should be cautious when using trend-line decision rules to predict future performance. Suggestions for alternate decision rules to explore with CATs are offered.
- Published
- 2024
- Full Text
- View/download PDF
38. Studying Between-Subject Differences in Trends and Dynamics: Introducing the Random Coefficients Continuous-Time Latent Curve Model with Structured Residuals
- Author
-
Julian F. Lohmann, Steffen Zitzmann, and Martin Hecht
- Abstract
The recently proposed "continuous-time latent curve model with structured residuals" (CT-LCM-SR) addresses several challenges associated with longitudinal data analysis in the behavioral sciences. First, it provides information about process trends and dynamics. Second, using the continuous-time framework, the CT-LCM-SR can handle unequally spaced measurement occasions and describes processes independently of the length of the time intervals used in a given study. Third, it is a hierarchical model. Thus, multiple subjects can be analyzed simultaneously. However, subjects might also differ in dynamics and trends. Therefore, in the present paper, we extend the CT-LCM-SR to capture these differences as well. This newly proposed "random coefficients continuous-time latent curve model with structured residuals" (RC-CT-LCM-SR) is introduced theoretically and technically. Additionally, we provide an illustrative example with data from the Health and Retirement Study (HRS), and we show how the RC-CT-LCM-SR can be used to study multiple sources of between-subject differences over time.
- Published
- 2024
- Full Text
- View/download PDF
39. The Multiple-Stimulus-Without-Replacement Preference Assessment Tool and Its Predictive Validity
- Author
-
Hugo Curiel, Emily S. L. Curiel, Santos Villanueva, Carlos Eduardo Garza Ayala, and Alexander S. Cadigan
- Abstract
This study demonstrates the use of two web-based programs, one to identify video preferences and the other to assess their reinforcing effects. We used the Multiple-Stimulus-Without-Replacement Preference Assessment Tool (MSWO PAT) to identify the video preference hierarchies of seven participants, ages 4-11 years old. We then used a customized reinforcer assessment program that arranged a concurrent-chains preparation with programmed conjugate schedules of reinforcement. Button presses emitted by participants modulated the quality (volume and opacity) of selected videos on a moment-to-moment basis, allowing us to identify the reinforcing effects of the videos in little time. The results showed that the preference assessment had predictive value for five of seven participants. We discuss the MSWO PAT, parameters that may affect the identification of preferences and the use of conjugate schedules to identify reinforcers.
- Published
- 2024
- Full Text
- View/download PDF
40. Predicting Language Use in Catalonia (Spain): The Role of Implicit and Explicit Language Attitudes
- Author
-
Maria Adelina Ianos, Ursula Hinostroza-Castillo, Fernando Senar, and Cristina Petreñas
- Abstract
This study examines the relationship between language attitudes and language use from a social cognition perspective in the context of Catalonia, a culturally and linguistically diverse Spanish Autonomous Community that has been implementing social and educational measures to promote the use of its own language, Catalan. We examined how implicit and explicit measures of attitudes towards Catalan and Spanish predict self-reported language use and spontaneous language choice. Participants (N = 98) completed an Implicit Association Test (IAT), self-reports of language attitudes and use, and a spontaneous language choice. Although both implicit and explicit measures correlated with the behavioural measures, the results showed that the IAT had predictive and incremental validity. Moreover, a partial dissociation pattern of prediction was identified, underscoring the importance of including implicit measures of language attitudes, as they predict variance in behaviour above and beyond what is predicted by explicit measures.
- Published
- 2024
- Full Text
- View/download PDF
41. One Score to Rule Them All? Comparing the Predictive and Concurrent Validity of 30 Hearts and Flowers Scoring Approaches
- Author
-
Tiffany Wu, Christina Weiland, Meghan McCormick, JoAnn Hsueh, Catherine Snow, and Jason Sachs
- Abstract
The Hearts and Flowers (H&F) task is a computerized executive functioning (EF) assessment that has been used to measure EF from early childhood to adulthood. It provides data on accuracy and reaction time (RT) across three different task blocks (hearts, flowers, and mixed). However, there is a lack of consensus in the field on how to score the task that makes it difficult to interpret findings across studies. The current study, which includes a demographically diverse population of kindergarteners from Boston Public Schools (N = 946), compares the predictive and concurrent validity of 30 ways of scoring H&F, each with a different combination of accuracy, RT, and task block(s). Our exploratory results provide evidence supporting the use of a "two-vector average" score based on Zelazo et al.'s approach of adding accuracy and RT scores together only after individuals pass a certain accuracy threshold. Findings have implications for scoring future tablet-based developmental assessments. [This is the online version of an article published in "Assessment."]
- Published
- 2024
- Full Text
- View/download PDF
42. Predictive Validity Study of Sustainable Leadership for Learning Questionnaire
- Author
-
Ishak, Marni and Hussin, Fauzi
- Abstract
The ultimate goal of this study is to test the validity of the newly developed Sustainable Leadership for Learning Questionnaire (SLLQ) in terms of predictive validity on teacher satisfaction towards teaching. Apart from that this study also to test the convergent and discriminant validity of SLLQ. To achieve the objective of the study, a cross-sectional survey design was utilized. A total of 190 teachers from national primary schools in Kedah, Malaysia was randomly selected to participate in this study. The Teaching Satisfaction Scale (TSS) were used to measure teachers' satisfaction as criterion variable. Using PLS-SEM analysis, this study found that the Sustainable Leadership for Learning Questionnaire (SLLQ) is a valid questionnaire to predict teachers' satisfaction towards teaching. This study has contributed to the establishment of newly constructed questionnaire namely SLLQ and also to the development of leadership literature in the context of primary school in Malaysia.
- Published
- 2022
43. Predicting Language Performance from Narrative Language Samples
- Author
-
Murphy, Kimberly A., Springle, Alisha P., Sultani, Mollee J., and McIlraith, Autumn
- Abstract
Purpose: Analysis of narrative language samples is a recommended clinical practice in the assessment of children's language skills, but we know little about how results from such analyses relate to overall oral language ability across the early school years. We examined the relations between language sample metrics from a short narrative retell, collected in kindergarten, and an oral language factor in grades kindergarten through 3. Our specific questions were to determine the extent to which metrics from narrative language sample analysis are concurrently related to language in kindergarten and predict language through Grade 3. Method: Participants were a sample of 284 children who were administered a narrative retell task in kindergarten and a battery of vocabulary and grammar measures in kindergarten through Grade 3. Language samples were analyzed for number of different words, mean length of utterance, and a relatively new metric, percent grammatical utterances (PGUs). Structural equation models were used to estimate the concurrent and longitudinal relationships. Results: The narrative language sample metrics were consistently correlated with the individual vocabulary and grammar measures as well as the language factor in each grade, and also consistently and uniquely predicted the language factor in each grade. Standardized path estimates in the structural equation models ranged from 0.20 to 0.39. Conclusions: This study found narrative language sample metrics to be predictive, concurrently and longitudinally, of a latent factor of language from kindergarten through Grade 3. These results further validate the importance of collecting and analyzing narrative language samples, to include PGU along with more traditional metrics, and point to directions for future research. [The article is written with the Language and Reading Research Consortium (LARRC).]
- Published
- 2022
44. Are Both Object and Alphanumeric Rapid Automatized Naming Measures Required to Predict Word Reading Fluency in German Prereaders?
- Author
-
Schoefl, Martin, Seifert, Susanne, Steinmair, Gabriele, and Weber, Christoph
- Abstract
Rapid automatized naming (RAN) powerfully predicts word-level reading fluency in the first 2 years of school as well as further reading development. Here, we analyze various RAN stimuli (objects and digits) and oral/silent word reading (OWR/SWR) modalities to find feasible measures for predicting early reading development. The RAN performances of 127 children starting first grade were assessed. The children's oral and silent word reading skills were then reassessed in the second grade. Linear regression models and relative weight analysis were used to compare reading and screening modalities and further precursors of reading such as phonological awareness and nonverbal IQ were controlled. Scores from the first grade RAN assessment did not differentially predict second grade OWR versus SWR levels. RAN digits predicted word reading development more strongly than RAN objects, which contributed uniquely when predictions used only RAN variables. However, when different precursors of reading were controlled, only RAN digits helped to predict early reading performance.
- Published
- 2023
- Full Text
- View/download PDF
45. French Study of Multidimensional Test Anxiety Scale in Relation to Performance, Age, and Gender
- Author
-
Fenouillet, Fabien, Prokofieva, Victoria, Lorant, Sonia, Masson, Julien, and Putwain, David W.
- Abstract
In the absence of a contemporaneous measure, the present study aimed to validate a French language translation of the Multidimensional Test Anxiety Scale (MTAS). In addition, measurement invariance was tested for gender and age. The participants were 520 secondary school students aged 11-16 years. A four-factor model of the MTAS showed a good fit to the data, good internal consistency, and the expected relations with trait anxiety and fear of assessment (positive), and examination performance (negative). Scalar invariance was shown for gender and age. Female students reported higher test anxiety scores. Older students reported higher scores on just one MTAS factor (cognitive interference); remaining factors were stable with age. The translated measure will greatly assist researchers and practitioners in the assessment of test anxiety in French language populations.
- Published
- 2023
- Full Text
- View/download PDF
46. The Effect of Student Race/Ethnicity on the Diagnostic Accuracy of the Social, Academic, and Emotional Behavior Risk Screener (SAEBRS)
- Author
-
Eklund, Katie, Kilgus, Stephen P., Barber, Alexandra, Dubose, Kortni, Collins, Bri'Anna, and Eikenmeyer, Madison
- Abstract
Universal screening has emerged as one strategy within schools for identifying students who demonstrate social emotional and behavioral (SEB) concerns to provide appropriate support. However, research has demonstrated little evidence of equity around diagnostic accuracy; that is, how students might be differentially identified as at-risk for SEB concerns based on race/ethnicity. The current study examined the diagnostic accuracy of the "Social, Academic, and Emotional Behavior Risk Screener" (SAEBRS) teacher screening measure as affected by student race/ethnicity, drawing from a sample of 730 students and 54 teachers from four urban, Midwestern elementary schools. Data were analyzed using nonparametric receiver operating characteristic (ROC) curve analyses, as well as ROC regression tests incorporating student race/ethnicity status as a covariate. Results indicated the SAEBRS (1) demonstrated borderline to optimal sensitivity and specificity across student racial/ethnic subgroups in predicting student SEB risk status, and (2) diagnostic accuracy did not vary across racial/ethnic subgroups to a statistically significant degree. Results highlight the need for further research on this topic.
- Published
- 2023
- Full Text
- View/download PDF
47. Teacher-Student Relationship Quality as a Barometer of Teaching and Learning Effectiveness: Conceptualization and Measurement
- Author
-
Jowett, Sophia, Warburton, Victoria E., Beaumont, Lee C., and Felton, Luke
- Abstract
Background: The Teacher-Student relationship (TSR) is instrumental for young children and adolescents' socio-emotional development and wellbeing as well as academic engagement and progress. Aims: The primary aim of this study was to test the psychometric properties, including reliability and factorial, convergent, and predictive validity, of the Teacher-Student Relationship Quality Questionnaire (TSRQ-Q) with two samples of students. Sample(s): Participants were 294 students from secondary schools in the East Midlands and the East of England. Participants were separated into two samples; those who completed the TSRQ-Q with their physical education teacher in mind (n = 150 students) and those who completed it with their mathematics teacher in mind (n = 144 students). Method: A multi-section questionnaire comprised of the TSRQ-Q and other validated measures was completed on one occasion by students in both samples to assess their perceptions of the quality of the TSR, positive and negative affect, intrinsic motivation, physical self-concept, enjoyment, and perceived competence. Results: In both samples, the TSRQ-Q demonstrated good internal consistency, factorial, convergent, and predictive validity. The quality of the TSR had both direct and indirect effects through positive affect on student outcomes in mathematics and physical education. Conclusions: The TSRQ-Q is a valid measure for assessing students' perceptions of the quality of the relationship with their teacher. The conceptual and practical significance of this unique relationship was reflected by its dual pathway effect on a range of student outcomes and via influencing students' positive affect in the classroom.
- Published
- 2023
- Full Text
- View/download PDF
48. Can a Commercial Screening Tool Help Select Better Teachers?
- Author
-
Chi, Olivia L. and Lenard, Matthew A.
- Abstract
Improving teacher selection is an important strategy for strengthening the quality of the teacher workforce. As districts adopt commercial teacher screening tools, evidence is needed to understand these tools' predictive validity. We examine the relationship between Frontline Education's TeacherFit instrument and newly hired teachers' outcomes. We find that a 1 SD increase on an index of TeacherFit scores is associated with a 0.06 SD increase in evaluation scores. However, we also find evidence that teachers with higher TeacherFit scores are more likely to leave their hiring schools the following year. Our results suggest that TeacherFit is not necessarily a substitute for more rigorous screening processes that are conducted by human resources officials, such as those documented in recent studies.
- Published
- 2023
- Full Text
- View/download PDF
49. Validation of the Child Observation Record Advantage 1.5 Assessment Tool for Preschool Children: A Multilevel Bifactor Modeling Approach
- Author
-
Akaeze, Hope O., Wu, Jamie Heng-Chieh, Lawrence, Frank R., and Weber, Everett P.
- Abstract
This paper reports an investigation into the psychometric properties of the COR-Advantage1.5 (COR-Adv1.5) assessment tool, a criterion-referenced observation-based instrument designed to assess the developmental abilities of children from birth through kindergarten. Using data from 8534 children participating in a state-funded preschool program for disadvantaged 4-year-olds, we employed a multilevel bifactor model to show that CORAdv1.5 has some potential as an unbiased assessment tool for tracking developmental progress in pre-kindergarten children, under a bifactor model parameterization, with fewer dimensions than were proposed by the test developer, HighScope. Examination of factor adequacy indices revealed that, despite the multidimensional structure, CORAdv1.5 items overwhelmingly reflect variance due to the general factors at the child and classroom levels. Evidence for multilevel reliability, response process validity, predictive validity, longitudinal measurement invariance, and generalizability of the items are also reported. The findings of this study have important implications for the use of COR-Adv1.5.
- Published
- 2023
- Full Text
- View/download PDF
50. Aspects of Technical Adequacy of an Early-Writing Measure for English Language Learners in Grades 1 to 3
- Author
-
Smith, R. Alex and Lembke, Erica S.
- Abstract
This study examined the technical adequacy of Picture Word, a type of Writing Curriculum-Based Measurement, with 73 English learners (ELs) with beginning to intermediate English language proficiency in Grades 1, 2, and 3. The ELs in this study attended schools in one midwestern U.S. school district employing an English-only model of instruction and spoke a variety of native languages. ELs completed two forms of Picture Word in the fall, winter, and spring. The criterion measure, a common English language proficiency assessment, was administered in the winter. Results indicated that Picture Word was not appropriate for the first-grade EL participants but showed promise for second- and third-grade ELs.
- Published
- 2021
- Full Text
- View/download PDF
Catalog
Discovery Service for Jio Institute Digital Library
For full access to our library's resources, please sign in.