Search

Measures of socioeconomic status (SES) are widely used in educational research and policy applications, in large part due to overwhelming evidence linking SES to student achievement. SES is usually conceptualized as an unobservable factor--a construct--measured using variables such as parental education, occupation, income/wealth, and home possessions to take into account disparities between students, classrooms, and schools. Researchers and policymakers agree on the importance of SES in educational settings, but their analyses typically rely on an invalid measure of it: eligibility for free and reduced-price lunch (FRL). This, along with other problems with FRL and other commonly used measures of SES, can call into question the validity of their conclusions. The brief offers several recommendations for how researchers can improve the measures they use for SES, and thereby promote both a deeper understanding and more effective use of SES in education research and policy.

Ratio and proportional relationships, along with the interrelated topics of fractions,decimals, and percent, provide a critical foundation for algebra (National Mathematics Advisory Panel, 2008). Proportional reasoning, which not only requires understanding the concept of ratios and that two or more ratios are equal, also requires the ability to extract relevant information to develop a representation of the problem situation and is challenging for many children and adolescents, especially students with mathematics difficulties (Özgün-Koca & Altay, 2009).The intervention investigated in this study represents an approach to proportional problem solving via schema-based instruction (SBI). Jitendra, Harwell, Dupuis, and Karl (2017a)examined the effectiveness of SBI for a subsample of students with mathematics difficulties (MD) selected from a randomized cluster study (Jitendra et al., 2015). The findings raise an important question: What is the range of proportional reasoning skills and general mathematical proficiencies for which SBI enhances student performance? The present study uses a regression discontinuity approach to identify the boundaries of the effectiveness of SBI for students with and without MD. Using a regression discontinuity approach provides evidence SBI is effective for a wide range of proportional reasoning skills and general mathematical proficiencies and thus broadens the population of students SBI can be used effectively with. Standardized slopes for the treatment effect also provide evidence the magnitude of the SBI effect is non-negligible. [SREE documents are structured abstracts of SREE conference symposium, panel, and paper or poster submissions.]

Ratio and proportional relationships, along with the interrelated topics of fractions, decimals, and percent provide a critical foundation for algebra (National Mathematics Advisory Panel, 2008). Solving even simple proportion problems is challenging for many children and adolescents. A small number of studies have examined the efficacy of schema-based instruction (SBI), a multicomponent approach to teaching proportional problem solving. The studies of SBI, with roots in schema theory of cognitive psychology and cognitive models of mathematical problem solving, have provided evidence of its promise in improving student learning. However all of these studies relied on data from the Upper Midwest of the U.S, and what is needed is evidence that the efficacy of SBI generalizes to a range of students and teachers located throughout the country. In this study, the authors pooled data from the three SBI studies by Jitendra et al. (2015, 2017a, 2017b) and asked a single research question: Is SBI effective in improving students' proportional problem solving at posttest and nine to 11 weeks later at delayed posttest (PPS), and in improving general problem solving (GMADE posttest) and, if so, is it equally effective for classroom/teacher and student characteristics such as teacher gender, percentage of students receiving special education services in a classroom, years of teaching experience, and student race and gender across three U.S. states? The pooled data findings clarify that SBI improves students' mathematical problem solving, more so for proximal than distal measures. These findings also show Black and Hispanic students continue to lag behind White students, teachers with more experience tend to be associated with higher performing classrooms, and increasing concentrations of FRL and students receiving special education services are generally associated with weaker performances. These and other findings provide an empirical adjudication of results from individual studies and represent an opportunity to enhance generalizability of the effectiveness of SBI. [SREE documents are structured abstracts of SREE conference symposium, panel, and paper or poster submissions.]

This randomized controlled study investigated the efficacy of a Tier 1 intervention, schema-based instruction (SBI), designed to help students with and without mathematics difficulties (MD) develop proportional reasoning. Twenty seventh-grade teachers/classrooms were randomly assigned to a treatment (SBI) or control (business-as-usual) condition. Participants included 373 students of whom 253 demonstrated MD. A measure of proportional problem solving (PPS) was administered at pre- and post-testing, and at 11 weeks following treatment, along with a general mathematical problem solving measure at pre- and post-testing. For the full sample, posttest differences favoring the treatment group were statistically significant for all measures. For students with MD, posttest differences favored the treatment group for the PPS posttest and PPS delayed posttest, but not for general problem solving posttest. [This paper was published in "Exceptional Children" (EJ1151254) and "Journal of Learning Disabilities" v83 n4 p340-358 2017.]

The purpose of this study was to replicate and extend the study of the schema-based intervention (SBI) conducted by Jitendra et al. (2015) that demonstrated impact in proportional problem solving for a homogeneous sample of Upper Midwest students and teachers who were predominantly White and middle class. This study was replicated in two geographically diverse locations (Northern Rocky Mountains and Southeast U.S.). The following research question was explored: What are the effects of the SBI intervention compared to business-as-usual instruction on students' proportional problem solving at immediate posttest and after 11 weeks as well as on students' general mathematical problem solving? Students from 52 seventh-grade classrooms (N = 1,379) and their teachers (J = 52) from 30 middle schools in four school districts participated in the study. The study used a randomized cluster design with longitudinal data. Students' proportional problem solving (PPS) performance was tested on three occasions (pre, post, delayed posttest given 11 weeks after the intervention). Results indicated statistically significant differences between treatment and control groups on the total score for fidelity-of-implementation items, with treatment teachers implementing SBI elements with more fidelity than topics implemented by control teachers (t(50) = 7.98, p < 0.001, g = 2.21). [SREE documents are structured abstracts of SREE conference symposium, panel, and paper or poster submissions.]

This study examined the effects of a research-based intervention, schema-based instruction (SBI), implemented by experienced- (taught SBI in previous study; Jitendra et al., 2015) and novice-teacher implementers (taught SBI for the first time with professional development) on the mathematics outcomes of seventh-grade students. SBI is a multicomponent intervention that emphasizes the mathematical structure of problems through the use of schematic diagrams and incorporates problem solving and metacognitive strategy instruction. Results indicated that both experienced- and novice-teacher implementers delivered SBI with similar levels of fidelity; there was no SBI experience effect on the immediate and 10-week retention tests of proportional problem-solving, on a general measure of problem solving, or on the end of the year state mathematics achievement test. These results provide evidence that the effectiveness of SBI generalizes over time to different cohorts of teachers and that the impact of SBI on student mathematics outcomes is maintained over time without additional PD. [This article was published in "Learning and Instruction" v44 p53-64 Aug 2016.]

Ratio and proportional relationships are of primary importance during the upper elementary and middle school grades (Kilpatrick, Swafford, & Findell, 2001; National Council of Teachers of Mathematics, 1989, 2000; National Mathematics Advisory Panel [NMAP], 2008). These relationships, along with the interrelated topics of fractions, decimals, and percent provide a critical foundation for algebra (NMAP, 2008). Proportionality involves the concept of ratio and is central to topics in mathematics such as linear functions, scale drawings, similarity, trigonometry, and probability. Proportionality is also at the core of many important concepts in the physical and social sciences. Solving even simple proportion problems is challenging for many children and adolescents when they lack understanding of the problem situation and whether a solution strategy is applicable (Adjiage & Pluvinage, 2007; Fujimura, 2001; Lamon, 2007; Lobato et al., 2010; Miyakawa & Winslow, 2009; NMAP, 2008; Tourniaire & Pulos, 1985; Weinberg, 2002). Although students require instruction that supports the development of underlying concepts and flexible procedures to solve proportion problems (NMAP, 2008; Tourniaire & Pulos, 1985), few intervention studies have been conducted to improve students' learning of ratios and proportions. The purpose of this study was to replicate and extend the study of the schema based instruction (SBI) intervention conducted by Jitendra et al. (in press) that demonstrated impact in proportional problem solving for a homogeneous sample. The following research questions were examined: (1) What are the effects of the SBI intervention on seventh-grade students' proportional problem solving immediately following the intervention and eight weeks following treatment; (2) What are the effects of the SBI intervention on problem solving immediately following the intervention and eight weeks following treatment for students receiving special education services; (3) What are the effects of the SBI intervention on seventh-grade students' general mathematical problem solving; (4) What are the effects of the SBI intervention on general mathematical problem solving for students receiving special education services; and (5) Does the treatment moderate students' learning trajectory over time? The consistent effects in previous randomized studies suggest confidence in the positive effects of SBI in enhancing proportional problem solving (PPS) for all students. This study replicated and extended earlier findings thus enhancing external validity. An SBI effect emerged for the immediate PPS test but unlike the original study (Jitendra et al., in press), participants did not maintain this learning over time for the PPS delayed posttest and the GMADE test. Future research, including a larger sample, is needed to confirm for whom and under what conditions the SBI intervention has positive effects. Tables are appended.

This experimental study evaluated the effectiveness of a research-based intervention, schema-based instruction (SBI), on students' proportional problem solving. SBI emphasizes the underlying mathematical structure of problems, uses schematic diagrams to represent information in the problem text, provides explicit problem solving and metacognitive strategy instruction, and focuses on the flexible use of multiple solution strategies. Eighty-two teachers/classrooms with a total of 1,999 seventh-grade students across 50 school districts were randomly assigned to a treatment (SBI) or control (business-as-usual) condition. An observational measure provided evidence that the SBI intervention was implemented with fidelity. Results of multilevel modeling indicated that the SBI group scored on average significantly higher than the control group on the posttest and retention test (9 weeks later) and also showed significantly more growth in proportional problem solving. There were no treatment effects on the Process and Applications subtest of the Group Mathematics Assessment and Diagnostic Evaluation. These results demonstrate that SBI can be more effective than the control approach in improving students' proportional problem solving. [This paper was published in the "Journal of Educational Psychology," (EJ1082754).]

When normality does not hold, nonparametric tests represent an important data-analytic alternative to parametric tests. However, the use of nonparametric tests in educational research has been limited by the absence of easily performed tests for complex experimental designs and analyses, such as factorial designs and multiple regression analyses, and limited information about the properties of these tests for realistic data conditions. Efforts to remedy this deficiency have begun with the introduction of general linear model-based nonparametric tests. The results of a computer simulation of the properties of several of these tests in hierarchical regression analyses indicated that, on balance, the top-performing test for the nonnormal distributions studied was the McKean-Hettmansperger F-test (J. McKean and T. Hettmansperger, 1976) using a confidence interval estimate of the scale parameter tau. The Serlin-Harwell aligned-rank chi-square test (R. Serlin and M. Harwell, 2001) performed almost as well, which, combined with the fact that it is easier to compute, makes it an attractive competitor to the Mc-Kean-Hettmansperger test. (Contains 5 tables and 59 references.) (Author/SLD)

This discussion of the use of statistical methods in educational research considers the frequent misuse of statistical procedures, the factors responsible for the misuse, and the remedies that might reduce the misuse of statistical procedures. There are longstanding concerns about the quality of statistical work in general as well as in educational research, but there is little formal documentation of the misuse of statistical methods by graduate students. Anecdotal evidence suggests that the misuse is not confined to students, but can be found in the work of faculty members and researchers. Misuse of statistical procedures by students is in large part due to inadequate instruction, sometimes resulting from poorly prepared teachers or inadequate texts, or even an incoherent curriculum. The magnitude of misuses of statistical procedures by researchers also suggests inadequate statistical training. Any remedies for these problems will need to be tailored to various types of students and educational researchers. Some possible solutions are: (1) put education back into educational statistics; (2) make instructors accountable for their teaching; (3) insist on adequate training for students; (4) include educational statisticians as reviewers for scholarly journals; (5) value research in statistical education; and (6) establish a professional "home" for educational statisticians. (Contains 35 references.) (SLD)

Educators in District #2 in New York City have focused on school reform for more than a decade, shaping their efforts at district-wide instructional improvement through subject-matter initiatives. This study was conducted to explore the influence of a variety of factors on student achievement in District #2 with a particular interest in the role of teacher professional development. Between 1988 and 1998, the percentage of District #2 students achieving at or about grade level rose from 56% to 73% in reading and from 66% to 82% in mathematics, suggesting that the professional development strategy may have had some effects. Achievement and demographic data for District #2's students were obtained, as was information on teachers' backgrounds and professional development experience. Questionnaires completed by 37 teachers in grades 3 and 5 were used, and analyses of the link between achievement and professional development were based on the 848 students of these teachers. Many variables that the school district cannot control, poverty, ethnicity, and proficiency in English, exerted an influence on student performance in reading and mathematics. Socioeconomic status had a strong relationship with classroom level reading performance, but teachers' reported engagement in professional development made a difference in that the relationship between gender and reading achievement was weakened when teachers discussed their literacy instruction with colleagues and with their principals frequently. Some possible reasons professional development did not show more powerful effects in reducing the achievement gap are discussed. This study has provided a foundation for other studies that will measure the linkages between various aspects of District #2's theory of action and student outcome measures. Two appendixes contain a discussion of study methodology and the teacher survey. (Contains 24 references.) (SLD)

The purpose of this paper is to illustrate how a quantitative literature review or research synthesis can be used to aggregate information in the item response theory literature to address specific research questions. The research questions focused on the magnitude of the contribution of the number of items and number of examinees to the accuracy of parameter estimates in the two-parameter item response theory model. The results from 7 studies used in the research synthesis suggest that more than 63% of the variables in a commonly used indicator of parameter estimation accuracy was attributable to these 2 factors, and that this result holds across a variety of factors. Other questions that a research synthesis might be used to address in the item response theory literature are described. (Contains 13 tables and 31 references.) (Author/SLD)

The effect of a nonlinear regression term on the behavior of the standard analysis of covariance (ANCOVA) F test was investigated for balanced and randomized designs through a Monte Carlo study. The results indicate that the use of the standard analysis of covariance model when a quadratic term is present has little effect on Type I error rates but produces a substantial power loss compared to theoretically expected values, often in excess of 20%. The extent of the power loss depends on the magnitude of the regression parameter associated with the nonlinear term. This finding appears consistently for varying numbers of groups and sample sizes and for various distributions. These results highlight the importance of plotting data and checking for nonlinearity prior to employing the standard analysis of covariance F test. (Contains 2 tables and 22 references.) (Author/SLD)

The learning difficulties described by students in statistics courses continue to engage researchers from several disciplines. One source of difficulty for graduate students in educational statistics courses is the reading difficulty of the textbook. Instructors making decisions about a textbook typically have little information about the reading difficulty of the text. Although reading difficulty is almost always assessed using a reading difficulty formula, such formulas have not been evaluated for use with graduate students majoring in the social sciences. An evaluation study examined the extent to which five reading difficulty formulas often used with populations in kindergarten through grade 12 consistently and validly differentiate among introductory statistics texts used in a school or college of education. The 5 formulas were applied to 10 textbooks commonly used in introductory statistics classes. The agreement among these formulas in rank ordering texts appears to be moderate at best. Preliminary findings suggest that these formulas are of only modest utility in assessing the reading difficulty of statistics texts, and that the development of new formulas is indicated. (Contains 5 figures, 4 tables, and 32 references.) (SLD)

A Monte Carlo study was used to examine the Type I error rates of five multivariate tests for the single-factor repeated measures model. The performance of Hotelling's T-squared and four nonparametric tests, including a chi-square and an "F" test version of a rank-transform procedure, was investigated for different distributions, sample sizes, and numbers of repeated measures. The results indicated that both Hotelling's T-squared and the F test version of the rank-transform test performed well, producing Type I error rates that were close to the nominal value. The chi-square version of the rank-transform test, on the other hand, performed poorly for virtually all conditions studied. The performance of the other nonparametric tests depended heavily on sample size. Based on these results, Hotelling's T-squared is recommended for the single-factor repeated measures model. Appendix A discusses computing the tests. (Contains 1 table and 48 references.) (Author/SLD)

The test of homogeneity developed by L. V. Hedges (1982) for the fixed effects model is frequently used in quantitative meta-analyses to test whether effect sizes are equal. Despite its widespread use, evidence of the behavior of this test for the less-than-ideal case of small study sample sizes paired with large numbers of studies is contradictory, and its behavior for nonnormal score distributions in primary studies is an open question. The results of a Monte Carlo study indicated that the Type I error rate and power of the homogeneity test were insensitive to skewed score distributions, but were very sensitive to smaller study sample sizes paired with larger numbers of studies. These findings extend earlier results and help to clarify the statistical behavior of the homogeneity test. Specifically, the pairing of small study sample sizes with large numbers of studies tends to produce conservative Type I error rates for the homogeneity test and underestimates its power, increasing the likelihood of Type II errors. (Contains 2 tables and 23 references.) (Author/SLD)

Monte Carlo studies of statistical tests are prominently featured in the methodological research literature. Unfortunately, the information from these studies does not appear to have significantly influenced methodological practice in educational and psychological research. One reason is that Monte Carlo studies lack an overarching theory to guide their interpretation. Also, the impressionistic nature of Monte Carlo studies can lead different readers to different conclusions. These shortcomings can be addressed using meta-analytic methods to summarize the results of Monte Carlo studies. Summarizing Monte Carlo studies in this fashion will generate guidelines for the appropriate use of particular statistical tests under specific assumption violations, and will allow an evaluation of the validity of the statistical results of previously published studies. This paper provides a methodological framework for, and an example of, quantitatively summarizing empirical Type I error and power values from Monte Carlo studies. The four-step summarization strategy covers problem formulation, data collection, data evaluation, and data analysis and interpretation. The example of a quantitative summary involves seven Monte Carlo studies of the parametric Bartlett test of the quality of independent variances. (Author/TJH)

Concern over the validity of statistical tests performed on data that may not satisfy underlying assumptions has prompted methodological researchers to perform Monte Carlo studies for frequently used tests. Unfortunately, these studies appear to have had little impact on methodological practice. One reason is the lack of an overarching framework to guide the interpretation of Monte Carlo studies for the same test. Another is the impressionistic nature of these studies, which can lead different readers to different conclusions. These shortcomings can be addressed using quantitative methods of research synthesis (e.g., meta-analysis) to summarize the results of Monte Carlo studies for a statistical test. In this paper, these methods are applied to a sample of Monte Carlo studies of the F-test in the oneway fixed-effects analysis of variance (ANOVA) model. The present analyses were based on Monte Carlo studies reported in 21 out of 30 journal articles. The results provide empirical support for the robustness of the Type I error rate of the F-test to certain assumption violations. However, the Type I error rate of the F-test was noticeably affected by unequal variances, even when sample sizes were equal. Recommendations for using this test when certain assumptions are violated are made. Four data tables and one bar graph are included. (Author/TJH)