Search

The purpose of this paper is to illustrate how displaying disattenuated correlation coefficients along with their unadjusted counterparts will allow the reader to assess the impact of unreliability on each bivariate relationship. The paper also demonstrates how a proposed new "what if reliability" analysis can complement the conventional null hypothesis significance test of bivariate relationships. The "what if reliability" procedure helps the researcher determine the extent to which the sample size, as opposed to the effect size, is responsible for the observed statistically significant, or not statistically significant, finding. The analyses illustrate how the sample size needed to detect a statistically significant bivariate relationship decreases as the observed score reliability coefficient pertaining to the independent or dependent measure (theoretically) increases, holding all other factors constant. As such, "what if reliability" analyses will help researchers interpret their results by considering the extent to which the reliability coefficients contribute or fail to contribute to the ability to achieve a statistically significant result. (Contains 3 tables and 28 references.) (SLD)

This paper suggests that although data from a homogenous sample might yield less reliable scores than did an inducted sample, these data should not be discarded until further examination of the data is conducted. The paper presents two statistics for monitoring data homogeneity and one statistic for correcting alpha when homogeneity is large. The first of the statistics for monitoring sample homogeneity is computed as the difference in mean item variances, expressed as a percentage. The second monitoring statistic is termed the "relative squared standard error of estimate index" (J. Roberts and A. Ongwuegbuzie, 2000). The new statistic for correcting alpha is developed for the situation in which the homogeneity is large and access to a previously normed sample is not available. The use of all three techniques is illustrated. (Contains 3 tables and 12 references.) (SLD)

A promising new teacher self-efficacy instrument, the Ohio State Teacher Efficacy Scale (OSTES) (M. Tschannen-Moran and A. WoolfolkHoy, in press) was critically evaluated regarding its development and submitted to confirmatory factor analysis (CFA). Participants were 183 inservice teachers in Texas and Washington, DC, most of whom taught primary grades. The hypothesized three-factor structure was not supported by the CFA, but a two-factor correlated structure was retained, which consisted of the Efficacy in Student Engagement and Efficacy in Instructional Practices factors. The third factor, Efficacy in Classroom Management, was eliminated in the strongest two-factor model. Recommendations are made to: (1) further refine the OSTES by removing the third factor; (2) seek increased use of CFA methodology in teacher efficacy instrumentation and (3) foster stronger exploratory factor analytic strategies, particularly regarding factor retention. The instrument is attached. (Contains 2 tables, 5 figures, and 51 references.) (SLD)

The appropriate use of exploratory factor analysis (EFA) necessitates thoughtful researcher judgment concerning a number of analytic decisions. This paper reviews some of the fundamental decisions necessary to conduct an EFA and examines reporting practice in published research across four journals. In general, insufficient information was given in published applications of EFA (n=60) to allow external verification of the results. Researchers often used poor strategies to determine the number of factors to retain. In one-third of the cases, a confirmatory factor analysis was warranted rather than EFA. Other errors in reporting and practice were noted. Several recommendations for improved EFA reporting practice are given. (Contains 4 tables and 105 references.) (Author/SLD)

This study examined the factorial invariance of scores from the Attitudes and Beliefs on Classroom Control Inventory (ABCC) (Martin and others, 1998) for 243 undergraduate preservice teachers. Although the original ABCC was developed with inservice teachers, use of the instrument to study the classroom beliefs of preservice teachers had not been studied. Confirmatory factor analyses were conducted to test competing hypothesized models concerning score structure. Results indicate overall poor fit of the data across models, calling into question the usefulness of the ABCC with preservice teachers. (Contains 1 figure and 28 references.) (Author/SLD)

This paper examines the differences between multilevel modeling and weighted ordinary least squares (OLS) regression for analyzing data from the National Educational Longitudinal Study 1988 (NELS:88). The final sample consisted of 718 students in 298 schools. Eighteen variables from the NELS:88 dataset were used, with the dependent variable being the science item response theory estimated number right standardized t-score. Results from the analyses yield no single criterion for choosing one method over the other, but they do illustrate some theoretical situations when multilevel models are preferred. As contextual effects grow larger, multilevel analyses tend to produce more accurate results of the data. Multilevel techniques also allow the researcher to use statistical analyses that are able to mine more complex data. (Contains 2 tables and 29 references.) (SLD)

This paper introduces a new self-efficacy instrument for science teachers and presents confirmatory factor analysis results for 247 elementary school teachers. Building on previous research on the Science Teaching Efficacy Belief Instrument (STEBI) (I. Riggs and L. Enochs, 1990), the new instrument, the Self-Efficacy Teaching and Knowledge Instrument for Science Teachers (SETAKIST), hypothesizes that science teacher self-efficacy exists in two constructs: teaching efficacy and knowledge efficacy. The second factor is based largely on the work of Lee Shulman. An appendix contains the SETAKIST instrument. (Contains 2 figures and 40 references.) (SLD)

Analysis of variance (ANOVA) designs typically use what is referred to as crossed design to test for differences between means of groups. In a balanced, or crossed, one-way ANOVA, each student (unit of analysis) would have a score in each of the experimental conditions. In a two-way design, the analysis is considered crossed if each level from one way is contained in each level of the other way. In this design, every person (unit of analysis) has a score in every cell. Many experimental designs in the behavioral sciences do not qualify as a crossed design. Units of measurement are rather "nested" inside other factors. A crossed experimental design would neglect the hierarchical structure of the data and produce incorrect interpretations of results. Neglecting a nested design when one actually exists will make the researcher: (1) wrongly attribute a main effect to an interaction effect when, in fact, no interaction exists; (2) divide by the wrong degrees of freedom when determining the mean square and F-value (and the statistical significance of the F-value); and (3) assume that a main effect has a smaller effect size (eta-squared) because the sum of squares for that effect is being partly attributed to the interaction effect. (SLD)

Researchers investigated the optimum length of teacher inservice activities where increasing teacher efficacy was the goal. Participants were elementary science teachers from seven teacher enhancement projects conducted from 1992-99. The length breakdown of each program was: 1992--6 weeks; 1994--6 weeks; 1995--4 weeks; 1996--4 weeks; 1997--4 weeks; 1998--3 weeks; and 1999--2 weeks. In each of the projects, teachers completed the Science Teaching Efficacy Belief Instrument (STEBI) on the first and last days of the inservice workshops. The STEBI examined personal science teaching efficacy (PSTE) and science teaching outcome expectancy. Data analysis indicated that there was no statistically significant difference between the mean PSTE gain scores on the three contrast variables among the four groups of teachers whose PSTE pretest scores were greater than or equal to 50. This may be due to the fact that teachers already scored high on the PSTE scale. Among teachers whose pretest PSTE scores were less than 50, there were significant gains when comparing the mean gain scores from teachers in the 2- and 3-week sessions and teachers in the 4- and 6-week sessions. It was found that inservice intervention programs had the greatest impact on the efficacy of those teachers who began the program with the lowest efficacy. Given the consistent relationships demonstrated between teacher efficacy and positive student outcomes, inservices impacting teachers' low efficacy are worth close examination. (Contains 32 references and 8 tables.) (SM)

According to some researchers canonical correlation results should be interpreted in part by consulting redundancy coefficients (Rd). This paper, however, makes the case that Rd coefficients generally should not be interpreted. Rd coefficients are not multivariate. Furthermore, it makes little sense to interpret coefficients not optimized as part of an analysis. A small heuristic data set using the Statistical Package for the Social Sciences is used to illustrate Rd. If the researcher's primary interest is to explore relationships between synthetic variable sets, then canonical correlation analysis should be used, and the interpretation should focus on the multivariate variance-accounted-for effect size, the standardized function coefficients, and the structure coefficients. (Contains 3 tables and 32 references.) (Author/SLD)

Many researchers acknowledge the prominent role that factor analysis can play in efforts to establish construct validity. Data can be analyzed with no preconceived ideas about the underlying constructs of structure of the data. This approach is exploratory factor analysis, Another approach is used when the researcher has an understanding of the constructs underlying the data. This approach, confirmatory factor analysis, is a theory-testing procedure. A primer on confirmatory factor analysis is presented. Elements discussed include matrices that can be analyzed correctly and various statistics for evaluating the quality of fit of models. The use of the AMOS software package to perform confirmatory factor analysis is illustrated. The use of confirmatory factor analysis is supported because it is a way to test the a priori expectations of the researcher, encouraging more meaningful and empirically based research. Appendixes contain the command syntax for the AMOS software package and the AMOS results printout. (Contains 3 tables, 1 figure, and 15 references.) (SLD)

Many school districts face the problem of evaluating new programs to train students in ethics and moral decision making. Using conventional personality tests in program evaluation may be helpful, but probably will not provide measures for the attitudes that are targeted by the intervention. The method of equal- appearing intervals developed by L. Thurstone (1959) provides a viable solution for this problem. How school districts could use this type of scaling to measure attitudes specific to the design of character education programs is discussed. The relative simplicity of Thurstone scaling means that most schools could develop and administer this method in-house. The relative ease of construction would allow school districts not only to measure the effectiveness of an intervention but also to identify children who might be best suited for character education programs. (SLD)

The COIN Basic Skills and Career Interest Survey is a two-part measure using the combined scores of the Wonderlich Basic Skills Test (E. Long, V. Artese, and W. Clonts, 1994) (WBST) and the COIN Career Interest Survey (R. Durgin, J. Ryan, and R. Ryan, 1995; COIN Educational Products, Inc.) (CIS). The third part of this package involves the combination of the two sets of scores linked to specific occupations. The overall result of the test package is an interpreted result that gives a student elements necessary for career decision making, educational planning, and curriculum selection. The entire package is designed for use with high school students. The package is attractively presented and appropriate for its audience. The WBST was normed with individuals in 86 occupations in 1992. Reliability analyses performed by the test developers indicate that the WBST is a reasonably reliable test. However, the test package does not contain any record of testing for criterion validity, and there is a lack of technical information about the CIS. If the instrument is intended to be used in a prescriptive way for future workers, then much work needs to be done to ensure that the questions that are asked and the prescriptions made are reliable and valid. Further work needs to be done by COIN to establish the reliability and validity of evidence from this package, but it is enjoyable to take, and might be a useful tool for graduating high school seniors once norms, validity, and reliability are established. (Contains five references.) (SLD)

This study examined the impact of independent practice of multiple-criteria text that targeted high-frequency words, decodability, and meaningfulness. Second-grade students, including at-risk students, were randomly assigned within classroom to a treatment group that read multiple-criteria text ("n" = 34), or contrast group that read authentic literature (i.e., children's books without intentionally imbedded scaffolds; "n" = 28) during daily 30-min independent reading sessions for 10 weeks. Pre-post data analysis indicated no statistically significant group differences, though a moderate effect size of 0.67 was found for the word reading of developing decoders in the treatment group. HLM analyses also provided preliminary evidence that practice with multiple-criteria text may be more effective than practice with authentic literature for developing decoders but not advanced decoders.