Search

This study, a collaborative undertaking between the college of health professions and the medical college at Thomas Jefferson University (Pennsylvania), was part of a larger project intended to examine whether a selected set of academic, demographic, and psychosocial variables are predictive of nursing and allied health student academic performance. The study compared psychosocial profiles of 71 female medical students with 182 female students in nursing and allied health fields. The survey instrument measured: loneliness, test anxiety, general anxiety, self-esteem, extroversion, external locus of control, neuroticism, stressful life events, and depression. Results indicated that medical, nursing, and allied health students differed considerably on a number of measures. Female allied health and nursing students appeared to be more depressed than female medical students, while female medical students had greater perceptions of general health and closer relationships with their fathers. The study also found a significant relationship in the expected direction between grade point average and stressful life events, test anxiety, perception of health, and self esteem. An unexpected finding was that the correlation between transfer grade point average and first semester grade point average was not significant. Six data tables are included. (Contains 21 references.) (CH)

Structural integrity of a measure is defined in terms of its replicability, constancy, invariance, and stability. Work completed in the development and validation of the Self Observation Scales (SOS) Primary Level (Stenner and Katzenmeyer, 1973) serves to illustrate one method of establishing structural integrity. The name of each scale of the SOS is a construct: an abbreviated thesis about the nature of the underlying variable which determines the configuration of indicants (questions) comprising the scale. SOS constructs (self acceptance, social maturity, school affiliation, and self security) are analyzed for different student groups. Traditional psychometrics assumes that the same questions measure the same underlying variables in all respondents; the validity of interperson and intergroup comparisons rests on this assumption. However, ample evidence suggests that this assumption is frequently in error. Procedures outlined in this paper provide a method for assessing the validity of this assumption, and a basis for developing reliable and valid instruments. (Author/MV)

This study investigated the effects of different structures of a synthesizer and formats of the generality component on the application and remember levels of learning. Seventy-three undergraduates participated. Four treatment groups were formed by combining two types of structure (complete versus partial) with two types of format in generality (visual versus verbal statement). A complete synthesizer contained a generality, an example, and some practice. A box-chart diagram was used to represent the visual format of the generality. Results suggest that, although the format of the generality does not make significant differences on either the application or the remember levels of learning, a complete synthesizer seems to benefit remember level learning. When comparing treatment groups with the control group, the post-hoc comparison again shows that complete synthesizers result in significantly better learning than no synthesizer at all. A discussion of the findings concludes the paper. Seventeen references are appended. (Author/RP)

Inconsistencies in the use of terminology to describe instructional treatments, confounding of discovery and expository methods with other aspects of instruction, and inconsistencies in defining independent and dependent variables within and between studies have all contributed to the lack of definitive conclusions about the effectiveness of different instructional methods. The purpose of this study was to compare the effectiveness of three instructional methods, i.e., expository, discovery, and a modified form of discovery, Component Display Theory, which incorporated the rule given in the expository lesson using a theoretical paradigm for analyzing the components of instruction. The subjects were 50 third-graders from Syracuse, New York, and a concept taken from beginning set theory in conceptual mathematics was selected as the task. While none of the instructional treatments proved to be significantly better on the overall test incorporating both application and transfer subscales, a few differences did emerge through analysis of the subscale results. The findings indicated that explicitly providing a generality, whether at the beginning or at the end of a lesson, will facilitate performance on applying that generality. A 15-item reference list is provided. (RP)

Subjects for an investigation of the effects of training with a computer and past computer experience on the computerized aptitude test performance of college students were 92 undergraduates at the University of North Carolina at Charlotte, who participated in a test study utilizing six Apple II+ microcomputers. The study was conducted to determine whether: (1) computerized test performance would be significantly enhanced and the concomitant anxiety level would be significantly lowered by increasing the amount of training with the computer prior to testing; and (2) increasing the amount of training with the computer would significantly reduce the difference between the mean anxiety level occurring with a computerized test and the mean anxiety level occurring with a paper-and-pencil test. Test performance was measured by the number-correct score on an arithmetic reasoning test. Results indicate that neither training immediately before testing nor past computer experience appear to significantly affect computerized test performance or the concomitant anxiety level. The mean paper-and-pencil test score was higher than the mean computerized score, however, and it is suggested that differences between pencil-and-paper and computerized test performance may be the result of human error and deficiencies of the computer software. A list of references completes the document. (JB)

Assessing people who operate in different languages necessitates the use of multiple language versions of an assessment. However, different language versions of an assessment are not necessarily equivalent. In this paper, the psychometric properties of different language versions on an international employee attitude survey are evaluated. This survey was administered to more than 50,000 employees of a large telecommunications company using both paper-and-pencil and Web administration formats. The structural equivalence of the survey was evaluated across language versions, cultural groups, and administration formats using multidimensional scaling. The statistical equivalence of English, French, and Spanish versions of the survey items was evaluated using analysis of covariance. The results indicate the structure of the survey is consistent across the groups studied, and that the different language versions of the items functioned similarly. The implications of the results for future research in this area are discussed. (Contains 3 figures, 4 tables, and 38 references.) (Author/SLD)

Despite repeated discussions of the dangers of covariance corrections, the use of analysis of covariance (ANCOVA) continues in situations that should preclude its use. Because the use of ANCOVA is particularly appealing in much counseling research (in which groups are often intact and sample sizes are often small for treatment interventions), the purpose of this paper was to examine the use and misuse of ANCOVA in the counseling literature. The use of ANCOVA was examined in three counseling journals from 1997 through 2001. The Journal of Counseling and Development, Counselor Education and Supervision, and the International Journal of Play Therapy were selected for the study. Overall, ANCOVA was used in 21 of the 250 articles reviewed. Researchers primarily used univariate applications of ANCOVA, and generally used intact groups and unbalanced cells. Most did not comment on some basic assumptions associated with ANCOVA, including the very important homogeneity of regression assumption. Recommendations for improved use of ANCOVA are presented. (Contains 2 tables and 32 references.) (SLD)

Researchers are often faced with the dilemma of having to use intact groups. Analysis of covariance (ANCOVA) is sometimes used as a means of leveling unequal groups. However, only after five assumptions have been met is it safe to use ANCOVA. Part and partial correlations also invoke statistical correlation procedures. This paper reviews these covariance correction procedures, explains the relevant assumptions and potential pitfalls related to the analysis, and discusses research situations when their use is warranted or ill-advised. (Contains 1 figure, 5 tables, and 13 references.) (Author/SLD)

The impact of computerized adaptive test (CAT) administration formats on student performance was studied with 645 medical technology students who also took a paper-and-pencil test. Analysis of covariance indicates no significant interactions among test administration formats and provides evidence for adjusting CAT test to more familiar modalities. (SLD)

This paper explains how analysis of covariance (ANCOVA) and related statistical corrections work and discusses difficulties with the use of these corrections under certain circumstances. ANCOVA is essentially a regression of a covariate variable on the dependent variable from the entire sample ignoring group membership, at least, if ANCOVA assumptions are perfectly met. Small heuristic data sets are used to illustrate when ANCOVA can and cannot be used correctly in educational research. ANCOVA generally can be used correctly with randomly assigned groups, but may not be needed in that situation. When groups are not randomly assigned, ANCOVA often cannot be used correctly. It is a paradox that ANCOVA often cannot be used when "correction" is most needed. For ANCOVA, meeting the homogeneity of regression assumption is critical in determining its viability as a statistical tool. An appendix contains a computer program for the ANCOVA heuristic analysis. (Contains four tables, five figures, and eight references.) (Author/SLD)

The present paper suggests that multivariate methods ought to be used more frequently in behavioral research and explores the potential consequences of failing to use multivariate methods when these methods are appropriate. The paper explores in detail two reasons why multivariate methods are usually vital. The first is that they limit the inflation of Type I error rates, and the second is that multivariate methods best honor the reality to which the researcher is purportedly trying to generalize. Three general analytic premises provide a framework for the discussion: (1) all statistical analyses are correlational; (2) the OVA (analysis of variance and covariance) analyses do not in and of themselves yield the capacity to make causal inferences; and (3) statistical significance tests do not evaluate the probability (p) that results will replicate. Eleven tables illustrate some analyses. (Contains 51 references.) (Author/SLD)

Although analysis of covariance (ANCOVA) is used fairly infrequently in published research, the method is used much more frequently in dissertations and in evaluation research. This paper reviews the assumptions that must be met for ANCOVA to yield useful results, and argues that ANCOVA will yield distorted and inaccurate results when these assumptions are violated. For ANCOVA to provide meaningful statistical control and to not obscure or mislead, it must be ascertained that the data set fulfills several requirements, especially those pertaining to the homogeneity of regression slopes. For ANCOVA to increase power against a Type II error, there must be a high correlation between the covariate and the dependent variable and no correlation between the covariate and the independent variable. ANCOVA practitioners should examine their data sets carefully to insure that ANCOVA is an appropriate analytic method rather than a method that generates more problems for a given data set than it actually solves. Numerous examples, five figures, and one table are provided. (Author/SLD)

This study compared electronic mail, traditional, and combination approaches for teaching graduate introductory statistics classes. The electronic course that was the focus of this study was offered in fall terms of 1995 through 1999. There were 23 participants in the electronic only classes, 69 in the traditional only classes, and 27 in the combination classes, with a majority membership of white females in all classes. Multiple choice pretests and posttests were given. An analysis of covariance (ANCOVA) was run using posttest scores as the response variable and pretest scores as the covariate. Random selection was not possible since participation in any version of the course was optional. Normality and homoscedasticty across all groups were verified. Homogeneity of regression was observed in scatterplots of pretest scores versus posttest scores and their trend lines by treatment and control groups. Therefore, the assumptions required for ANCOVA seemed to be reasonably well met. The test indicated that the null hypothesis of no statistically significant difference among the traditional, electronic, and both traditional and electronic classes' scores could not be rejected at the 0.05 level, with an effect size of f=0, a negligible effect according to J. Cohen. It is concluded that offering the course through electronic mail or a combination of electronic mail and the traditional approach did not appear to hinder the performance of the students, to the extent measured by the multiple-choice tests. Attachments include the ANCOVA report and a course description and syllabus. (Contains 26 references.) (SLD)

This paper analyzes what happens to the effect size of a given dataset when the variance is removed by categorization for the purpose of applying "OVA" methods (analysis of variance, analysis of covariance). The dataset is from a classic study by Holzinger and Swinefors (1939) in which more than 20 ability test were administered to 301 middle school students to discern which abilities determined overall academic performance. One-way analyses of variance (ANOVA) were run with the dichotomized variable as the independent factor to compare the reduction in effect size in the ANOVAs with the effect size of regression run with the intact continuous independent variable. The effect sizes of the ANOVAs that were run with the dichotomized variable were about half of the regression effect sizes run with the unaltered independent variable. Each of the effect sizes after dichotomization would fit into a lower category, resulting in a serious underestimation of the substantive effect. (Contains 3 tables and 16 references.) (SLD)

A study was conducted to compare electronic and traditional versions of a graduate-level research class. There were 73 participants in the electronic classes and 35 in the traditional classes, with a majority membership of white females enrolled in graduate education programs. Multiple choice pretests and posttests were administered to all groups. An analysis of covariance indicated that there was no statistically significant difference, at the 0.05 level, between the adjusted means of the two groups, suggesting that the electronic option allows for comparable learning outside the classroom. Attachments contain the course description and the analysis of covariance report.(Contains 26 references.) (Author/SLD)

This research explored the degree to which group sizes can differ before the robustness of analysis of variance (ANOVA) and analysis of covariance (ANCOVA) are jeopardized. Monte Carlo methodology was used, allowing for the experimental investigation of potential threats to robustness under conditions common to researchers in education. The effects of unequal group sizes were explored under the following data set conditions: (1) heterogeneity of group variances; (2) skew; (3) kurtosis; and (4) in ANCOVA, heterogeneity of regression slopes. Two independent sets of simulations were conducted, one using a total group of 90 and the other a total of 60. Experimentation was limited to simulations using three groups, with the total number divided in a systematic fashion. Results of these studies produced results consistent with previous research. In the analyses having homogeneity of group variances, the only simulations that emerged as statistically significant from the theoretical F test were those that had a large degree of difference in group numbers and unequal regression slopes. No significant differences emerged in simulations with near equal numbers. Tables in this paper document the simulation results, but also offer the research practitioner some idea of the true risk of Type I error in such situations. (Contains 7 tables and 24 references.) (SLD)

This paper summarizes a study that assessed the effects of an extended case study that focused on wetland issues with seventh and eighth grade students. The extended case study is an instructional methodology that incorporates the issue investigation/evaluation and action training model. A modified pretest-posttest nonequivalent control group design was utilized with fifteen intact classes from Illinois and Missouri. Posttest data were collected on the variables of overt environmental behavior, knowledge of ecological foundations, individual locus of control, group locus of control, knowledge of citizenship action skills, and perceived skill in the use of citizenship action skills. Analysis of covariance was used to compare treatment groups and control group means. Statistically significant differences were found with the variable overt environmental behavior. The two experimental treatments were found to be more effective than the control, and the full treatment was found to be more effective then the partial treatment in increasing overt environmental behavior. (Author)

Among the plethora of cooperative learning studies, several investigations of cooperative or group assessment (CA) have appeared. In this investigation, the methodology of R. R. McCown (1992) is refined to examine CA in a classroom setting, primarily examining achievement outcomes and student perceptions of the CA process. Fifty-six undergraduates in educational psychology participated in a nonequivalent control group design. It was hypothesized that students taking a test using a CA procedure would perform significantly better on a posttest of course concepts than would students completing the test in traditional format. The CA group completed examinations individually and in groups, while the traditional group (TA) took examinations individually. Analysis of covariance did not support the hypothesis. However, student reactions to the CA process were overwhelmingly positive. An appendix gives means, modes, and standard deviations for the group assessment survey. (Contains 13 references.) (Author/SLD)

By employing a concomitant variable, researchers can reduce the error, increase the precision, and maximize the power of an experimental design. Blocking and analysis of covariance (ANCOVA) are most often used to harness the power of a concomitant variable. Whether to block or covary and how many blocks to be used if a block design is chosen become important. This paper provides an historical review of the problem and recommends future research to examine the problem based on how subjects are assigned, how data are analyzed, and the distributions of the variables. In this study, subjects were randomly assigned to treatments ignoring the concomitant variable, and data were analyzed by one-way analysis of variance (ANOVA), post-hoc two-block, four-block, and eight-block ANOVA and ANCOVA. Distributions of the concomitant and dependent variables were normal. The Monte Carlo method was used to generate 20,000 data sets for 8 experimental conditions (2 levels of subject and 4 levels of correlation between concomitant and dependent variables. The five analysis procedures were examined under each experimental condition. Results show that ANCOVA is more powerful than post-hoc rank blocking. Eight tables present analysis results. (Contains 36 references.) (Author/SLD)

The Pearson product-moment correlation, r, is commonly applied in educational research. Almost all researchers realize that r ranges between -1.00 and +1.00, and that negative coefficients indicate that the bivariate relationship is inverse. Researchers also recognize that the Pearson r only evaluates linear relationship, and is not sensitive to curvilinear relationship. However, few researchers, if pressed, could explain exactly what makes r negative or positive, from a mathematical point of view, even though most researchers know what such results mean. The present paper explores the factors that affect r, including those that impact its sign. The reasons for preferring r over the covariance are explored. Small data sets and graphs are employed to make the discussion concrete. Four figures, four tables. (Contains 8 references.) (Author)

The purpose of this study was to investigate whether systematic, non-zero differences between pairs of item bank b-values have occurred in the recent history of two licensure examinations. Licensing examinations were studied for two related health care professions (Program 1 and Program 2). A series of analysis of covariance models was fit to the data in order to investigate the magnitude of changes in item bank b-values and the relationship of any changes to variables indexing factors that have been documented to affect the stability of item parameters. For Program 1, one 300-item form was used, and for Program 2, two 203-item forms were used to obtain a larger number of items. Analyses were first performed to evaluate the relationship, if any, between the dependent variable and the covariates. For both examinations, b-value differences across pairs of item administrations were not influenced by the changing position of the item in different forms. There were systematic changes as a function of time elapsed from a baseline, as discussed in the paper. Item parameter or scale drift was noted for one examination, while bank or pool drift was found for both. Factors that moderated bank drift, and differences between actual form cutscores and cutscores adjusted for documented bank and scale drift ranged from one to five points for the two examinations. Nine tables and five figures present details of the analyses. (SLD)

Conventional statistical significance tests do not inform the researcher regarding the likelihood that results will replicate. One strategy for evaluating result replication is to use a "bootstrap" resampling of a study's data so that the stability of results across numerous configurations of the subjects can be explored. This paper illustrates the use of the bootstrap in a canonical correlation analysis. Canonical correlation analysis is the most general case of classical general linear model analyses, subsuming other univariate and multivariate parametric method (e.g., t-tests, analysis of variance, analysis of covariance, regression, multivariate analysis of variance, and discriminant analysis) as special cases. A sample of 50 out of 301 subjects from a study by K. J. Holzinger and F. Swineford (1939) is used. Since bootstrap analyses capitalize during resampling on the commonalities inherent in a given sample, they yield somewhat inflated evaluations of replicability. However, inflated empirical evaluations of replicability are often superior to a mere presumption of replicability. Ten tables and one figure present details of the analysis. A 63-item list of references and an appendix listing the 50 analysis cases are included. (Author/SLD)

Various realizations have led to less frequent use of the "OVA" methods (analysis of variance--ANOVA--among others) and to more frequent use of general linear model approaches such as regression. However, too few researchers understand all the various coefficients produced in regression. This paper explains these coefficients and their practical use in formulating interpretations of regression results. A small heuristic data set of 20 subjects is used to make the discussion more concrete and accessible. It is argued that sensible interpretation of regression results usually must invoke an examination of both beta weights and structure coefficients. Six tables and two figures illustrate the discussion. Three appendices provide details of the calculations, and there is a 20-item list of references. (Author/SLD)

Analysis of covariance (ANCOVA) is a data analysis method that is often used to control extraneous sources of variation in non-equivalent group designs. It is commonly believed that as long as the covariate is highly correlated with the dependent variable there is nothing to lose in using ANCOVA, even in non-randomized studies. This paper examines some of the conditions that lead to successful and unsuccessful criterion source adjustments, and demonstrates that under certain circumstances, ANCOVA may perform in a manner that is antithetical to its intended purpose. Several hypothetical data sets were constructed, each with 70 observations, to illustrate two examples of appropriate ANCOVA use and two examples of inappropriate results. ANCOVA may serve to introduce confounding variables into the analysis when covariates represent differences between groups that are unrelated to outcome measures. Two tables present information from the analyses. A 26-item list of references is included. (Author/SLD)