Search

The purpose of this study was to determine the measurement comparability of paper-and-pencil and multimedia versions of two career assessment components in the DISCOVER multimedia career and education planning system. Students in grade 9 (N=606) and 11 (N=416) completed paper-and-pencil and Compact Disc-interactive (CD-i) versions of the Unisex Edition of the American College Testing Inventory (UNIACT) and the Inventory of Work-Relevant Abilities (IWRA). Measurement comparability of the versions was evaluated by comparing mean scale score differences, correlations between corresponding scales, and patterns of scale intercorrelations. In addition, targeted factor analyses were run for UNICAT and principal components analysis for UNIACT and IWRA. The results indicate that the use of pictures and voice-overs in the multimedia versions of the assessments yields scores comparable to those obtained with the paper-and-pencil versions. Appendixes contain the career guidance survey and a discussion of test equating. (Contains 4 figures, 18 tables, and 52 references.) (Author/SLD)

In recent times social capital has received considerable attention because it is seen as having the potential to address many of the problems facing modern society, including the poor educational outcomes of considerable numbers of young people. This paper uses data from the Longitudinal Surveys of Australian Youth (LSAY) to explore the relationship between social capital at age 15 and participation in education and training at age 17. The issue is whether social capital is yet another factor which advantages the already advantaged, or whether social capital operates separately from family background. Findings reveal that social capital influences educational participation "over and above" the effects of background characteristics such as parents' education levels, parental occupation, geographic location, cultural background, school sector and academic achievement. For both males and females, participation in a diverse range of activities has the greatest influence on participation in education and training, followed by the strength of the relationship students have with their teachers. Increasing rates of participation in sport also increase educational participation for females. The authors note that, if anything, the findings underestimate the net effects of social capital because the results cannot fully account for the accumulation of social capital prior to the age of 15. The finding that social capital matters for school education is a very positive one. It implies that activities that promote and encourage engagement at school can go some way to redressing economic and social disadvantage. Appended are: (1) Factor analysis; (2) Distribution of social capital; (3) Logistic regression (1); (4) Logistic regression (2); and (5) Distribution of social capital by academic orientation. (Contains 10 tables, 12 figures and 6 footnotes.)

While the importance of effective principals is undisputed, few studies have addressed what specific skills principals need to promote school success. This study draws on unique data combining survey responses from principals, assistant principals, teachers and parents with rich administrative data to identify which principal skills matter most for school outcomes. Factor analysis of a 42-item task inventory distinguishes five skill categories, yet only one of them, the principals' organization management skills, consistently predicts student achievement growth and other success measures. Analysis of evaluations of principals by assistant principals confirms this central result. Our analysis argues for a broad view of instructional leadership that includes general organizational management skills as a key complement to the work of supporting curriculum and instruction. Two appendices are included: (1) Factor Loadings Matrix for Principal Effectiveness Factors; and (2) Factor Loadings Matrix for Assistant Principal Effectiveness Factors. (Contains 3 figures, 5 tables and 6 footnotes.) [This paper was supported by the Stanford University K-12 Initiative.]

The purpose of this paper is to help higher education institutions determine how well their institutional effectiveness models match the North Central Association of Colleges and Schools Commission on Institutions of Higher Education Criterion Four requirements. The paper describes each of the major requirements as a domain in a matrix with cross-related subcategories. The matrix is built around six issues: (1) activities to be measured need to be established, (2) perspectives of the constituencies and related criteria need to be identified, (3) the level of analysis should be dependent on constituent perspectives and expectations, (4) the time frame determines how data are measured, (5) the types of data needed are related to perspectives and levels of analysis, and (6) the referent used to assess indicators is determined by the dominant constituency. The paper suggests that institutions use this matrix to assess the degree to which their institutions match the elements required in Criterion Four, and also to identify strengths and areas for improvement. (Contains 30 references.) (CH)

This study examined the internal consistency of the Student Satisfaction Inventory (SSI) (L. Schrener, S. Juillerat, and Noel-Levitz Group, 1997) and the latent structures of the instrument. The SSI is an instrument that measures students' satisfaction with their college experiences. The SSI was administered to 251 students randomly selected from the student population. Findings indicate that the SSI is internally reliable both as an entity and on the subscale level. They also indicate that the SSI appears to elicit two latent factors. (Author/SLD)

This report describes a study conducted by the Audio-Visual Media Research Group of the Open University to investigate student attitudes, over time, to the value of radio as a course component. Results of a January 1979 survey using a questionnaire are reported with tables and analyzed according to (1) whether a student had access to a VHF radio at home, (2) the way in which a student already used radio, (3) any experience a student may have had with Open University radio before beginning a foundation course, and (4) any expectations students might have of the role radio could play as a course component and how confident they were that they could use radio in this way. The appendices include the questionnaire, data on the samples used, and the percentage of responses to individual questionnaire items. (CHC)

Over the past 15 years a body of research has investigated organizational transitions in adolescent development. Researchers studying school transitions exclusively ask whether significant transition affects students' social and psychological development. To test the presence of transition effects, researchers have typically observed mean differences on certain observed outcomes between the students who experienced school transition and those who did not. This paper shows that much more could be learned about organizational transitions if investigators change their operational definition of transition effects from "mean difference" to "an effect being in a transition group." The new definition changes the research question from "whether" to "how much," and accounts for the uncertainties of transition effects as well as the outcome under investigation by multilevel modeling. The paper demonstrates multilevel modeling by analyzing longitudinal data from NELS:88 to identify the effects of between-school transitions on student self-esteem. The final sample for this paper was composed of 483 students in 100 middle schools and 113 high schools, who were observed at the 8th and the 10th grades. The analyses show the estimates of transition effects and identify the components of transition effects, the effects of individual background variables, past and present organizational characteristics, and the interaction effects between past and present organizations. Multilevel modeling also allows investigation of the interaction effects between individual backgrounds and school characteristics. Twelve tables and 3 figures are included. (Contains 19 references). (LMI)

The purpose of this study was twofold. First, it investigated preservice and inservice teachers' observed and experienced violence at various educational levels. Second, the study determined by gender, these individuals' knowledge and awareness of violence factors. The subjects consisted of 26 university preservice and four inservice teachers in two sections of an introductory special education class at a small southeastern university. The preservice teachers consisted of elementary, secondary, and K-12 special education majors. The inservice teachers consisted of 2 males and 2 females. A survey questionnaire was developed, validated, and administered. Participants indicated their school safety-violence observations and experiences from grade school through college, and their agreement or disagreement to a minimum of 10 statements listed under five safety-violence factors. The results indicated that subjects had differences in violence experienced in schools. The results also suggested that subjects had similar gender perceptions of school violence, but differences occurred in their perceptions of acts of school violence. Limitations and implications for future research and school safety-violence prevention programs are presented. (Contains 7 tables.) (Author/JDM)

In an earlier study, I. Newman and D. Waechter (1999) investigated graduate education students' perceptions and attitudes toward cheating in scientific research. A questionnaire was developed that was based on reactions and concerns related to cheating behavior seen in a public television video about faking scientific data. This study examined the factor structure of the developed Cheating Scale, which was administered to 167 graduate education students. Analysis identified a three-factor solution that accounted for approximately 65% of the trace. The factors were identified as Ethical/Critical Evaluation, Academic Pressure, and Human Nature. Students who scored above the median on questions about their personal dishonesty had a more complex (four-factor) factor structure on the Cheating Scale. Males and females had significantly different perceptions of cheating, with females more accepting of these behaviors than males. (Contains 20 tables and 13 references.) (SLD)

Adapting credentialing examinations for international uses involves translating tests for use in multiple languages. This paper explores methods for evaluating construct equivalence and item equivalence across different language versions of a test. These methods were applied to four different language versions (English, French, German, and Japanese) of a Microsoft certification examination with samples ranging from 1,329 to 2,000 examinees per test. Principal components analysis, multidimensional scaling, and confirmatory factor analysis of these data were conducted to evaluate construct equivalence. Detection of differential item functioning across languages was conducted using the standardized p-difference index. The results indicate that these procedures provide a great deal of information useful for evaluating test and item functioning across groups. Some differences in factor and dimension loadings across groups were noted, but a common, one-factor model fit the data well. Four items were flagged for differential item functioning across all groups. Suggestions for using these methods to evaluate translated tests are provided. (Contains 8 tables, 3 figures, and 13 references.) (Author/SLD)

This paper presents the development and initial validation of new measures of critical thinking and noncognitive attributes that were designed to supplement existing standardized tests used in the admissions system for higher education in Chile. The importance of various facets of this process, including the establishment of technical rigor and political support of instrument development, piloting, and implementation is also emphasized because the results from this pilot study could affect the admissions process for some of the most prestigious and competitive universities in the country. The new instruments were pilot tested on a group of high school seniors (n = 1,568) and first-year college students (n = 1,443) during 2008. The sample included students from 20 secondary schools and 4 universities. Initial results were promising and showed that the measures were effectively assessing attributes not currently considered in the Chilean higher education admissions process. In addition, performance on the new instruments was found to be less correlated with students' sociodemographic characteristics than were measures currently used for undergraduate admissions. Appended are: (1) Factor Analysis of Critical Thinking Essay; (2) Factor Analysis of the Personal Reflection Instrument; (3) Pearson Partial Correlation Coefficients Among Standardized Questionnaires Controlling for Social Desirability; (4) Factor Analysis of All Likert Scale Self-Ratings' Items Combined; and (5) Factor Analysis of Teacher Survey. (Contains 19 tables, 4 figures and 20 notes.)

ETS has recently instituted the Cognitively Based Assessments of, for, and as Learning (CBAL) research initiative to create a new generation of assessment designed from the ground up to enhance learning. It is intended as a general approach, covering multiple subject areas including reading, writing, and math. This paper is concerned with the writing assessment being developed within the CBAL framework, and in particular, with the potential for using automated scoring techniques effectively within such an assessment to support learning. Appended are: (1) Feature Sources; (2) Additional Tables (e-rater Microfeatures); and (3) Existing and Proposed Scoring Rubrics for Cognitively Based Assessments of, for, and as Learning (CBAL) Writing Strands I and II (as Used in the Pilot and Proposed Revision). (Contains 48 tables, 1 figure and 30 notes.)

Real world data often fail to meet the underlying assumption of population normality. The Rank Transformation (RT) procedure has been recommended as an alternative to the parametric factorial analysis of Covariance (ANCOVA). The purpose of this study was to compare the Type I error and power properties of the RT ANCOVA to the parametric procedures in the context of a completely randomized balanced 3 x 4 factorial layout with one covariate. This study was concerned with tests of homogeneity of regression coefficients and interaction under conditional (non)normality. Both procedures displayed erratic Type I error rates for the test of homogeneity of regression coefficients under conditional nonnormality. With all parametric assumptions valid, the simulation results demonstrate that the RT ANCOVA failed as a test for either homogeneity of regression coefficients or interaction due to severe Type I error inflation. The error inflation was most severe when departures from conditional normality were extreme. Also associated with the RT procedure was a loss of power. It is recommended that the RT procedure not be used as an alternative to factorial ANCOVA despite its encouragement from publishers of statistical software packages. (Contains 19 tables and 38 references.) (Author/SLD)

The central objective of factor analysis is to explain the greatest amount of variance in a data set with the smallest number of factors. Higher-order analysis is an invaluable tool that offers the benefit of parsimony provided by first-order analysis with the opportunity to make data-based generalizations beyond the first-order. Higher-order analysis provides a hierarchical framework that better honors the reality with which many phenomena in the social sciences are perceived. Interpretation of higher-order factors requires careful understanding and consideration on the part of the individual researcher. A step-by-step discussion of a real factor analysis is provided to make computer-based results more clear. (Contains 1 figure, 12 tables, and 8 references.) (SLD)

Career decision making is an important aspect of life. At some point one must decide what type of job to apply for or whether to seek education for a specific career. Many individuals struggle with these decisions. Unfortunately, multidimensional measures of uncertainty are needed in providing guidance to such persons, because it appears the indecision phenomenon is multidimensional. The present study involved the development of a 97-item multidimensional measure of 6 career-indecision factors. Based on data provided by 364 college students, scale scores on the Career Assessment Diagnostic Inventory (CADI) had reliability coefficients ranging from 0.83 to 0.94. Factor analysis results were supportive of a conclusion that CADI scores are construct valid. These results suggest that the CADI has considerable potential as a multidimensional measures of factors thought to be related to career indecision. An appendix contains the items retained in the final CADI version. (Contains 9 tables and 58 references.) (Author/SLD)