Search

Standardized testing for college admissions has grown exponentially since the first administration of the old "College Boards" in 1901. This paper surveys major developments since then: the introduction of the "Scholastic Aptitude Test" in 1926, designed to tap students' general analytic ability; E.F. Lindquist's creation of the ACT in 1959 as a competitor to the SAT, intended as a measure of achievement rather than ability; the renewed interest on the part of some leading colleges and universities in subject-specific assessments such as the SAT Subject Tests and Advanced Placement exams; and current efforts to adapt K-12 standards-based tests for use in college admissions. Looking back at the evolution of admissions tests, it is evident that we have come full circle to a renewed appreciation for the value of achievement tests. The original College Boards started out as achievement tests, designed to assess students' mastery of college-preparatory subjects. A century of admissions testing has taught us that this initial premise may have been sounder than anyone realized at the time. But the journey has been useful, since we now have much better understanding of why assessment of achievement and curriculum mastery remains vital as a paradigm for admissions testing. Curriculum-based achievement tests are the fairest and most effective assessments for college admissions and have important incentive or "signaling effects" for our K-12 schools as well: They help reinforce a rigorous academic curriculum and create better alignment of teaching, learning, and assessment all along the pathway from high school to college. [This paper served as the basis for an invited address by the first author at the annual meeting of the American Educational Research Association in San Diego, California on April 15, 2009.] (Contains 3 footnotes.)

This study examined the relationship between students' self-reported high school grade point average (HSGPA) from the SAT Questionnaire and their HSGPA provided by the colleges and universities they attend. The purpose of this research was to offer updated information on the relatedness of self-reported (by the student) and school-reported (by the college/university from the high school transcript) HSGPA, compare these results to prior studies, and provide recommendations on the use of self-reported HSGPA. Results from this study indicated that even though the correlation between the self-reported and school-reported HSGPA is slightly lower than in prior studies (r = 0.74), there is still a very strong relationship between the two measures. The following tables and figure are appended: (1) HSGPA Grading Scales Across Higher Education Institutions in the Study; (2) Recoding of School- and Self-Reported HSGPA; (3) Descriptive Statistics for the Academic Measures (N = 40,301); (4) Self-Reported Versus School-Reported HSGPA Accuracy: Correlations, Percentage of Exact HSGPA Match, Underreporting and Overreporting of HSGPA in Grade Steps by Race/Ethnicity, Parental Income, Parental Education Level and SAT Score Band; (5) Accuracy of Self-Reported HSGPA by HSGPA Value; (6) Percentage of Exactly Matching, Underreporting and Overreporting of HSGPA by Demographic Characteristics in the SAT Score Band; and (7) Self-Reported HSGPA Item on the SAT Questionnaire (2005-2006).

Performance was examined for five cohorts of 1998-2002 Texas public high school graduates through their first year and 1998-2001 cohorts through their fourth year of Texas public higher education. Student performance on college outcomes included (a) first- and fourth-year grade point averages (GPAs), (b) first- and fourth-year credit hours earned, and (c) four-year graduation status. Outcomes were compared across students who varied by three types of AP® (course only, exam only, and both course and exam) and two types of non-AP (dual enrollment only and other course only) experiences in high school. Phase 1 analyses addressed each of seven AP Exams separately. Within each cohort year and within each cohort related to the specific AP Exam, students in the various AP and non-AP groups were matched on ability (ranked SAT® score category) and free or reduced-price lunch (FRPL) participation status. Students in each of the separately analyzed AP English Language and Composition, AP English Literature and Composition, AP Calculus AB, and AP U.S. History course and exam groups, for example, significantly outperformed their respective AP course only, dual enrollment, and other course peers on the GPA, graduation rate, and most credit hours outcomes. Phase 2 analyses aggregated AP course/ exam and non-AP experiences, taking into account SAT score category and FRPL status. Students who took AP courses and exams significantly outperformed students who took AP courses only and other courses only on all outcomes in all years and, in general, exceeded AP Exam only and dual enrollment only group performances on the outcomes. They also graduated from college within four years at consistently higher rates than those in the other groups. Results provide strong evidence of benefits to students who participate in both AP courses and exams in terms of higher GPAs, credit hours earned, and four-year graduation rates. The following are appended: (1) Correspondences Between AP Exams/Courses and Dual Enrollment Courses in Phase 1; and (2) Correspondences Between AP Exams/Courses and Dual Enrollment Courses in Phase 2.

The report provides detailed information about the methods and instruments used to evaluate school readiness initiatives, discusses important considerations in selecting instruments, and provides resources and recommendations that may be helpful to those who are designing and implementing school readiness evaluations. Study results indicate that state and local evaluators have used a variety of instruments to collect child outcome data, some that are well known and others that are not. In general, many of the better-known instruments demonstrate adequate psychometric properties (reliability and validity, which ensure that the instruments consistently measure what they were intended to measure), but a number of issues, such as the appropriateness of the measure to the study's purpose and sample, appear to present substantial challenges in evaluations of state- and locally-funded school readiness programs. Recommendations based on the data collected from this sample are provided to help school readiness programs and evaluators as they select instruments for assessing programs and implementing the evaluations include: (1) Careful selection of outcomes for assessment that match the goals of the program and address the components of children's learning and development that are linked with later success in school; (2) Clear definition of the purpose for which assessment data will be collected, and selection of instruments designed and validated for that purpose; (3) Selection of instruments that have a proven track record; (4) Selection of instruments that are culturally and linguistically appropriate; (5) Consideration of data collection (internal/external); (6) Assessment administration, including training and reliability studies; and (7) Data collection in context. Report findings highlight the challenges that evaluators face in ensuring that data are collected in a manner that yields credible, trustworthy, and meaningful information about child outcomes. The report cites a number of resources that can assist evaluators in making decisions about child assessments: resources to guide decisions about how to assess child outcomes, reviews of measures, and web sites with technical information related to measures used in large federal studies. Four appendixes include: (A) Methodology; (B) Coding Sheet and Evaluations Reviewed; (C) Evaluation Reports Reviewed, by State and Program; and (D) Descriptions of Most Commonly Used Instruments. (Contains 4 boxes, 6 figures, 4 tables, and 28 notes.) [This report was prepared for the Institute of Education Sciences (IES) by Regional Educational Laboratory Southeast administered by SERVE Center, University of North Carolina at Greensboro.]

In September 2005 the National Foundation for Educational Research (NFER) in association with the Department for Education and Skills (DfES), the Sutton Trust and the College Board, began a five-year research study to examine the validity of an aptitude test in higher education admissions. This report describes and explores the relationships between scores on the SAT Reasoning Test[TM] (also known in brief as the SAT[R]), attainment measures (A levels and GCSEs [General Certificates of Secondary Education]) and background characteristics of the student sample. The SAT[R] has three measures: Critical Reading, Mathematics and Writing. This report contains nine sections. Following an executive summary, section 2 describes the background to the research and the aims and objectives of the study. Section 3 outlines the methodologies employed and section 4 details the representation and background characteristics of the main SAT[R] sample and the subsequent sub-samples who took part in the student surveys in spring and autumn 2006. Section 5 reports the findings of the analysis of the quantitative data examining the relationships between the SAT[R] and both A level and GCSE attainment. These results are then explored in relation to two key research questions in sections 6 and 7, namely: Can the SAT[R] identify students with the potential to benefit from higher education whose ability is not adequately reflected in their A level results because of their economically or educationally) disadvantaged circumstances (section 6); and can the SAT[R] distinguish helpfully between the most able applicants who get straight As at A level (section 7)? Section 8 presents the findings from the questionnaire surveys and section 9 outlines future potential phases of this research study and suggested milestones when further results will be disseminated. Appended to this report are: (1) Spring 2006 Student Questionnaire; (2) Autumn 2006 Student Questionnaire; (3) SAT[R] Descriptive Statistics; and (4) Functioning of the SAT[R]. (Contains 44 tables and 24 figures.) [This report was produced by the Department of Education and Skills. Additional funding for this project was provided by the Sutton Trust.]

Part of the third-largest school district in the nation, 77 percent of Chicago's nearly 427,000 public school students are poor, as defined by eligibility for participation in free or reduced-cost lunch programs. Many are members of minorities; many have limited proficiency in English. To make a successful transition to college--even to graduate from high school--many young people in the Chicago school system face hurdles unimaginable in the nearby suburbs. The Chicago Public School system (CPS) wants to increase the college readiness of the city's students. Its goal is to increase the numbers of students who take a rigorous curriculum and set their sights on getting into the most challenging college for which they are qualified. The CPS also wants to improve high school graduation rates and college attendance rates, and to provide a basis for curricular interventions to improve student performance. This paper describes how CPS was able to monitor the effectiveness of its curriculum and improve its college readiness through the implementation of EXPLORE[R], PLAN[R], and the ACT programs.

Our research question was whether we could develop a feasible technique, using Bayesian networks, to diagnose gaps in student knowledge. Thirty-four college-age participants completed tasks designed to measure conceptual knowledge, procedural knowledge, and problem-solving skills related to circuit analysis. A Bayesian network was used to model the knowledge dependencies among the circuit analysis concepts. Preliminary results suggested that the Bayesian network was generally working as intended. When high- and low-performing groups were formed on the basis of posterior probabilities, significant group differences were found favoring the high-performing group with respect to circuit definitions and circuit analysis problems, for both actual and self-assessments, and higher major GPA. The Bayesian network was able to predict participants' performance on a problem-solving item on average 75% of the time. The findings of this study are promising for our long-term goal of developing scalable and feasible online automated reasoning techniques to diagnose student knowledge gaps. (Contains 12 tables and 2 figures.) [Appended are: (1) Node-Voltage Analysis Problem-Solving Procedure (Kaiser, 2003); and (2) Bayesian Network.]

This Policy Brief describes the methods a growing number of states are using to place college readiness indicators in high school coursework and assessments. To help ensure that students are prepared for college-level work, a number of states embed college readiness indicators in curriculum and assessments. This policy brief presents how several states have implemented this practice--often through legislation--at the local, state, and district levels. It is concluded that, while there is no "right path" for states to follow, a variety of options in curriculum and assessment alignment have successfully identified students needing remediation or additional coursework before college, and inspired students who did not consider themselves "college material" to go on to complete postsecondary education. (Contains 15 endnotes.)

Computer Adaptive Sequential Testing (CAST) is a test delivery model that combines features of the traditional conventional paper-and-pencil testing and item-based computerized adaptive testing (CAT). The basic structure of CAST is a panel composed of multiple testlets adaptively administered to examinees at different stages. Current applications of CAST reply on the item response theory (IRT) and assume a unidimensional IRT model for scoring. This study evaluated the robustness of CAST when tests were constructed, administered, and scored by a unidimensional IRT model but item responses were multidimensional. Various conditions of multidimensionality were simulated in item pools, as well as different levels of content misclassification through manipulation of the correspondence between content area and dimension of items. An automated test assembly (ATA) process constructed CAST panels from the item pools, each representing a unique combination of multidimensionality and content misclassification. Administration of the panels was simulated and multidimensional response data were scored by the unidimensional IRT model. The ability scores, routing decisions, and pass-fail decisions were evaluated against "true" ability scores and decisions to assess the impacts of multidimensionality and content misclassification. Results showed that, when multidimensionality was mild as measured by the angle distance between item clusters, unidimensional ability estimates and routing decisions were not sensitive to the level of content misclassification in item pools. (Contains 8 tables and 14 figures.)

This study employed logistic regression to predict the probability that a student would be successful in achieving a FGPA at various levels, based on that student's SAT scores and high school grade point average. Tables of accuracy rates are appended.

The cultural and socioeconomic diversity of the U.S. school population is now and long has been underrepresented in programs for academically advanced students (see, e.g., Donovan & Cross, 2002; Marland, 1972). In the past decade, however, educators have offered several proposals for increasing the diversity of programs for the gifted and talented (see Boothe & Stanley, 2004, for one compendium of views). Some educators would make greater use of parent, teacher, and self-rating scales. Some would emphasize nonacademic talents--such as musical, athletic, and leadership abilities. Others call for the use of different measures--especially nonverbal ability tests. Although each of these proposals has something to offer, none directly addresses the central problem in identifying academic talent. All hope to find ways of measuring academic promise that will simultaneously reduce average differences between ethnic groups and identify those minority students who currently display or who are most likely to develop academic excellence. Unfortunately, just because a measure shows a smaller average difference between groups does not mean that it is a better tool for identifying academically talented students. Indeed, research has consistently shown that academic achievement in minority children is best predicted by the same cognitive and affective characteristics that predict academic achievement in majority children (Keith, 1999; Lohman, 2005). Therefore, although attempts to select students using other criteria may identify more minority students, many of the students identified--minority or majority--will not be the ones who currently display or who will someday develop academic excellence. The problem, then, is to find a way to identify and assist these children without compromising the ability of programs to serve children who already display high levels of academic and cognitive development. The goals of this monograph are (a) to explain why an aptitude approach to talent identification accomplishes these goals and (b) to illustrate how schools can implement this approach. (Contains 8 footnotes, 3 tables and 8 figures.)

The Education Policy Institute (EPI) conducted an analysis of recent academic data collected from 24 KIPP (Knowledge is Power Program) schools on behalf of the KIPP Foundation. The purpose of this study was to evaluate these data to determine whether KIPP schools have had a positive impact on the student learning of 5th-grade students as demonstrated by standardized test results in comparison to national norms. Findings from this study show that 5th-grade cohorts at KIPP schools post substantially greater academic gains on the Stanford Achievement Test (SAT) than what is considered normal, a finding consistent with prior research on KIPP schools (Doran and Drury, 2002). Given that a growth score of zero on the normal curve equivalent (NCE) is considered "normal growth," schools administering the tests first in the fall with a follow-up test in the spring enjoyed a mean gain of 10.1 in reading, 10.9 in language, and 17.4 in mathematics. Schools that first administered the test in the fall and then again the following fall recognized score gains of 7.5 in reading, 9.1 in language, and 11.6 in mathematics. As the data indicate, KIPP 5th-grade cohorts experienced average gains of 9 to 17 points across all tests. Methodology and School-by-School Results are appended. (Contains 8 footnotes, 5 figures and 8 tables.) [This document was published by the Educational Policy Institute (EPI) for the KIPP Foundation.]

The aim of this study was to report the development and testing of a model explaining gender differences in the interrelationships among aptitude measures, high school mathematics and science preparation, college academic level, motivational and self-regulatory variables, and grade point average using structural equation modeling and multiple groups analysis for undergraduate females (n=312) and males (n=158). Data were gathered using a new instrument entitled Learning Strategies and Study Skills survey (Ruban and Reis, 1999). First, the study investigated gender differences in precollege scholastic and academic factors, motivation, self-regulation, and academic achievement of postsecondary students. Second, the study simultaneously examined the interrelationships among these cognitive and noncognitive variables for the entire sample of students and for males and females within the structural modeling framework. Tests of interrelationships helped to attain a more complete understanding of the factors that impact college students academic performance, and whether these factors exert similar or differential influence on the academic attainment of males and females. Results of the study provide support for gender invariance for the measurement and structural model for males and females. Even though no gender differences were found, interesting patterns emerged regarding the impacts of precollege scholastic and academic factors and self-regulated learning factors on college academic achievement among male and females collegians. These paths of influence are interpreted in terms of social cognitive theory and research conducted within the framework of gender differences. (Contains 2 figures, 6 tables, and 92 references.) (Author/SLD)

Even though research on academic self-regulation has proliferated in recent years, no studies have investigated the question of whether the perceived usefulness and the use of standard self-regulated learning strategies and compensation strategies provide a differential prediction of academic achievement for college students with and without learning disabilities (LD), after controlling for standardized aptitude measures (such as the Scholastic Assessment Test) and academic level in college. This study reports the development and testing of a model explaining the interrelationships among aptitude measures, self-regulatory variables, and grade point average using structural equation modeling and multiple groups analysis for students with LD (n=53) and students without LD (n=417). Data were gathered using a new instrument survey entitled "Learning Strategies and Social Skills" (Ruban and Reis, 1999). The results indicate that students with LD differed significantly from students without LD in the relationships among standardized aptitude measures, academic level, and the use of standard self-regulated learning strategies and compensation strategies, which, in turn, provided a differential explanation of academic achievement for students with and without LD. These paths of influence and idiosyncrasies of academic self-regulation among students with LD are interpreted in terms of social cognitive theory and research conducted in the field of learning disabilities. (Contains 4 figures, 3 tables, and 79 references.) (SLD)

This paper examines the differences between student performance on two separate measures, the Spatial Visualization portion of the "Differential Aptitude Test" and the "Geometry Content Knowledge Test." Results from the hybrid quantitative/qualitative study indicate that although there were no differences in performance on spatial visualization for males and females or across ethnicities, differences in performance on geometry content knowledge tasks for ethnicity were evident. (Contains 35 references.) (Author/ASK)

Using data from a sample of 10 colleges at which most students had taken both SAT I: Reasoning Test and SAT II: Subject Tests, the authors simulated the effects of making selection decisions using SAT II scores in place of SAT I scores. Specifically, they treated the students in each college as forming the applicant pool for a more select college, and then selected the top two-thirds (and top one-third) of the students using high school grade-point average combined with either SAT I scores or the average of SAT II scores. Success rates, in terms of freshman grade-point averages, were virtually identical for students selected by the different models. The percent of African American, Asian American, and White students selected varied only slightly across models. Appreciably more Mexican American and Other Latino students were selected with the model that used SAT II scores in place of SAT I scores because these students submitted Subject Test scores for the Spanish test on which they had high scores.

Purpose: This paper will: (1) review the validity of the Graduate Record Examination for predicting international students' Graduate Grade Point Average (GGPA); and (2) inspect possible social, cultural and language bias or learning style differences in the prediction of international students' performance from Graduate Record Examination (GRE) test scores when used in admission / selection decisions at American universities. Results: Different studies demonstrated that: (1) International Students who were not homogeneous with respect to linguistic, cultural and educational background; (2) Subgroups with homogeneous country of origin and background variables; and (3) Subgroups classified according to English proficiency, as indicated by Test of English as a Foreign Language (TOEFL) scores; GRE-Verbal, Analytic, and Quantitative scores; and self-reported English language proficiency. Research indicated that for all subgroups of quantitatively-oriented departments, GRE-Quantitative scores and GRE-Analytical scores were more highly correlated with first year average grades (FYA) than were GRE-Verbal scores. Their correlation coefficients were .311, .275, and .097 for quantitative, analytical, and verbal cores, respectively Conclusions: Every year hundreds of students from foreign countries enroll in American graduate schools and this trend will probably continue in the future. The decision to admit foreign students into a graduate program affects their advanced education opportunity and future career. The admission of foreign students to graduate school is a very complicated issue. Unlike their American counterparts, foreign students often lack proficiency in the English language due to differences in their native language, learning style and their cultural, social and economical background. The appraisal of the foreign candidate's aptitude for graduate study by standardized admission tests also has pitfalls. Poor performance may be due to factors not directly related to aptitude for graduate study. Recommendations: In order for foreign students in the US universities to be appropriately classified and their academic characteristics properly assessed, the admission committees need to create or be provided an effective evaluation system. In order to do that, confounding problems related to the GRE and its validity and usability with foreign students must be examined. (Contains 4 tables.)

Data on the sophomore cohort of the original High School and Beyond (HS&B) study were analyzed to determine the value of the General Educational Development (GED) credential versus that of the conventional high school diploma in explaining the earnings of 27-year-old males in the early 1990s. The study sample consisted of those 4,899 male HS&B sample members who were in grade 10 in 1980 and who participated in the fourth follow-up survey in 1992. The study replicated the basic findings of prior studies, which implicitly assumed that the GED's labor market value does not depend on the skills with which dropouts left school. The GED's effects on subsequent employment and earnings varied according to individuals' academic aptitude as measured by a standardized test administered during the original study. Obtaining a GED was associated with higher earnings at age 27 for those male dropouts who had very weak cognitive skills in grade 10, but not for those who had stronger cognitive skills in grade 10. The GED was deemed a key step for further education and training. (The document contains 24 references and 35 endnotes. The procedures used to select the HS&B sample and identify sample members' educational status are appended.) (MN)

It has been proposed that the federal government's General Aptitude Test Battery (GATB) might be used to provide private employers with the same kind of useful information the federal government and college counselors receive about job applicants. The U.S. Department of Labor asked the National Research Council (NRC) to review the issues related to use of the GATB for screening all job applicants and providing employers with the job applicant's GATB scores. The NRC formed a committee to review the issue, but the committee did not include any industrial organizational psychologists who had studied the GATB. The Committee recommended against the use of the GATB for a number of reasons. These reasons are reviewed, and reasons the report development process and the report itself are flawed are discussed. Criticism of the NRC process centers on the unusual composition of the review committee, the argument the committee made about the meager evidence of benefits from using the GATB, disagreement with the committee's conclusion about the "zero-sum labor market," and some contradictory positions taken in the report. The NRC committee is charged with bias in its evaluation. (Contains 1 figure and 22 references.) (SLD)

The relationship between coaching and subsequent performance on the SAT I is a complex issue. Overall, anecdotal reports from one or two students and even studies that gauge the effects of coaching by examining the score gains for students who have enrolled in these programs are highly misleading and will not present an accurate picture of the true effects of coaching. Collectively, these studies indicate that coaching does have some benefit. However, though consistent, the effect is small. A typical student enrolling in a formal test preparation program can expect a total score gain of between 25 to 32 total points on the verbal and math sections beyond what they could expect to gain without attending such a program.

An important but too infrequently considered methodology that can be used in research, especially research focusing on Pearson's "r" (B. Walsh, 1996) or the multiple "R" (see B. Thompson, 1992) involves testing for moderator or mediator variable effects. As explained by M. Gall, W. Borg, and J. Gall (1996), moderator analysis "involves identifying a subgroup for whom the correlation between a criterion and a predictor variable is significantly greater than the correlation for the total sample from which the group was formed" (p. 425). A study by M. Zeidner (1987) provides a concrete example of this methodology. Zeidner studied the relationship between aptitude test performance and grade point average. However, he believed that this relationship would be moderated by age (be different across age groups). Therefore, he computed "r" across four different age groups, and found that values differed considerably across age, as expected. Techniques for evaluating these effects are explained and illustrated. In a sense, these dynamics can be related to the "restriction of range" effect (an influence on "r" caused by the variance in the sample; correlation will increase or decrease depending on the range of subjects studied) so widely misunderstood within the social sciences. (Contains 2 figures and 17 references.) (Author/SLD)

This document describes 32 recently developed career assessment instruments that have been deemed useful for school-to-work programs. The following instruments are among those profiled: Ability Explorer; Adult Measure of Essential Skills; Aptitude Interest Inventory; Ashland Interest Assessment; Barriers to Employment Success Inventory; Basic Skills Locator Test; Career Anchors: Discovering Your Real Values, Revised; Career Planning Survey; Career Profile Assessment; Career Scope; Career SnapShot 2001; Career Thoughts Inventory; COIN Basic Skills and Career Interest Survey; Employability Maturity Interview; Employee Reliability Inventory; Guide for Occupational Exploration Inventory; Job Search Attitude Inventory; Leisure to Occupations Connective Search; Leisure/Work Search Inventory; Magellan; Occupational Clues; School to Work Career Survey; Strategic Assessment of Readiness for Training; Transition Planning Inventory; Vocational Interest Assessment System; Wonderlic Basic Skills Test; Work Adjustment Inventory; Work Adjustment Scale; Workmate; and Work Temperament Inventory. A table that constitutes 75% of the document presents the following information for each instrument: name; publisher; date of publication; cost of an individual test; intended audience; publisher's phone number and Web page address; number of scales, administration time; availability or machine or hand scoring; availability of a computer-based version; types of scores reported, references to reviews; brief description, and publisher's E-mail address. (MN)