Search

Nearly two decades into the "completion agenda" in higher education, many community colleges have adopted collegewide reforms designed to improve stubbornly flat rates of student success and address persistent equity gaps. The longer-term effects of such collegewide reforms may take years to observe. In the meantime, college leaders need to know whether changes they make in the short run are associated with longer-term student success. Measuring the progress and effects of institutional reform is particularly vital in economically important STEM fields. Drawing on administrative records from transfer-intending community college starters across three states, this study develops and explores potential indicators of early STEM program momentum. We find that a relatively simple set of STEM momentum metrics--notably early completion of calculus or non-math STE coursework specified in statewide STEM transfer pathways and, to a lesser degree, the prerequisites to such courses--are reliable indicators of subsequent STEM transfer and bachelor's degree attainment. Our findings provide support for the use of the STEM momentum metrics to formatively evaluate reforms aimed at strengthening STEM transfer outcomes and closing equity gaps in STEM bachelor's degree attainment.

As part of its regulatory roles, the Australian Government Tertiary Education Quality and Standards Agency (TEQSA) is required to assess evidence of 'scholarship' that is put forward by providers, whether evidence of scholarship by individuals, such as teaching staff within the provider, or for the provider as a whole. TEQSA is seeking to review whether its current approach to assessing claims of scholarship and scholarly activity (as described in the Guidance Note on Scholarship) is adequate, or if the approach needs to be reconceptualised. The purpose of this discussion paper is to set out, for consideration by the sector and other stakeholders, draft principles that are proposed to guide providers in making claims related to scholarship, and to inform TEQSA's assessments of such claims. TEQSA's Guidance Note describes 'scholarship' as 'activities concerned with gaining new or improved understanding, appreciation and insights into a field of knowledge, and engaging with and keeping up to date with advances in the field.' This includes advances in ways of teaching and learning in the field and advances in professional practice, as well as advances in disciplinary knowledge through original research.' This discussion paper proposes a set of principles that would guide providers when offering evidence of their capacity to meet the requirements of the Higher Education Standards (HES) Framework in relation to scholarship. [For "Guidance Note: Scholarship. Version 2.5," see ED610488.]

The Community Eligibility Provision (CEP) is a policy change to the federally-administered National School Lunch Program that allows schools serving low-income populations to classify all students as eligible for free meals, regardless of individual circumstances. This has implications for the use of free and reduced-price meal (FRM) data to proxy for student disadvantage in education research and policy applications, which is a common practice. We document empirically how the CEP has affected the value of FRM eligibility as a proxy for student disadvantage. At the individual student level, we show that there is essentially no effect of the CEP. However, the CEP does meaningfully change the information conveyed by the share of FRM-eligible students in a school. It is this latter measure that is most relevant for policy uses of FRM data. [Portions of this paper were previously circulated under the title "Using Free Meal and Direct Certification Data to Proxy for Student Disadvantage in the Era of the Community Eligibility Provision" (ED595218). The authors have since split the original paper into two parts. This is the first part.]

Among community college leaders and others interested in reforms to improve student success, there is growing interest in adopting machine learning (ML) techniques to predict credential completion. However, ML algorithms are often complex and are not readily accessible to practitioners for whom a simpler set of near-term measures may serve as sufficient predictors. This study compares the out-of-sample predictive power of early momentum metrics (EMMs)--13 near-term success measures suggested by the literature--with that of metrics from ML-based models that employ approximately 500 predictors for community college credential completion. Using transcript data from approximately 50,000 students at more than 30 community colleges in two states, I find that the EMMs that were modeled by logistic regression accurately predict completion for approximately 80% of students. This classification performance is comparable to that of the ML-based models. The EMMs even outperform the ML-based models in probability estimation. These findings suggest that EMMs are useful predictors for credential completion and that the marginal gain from using an ML-based model over EMMs is small for credential completion prediction when additional predictors do not have strong rationales to be included in an ML-based model, no matter how large the number of those predictors may be.

This study compares the differences in the mismatch between the education and occupations of immigrants in Canada and the United States, operationalized by over-education. It further explores how the cross-country differences may be related to the supply of and demand for university-educated immigrants and the way they are selected. Using comparable data and three measures of over-education, this study found that university-educated recent immigrants in Canada were much more likely to be overeducated than their U.S. peers. The over-education rate gap between recent immigrants and the native-born was much more pronounced in Canada than in the United States. In addition, while labour market demand was associated with a lower level of over-education in both countries, a greater supply of university-educated recent immigrants was positively associated with a likelihood of over-education among recent immigrants in Canada, but not in the United States. Furthermore, in Canada, the over-education rate was significantly lower among immigrants who were admitted through some form of employer selection (e.g., immigrants who worked in skilled jobs in Canada before immigration) than those who were admitted directly from abroad. Overall, this study provides insight into how the immigration system interacts with broader aspects of the labour market to shape the labour market outcomes of immigrants.

Private school choice options are being proposed and adopted in numerous states across the country. As of the spring of 2019, 62 private school choice programs were in operation in 29 states and the District of Columbia, serving more than 400,000 children. Although growth in private school choice programs and enrollment has been considerable over the past decade, the percentage of private schools participating in school choice options varies considerably by program. Understanding how program design impacts the supply of private schools participating in these programs will be critical for policymakers as they work to create and expand education choice options for families. This report considers whether certain regulations governing school choice programs reduce specialization and diversity in the supply of private schools that participate. Expanding on the prior analysis of the impact of regulations in three locations, the authors chart how schools self-identify prior to and after switching into private school choice program environments in each of the five locations. The authors found that private school leaders in all five locations are more likely to classify their schools as less specialized than they were prior to entering the programs. The authors also found evidence to suggest that more homogenization occurred in more highly regulated voucher programs. These findings suggest a potential homogenizing effect of regulations on school supply, limiting the diversity of the private school market. These findings also suggest policymakers should carefully weigh the costs of regulating private school choice programs, as overregulation could reduce specialization and diversity of school supply in a given school choice program, limiting the options that are available to families.

This study develops, validates, and applies a typology of teachers using labor market data. We construct our typology by applying a correlated topic model to 17,000 personal statements teachers submitted as part of their applications to open positions in Wisconsin public schools. We identify seven types of teachers active on the labor market: Inclusivists, Idealists, Nurturers, Generalists, Classroom Experts, Guides, and External Experts. Using a combination of quantitative and qualitative methods, we explore trends within and among types as well as demographic relationships and labor market behaviors. This research provides novel insights into the philosophies, mission, vision, and values of teachers; links these characteristics to teachers' job market preferences; and provides a sound psychometric foundation for these measures of teacher type to be applied in subsequent research.

The aim of the paper is to identify and analyse patterns and trends in enrolment in upper secondary initial vocational education and training (IVET) in Europe over the period 1995 to 2015. The research produced findings on comparability of the international data as well as trends in enrolment in upper secondary IVET and its share of enrolment in upper secondary education. It found that, in countries where there has been a decline in VET enrolment, most of this is due to a declining youth population. It also found that many fluctuations in enrolment figures are artificial in the sense that they are the result of changing national classification and/or reporting practices. When correcting for these factors, findings show that the development of absolute enrolment in VET, as well as its share of enrolment in upper secondary education, has been quite stable in most countries. [The research was carried out by a consortium led by 3s Unternehmensberatung GmbH, and includes the Danish Technological Institute, the Institute of Employment Research (University of Warwick), the Institute of International and Social Studies (Tallinn University) and Fondazione Giacomo Brodolini. The Federal Institute for Vocational Education and Training (BIBB) in Germany is supporting the project as a subcontractor.]

Capitalizing on the findings in our preceding study of a purely theoretical model, this paper aims to empirically examine whether and to what extent public universities' institutional missions have transformed in recent years in the States of California and New York by quantifying a degree of functional diversification of universities. We focus on research funding and productivity, and public service activities, and have developed a Concentration Equality Index (CEI) to help in this analysis. We then apply the CEI over time to a selected group of public university-system campuses within the State University of New York (SUNY) system, the City University of New York (CUNY), and the California State University (CSU) and the University of California (UC) systems. Among our findings: a select group of CSU campuses which all have roles at teaching-intensive schools, have gained increasingly versatile roles with rapidly expanded spending capacity in research and public service. These focal shifts resulted in some CSU campuses transforming into "UC-like universities", that is, a trend toward an institution with multi-functional operations of equally weighted instruction, research, and public service. In contrast, several campuses of both SUNY and CUNY systems have come to place varied weights on chosen missions rather than evening out their roles in instruction, research, and public service.

The National Student Outcomes Survey (SOS) collects information about students who completed their vocational education and training (VET) in the previous calendar year. The gathered information on the surveyed VET students includes their reasons for training, employment outcomes, satisfaction with training, and further study outcomes. The survey covers students who have completed a qualification (graduates) and those who have completed only part of a course and then left the VET system (subject completers). The National Centre for Vocational Education Research (NCVER) has conducted the survey with government-funded VET students annually since 1999. In 2016, the scope of the survey was expanded to report on the outcomes of graduates whose training was Commonwealth--or state-funded as well as fee-for-service graduates. These graduates were referred to as total VET graduates. An explanation of the difference between total VET and government-funded student outcomes can be found in appendix A. The expanded scope was applied to the 2017 survey for graduates (following a successful trial in 2016) and for the first time for subject completers and the series renamed VET student outcomes. At the time of sample selection, insufficient information is available from the National VET Provider Collection to identify 'actual' subject completers. Instead, a sample of potential subject completers is chosen, which includes students who are continuing in the VET system. The status of respondents is determined through the survey responses. As such, respondents to the SOS include a number of students who were sampled as subject completers based on administrative data reported to the National VET Provider Collection, but self-identify in the questionnaire as graduates. For many years these 'self-reported graduates' (SRGs) were categorised as graduates in survey outputs, because the self-report was deemed to be more reliable than the collection data. However, it became apparent that many SRGs were not, in fact, graduates. In response to this issue, in 2012, NCVER created a logistic model that predicted the eligibility of a SRG being an 'actual' graduate based on their personal and training characteristics (Braysher 2012). This model has since been run annually for each SOS to assign group membership to SRGs. Those SRGs that were not predicted by the model to be a graduate were re-assigned to their original subject completer status. One of the conditions of the initial analysis was that the model should be reviewed at least every four years to assess its ongoing validity and to make possible modifications should demographics and administrative data change and alter the predictive power of the model. The model was reviewed by NCVER in 2015 (unpublished). The review found some changes in data quality, but found no evidence that these changes were affecting the estimates. At the time no changes were recommended to the logistic model or graduate reclassification procedure, but it was recommended that the model be reviewed again at a later stage. In relation to data quality, the proportion of subject completers claiming to be self-reported graduates has declined significantly since 2007, particularly from 2015 to 2017, highlighting the improvement in the quality of the National VET Provider Collection data since the need to run the model arose (appendix B). Hence, it was appropriate to review the model again. This report presents the findings of an additional comprehensive review of the model and methodology.

A singular vision has propelled higher education and ministries of education in Asia since the new millennium. It is a vision launched by the once rising tide of a globalized world order that spilled into higher education: in order to be competitive on the world scene, each Asian country had to build "World Class Universities," which could be compared and rank-ordered with the pre-eminent research universities of America, Britain and elsewhere. And if the pre-eminent American and British research universities could not be quickly surpassed, punctuated jumps up the status ladder of global higher education would establish an "heir-apparent" trajectory. Now, however, the rising tide of globalism has perhaps crested. Indeed, as some scholars of globalism and higher education point out, in certain aspects of national policy the rising tide is turning into a receding one. Nation-states are recoiling from an elite-driven, transnational world order into an "inside economy" that is concerned with its own issues of development and sustainability, quite apart from measuring itself against the rest of the world. In time, higher education in Asia is likely to follow suit. Globalism created the vision of "World Class Universities" ranked in an imitative hierarchical order; its demise suggests the need for other ideals that can drive the quest for excellence in higher education in Asia. The bedrock of educational quality for the New Flagship University in Asia is excellence in all of the forms of knowledge and understanding of human experience. The stronger the excellence in each form, the better the university's earned standing amongst its peers. But this is not a relative relationship, like the World Rankings, where worth is measured against other institutions in a game of winners and losers. Rather, it is measurement against the standards of excellence inherent in each form itself.

Childcare quality is often thought to be important for influencing children's subsequent attainment at school. The English Government regulates the quality of early education by setting minimum levels of qualifications for workers and grading settings based on a national Inspectorate (OfSTED). This paper uses administrative data on over two million children to relate performance on national teacher assessments at ages 5 and 7 to the quality characteristics of the nursery they attended before starting school. Results show that staff qualifications and childcare quality ratings have a weak association with teacher assessments at school, based on comparing children who attended different nurseries but attended the same primary school. Our results suggest that although children's outcomes are related to the nursery they attend, which nurseries are good cannot be predicted by staff qualifications and OfSTED ratings; the measures of quality that Government has focused on. Supplementary tables are appended. [This paper was produced as part of the Centre for Economic Performance's Education and Skills Programme.]

This white paper explores the variation among Part C exiting categories as reported by the 50 states and Washington, D.C., using 2012-13 exiting data. Examining factors that may contribute to variation can lead to improved Part C exiting data quality within and across states. Examination of the Part C exiting data shows that 8 of the 10 categories (categories 1, 2, 4, 5, 6, 8, 9, and 10) had relatively wider variation among states. The paper investigates state characteristics that may influence exiting data variation, as well as possible strategies for improving data quality. This white paper includes four sections. Part 1 summarizes Part C exiting definitions. Part 2 reviews broad, general trends in exiting data variation. Part 3 examines potential reasons for variation, with possible strategies to improve exiting data quality. Finally, Part 4 summarizes the findings of examination of exiting data variation and highlights key findings for state data personnel. [The IDEA Data Center is one of the centers in the Technical Assistance and Dissemination Network (TA&D Network), funded by the Office of Special Education Programs (OSEP).]

Many postsecondary institutions, and community colleges in particular, require that students demonstrate specified levels of literacy and numeracy before taking college-level courses. Typically, students have been assessed using two widely available tests--ACCUPLACER and Compass. However, placement testing practice is beginning to change for three reasons. First, the Compass test will no longer be offered as of the end of 2016. Second, questions have been raised about the validity of commonly used placement tests. Third, there are emerging discussions about the need to consider other aspects of students' readiness to succeed in college, especially so-called noncognitive skills. In this paper, we discuss the history of college placement testing, with a focus on nonselective colleges. We describe the limitations of placement tests, the consequences of placement errors, and the movement toward changing systems of placement. The use of multiple measures is suggested as an approach that will result in more accurate placement. A typology of approaches to assessment and placement is described, including the identification of individual measures (e.g., alternative math and English tests, noncognitive assessments, and high school transcript information), ways to use them in combination (e.g., waivers, decision rules, and placement formulae), and how colleges might use assessment results in more varied ways (e.g., for placement into different course types or services). Finally, we include a discussion of emerging issues affecting assessment and placement practices.

The digital revolution and automation are accelerating changes in the labour market and in workplace skills, changes that are further affected by fluctuations in international and regional economic cycles and employment opportunity. These factors pose a universal policy challenge for all advanced economies and governments. In the workplace, people seek to acquire contemporary and relevant skills to gain employment and retain transferable skills to maintain employment. The central purpose of this paper is to investigate how other nations or regions are dealing with these issues. What approaches are they taking to understanding the mix and dynamics of the skills attained by individuals and, more broadly, the totality of skills that in aggregate constitute a highly capable and adaptable labour force, one that supports firm viability and greater national productivity. This research has examined a range of initiatives and approaches being developed or in use in selected countries, including the United States, Singapore and New Zealand, and agencies/organisations; for example, the European Commission and the Skills for the Information Age Foundation. In doing so, it showcases the good practices used to ensure that occupational-level skills information remains current and widely accessible. [For "Identifying Work Skills: International Case Summaries. Support Document," see ED579875.]

ACECQA has published the second in its series of occasional papers, analysing one of the most challenging quality areas -- Children's Health and Safety. Quality Area 2 addresses one of the primary objectives of the National Quality Framework -- to ensure the health, safety and wellbeing of children attending education and care services. The paper examines performance across service types, socio economic and remoteness classifications, jurisdictions and management types. It also breaks down Quality Area 2 and highlights that standard 2.3 (each child is protected) and element 2.3.3 (incident and emergency planning) are the most challenging aspects of the Quality Area. One of the findings from the analysis is that services in remote and very remote areas may benefit from more support to understand and comply with the requirements of Quality Area 2. The paper also summarises recent state, territory and Commonwealth initiatives around child safe organisations, as well as some examples of the types of compliance and enforcement action that state and territory regulatory authorities have taken relating to children's health and safety.

Calls to increase the number of undergraduate STEM credential recipients have largely failed to differentiate between sub-baccalaureate and four-year credentials at the undergraduate level, which is problematic for workforce development. In this paper, the authors develop a classification system for sub-baccalaureate STEM credentials that is incorporated into an analysis of administrative data from the Virginia Community College System. The authors first describe sub-baccalaureate STEM students and then examine the relationships between STEM matriculation and short-term outcomes for six cohorts. The authors use Mincerian regressions to estimate the earnings associated with completing a STEM credential four years after initial enrollment. In addition to confirming that students with career-oriented credentials drive short-term STEM earnings benefits, and that full-time students are more likely to complete credentials than their part-time peers, this study also finds relative homogeneity between STEM and non-STEM community college students, suggesting that ability may not be the primary factor inhibiting middle-skill STEM workforce preparation. The authors conclude by discussing the findings and suggesting how these data could be useful in better aligning Virginia's economic development plans and postsecondary educational offerings. A table on VCCS STEM Majors by Program Category and Credential Orientation is appended.

We investigate the extent to which graduate returns vary according to the class of degree achieved by UK university students and examine changes over time in estimated degree class premia. Using a variety of complementary datasets for individuals born in Britain around 1970 and aged between 30 and 40, we estimate an hourly wage premium for a "good" (relative to a "lower") class of degree of 7% to 9%, implying a wide spread around the average graduate premium. We also estimate the premium for a good relative to a lower degree for different cohorts (those born between the mid-1960s and early-1980s) and find evidence that the premium for a good degree has risen over time as the proportions of cohorts participating in higher education have increased. Tables and a list of university types are appended.

Of the copious research on the labor market returns to college, very little has adequately modeled the pathways of non-completers or compared their outcomes with those of award holders. In this paper, we present a novel method for linking non-completers with completers according to their program of study. This method allows us to calculate the labor market returns to programs of study, accounting both for those who obtain an award and those who do not. We use a large dataset of community college transcripts matched with earnings data. We find that different classification systems--by algorithm, intent, or goal--yield very different enrollment patterns across programs. These classifications make a substantial difference to earnings patterns. Our results show that returns vary not only by program completion, but they also vary by program non-completion. Consequently, combining completers and non-completers yields a new pattern of returns. For some awards, this leads to wider earnings differentials. However, we find that the variance in returns by subject of study is reduced when we combine data on completers and non-completers. Finally, we find that progression in a program per se does not lead to higher earnings for students who do not complete (even as it demonstrably does for students who complete their program). If validated, these findings have significant implications for policies on program choice and on student retention policies. A supplementary figure is appended.

Of the copious research on the labor market returns to college, very little has adequately modeled the pathways of non-completers or compared their outcomes with those of award holders. In this paper, we present a novel method for linking non-completers with completers according to their program of study. This method allows us to calculate the labor market returns to programs of study, accounting both for those who obtain an award and those who do not. We use a large dataset of community college transcripts matched with earnings data. We find that different classification systems--by algorithm, intent, or goal--yield very different enrollment patterns across programs. These classifications make a substantial difference to earnings patterns. Our results show that returns vary not only by program completion, but they also vary by program non-completion. Consequently, combining completers and non-completers yields a new pattern of returns. For some awards, this leads to wider earnings differentials. However, we find that the variance in returns by subject of study is reduced when we combine data on completers and non-completers. Finally, we find that progression in a program "per se" does not lead to higher earnings for students who do not complete (even as it demonstrably does for students who complete their program). If validated, these findings have significant implications for policies on program choice and on student retention policies. Appended is Figure A.1: Assignment of Students to Programs.

One in five school-age children in the U.S. speaks a language other than English at home (Zeigler & Camarota, 2014). Roughly half of these emerging bilingual students (Garcia, 2009) are classified as English learners (ELs) when they enter school, meaning they do not meet state or district criteria for English proficiency (NCES, 2015). As the fastest growing official subgroup of students, ELs are transforming schools across the country, in cities as well as suburban and rural communities; in traditional immigrant-receiving areas as well as in new immigrant destinations. Emerging bilingual students, and the subset of them that are classified as ELs, bring with them important linguistic, social, cultural, and intellectual assets that can enrich and strengthen education for all students (González, Moll & Amanti, 2013). But questions persist around how best to ensure that students who are not yet proficient in English can thrive in school, academically, linguistically, and socially. Should ELs be taught in bilingual classrooms that promote fluency in their home language while ensuring access to core academic content and developing English language skills? Or should they be taught in English immersion classrooms in order to maximize exposure to English? How do we ensure that emerging bilingual students develop both English proficiency and strong academic skills, while maintaining and developing literacy in their home language? How can schools best build on ELs' linguistic assets and support their educational needs?

When lotteries are infeasible, researchers must rely on observational methods to estimate charter effectiveness at raising student test scores. Considerable attention has been paid to observational studies by the Stanford Center for Research on Education Outcomes (CREDO), which have analyzed charter performance in 27 states. However, the sensitivity of CREDO's findings to its methodology has not been subject to a thorough, independent evaluation. Our analysis of the CREDO estimates of charter effectiveness in Florida between 2001 and 2009 reveals four main findings that increase confidence in its results: (1) Its use of multiple observations when matching students does not materially affect results; (2) Student participation in programs such as special education and free/reduced price lunch is inconsistently measured across sectors, but matching on these variables only modestly affects CREDO's estimates; (3) Exogenous instrumental variable estimates of charter effectiveness produce qualitatively similar results to CREDO's observational estimates; and (4) Impact estimates differ for oversubscribed and undersubscribed charters, which helps explain why lottery-based studies tend to find larger charter impacts than CREDO does. The following are appended: (1) Other Potential Concerns with CREDO's Methodology; (2) Differential Program Participation; (3) Analysis of Transfer Students Only; and (4) Summary Statistics for Over and Undersubscribed Charter Schools.

The purpose of this study was to determine the trends of recent research papers in foreign language teaching in Turkish context and to give ideas to researchers and policy makers for future studies. Content Analysis method was used in this study. The focus of the study was 189 research papers published between 2009-2013 years in journals indexed in SSCI (94) and the ULAKBIM (95) database in Turkey. Research Papers Classification Form was used as data collection instrument. The papers were scanned in terms of year of the journal, authors, language of the paper, journal index, topic of the paper, research design, data collection tools, sample, sample size and data analysis method. The results revealed that the most frequently studied topics on foreign language teaching and learning from 2009 to 2013 were concept analysis, teaching and learning, the highest number of articles were published in 2013, most of the authors were Turkish, majority of the articles were published in English, quantitative method was used more than qualitative method in research design, undergraduates were the focus of attention as sample group and 31-100 sample size was preferred more than others. An appendix presents the titles of the journals scanned in this study (2009-2013).

School districts and state departments of education frequently must choose between a variety of methods to estimating teacher quality. This paper examines under what circumstances the decision between estimators of teacher quality is important. We examine estimates derived from student growth percentile measures and estimates derived from commonly used value-added estimators. Using simulated data, we examine how well the estimators can rank teachers and avoid misclassification errors under a variety of assignment scenarios of teachers to students. We find that growth percentile measures perform worse than value-added measures that control for prior year student test scores and include teacher fixed effects when assignment of students to teachers is nonrandom. In addition, using actual data from a large diverse anonymous state, we find evidence that growth percentile measures are less correlated with value-added measures with teacher fixed effects when there is evidence of nonrandom grouping of students in schools. This evidence suggests that the choice between estimators is most consequential under nonrandom assignment of teachers to students, and that value-added measures controlling for teacher fixed effects may be better suited to estimating teacher quality in this case.

The fields of science, technology, engineering, and mathematics, more ubiquitously known by the acronym "STEM," have received a substantial amount of attention in recent years. As part of a research study investigating the alignment (or lack thereof) between the goals and priorities of educators and employers, we found it difficult to ascertain precisely what constituted a STEM occupation and to reconcile vastly different job, wage, and educational attainment projections. In this paper we explore the ways in which agencies and labor market researchers define STEM in six widely cited reports, and how these definitions influence the estimated and projected number of STEM jobs, estimates of how much these jobs pay, and what levels of education are necessary to attain these jobs. We find that agencies and researchers use different methods to classify STEM jobs (e.g., work tasks performed, knowledge, skills, education, type of work), which result in STEM job estimates ranging from 5.4 million to 26 million. Wage estimates exhibit a similarly wide range, from $50,000 to $96,000. In addition, jobs that require some STEM knowledge are frequently excluded from labor market analyses of STEM occupations, resulting in the under-counting of STEM-related jobs, the over-estimation of potential wages, and under-valuing of a sub-baccalaureate degree. Finally, we emphasize how groups of occupations (e.g., STEM) and industries (e.g., manufacturing) cannot be conflated with specific occupations. We recommend that researchers and agencies adopt a universal definition of STEM occupations and/or be more precise when speaking about the labor market in STEM related fields.

The past five years have seen unprecedented changes in freshman admissions at the University of California, reflecting steep cuts in state funding that UC sustained during that period as well as changes in UC's definition of who is eligible to enter the university. The number of California applicants who were "not" admitted to the UC system more than doubled between 2010 and 2012, although part of that increase also reflected a change in admissions policies and procedures. The number of "no shows"--applicants who were admitted but did not attend--increased sharply and for the first time exceeded the number of admits who enrolled at UC. Most troubling, UC's "participation rate"--the percentage of California high school graduates who entered UC as freshmen, a key indicator of college access--fell to its lowest level in three decades. It appears the university may be nearing a pivotal moment in its "social contract" with the people and State of California. The present study is based on a sample of 1,144,047 California high school graduates who applied for freshman admission at UC between 1994 and 2011; the sample data are augmented, wherever possible, with published data for the 2012 admissions cycle as well. The study traces the impact of both internal and external factors on UC admissions, including the introduction of multiple filing, changes in university policies on eligibility and admissions, and the long-term decline in state funding for UC that accelerated precipitously with the 2009 recession. The paper concludes with policy commentary and proposals. The funding model that has driven UC's growth since the advent of the 1960 Master Plan for Higher Education is broken, and a new model is needed if the university is to renew its social contract with California. In seeking a new funding model, however, the traditional Master Plan construct of "eligibility" for admission to UC remains as relevant today as it was in 1960 as a foundation upon which to rebuild.

Reliable and consistent time series are essential to any kind of economic forecasting. Skills forecasting needs to combine data from national accounts and labour force surveys, with the pan-European dimension of Cedefop's skills supply and demand forecasts, relying on different international classification standards. Sectoral classification (NACE) and occupational classification (ISCO) standards underwent substantial changes in recent years, related not only to internal structure but also the logic of how companies or workers are classified. It is natural that changes to key classification standards had significant impact on this pan-European exercise but Cedefop has developed methodology to overcome this problem and to find the best match between the old and new classifications. This publication details Cedefop's action, which should also have value for those dealing with labour market analysis at national level. The following are appended: (1) General limits of the LFS-based database; (2) Cedefop pan-European forecasting model of skills supply and demand; and (3) Detailed results of employment share analysis. [This research paper is based on contributions from Terry Ward, Robert Stherer (Alphametrics), and Rob Wilson (Institute for employment research University of Warwick. This project is supported financially by the progress programme, a European Union programme to promote employment and social solidarity (2007-13).]

The Community Engagement Classification is an elective classification offered by the Carnegie Foundation for the Advancement of Teaching. In order to be classified, campuses provide evidence documenting engagement through an application process. Campuses were classified in 2006, 2008, and 2010, and will be classified on five-year cycles from 2015 onward. (Information about the classification can be found on the Carnegie Foundation website.) This mixed-methods, two-part study sought to discover how institutions that received the Carnegie Community Engagement Classification in 2010 approached their application process and to examine the longer term outcomes of that process. How did they undertake a "full inventory" of their community engagement efforts, and what were the outcomes of undertaking such an inventory, beyond receiving the Classification? The study had two purposes: (a) To document the strategies and methods used by successful applicants for the 2010 Carnegie Community Engagement Classification, and (b) to document the cultural shifts experienced by institutions as they developed their successful applications for this Classification.

The federal government's Race to the Top competition has promoted the adoption of test-based performance measures as a component of teacher evaluations throughout many states, but the validity of these measures has been controversial among researchers and widely contested by teachers' unions. A key concern is the extent to which nonrandom sorting of students to teachers may bias the results and lead to a misclassification of teachers as high or low performing. In light of this, it is important to assess the extent to which evidence of sorting can be found in the large administrative data sets used for VAM estimation. Using a large longitudinal data set from an anonymous state, we find evidence that a nontrivial amount of sorting exists--particularly sorting based on prior test scores--and that the extent of sorting varies considerably across schools, a fact obscured by the types of aggregate sorting indices developed in prior research. We also find that VAM estimation is sensitive to the presence of nonrandom sorting. There is less agreement across estimation approaches regarding a particular teacher's rank in the distribution of estimated effectiveness when schools engage in sorting. The following is appended: Performance of Panel Data Value-added Estimates.

In recent years, "continuous improvement" has become a popular catchphrase in the field of education. However, while continuous improvement has become commonplace and well-documented in other industries, such as healthcare and manufacturing, little is known about how this work has manifested itself in education. This white paper attempts to map the landscape of this terrain by identifying and describing organizations engaged in continuous improvement, and by highlighting commonalities and differences among them. The findings classify three types of organizations engaged in continuous improvement: those focused on instructional improvement at the classroom level; those concentrating on system-wide improvement; and those addressing collective impact. Each type is described in turn and illustrated by an organizational case study. Through the analysis, six common themes that characterize all three types of organizations (e.g., leadership and strategy, communication and engagement, organizational infrastructure, methodology, data collection and analysis, and building capacity) are enumerated. This white paper makes four concluding observations. First, the three case studies provide evidence of organizations conducting continuous improvement work in the field of education, albeit at different levels and in different ways. Second, entry points to continuous improvement work are not mutually exclusive, but are nested and, hence, mutually informative and comparative. Third, continuous improvement is not synonymous with improving all organizational processes simultaneously; rather, research and learning cycles are iterative and gradual in nature. Fourth, despite being both iterative and gradual, it is imperative that improvement work is planned and undertaken in a rigorous, thoughtful, and transparent fashion. The following are appended: (1) Selected Continuous Improvement Methodologies; (2) Baldrige Education Criteria for Performance Excellence Framework; (3) Examples of Improvement Artifacts from the School District of Menomonee Falls, Wisconsin; (4) Examples of Improvement Artifacts from Montgomery County Public Schools, Maryland; and (5) Examples of Improvement Artifacts from Strive Partnership Cincinnati.

Historically, educational policymakers have focused on and invested heavily in accountability testing of "learning" to leverage improvement in student learning. Through accountability testing, policy makers aim to communicate standards, establish performance goals, provide data through which educators can analyze and improve school programs and student performance, and establish incentives and sanctions to motivate action. Today, however, there is growing recognition of the limitations of accountability testing of learning, and wide acknowledgment and accumulating evidence of the crucial role that formative assessment--"assessment for learning"--can play in helping all students achieve rigorous standards. This paper pairs an in-depth exploration of current formative assessment practices and research with guidance on application of the presented conceptual model to the Next Generation Science Standards (NGSS). The paper includes a selective review of diverse literature focused on effective formative assessment practice, and uses this literature to describe and justify a proposed model. The paper highlights special considerations for, and provides examples of the model's application in Next Generation Science Standards. Implications for the design of coherent assessment systems that support student learning are included.

A strong VET system is increasingly seen as essential to overcoming the current economic crisis in Europe. VET is seen as a powerful tool to assist in balancing labour market inefficiencies, increasing youth employment possibilities, and reducing skills mismatch. Its inherent flexibility and closeness to the labour market place VET in a good position to contribute to a faster economic recovery and long-term sustainable development. However, crucial for this role is continuous and systematic VET renewal that assures its relevance for the labour market. This publication explores 15 European national approaches to feedback mechanisms between VET and the labour market. It illustrates the diversity of solutions currently applied across Europe and how they are embedded in national traditions and education philosophy. The study asks three fundamental questions: how inclusive are national mechanisms for feedback between the VET system and the labour market; how responsive are existing mechanisms; and how transparent? Three annexes provide: (1) Case Studies; (2) List of interviewees; and (3) List of experts responsible for the country overviews. Bibliography and references are included. [This paper is the result of a team effort reflecting the work of a research consortium led by Jörg Markowitsch from 3s Research Laboratory who together with Tanja Bacher, Carol Costley, David Etherington, Gerhard Geiger, Günter Hefler, Jelena Helemäe, Triin Roosalu, Ellu Saar, Auni Tamm, and Odd Bjørn Ure conducted the research and fieldwork and drafted the report. This work was carried out under Cedefop's service contract No 2011-0161/AO/ECVL/JB-IPS/Cooperation Labour market--VET/007/11.]

This paper reviews the new directions in institutional research on undergraduate racial/ethnic and socioeconomic diversity at the University of California, Berkeley. The use of SERU/UCUES and other web-based census surveys has made possible more detailed and extensive analysis of student diversity. Included is research on an expanded number of racial/ethnic groups and on multiracial students, the significance of the African American experience, implications of the new IPEDS racial/ethnic reporting requirements, and a closer examination of Pell Grant and first-generation college students. UCUES survey results are used to development a more comprehensive parental education and immigrant generation diversity (EID) typology that is then used to examine the interrelationships among student demographics and various facets of the undergraduate academic experience. Finally, an analysis of student accounts of the experience of diversity at Berkeley provides an example of how web-based census surveys afford new opportunities for cost-effective qualitative diversity research. (Contains 19 tables, 1 figure and 1 footnote.)

This paper examines the current academic and policy literatures concerning school evaluation in primary and secondary education within the OECD countries. First, it provides a typology of the existing systems of school evaluation across the OECD. It encompasses the diverse criteria and instruments commonly used to carry out schools evaluation, as well as the players involved in the design and implementation of school evaluation. It also describes potential consequences for schools. Second, this paper analyses how school evaluation schemes are interrelated with other components of the evaluation framework, such as teacher evaluation and system evaluation. The potential complementarities, duplication and inconsistency of objectives stemming from these interrelations are discussed. Third, this paper presents the advantages and drawbacks of different approaches to school evaluation, the resistance and implementation difficulties resulting from misalignment of interests between different stakeholders, and possible ways to overcome impediments to implementation. Finally, it reviews the quantitative and qualitative evidence available on the impact of different school evaluation schemes on school performance, student learning and the incentives for the teaching staff. It concludes by considering the circumstances under which school evaluation schemes seem to be more conducive to school improvement. The effectiveness of school evaluation schemes relies on developing competencies for evaluation and for using feedbacks. Alignment of stakeholders' interests is also critical to have the support of those being assessed.

Community college students typically have access to a large selection of courses and programs, and therefore the student transcripts at any one college or college system tend to be very diverse. As a result, it is difficult for faculty, administrators, and researchers to understand the course-taking patterns of students in order to determine what programs of study they appear to be pursuing. Attempting to examine these patterns and then comparing them with listed program requirements would be a very time-consuming activity. The most common way of assigning a program of study to a student--picking the subject in which she has taken the most courses--is overly simple, because many programs require courses across several subjects. However, because students who have similar patterns of course-taking in terms of subjects and particular courses taken are likely to be in similar programs, clustering can be a useful way to make sense of the relevant data. Clustering allows researchers to group similar items into clusters, relying only on a measure of the similarity of those items. In this paper, we apply a clustering algorithm to the problem of understanding college transcripts, which serve as the items to be clustered. To our knowledge, this is the first effort to organize transcripts based on their course content using clustering. We base the measure of similarity on the proportion of curricular subjects that each transcript has in common with every other one. Our data are community and technical college transcripts for a cohort of students who first entered the Washington State system during the fall of the 2005-06 academic year and who had no prior postsecondary experience. We used our clustering algorithm to separately cluster liberal arts and career-technical students. We found that the algorithm did a good job of separately clustering each of these groups. The clusters roughly corresponded to programs of study, so we were able to estimate how many students were undertaking each program and what subjects students were studying within each cluster. We were also able to examine the demographics and the completion and transfer rates of the students within each cluster, in order to get an idea of what types of students were in each program of study and how successful they seemed to be in college. We found substantial variation on these dimensions as well as on the extent to which students' programs were either concentrated in a single subject or spread across several subjects. Clustering is a powerful way to understand the course-taking patterns of students and assign programs of study. It makes few prior assumptions about the data; rather, it allows the data to organize itself based on a similarity measure. It relieves the analyst of determining what the program categories should be. It has the ability to detect patterns of activity across subjects within student transcripts. Note that although we have applied this method to community college students, it is applicable at all levels of postsecondary education. We conclude that this method would be useful to researchers throughout education who are trying to understand student course-taking patterns and programs of study, and who need to organize large amounts of transcript data. Appended are: (1) Details of the Cluster Analysis; (2) Details of the Cluster Analysis; and (3) Five Clusters of Liberal Arts Student Transcripts. (Contains 10 tables and 3 footnotes.)

Data relating to occupations has been collected in the national apprentice and trainee collection since 1994. The coding used conforms to classifications endorsed by the Australian Bureau of Statistics (ABS). The latest version issued from the ABS is the Australian and New Zealand Standard Classification of Occupations (ANZSCO). The classification previous to ANZSCO is the second edition of the Australian Standard Classification of Occupations (ASCO). Currently occupation data is collected using both ANZSCO and ASCO codes. Although ANZSCO was only introduced to the Apprentice and Trainee Collection in 2007, states and territories were asked to back-code ANZSCO on all contracts that were active as at 1 July 2000. Therefore while ASCO codes appear on records from the beginning of the collection, ANZSCO codes, only appear on records from 1 July 2000. Thus, although the database has a field for ANZSCO, the value is missing for all records prior to this date. This technical paper is a description of the methodology used to impute values for records in the National Apprentice and Trainee database that have missing ANZSCO codes. Appendices include: (1) Method (detailed); and (2) System charts. [For the accompanying report, "Apprentices and Trainees: September Quarter 2009," see ED508817.]

"Going Further" presents a roadmap to the works of the ACCLAIM (Appalachian Collaborative Center for Learning, Assessment, and Instruction in Mathematics) Research Initiative, the research effort of one the Centers for Learning and Teaching (CLTs) created with a grant (2001-2005) from the National Science Foundation. The Center began with a broad mission to include practitioners, a doctoral program, and a research program. This report focuses on the most substantive "works" produced for the Center's research program (known as the "Research Initiative"). This report also offers an internal evaluation of the works produced (the present authors for the Center), but it relies rather than more heavily on descriptions of the works, and less on nuanced judgments than might an essay review. One section deals with specifically with the evaluation of individual works, as to be explained shortly, but one section engages a wider discussion around the critique of several key ideas that surface across the ACCLAIM opus. Both these presentations are best read as part of the roadmap mission; they give a somewhat deeper engagement with the scope of the works than the simple descriptions initially presented. The report synthesizes the results of ACCLAIM's empirical work, summarizing the findings of individual studies (14 quantitative studies and 20 qualitative studies; unduplicated accounting) but also characterizing the overall findings. Appended are: (1) Table of ACCLAIM works replaced by later versions; (2) ACCLAIM's Theoretical Framework; (3) Background Information; (4) Numerical Codes for ACCLAIM Works Listed in Table 2, Alphabetized by Year; (5) Evaluation Chart for High-Scoring Theoretical Works; and (6) Glossary of Key Concepts. (Contains 5 tables and 60 footnotes.)

Over recent years governments have been placing more emphasis on innovation as a source of national competitiveness. Governments now assess their investments across many areas in terms of the contribution that such investments make to increasing innovation. This has been especially significant for education and in particular for the development of policies for universities because universities perform research as well as provide learning. The measures governments use to assess the performance of universities frequently include indicators of research-related engagement with business, or of the level of patenting, licensing and start ups. Assessing university performance by focusing on just some of the direct services or outputs that universities provide, or on narrow aspects of their individual services, seriously underestimates the contributions that universities make. Appended are: (1) The health innovation system; (2) Sources of ideas or information for business innovation; (3) Innovation-Active Businesses; and (4) Carnegie Classification of Institutions of Higher Education in the USA. (Contains 33 footnotes.)

This paper discusses routes by which HIV/AIDS money is dispersed and received. It notes that capturing accurate data on actual spending patterns of large donors can be difficult, as there is no uniform tracking or reporting system and much HIV/AIDS money is spent under the broader category of sexual and reproductive health. Most of the information contained in the first two sections is based on main reports that assess the general manner in which HIV/AIDS money as a whole is being distributed. Moving on from who is providing funds for HIV/AIDS initiatives at global level, it tracks sources and flows from governments, through bilateral and multilateral channels. It does not include estimates of household spending on care and treatment, which cannot be realistically quantified. Information follows on top US and European donors, the international business community and pharmaceutical companies. Later sections look into ways HIV/AIDS funding is being spent, with the proviso, as before, that detailed breakdowns of actual spending are rare. The broadest categories are prevention, care and treatment, orphan support and research. Within the field of ECD vis-a-vis HIV/AIDS, funds are being directed through two main areas of concern--prevention of mother-to-child transmission and the care of orphans and vulnerable children. This paper describes major players in these arenas, showing that efforts are being made by a few agencies through the amount of funding directed along these channels is minimal in contrast to other target areas. Fundamental questions are raised about current donor priorities and there follows some discussion touching areas where new or reallocated HIV/AIDS funding could be directed. Obvious gaps in the provision of money for ECD and HIV/AIDS support are then identified along with opportunities to carry out work fill such gaps. The final section examines what it would take to direct more money to support young children living in the shadow of HIV/AIDS. Including very young children in HIV/AIDS response strategies will ultimately depend on individual communities devising their own solutions. A further section highlights the critical importance of sharing knowledge through networks that communicate and disseminate evidence-based research findings and project evaluations. In conclusion, this paper calls for advocacy to urge that more funding should go to ECD-HIV/AIDS needs and that current funding approaches to dealing with the crisis need, in addition, to be tracked and evaluated, with a view to promoting more and better ways of meeting the unfulfilled needs of very young children affected by HIV/AIDS. (Contains 2 figures.)

Capturing the conditions of children of color living in single-parent families has become more complex due to the growing presence of interracial households. This analysis assesses the size and poverty status of single-female headed families housing multiracial children. Using data from the 2000 Census, we find that 9 percent of female-headed families house either children who are classified with more than one race or are classified as a single race different than their mother's compared to only 3 percent of married couple families. Logistic regression analyses assessing the odds of poverty status for families finds that being a multiracial family does not constitute a uniform advantage or disadvantage for female headed households. Rather, these families, like most families of color, are more likely to experience poverty than white monoracial families. The two exceptions are White multiracial families who are more likely to be in poverty relative to this reference group and Asian multiracial families who have similar poverty rates as white monoracial families (and a lower rate than Asian monoracial families). (Contains 5 tables and 2 endnotes.) [This analysis was supported by the University of Kentucky Poverty Center Young Investigator's grant and the Woodrow Wilson Fellowship Foundation both awarded to the first author.]

An analysis of student verbatim responses in the Student Outcomes Survey provides suggestions for how the vocational education and training (VET) system could be improved. Eight major domains are discussed: course relevance and design; staff attributes and behaviours; teaching and learning practices; access to courses; facilities and services; assessment practices; learning resources; equipment and materials; initial information provision; and administration and learning support services. Aspects of training questions are appended. (Contains 11 tables and 6 footnotes.) [For Support Document, see ED507620.]

At the 18th meeting of Ministerial Council on Education (MCEETYA) (12th-13th May 2005), Council urged the Australian Government to: (1) increase per capita assistance to ESL-NA students with high educational needs, in particular, refugees and humanitarian entrants; (2) provide a new per capita allocation so that specific ethnic and cultural issues faced by those children and families entering into Australia under the refugee and humanitarian category can be addressed; and (3) expand the visa classifications eligible for new arrivals funding. Ministers Nelson and Hardgrave noted the points raised but indicated that a more evidence-based discussion of the issue was warranted to justify any Australian Government increase in funding. The MCEETYA Schools Resourcing Taskforce (SRT) convened a working party of Australian Government, Victorian Government and Catholic Education Commission representatives to work with the SRT secretariat to investigate this issue. A survey of the government state and territory education systems and Catholic education systems across Australia was conducted in March 2006. The survey was structured to collect data about the different modes of delivery of ESL tuition provided by an education system for its newly arrived students. Financial and non-financial data were collected in the survey. All states and territories participated except the Northern Territory. The Catholic Education Commission (CEC) provided data on two dioceses with substantial ESL-NA students. This report includes: (1) an overview of the existing policy settings for providing ESL tuition to newly arrived students, and of recent permanent and temporary immigration trends; (2) a national mapping of the services and different modes of delivery of ESL tuition provided by a system for its newly arrived students; and (3) an analysis of the current per capita costs of providing ESL tuition to newly arrived students and information on the adequacy of current provision for both groups of newly arrived students based on survey data. The Australian Government has expressed reservations about the methodology and conclusions drawn in this Discussion Paper. While these reservations remain, the critical importance of English language skills in improving workforce participation, productivity and aggregate economic outcomes for Australia, and to the well-being of individuals, is readily acknowledged. The Australian Government recognises the need to build on such common ground with further work to ensure areas of significant and specific ESL need are addressed. Appended are: (1) Visa Sub-Classes Eligible to Receive Commonwealth ESL-NA Program Funding; (2) Visa Sub-Classes Ineligible to Receive Commonwealth ESL-NA Program Funding; (3) SA Typical and Agreed Patterns of Entry and Exit Scales: DECS--ESL Scales; and (4) Survey Instrument. (Contains 19 tables, 9 figures and 63 footnotes.)