Search

This paper presents credible and comparable evidence on intergenerational educational mobility in 53 developing countries using sibling correlation as a measure, and data from 230 waves of Demographic and Health Surveys. It is the first paper to provide estimates of sibling correlation in schooling for a large number of developing countries using high quality standardized data. Sibling correlation is an omnibus measure of mobility as it captures observed and unobserved family and neighborhood factors shared by siblings when growing up together. The estimates suggest that sibling correlation in schooling in developing countries is much higher (average 0.59) than that in developed countries (average 0.41). There is substantial spatial heterogeneity across regions, with Latin America and Caribbean having the highest (0.65) and Europe and Central Asia the lowest (0.48) estimates. Country level heterogeneity within a region is more pronounced. The evolution of sibling correlation suggests a variety of mobility experiences, with some regions registering a monotonically declining trend from the 1970s birth cohort to the 1990s birth cohort (Latin America and the Caribbean and East Asia and Pacific), while others remained trapped in stagnancy (South Asia and Sub-Saharan Africa). The only region that experienced monotonically increasing sibling correlation is the Middle East and North Africa. The recent approach of Bingley and Cappellari (2019) is used to estimate the share of sibling correlation due to intergenerational transmission. The estimates show that when the homogeneity and independence assumptions implicit in the standard model of intergenerational transmission are relaxed, the estimated share is much larger. In the sample of countries, on average 74 percent of sibling correlation can be attributed to intergenerational transmission, while there are some countries where the share is more than 80 percent (most in Sub-Saharan Africa). This suggests a dominant role for parents in determining the educational opportunities of their children. Evidence on the evolution of the intergenerational share, however, suggests a declining importance of the intergenerational transmission component in many countries, but the pattern is diverse. In some cases, the trend in the intergenerational share is opposite to the trend in sibling correlation. [This report was prepared by the World Bank Group's Development Research Group, Development Economics.]

Prior work on teacher candidates in Washington State has shown that about two thirds of individuals who trained to become teachers between 2005 and 2015 and received a teaching credential did not enter the state's public teaching workforce immediately after graduation, while about one third never entered a public teaching job in the state at all. In this analysis, we link data on these teacher candidates to unemployment insurance data in the state to provide a descriptive portrait of the future earnings and wages of these individuals inside and outside of public schools. Candidates who initially became public school teachers earned considerably more, on average, than candidates who were initially employed either in other education positions or in other sectors of the state's workforce. These differences persisted at least 10 years into the average career and across transitions into and out of teaching. There is therefore little evidence that teacher candidates who did not become teachers were lured into other professions by higher compensation. Instead, the patterns are consistent with demand-side constraints on teacher hiring during this time period that resulted in individuals who wanted to become teachers taking positions that offered lower wages but could lead to future teaching positions.

This paper uses recent evidence from international early grade reading programs to provide guidance about how best to create appropriate targets and more effectively identify improved program outcomes. Recent results show that World Bank and US Agency for International Development-funded large-scale international education interventions in low- and middle-income countries tend to produce larger impacts than do interventions in the United States, as measured by effect sizes. However, these effect sizes rarely translate into large gains in mean oral reading fluency scores and are associated with only small increases in the proportion of students meeting country-level reading benchmarks. The limited impact of these low- and middle-income countries' reading programs on the proportion of students meeting reading benchmarks is in large part caused by right-skewed distributions of student reading scores. In other words, modest impacts on the proportion of students meeting benchmarks are caused by low mean scores and large proportions of nonreaders at baseline. It is essential to take these factors into consideration when setting program targets for reading fluency and comprehension. We recommend that program designers in lower-performing countries use baseline assessment data to develop benchmarks based on multiple performance categories that allow for more ambitious targets focused on reducing nonreaders and increasing beginning readers, with more modest targets aimed at improving oral reading fluency scores and increasing the percentage of proficient readers.

The field of learning analytics holds considerable promise for higher education, with reports of successful uses now emerging in selected institutions. At the same time, critics have expressed concerns regarding privacy, ethics, and intrusions into teachers' pedagogy. Without attentive planning, higher-education professionals applying learning analytics may inadvertently undermine their institutions' core teaching and learning missions. The authors offer a framework for moving forward with learning analytics, organized around three principles: (a) Institutions should take the lead in their conversations with vendors, emphasizing the distinctive values of higher education; (b) Learning analytics data should be balanced with other forms of evidence that analytics cannot capture, especially participant experiences; and (c) Successful implementations will leave room for adaptations by people on the ground--to notice what is working and integrate the tools into their practices. Only by empowering students, faculty, and staff can these tools fulfill their potential in higher education.

Generally speaking, when one writes about their research they are making a contribution to the scientific community and disseminating the results of findings in scientific articles. This means that other researchers have access to the research produced and can examine the subjects raised in greater depth to advance scientific knowledge. This paper discusses the format of papers that are strictly academic. The specific structure of the text will be determined by whether it is for a master's dissertation, a doctoral thesis, a chapter of a specialist book or an article for a scientific journal. In the case of qualitative research, it is necessary, when writing the text, to bear in mind a series of processes that will be explained in this handbook, such as: (1) the justification for the research in terms of its social and educational interest, and in theoretical terms; (2) the gathering of information or data; (3) the treatment and organization of the data; (4) the adoption of a theoretical and methodological framework; (5) data analysis; (6) the interpretation of data in an original and/or creative way, and obtaining the findings; (7) setting out a discussion on the relevance of the results; and (8) setting out the conclusions. Differences between a master's dissertation and a thesis are also described. A discussion of differences between articles in scientific journals and chapters of a book are discussed. [A Catalan version of this chapter is also included in the book. The transcription symbols used in this chapter are based on conventions developed by the GREIP group (see Moore & Llompart, this volume) and are included in the annex.

Drawing on insights from culturally responsive K-12 education frameworks and dual process theory, this qualitative study explores the situated nature of reflective practice among postsecondary faculty and calls for increased critical awareness and institutional support. Through interviews with 21 California research university faculty, this study found that instructors drew on both numeric and non-numeric data forms to engage in reflective practice. This tendency indicates a need for a more holistic, multi-disciplinary, and critical understanding of "data" than what the current accountability movement has imagined. This study's findings also show that although faculty consistently engaged in reflective practice, the outcomes of this reflection were severely limited by both individual bias and institutional constraints. Thus, while we recognize the current budgetary struggles many universities face, we argue that in order to better serve postsecondary students, particularly those from historically underrepresented groups, more institutional support is needed. Postsecondary institutions can play a significant role in facilitating critical examination by providing faculty the necessary space, time, and guidance to engage in critical reflection as well as the appropriate institutional mechanisms to voice concerns and enact change.

This report presents the underlying theory and methodology of the first framework to operationalize culture and culturally responsiveness in the context of Positive Behavioral Interventions and Supports. Created following a systematic review of literature, this framework was created as a cultural artifact to expand the conceptualization of the role of culture in the implementation of PBIS and other education programs. The author hopes the framework will start a movement to address the systemic contradictions that researchers and practitioners in the field experience regarding racial disparities in behavioral and academic opportunities and outcomes and the locally meaningful implementation of PBIS and other top-to-bottom initiatives and programs (e.g., Response to Intervention [RTI]).

Teacher and student sorting within and between schools has historically garnered national attention and controversy. Scholars have recently examined how school leaders make data-driven decisions to create more equitable schools, particularly through teacher and student assignment practices. School leaders use student test scores to strategically sort students into classes to create "balance." Student achievement data is also linked to teacher reassignment, in which ineffective teachers are strategically reassigned to non-tested grades or subjects, while effective teachers may be promoted to leadership positions. Scholars have found that a combination of data-driven strategic staffing, accountability pressures, and micro-politics are prime drivers of assignment strategies within schools. Within the literature on assignment practices, one strand examines the impact of assignment practices on students but does not engage with principals' decision-making strategies despite the central role principals play in school decisions. The second strand of research delves deeply into leadership decision making but seldom provides data on the actual assignment practices. This includes observed data regarding the two types of assignment strategies dominant within the literature: privilege- and learning-centric. A learning-centric assignment is the placement of highly knowledgeable teachers with students who need effective teachers more than do other peers (Donaldson, 2011, In contrast, privilege-centric assignments place experienced teachers in classrooms with higher achieving students. School leaders may also use privilege-centric staffing to appease affluent parents and increase retention of their more experienced teachers. Although privilege-centric assignments may be counterproductive to the student experience, many schools employ this strategy despite principals' reports of using data to increase school equitability. The assignment of teachers to students and classes is presumed to be an entirely local (school-level) decision, with school leaders having substantial authority over this process. The findings of the preceding research underscore the importance of principals in this process, and yet we know little about the ability of leadership to interrupt systems of privilege and institute more equitable staffing arrangements. Because of this potential inconsistency between principals reporting that they make data-driven decisions and their actual assignment practices, we see the need for continued research in this area. The purpose of this study is to fill this gap within the literature by first examining the relationship between principals' reported data-driven decisions and teacher and student assignment strategies within schools, and then looking at the relationship between assignment practices and student learning. This study couples survey data from 213 Florida elementary and middle schools with 8 years of longitudinal, statewide data to address the following questions: (1) To what extent are privilege- and learning-centric assignment practices used within schools? Are these practices changing over time?; (2) How do privilege- and learning-centric assignments impact student-learning gains?; (3) What drives student-assignment practices--is this phenomenon embedded in the culture of a school or do principals dictate assignment practices?; and (4) How are principals' reported uses of data related to observed privilege- and learning-centric sorting?

The State Higher Education Executive Officers Association's (SHEEO) "Communities of Practice" project builds upon SHEEO's ongoing efforts to measure the capacity and effective use of state postsecondary data systems and provides states with opportunities to develop solutions to common issues with those systems. The sixth Community of Practice convening, "Aligning K-12 and Postsecondary Education," was held December 7-8, 2021, in Denver, Colorado. The two-day meeting included representatives from 13 states: Delaware, Georgia, Hawaii, Idaho, Kentucky, Louisiana, Maryland, Missouri, Pennsylvania, Rhode Island, Tennessee, Utah, and Washington. Teams included representatives from SHEEO agencies, state K-12 agencies, P-20 partnership organizations, state longitudinal data systems, and others. The Community of Practice also addressed the impacts of the COVID-19 pandemic on student transitions and how state data systems can be used to promote equitable outcomes for low-income students and students of color. During the convening, teams explored practical uses of state P-20 data for improving college access, equity, and success and how state postsecondary data can better inform the K-12 to postsecondary pipeline. This white paper highlights key themes and findings of the convening, including challenges and lessons learned from the participating state teams and suggestions of topics for further consideration. Case studies describing ongoing efforts in Georgia and Pennsylvania are included in the appendices. Presentations from the December 2021 convening are available on SHEEO's state postsecondary data website.

This white paper analyzes the potential effects of requiring income-driven repayment for all federal loans as well as relying on paycheck withholding for loan payments, with particular attention to the implications for low-income students and families. The Institute for College Access & Success (TICAS) also examines the relevance and evolution of mandatory IDR ["income-driven repayment"] systems in Australia and the United Kingdom, and the paper includes specific recommendations to streamline and improve student loan repayment options in the United States. Two appendices are included: (1) Citation List of Figure 2: "Key Comparisons of IDR Systems and Context: U.S., U.K., and Australia"; and (2) Borrower Example Details.

A defining characteristic of current U.S. educational policy is the use of data to inform decisions about resource allocation, teacher hiring, and curriculum and instruction. Perhaps the biggest challenge to data-driven decision making (DDDM) is that data use alone does not automatically result in improved teaching and learning. Research indicates that translating raw data into useable information and actionable knowledge for teachers requires not only adequate technical and social supports, but also an awareness of how educators in real-world settings actually use information to make decisions. Yet, little is known about DDDM in higher education, in general, and how postsecondary faculty make sense of and use data in their instructional decision-making processes, in particular. In this paper, we use naturalistic decision-making theory to generate practice-based descriptions of how 59 STEM faculty at three large public research universities used data as part of their course planning. Interview transcripts and notes taken while observing planning meetings were analyzed using an inductive approach to content analysis. In practice, respondents used different types of data and other information obtained from, for example, student assessments, end-of-semester evaluations, and conversations with colleagues. Results indicate that faculty generally collect and analyze data in informal, ad hoc scenarios ungoverned by institutional policy. Exceptions include disciplines with accreditation pressures and team-taught courses where structured (and supported) opportunities exist for faculty to collect, analyze, and reflect upon data about student learning. Thus, while numeric data are clearly viewed by this population of faculty as the most rigorous, in practice, even those that use quantitative data also use other sources of information. These results suggest an opportunity for educational leaders to design policies and professional development initiatives that facilitate a more formal collection of and reflection on data by faculty. In pursuing such technical solutions, however, policymakers and educational leaders must carefully negotiate the tension between rigor and relevance, and learn from the challenges experienced in the K-12 sector regarding DDDM.

Accountability in the private and public sectors of society has received significant attention in both research and practice, partly because of its importance, but also because it is challenging to define, measure and implement. The nature of accountability is complex, ambiguous and highly context-dependent. As related to postsecondary education (PSE), multiple stakeholders across the nation have been pushing for greater accountability for at least three decades. Various stakeholders, including elected officials at the national and state level seemingly obsessed with achieving a "one size fits all" kind of accountability system, continue to produce proposals and in some cases legislation, for our institutions of postsecondary education to provide, among other matters, evidence of improved degree production, cost control, and student learning. This paper identifies who some of the stakeholders are and provides a range of accountability initiatives underway since a Center for Studies in Higher Education report, "Accountability in Higher Education: A Public Agenda for Trust and Cultural Change," was presented in 2006, including a specific focus on California. [The 2006 report is available in ERIC at ED503070.]

ED"Facts" is an initiative of the U. S. Department of Education to base education policy on reliable performance data provided by state education agencies. Among its many data items, ED"Facts" houses school-level counts of students disaggregated by state-defined student economic status, typically free and reduced-price lunch (FRL) eligibility, that rely upon a link between economic status and some other measure, such as an individual student's reading test score. It is important for the U.S. Department of Education to consider what changes to the accessibility and quality of FRL eligibility data may occur as a result of the "Healthy, Hunger-Free Kids Act of 2010" ("PL 111-296"), and what other measures of economic disadvantage might be feasible (or improved) alternatives to FRL eligibility. PL 111-296 amends the Richard B. Russell National School Lunch Act (42 U.S.C. 1759) and includes new provisions for determining FRL eligibility that have the potential to affect the reliability and availability of data to U. S. Department of Education programs participating in ED"Facts". These potential changes are important to federal program and statistical data users and those members of the public who use data on FRL eligibility that the Department publishes. There are three major areas in PL 111-296 that affect data. First, the law encourages more extensive use of direct certification--that is, determining a student's eligibility through documented eligibility for other services such as the Supplemental Nutrition Assistance Program (SNAP; formerly known as the food stamp program) by schools and local education agencies (LEAs). Because direct certification of individual students is based on data that already have been approved by other programs, the method is likely to improve data quality. Second, PL 111-296 introduces the Community Eligibility Option (CEO) that eliminates the requirement for individual eligibility information once a school has determined a baseline percentage of FRL-eligible students. When added to existing eligibility Provisions 2 and 3 of the "National School Lunch Act," which do not require the annual certification of individual students, use of the CEO may result in missing or out-of-date individual FRL eligibility information. Finally, the law directs the U.S. secretary of agriculture to identify alternatives to annual FRL eligibility applications, citing the American Community Survey (ACS) as a possible source of community income statistics that could obviate the need to determine the eligibility of individual students. ED"Facts" does not collect student-level data. However, some of the data reported to ED"Facts", such as the academic performance of different groups of students, are based on student-level information maintained by the state or local education agencies. Federal education policy and program planners will address a number of issues in deciding what, if any, action to take in advance of FRL data changes resulting from the new law. This paper provides background information that is intended to support discussion about the following questions: (1) Is individual student-level FRL eligibility status required by all or only some of the programs that rely on ED"Facts" for their data? Would school estimates of FRL eligible percentages be sufficient for some of these programs? (2) Could state education agencies continue to collect the information now used to directly certify students for FRL--such as SNAP or Temporary Assistance for Needy Families (TANF) status--if a state or LEA adopted a school lunch program certification method that no longer required individual FRL eligibility data? (3) How do states currently deal with variety in certification methods among their own LEAs and schools? Would these methods be acceptable as variety in certification approaches presumably increases? The purpose of this paper is to examine the current FRL eligibility measure used by ED and the states in order to infer what changes in this measure are likely under PL 111-296 and identify any existing or proposed alternative measures that ED might wish to consider. The report will not address statistical or methodological issues (e.g., the design of proposed validation studies), but will discuss the findings of such studies where they are relevant to FRL eligibility data. The following are appended: (1) Allowed Access to Individual Eligibility Data; (2) State Definitions of Economically Disadvantage for No Child Left Behind Accountability Reporting: School Year 2010-11; and (3) Short Description of the American Community Survey.

The idea that teaching practices could be improved through better data systems has captured the imagination of policy makers, researchers and foundations. However, the concept of data use itself is so ill-defined as to suggest that it is more of a panacea than a coherent reform proposal. This paper examines the concept with an eye toward what we would need to know about teacher data use to ensure that it would actually yield productive improvements in teaching and would not turn out to be an ill-thought-through panacea. The paper offers a research agenda organized around two central issues: The difference between psychometric feedback and clinical feedback, and the difference between productive teaching and defensive teaching. (Contains 1 table.)

The top priority for the U.S. Department of Education's new Race to the Top-District (RTT-D) competition is to create personalized-learning environments to bolster student achievement: "Absolute Priority 1: Personalized Learning Environments. To meet this priority, an applicant must coherently and comprehensively address how it will build on the core educational assurance areas (as defined in the notice) to create learning environments that are designed to significantly improve learning and teaching through the personalization of strategies, tools, and supports for students and educators that are aligned with college- and career-ready standards (as defined in the notice) or college- and career-ready graduation requirements (as defined in the notice); accelerate student achievement and deepen student learning by meeting the academic needs of each student; increase the effectiveness of educators; expand student access to the most effective educators; decrease achievement gaps across student groups; and increase the rates at which students graduate from high school prepared for college and careers." Using online learning in blended-learning environments will be critical for most Local Education Agencies (LEAs) to realize this priority. As Innosight Institute has written in several reports, there are a growing number of schools and districts across the country that are implementing blended-learning models. Innosight Institute has collected dozens of case studies of blended-learning programs on its website. These profiles may be an important resource for those LEAs seeking to apply to the competition, as the profiles chronicle the history of how schools and districts started their blended-learning programs, the effect of these programs on student achievement, the blended-learning models they use, and the software or Internet tools that power these programs. This paper describes how personalized learning looks like in practice and discusses the essential elements of an effective transformation. (Contains 2 figures.)

The Australian Vocational Education and Training Management Information Statistical Standard (AVETMISS) for vocational education and training (VET) providers (referred to as "the Standard") is currently being reviewed by the National Centre for Vocational Education Research (NCVER). The Standard is the framework that underpins the National VET Provider Collection. It provides consistency in data definitions, which ensures accurate data for use in national data collections, where information is compiled from many different sources. This discussion paper is the first step in the review. It provides a framework for feedback and identifies key issues for consideration. There are many issues associated with the specification of a Standard, such as what information is to be collected, from whom and how often. Decisions about implementation of the AVETMIS Standard are made by a range of different bodies and, while the focus of this paper is primarily on the information to be collected, comments on the timing and coverage are also welcome. Two appendices are included. (Contains 1 figure.)

Enrollment is rising across the nation's community colleges, but completion rates remain untenably low. Reformers are focusing on the importance of using comprehensive, high-quality data on student progress and completion to bring about change. A core tenet of Achieving the Dream: Community Colleges Count has been to embed a culture of data-informed decision making on campuses and in state capitals. To move this work forward, a group of data experts from Achieving the Dream's state policy teams came together in 2005 to design and test a valuable set of intermediate measures that demonstrate how community college students progress--or, as is too often the case, fail to progress--toward a credential or further education. Most important, this Cross-State Data Work Group showed the predictive power of interim indicators for identifying students in great need of intervention early enough to help boost their chances of staying in and finishing college. "On the Road to Success" presents the next step taken by the Cross-State Data Work Group: a comparative analysis of a comprehensive set of intermediate milestones and final measures of success that practitioners and policymakers can use to rethink and remake their approaches to increasing student achievement. Any institution in the country can use these "Benchmarks of Student Success"--a set of key indicators of community college student progression and completion--to frame strategic planning. This is one of the first times such comparative data are being reported publicly. The state-level data in this brief represent only part of what the members of the Data Work Group have collected and reported, and their cross-state conversations will continue. Perhaps an even more powerful impact will be derived as states turn this analytic approach inward and examine the comparative effectiveness of colleges within a single state. Benchmarks for Student Success are appended. (Contains 11 figures, 1 table, and 10 endnotes.) [For "On the Road to Success: How States Collaborate and Use Data to Improve Student Outcomes. Executive Summary," see ED537263.]

This report describes a conversion experiment and subsequent reader survey conducted by the American Council of Learned Societies (ACLS) Humanities E-Book (HEB) in late 2009 and early 2010 to assess the viability of using scholarly monographs with handheld e-readers. As sample content, HEB selected six titles from its own online collection, three in a page-image format with existing OCR (optical character recognition)-derived text and three encoded as XML files, and had these converted by an outside vendor with minimal editorial intervention into both MOBI (prc) and ePub files. During its in-house assessment phase, HEB experienced some navigational difficulty with both formats and found that annotation and other interaction with the text was difficult using a number of popular e-readers. HEB also found the XML titles to be of limited functionality in the MOBI format and therefore opted not to further poll readers on this subset. About 88% of the 142 survey participants expressed overall satisfaction with the appearance and functionality of the three remaining handheld samples, although roughly half reported some level of frustration with the search function using either format, and only 26% felt they would have an easy time citing and referencing these editions. Satisfaction with other interactive features, such as adding notes, bookmarking and highlighting, was noticeably higher; however, the "n/a" option was also selected frequently for these categories, and it appears that a large number of participants were unable to perform the tasks in question due to confusing or insufficient instructions from the device manufacturer. As formats evolve, future satisfaction with these features may increase. Irrespective of specific limitations, 75% of participants were interested in potentially downloading additional similar titles for free or if priced below $10. HEB's initial findings in this study indicate that titles formatted for existing handheld devices are not yet adequate for scholarly use in terms of replicating either the benefits of online collections--cross-searchability, archiving, multifarious interactive components--nor certain aspects of print editions that users reported missing, such as being able to mark up and rapidly skim text. A turnaround is underway once a common and more robust format optimized for handheld readers is determined and devices themselves evolve, adding improved display options and better and more intuitive web-access, searching and other interactive use of content. Survey Results is appended. (Contains 30 footnotes.)

What do you want to accomplish? Whatever it is, you need money to do it, and on most campuses, the biggest source of funding is enrollment-related revenue from students. Yet, interestingly, perhaps because most chief executives are more familiar with fundraising, opportunities to strengthen enrollment revenue often receive less attention than major donors. This paper is designed to help bridge this gap. In 10 succinct points, readers who know and understand fundraising will learn how to easily transfer that knowledge to understand the enrollment revenue stream. You'll see how you can work more closely with your enrollment team to exert greater influence over enrollment, its revenue, and strategic enrollment planning. After reading this paper, the hope is that you will: (1) Gain a better understanding of how to manage enrollment to accomplish strategic institutional goals; (2) Know where your support and leadership is crucial to your enrollment team; and (3) Identify ways to exert more influence over enrollment-related revenue.

The recent demand for schools to respond to external accountability measures challenges school leaders to create school instructional systems that use data to guide the practices of teaching and learning. This paper considers how local school leaders build data-driven instructional systems (DDIS) by developing new programs and using existing school functions to create an information flow through a school. The paper considers how leaders work with teachers and students to create DDIS to intentionally and systematically improve student learning. The paper begins by presenting a theoretical and analytical framework for understanding the systems that local school leaders create to develop a DDIS to facilitate an information flow about student achievement in schools. The DDIS is presented as six component functions involving (1) data acquisition, (2) data reflection, (3) program alignment and integration, (4) program design, (5) formative feedback, and (6) test preparation. The second part of the paper reviews data collected in a year-long study of four schools, using the DDIS cycle as a framework for describing how school leaders structure opportunities to engage in data-driven decision making. The data provide rich examples of how schools facilitate and impede the flow of student achievement information, and call into question any simple definition of data-based decision making in schools. The following are appended: (1) DDIS Interview Protocol; and (2) DDIS Coding Scheme. (Contains 1 table, 1 figure, and 2 footnotes.) [This paper was prepared for the Annual meeting of the National Council of Professors of Educational Administration (Washington, DC, July 2005).]

Feedback on how students progress through completing subgoals can improve students' learning and motivation in programming. Detecting subgoal completion is a challenging task, and most learning environments do so either with "expert-authored" models or with "data-driven" models. Both models have advantages that are complementary -- expert models encode domain knowledge and achieve reliable detection but require "extensive authoring efforts" and often cannot capture all students' possible solution strategies, while data-driven models can be easily scaled but may be less accurate and interpretable. In this paper, we take a step towards achieving the best of both worlds -- utilizing a data-driven model that can intelligently detect subgoals in students' correct solutions, while benefiting from human expertise in editing these data-driven subgoal rules to provide more accurate feedback to students. We compared our hybrid "humanized" subgoal detectors, built from data-driven subgoals modified with expert input, against an existing data-driven approach and baseline supervised learning models. Our results showed that the hybrid model outperformed all other models in terms of overall accuracy and F1-score. Our work advances the challenging task of automated subgoal detection during programming, while laying the groundwork for future hybrid expert-authored/data-driven systems. [For the full proceedings, see ED615472.]

This timely report focuses on accountability -- the current lingua franca of higher education -- and the question of the public trust as a reflection of the respect and confidence of the people that are served by the nation's colleges and universities. Designed to assist policymakers and educational leaders, the report identifies the components of a state-level higher education accountability system: acting on a public agenda, maintaining the public trust of the people served by higher education, instituting well-conceived and meaningful performance indicators, and enabling data-driven decision making are each considered essential for engaging all stakeholders in a state's higher education accountability system. Elected officials, members of the higher education system, and stakeholders all must work in partnership with one another to carry out their respective roles. The report also highlights public trust and its import as it relates to policymaking and higher education. Appended are: (1) Illustrative State Programs; (2) Performance indicators used in several states; (3) Selected performance measures in other countries; and (4) Definitions. (Contains 1 figure and 158 footnotes.)