Search

This appendix is a companion to the "Study of Enhanced College Advising in Upward Bound: Impacts on Where and How Long Students Attend College" (ED615732) report. The appendix provides additional information about the "Find the Fit" enhanced advising strategies and the study that assessed its effectiveness when implemented within Upward Bound projects. This additional information includes details on how the study was designed and conducted, the statistics that support key findings in the report, and exploratory analyses performed to further investigate and understand those key findings. The content of the appendix is referenced throughout the main report. The following sections are included: (1) Additional details about "Find the Fit; (2) Additional information about how the study was designed and implemented; (3) Supplemental tables and information on study findings; and (4) Effects by level of "Find the Fit" Implementation. [For the study highlights, see ED615734.]

Decisions about whether and where to go to college can make a difference in students' later success. However, many students from low-income families--"undermatch"--they do not enroll at all or do not enroll in the most selective college they likely could attend. This study investigated whether promising advising strategies, bundled together in a package called "Find the Fit," could improve college choices for rising high school seniors in the federal college access program Upward Bound. "Find the Fit" includes customized information about college going and costs, text messaging of key application and financial aid deadlines, and specialized training for the students' advisors. About 200 Upward Bound projects across the country volunteered to test "Find the Fit," with half randomly selected to get access to it during the study period. This final report compared the college choices of 4,500 students in projects with and without access to "Find the Fit" to determine its effects on college attendance. Key findings include: (1) Despite its initial benefits -- increasing the number and selectivity of the colleges to which students applied -- "Find the Fit" did not change whether these students, mostly from low-income families, undermatched; (2) "Find the Fit" did shift some students' enrollment choices to more selective colleges, regardless of whether students were undermatched. This shift did not seem to come with higher costs or a greater risk of dropping out, potential consequences of attending a more selective college; and (3) But there was no significant effect on college persistence through the third year after high school. [For the study highlights, see ED615734. For the appendix, see ED615736.]

Abt Associates evaluated the Bridges to Career Opportunities (BCO) program, developed by the Local Initiatives Support Corporation (LISC), to determine whether adults with low incomes and low skills who participate in BCO services are more likely to increase their employment and financial stability than do adults who participate in LISC's Financial Opportunity Center® (FOC) services alone. Overall, participants who were in the BCO programs were 19 percentage points more likely to have a "job improvement" than were participants in FOC-only programs (56 percent versus 37 percent). [This report was submitted to and supported by the Local Initiatives Support Corporation (LISC).]

These are the appendices for the report "Study of College Transition Messaging in GEAR UP: Impacts on Enrolling and Staying in College." Five appendices are included in the document: (1) Additional Details about College Transition Messaging; (2) Additional Information about How the Study Was Designed and Implemented; (3) Supplemental Tables and Information on Study Findings; (4) Bayesian Analysis of Immediate College Enrollment Result; and (5) College Advising Experiences. [For "Study of College Transition Messaging in GEAR UP: Impacts on Enrolling and Staying in College. Evaluation Report. NCEE 2021-005," see ED610968. For "Study of College Transition Messaging in GEAR UP: Impacts on Enrolling and Staying in College. Study Highlights. NCEE 2021-005h," see ED610970.]

Many high school seniors who plan to attend college do not enroll, and others enroll but do not return for a second year. Low-income students are more likely than their higher-income peers to fall off track after high school ends, when they often have limited connections to advisors, insufficient information about key college transition steps, and concerns about falling short and fitting in. Text-message-based advising is an increasingly popular strategy to address these challenges, including among states and districts that participate in the federal college access program Gaining Early Awareness and Readiness for Undergraduate Programs (GEAR UP). In this study about 4,800 college-intending seniors in high-need high schools across the country were randomly divided into two groups: one received their regular GEAR UP supports in the summer before and during their first year of college, and the other group received these regular supports with the addition of text messaging designed to support a successful transition to college. The study compared the experiences and outcomes of the two groups to determine the effectiveness of the transition messaging. [For "Study of College Transition Messaging in GEAR UP: Impacts on Enrolling and Staying in College. Appendix. NCEE 2021-005a," see ED610969. For "Study of College Transition Messaging in GEAR UP: Impacts on Enrolling and Staying in College. Study Highlights. NCEE 2021-005h," see ED610970.]

"Find the Fit" advising increased the number and selectivity of colleges applied to by disadvantaged students in a federal college access program. "Find the Fit" includes customized information about college going and costs, text messaging of key application and financial aid deadlines, and related advisor training. These research-based strategies are designed to help students enroll in the best college they can and stay in until they complete a degree. This snapshot of the first report of the study looks at "Find the Fit's" effects on students' steps toward enrolling in a more selective college. [For the full report, "Study of Enhanced College Advising in Upward Bound: Impacts on Steps toward College. NCEE 2019-4002," see ED588782. For the study highlights, "Study of Enhanced College Advising in Upward Bound: Impacts on Steps toward College. Study Highlights. NCEE 2019-4002," see ED588786.]

The U.S. Department of Education tested a set of promising, low-cost advising strategies, called "Find the Fit," designed to help low-income and "first generation" students enrolled in the Department's Upward Bound program choose more selective colleges and stay in until they complete a degree. About 200 Upward Bound projects with 4,500 seniors agreed to participate. The projects were randomly assigned to receive "Find the Fit" to supplement their regular college advising (treatment group) or to offer their regular advising (control group). This first of three reports looks at "Find the Fit's" effects on students' steps toward enrolling in a more selective college. The study found that the enhanced advising increased the number and selectivity of colleges to which students applied. [For the study snapshot, "Study of Enhanced College Advising in Upward Bound: Impacts on Steps toward College. Study Snapshot. NCEE 2019-4002," see ED588785. For the study highlights, "Study of Enhanced College Advising in Upward Bound: Impacts on Steps toward College. Study Highlights. NCEE 2019-4002," see ED588786.]

The National Science Foundation contracted with Abt Associates to conduct an evaluation of its Partnerships for International Research and Education (PIRE) program, which supports intellectually substantive collaborations between U.S. and foreign researchers in which the international partnership is essential to the research effort. The evaluation compared the research outcomes of PIRE and comparison group awards (i.e., projects) and personnel (Principal Investigators, postdoctoral and graduate student participants); compared participant experiences, especially collaborations with foreign investigators during and subsequent to the award period; examined the role of U.S. institutional policies and practices in the implementation of PIRE awards; and explored potential contributions to global societal challenges. Findings suggest that the PIRE program is fostering the global engagement of U.S. scientists and engineers with no loss of research productivity or quality relative to the comparison group. A quasi-experimental comparative interrupted time series analysis showed that PIRE increases postdoctoral researchers' and graduate students' research productivity and the citation impact of postdocs' publications. This is the first of three volumes of the final report for the Evaluation of NSF's Partnerships for International Research and Education (PIRE) program. Volume 1 includes an Executive Summary, an overview of the PIRE program and the objectives of the evaluation, a brief explication of study methods, and main findings. [This report was written with assistance from Lisa Setrakian and Maureen Sarna. For volume 2 "Evaluation of the National Science Foundation's Partnerships for International Research and Education (PIRE) Program, Volume 2: Supplementary Materials. Final Report," see ED563796. For volume 3, which contains separate reports of each survey, visit: http://abtassociates.com/Reports/2015/Evaluation-of-the-National-Science-Foundation%E2%80%99-(8).aspx.]

The National Science Foundation contracted with Abt Associates to conduct an evaluation of its Partnerships for International Research and Education (PIRE) program, which supports intellectually substantive collaborations between U.S. and foreign researchers in which the international partnership is essential to the research effort. The evaluation compared the research outcomes of PIRE and comparison group awards (i.e., projects) and personnel (Principal Investigators, postdoctoral and graduate student participants); compared participant experiences, especially collaborations with foreign investigators during and subsequent to the award period; examined the role of U.S. institutional policies and practices in the implementation of PIRE awards; and explored potential contributions to global societal challenges. Findings suggest that the PIRE program is fostering the global engagement of U.S. scientists and engineers with no loss of research productivity or quality relative to the comparison group. A quasi-experimental comparative interrupted time series analysis showed that PIRE increases postdoctoral researchers' and graduate students' research productivity and the citation impact of postdocs' publications. This report contains supplementary materials to the final report. This is the second of three volumes of the final report for the Evaluation of NSF's Partnerships for International Research and Education (PIRE) program. Volume 2 includes additional material: technical details on the study methods and survey response rates, supplementary information about the research outcomes of the PIRE program, and a comparison of PIRE participants' employment outcomes to national trends for science and engineering (S&E) degree recipients. [For volume 1 "Evaluation of the National Science Foundation's Partnerships for International Research and Education (PIRE) Program, Volume 1: Final Report," see ED563795. For volume 3, which contains separate reports of each survey, visit: http://abtassociates.com/Reports/2015/Evaluation-of-the-National-Science-Foundation%E2%80%99-(8).aspx.]

Over a decade ago, the National Science Board (NSB) highlighted the importance of international collaboration in its call for increased government commitment to promoting international science and engineering (S&E) research and education. The NSB also identified the National Science Foundation (NSF) as having an important leadership role in international S&E research and education activities. Among NSF's postdoctoral programs, the International Research Fellowship Program (IRFP) is unique in its emphasis on providing postdoctoral fellows with international research experiences. Established in 1992, IRFP provides financial support to postdoctoral scientists for a research experience abroad lasting from 9 to 24 months; there is no restriction on the geographical location of the hosting institution. NSF contracted with Abt Associates to conduct an evaluation of IRFP, to gather evidence about whether the IRFP program has achieved its goal of furthering the collaborative activities and international partnerships of early career STEM researchers. The evaluation found evidence that the IRFP program is meeting its goals. This report summarizes the findings from the evaluation of the International Research Fellowship Program (IRFP), which was designed to investigate the characteristics of IRFP applicants and their motivations for participation, the host researchers' motivations for participating in the program, participants' experiences in and perceptions of the program, and outcomes of the program. Findings from the evaluation of IRFP revealed important benefits of the program, on both an individual and a collective basis.

Students in science and engineering (S&E) are preparing for careers in fields where international partnerships are increasingly important to advancing knowledge and discoveries. It has been over a decade since the National Science Board (NSB) highlighted the importance of international collaboration and called for increased government commitment to promoting international S&E research and education. NSB called for the National Science Foundation (NSF) to take a leadership role in international S&E research and education activities promoting "…increased participation in international S&E activities by younger U.S. scientists and engineers from diverse backgrounds, especially those in the early stage of their careers, in order to develop an internationally competitive and globally-engaged S&E workforce." NSF's East Asia and Pacific Summer Institutes (EAPSI) program, which provides international fellowships to U.S. graduate students in S&E, is one program that helps to further NSF's commitment to support the active engagement of early career S&E researchers in international collaborations. NSF contracted with Abt Associates to conduct an evaluation of the EAPSI program to investigate whether it was meeting its goal of providing U.S. graduate students with international experiences that will enable future collaboration with foreign peers. This report summarizes the findings from that evaluation, which was designed to investigate the characteristics of EAPSI applicants and their motivations for participation, the host researchers' motivations for participating in the program, participants' experiences in and perceptions of the program, and outcomes of the program.

Students in science and engineering (S&E) are preparing for careers in fields where international partnerships are increasingly important to advancing knowledge and discoveries. It has been over a decade since the National Science Board (NSB) highlighted the importance of international collaboration and called for increased government commitment to promoting international S&E research and education. The National Science Foundation (NSF) contracted with Abt Associates to conduct an evaluation of the East Asia and Pacific Summer Institutes (EAPSI) program to investigate whether it was meeting its goal of providing U.S. graduate students with international experiences that will enable future collaboration with foreign peers. This report presents the findings from this evaluation. The evaluation found evidence that the EAPSI program is meeting its goals to: (1) introduce U.S. graduate students to East Asia and Pacific science and engineering in the context of a research setting; and (2) help students initiate scientific relationships that will better enable future collaboration with foreign counterparts. The following are appended: (1) NSF-Generated EAPSI Descriptions; (2) Non-Response Bias Study; (3) Detailed Description of Impact Analysis; (4) Detailed Description of Benchmarking Analysis; (5) Surveys; and (6) Benchmark of Applicants to National Data.

Citation: Ricciuti, A.E., R.G. St.Pierre, W. Lee, A. Parsad & T. Rimdzius. Third National Even Start Evaluation: Follow-Up Findings From the Experimental Design Study. U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance. Washington, D.C., 2004. Background: The Even Start Family Literacy Program has provided instructional services to low-income children and their parents since 1989. A previous randomized controlled trial in the early 1990s did not show this program to have positive impacts. Purpose: To assess the effectiveness of Even Start in a group of grantees around the country. An earlier report from this study presented impact findings based on pretest and posttest data at the start and end of a school year. No program impacts were found. The purpose of the current report is to present impact analyses of follow-up data collected one year after posttest data. Setting: 18 Even Start grantees in 14 states that operated in the 1999-2000 and 2000-2001 school years. Subjects: 463 families eligible for and interested in participating in Even Start family literacy services. Intervention: Even Start families were offered family literacy services, defined as (1) interactive parent-child literacy activities, (2) parenting education, (3) adult education, and (4) early childhood education. Research Design: Randomized controlled field trial in which families were randomly assigned either to Even Start (309 families) or a control group (154 families). Control or Comparison Condition: Control families could participate in any educational and social services to which they were entitled, but they were not allowed to participate in Even Start for one year. Data Collection and Analysis: Pretest data on child and adult literacy skills were collected in the fall, posttest data were collected in the spring/summer, and follow-up data were collected the next spring. Measures included direct assessment of children (Peabody Picture Vocabulary Test, Woodcock-Johnson Battery, Story & Print Concepts), direct assessment of parents (Woodcock- Johnson Battery), teacher report on children (Social Skills Rating System), parent reports on economic and educational status, child literacy-related skills, home literacy environment and activities, parent assessment of children (Vineland Communication Domain), and school records. A longitudinal sample (data at all three waves) of children and parents was created for each outcome measure, and t-tests were conducted to assess differences in gains between Even Start and control groups. The sample size for the analysis of any given outcome depends on several factors including attrition, age of the child, exclusion of families who were assessed in Spanish, and the need for longitudinal data. For example, the PPVT analysis for children was done with samples of 97 Even Start and 44 control children, and the Woodcock-Johnson analysis for parents was done with samples of 149 Even Start and 65 control parents. Findings: As was the case at posttest, Even Start children and parents made gains on a variety of literacy assessments and other measures at follow-up, but they did not gain more than children and parents in the control group. It had been hypothesized that follow-up data might show positive effects because (1) Even Start families had the opportunity to participate for a second school year, and (2) change in some outcomes might require more time than others. However, the follow-up data do not support either of these hypotheses. Conclusion: The underlying premise of Even Start as described by the statute and implemented in the field was not supported by this study.

The Even Start Literacy Program, established in 1989, aims to simultaneously improve the literacy of children and their parents through (1) early childhood education; (2) parenting education; (3) adult education; and (4) parent-child joint literacy activities. The report details findings from the third national Even Start evaluation. The evaluation included two complementary studies: the Even Start Performance Information Reporting System (ESPIRS), which provided annual data on the universe of Even Start projects, and the Experimental Design Study (EDS) which examined Even Start effectiveness in 18 projects. The report draws on data collected in all four years of the ESPIRS as well as pretest and posttest data for 1999-2000 and 2000-2001 collected from the 18 EDS projects. Findings include the following: (1) While Even Start children and parents made gains on literacy assessments and other measures, children and parents in the 18 programs did not gain more than those in the control group, about one-third of whom received early childhood education or adult education services; (2) compared to Head Start, Even Start parents are much less likely to have a high school diploma, and Even Start families have substantially lower household income; (3) Even Start children and parents made small gains on literacy measures and scored low compared to national norms when they left the program; (4) families do not take full advantage of the services offered by Even start projects, participating in a small amount of instruction relative to their needs and program goals; (5) while early childhood classroom experiences provided by the EDS projects were of overall good quality, there was not sufficient emphasis on language acquisition and reasoning to produce measurable impacts and hence to achieve legislative outcomes; and (6) the extent to which parents and children participated in literacy services is related to child outcomes. (Contains 67 references.) (HTH)