Search

Those who study education policy have struggled to understand how money matters in education and why educational research finds no strong or consistent relationship between expenditures and student performance. The papers in this publication were commissioned by the National Center for Education Statistics (NCES) to address the twin concerns of what additional school-finance information NCES should collect and report, and how extant data might be analyzed. The publication is dedicated to Charles S. Benson, University of California, Berkeley, who dedicated his life's work to improving education finance for students in poor school districts. Following the foreword by Paul D. Planchon, and acknowledgments, contents include the following papers: (1) "Introduction and Overview" (William J. Fowler, Jr.); (2) "Does Money Matter in Education? A Policymaker's Guide" (Lawrence O. Picus); (3) "The Effect of Constitutional Litigation on Educational Finance: A Further Analysis" (G. Alan Hickrod); (4) "Student-Level School Resource Measures" (Robert Berne and Leanna Stiefel); (5) "Proposed 'Good Practices' for Creating Data Bases from the F-33 and CCD for School Finance Analyses" (Michael O'Leary and Jay Moskowitz); and (6) "The Empirical Argument for Educational Adequacy, the Critical Gaps in the Knowledge Base, and a Suggested Research Agenda" (William H. Clune). Each article contains references. (LMI)

This paper discusses the use of telephone surveys as a substitute for, or addition to, the use of door-to-door surveys in assessing adult literacy ability. Part 1 addresses issues of validity in adult literacy assessment, including the concepts of construct or inferential validity and action or use validity. Part 2 is a review of related research that has used telephone and mail surveys to assess the cognitive abilities of adults. Part 3 discusses the theoretical model of literacy that the National Adult Literacy Survey (NALS) and International Adult Literacy Survey (IALS) have used and how that model can be modified to incorporate a developmental and componential model of literacy that also includes the oral language aspects of literacy given in the Adult Education and Family Literacy Act of 1998. Part 4 addresses several issues about the use of telephone and mail surveys, including concerns about using these survey methods in such a way as to make continuity in trend data from previous surveys of adult literacy possible. Appendixes include 69 references and a summary of an empirical study of the proposition that at a certain stage adults' comprehension of language can be assessed as well by listening as by reading. (Author/YLB)

A study focused on researchers' use of the English-Language Background Questionnaire (EBQ) portion of the National Adult Literacy Survey (NALS), the results of which were released in 1993. Data were gathered by surveying NALS data users about how they have used the EBQ data, their perspectives on the strengths and weaknesses of the EBQ, and their recommendations for changes to the EBQ. A content analysis was also conducted of published research papers that have involved analyses of the EBQ data to determine how the data were used in these research analyses. The study also drew upon recommendations garnered by a series of focus groups conducted by the American Institutes for Research in 1998. These sources were integrated with the study's own analysis of the NALS data. The following changes to the EBQ for future adult literacy assessments were recommended: expand the questions pertaining to educational experiences; expand the section on social and political participation or move the current items to other areas and delete this section; add an item on voter registration; acquire more detailed information regarding work history, wages, and parents' occupations; expand the section pertaining to literacy practices; and gather information on technological literacy practices. (Appendixes include 34 references; instruments; and studies examined for content review.) (YLB)

The Workforce Investment Act of 1998 shifts decision-making authority for funding of local job training programs from the federal government to state and local boards, which will need local decision-making tools to inform policy. One such tool is a method proposed by S. Goetz and D. Debertin to estimate demand for educational attainment at the county level utilizing nonlocal secondary data on employment patterns. The method focuses analysis on the demand of private businesses and makes possible a simple forecast of future demand. This paper estimates local demand for educational attainment in two West Virginia counties using the Goetz-Debertin (nonsurvey) method and compares results with those from a survey of local businesses. Two assumptions of the nonsurvey method are fixed proportions of occupations per sector and fixed proportions of educational attainment per occupation. Results indicate that differences in the distribution of businesses by size and by subsector across counties might affect final demand, but the nonsurvey method does not take this into account. Indications that proportions of input might change over time suggest that the nonsurvey method should use annual data sets such as the Current Population Survey. Conflicting with the nonsurvey method's second assumption is the finding that certain occupations in different sectors differ in demand for educational attainment. The study also found that the West Virginia counties' proportion of top-level occupations was substantially below national averages, suggesting underinvestment in education, and that businesses saw a real need for more computer training. (Contains 15 references, many data tables, and the business survey questionnaire.) (SV)

Current research into youth transitions in Australia documents an increasingly individualized process in which significant numbers of youths are deemed at risk of not making a successful transition from school to work. Many theorists are questioning the applicability of the linear model of transition to current conditions. Other theorists are questioning whether the model was ever applicable to all students (especially "nonmainstream" students). The literature also documents the perceived failure of policy in ensuring successful transitions through recognized "pathways" of vocational learning and experience. It may be argued that, by broadening their focus to include the informal vocational experiences young people initiate and the type of learning that occurs in such instances, educational researchers may provide useful insights into how young people experience the transition process and how they seek to position themselves in the youth labor market. Research on this area is being conducted as part of the Research Centre for Vocational Education and Training's national key center program supported by the Australian National Training Authority. It is hoped that this research will shed new light on the increasingly complex transition process experienced by noncollege-bound young people and help policymakers devise more effective policies to assist this transition. (Contains 22 references.) (MN)

Cognitive interviews were conducted with 12 teachers and 66 of their students to learn how to improve National Assessment of Educational Progress (NAEP) questionnaire items asking about instructional practices and teacher development experiences. Items were purposively selected from those used in the 1996 and 1998 NAEP student and teacher background questionnaires in fourth grade reading, mathematics, and science, and eighth grade mathematics and science. NAEP items that were administered in analogous forms to teachers and their students were chosen, to facilitate validation, and items known or thought to be problematic were selected. Several general types of item problems were found: (1) behavioral frequency items; (2) time frame problems; (3) problems with response options; (4) comprehension problems; (5) problems with list format items (loss of content); (6) problems with check all that apply items; and (7) other problems and issues. The cognitive interviews, with validation components, enabled detection of many survey design problems and an understanding of the reasons for their occurrence so that these problems can be avoided in the development of future NAEP and other background survey items. (Contains 9 tables and 16 references.) (SLD)

A study explored which nonexperimental comparison group methods provide the most accurate estimates of the impacts of mandatory welfare-to-work programs and whether the best methods work well enough to substitute for random assignment experiments. Findings were compared for nonexperimental comparison groups and statistical adjustment procedures with those for experimental control groups from a large-sample, six-state random assignment experiment--the National Evaluation of Welfare-to-Work Strategies. The methods were assessed in terms of their ability to estimate program impacts on annual earnings during short-run and medium-run follow-up periods. Findings with respect to the first issue suggested in-state comparison groups perform somewhat better than out-of-state or multi-state, especially for medium- run impact estimates; a simple difference of means or ordinary least squares regression can perform as well or better than more complex methods when used with a local comparison group; impact estimates for out-of-state or multi-state comparison groups are not improved substantially by more complex estimation procedures but are improved somewhat when propensity score methods are used to eliminate comparison groups that are not balanced on their baseline characteristics. Findings with respect to the second issue indicated the best methods did not work well enough to replace random assignment. Statistical analyses are appended. (Contains 51 references.) (YLB)

The issue of increasing productivity was examined from an interdisciplinary perspective focusing on the impact of workplace practices on various productivity-related outcomes. First, the following methodological issues were discussed: defining workplace practices that affect productivity; linking employer behavior and organizational performance; dealing with the complexity of interrelated factors; reverse causality; bias from selection into the program; bias from the research and publication process; biases from reverting to normal; the Hawthorne effect; and short-run versus long-run effects. Next, the impacts of the following workplace practices on productivity were analyzed with consideration for those methodological issues: job design; employee involvement; compensation; alternative work time arrangements; training; diversity management; and workplace well-being programs. Most of those workplace practices had positive effects on employees, which in turn positively affected firm performance, productivity, and competitiveness. Success of the workplace practices was enhanced when they were combined in clusters, integrated to fit overall corporate strategy, and supported by managers, supervisors, and unions. The analysis identified 11 barriers to adoption and diffusion of "best" workplace practices, including the following: managerial resistance, employee resistance, union resistance, legislative barriers, short-term focus, workplace practices as a source of competitive advantage, barriers to cooperative actions, and externalities and the fact that trained employees may be lured away by other companies. (Contains 433 references.) (MN)

This paper discusses the initial development and testing of a Web-based instrument for collecting daily data on instruction. This teacher log was developed for use in the Study of Instructional Improvement, a longitudinal study on school improvement in high poverty areas. The researchers wanted to further develop the potential of teacher logs by experimenting with Web-based technology and computer branching. The 1998 teacher log study tested the feasibility of using a Web based instrument to collect data on instruction. Seven elementary teachers used the log to report on 29 mathematics and reading lessons. Researchers observed classroom instruction, completed log reports, and wrote detailed narrative descriptions for 24 lessons. All teachers were able to use the log to report on instruction. Data were successfully entered through a Web interface into a database and subsequently used to conduct provisional analyses of both instruction and instrument reliability and validity. The researchers conclude that a Web-based instrument can be used successfully to capture important information about teaching, some of which has not been available before (lesson duration, student attendance for each lesson, and instructional groupings or formats for each lesson). An appendix offers teacher log pilot help pages. (SM)

This report describes and evaluates the methods and procedures used for the field test of the Baccalaureate and Beyond Longitudinal Study (B&B:2000/01). The B&B:2000/01 field test included important changes from previous B&B surveys (conducted in 1994 and 1997) in its sample design and collection of data. The introductory chapter describes the background, purposes, schedule, and products of the B&B:2000/01 study and the unique purposes of the field test. In chapter 2, the field test design and method are described. Overall outcomes of data collection, as well as results of special studies, are presented in chapter 3. Chapter 4 documents the quality of the data collected, and chapter 5 summarizes the major recommendations for changes in design for the full-scale study. Materials used during the field test are provided as five appendixes to the report, and cited, where appropriate, in the text. (Contains 33 tables and 6 figures.) (SLD)

Many imputation techniques and imputation software packages have been developed over the years to deal with missing data. Different methods may work well under different circumstances, and it is advisable to conduct a sensitivity analysis when choosing an imputation method for a particular survey. This study reviewed about 30 imputation methods and 5 imputation software packages. Eleven of the most popular imputation methods were evaluated through a Monte Carlo simulation study. The first four chapters of this report are methodology discussions based on a review of the literature on imputation. Chapter 1, describes about 30 commonly used methods, including those used by the National Center for Education Statistics, and discusses their strengths and weaknesses. Chapter 2 focuses on five software packages for imputation, Nonresponse bias correction through imputation is addressed in chapter 3, and variance estimation with imputed data and multiple imputation inference is discussed in chapter 4. Chapter 5 reports the results of the simulation study, which evaluated 11 methods according to 8 evaluation criteria for 4 types of distributions, 5 types of missing mechanisms, and 4 types of missing rates. (Contains 31 tables and 45 references.) (SLD)

This report describes the imputation procedures used to deal with missing data in the National Education Longitudinal Study of 1988 (NELS:88), the only current National Center for Education Statistics (NCES) dataset that contains scores from cognitive tests given the same set of students at multiple time points. As is inevitable, cognitive test data are missing for some individuals in each test round, especially the second followup (F2). The NCES has decided to use imputation to reduce the bias caused by nonresponse. The first step was to conduct a simulation study to evaluate two different imputation procedures currently used at NCES: the model-based random imputation method (PROC IMPUTE) and a within-class random hot-deck imputation. Simulation findings confirmed that PROC IMPUTE performed better. The second step involved using PROC IMPUTE to impute missing F2 test scores in four subject areas: mathematics, reading, science, and history/citizenship/geography. Results provide end users with complete cognitive test data for both cross-sectional and longitudinal research. (Contains 18 tables, 4 figures, and 7 references.) (SLD)

In large multipurpose surveys, it is common to select the sample systematically proportional to some measure of size (PPS) that is correlated with an important variable of interest. Assuming the frame is sorted in a useful deterministic manner, systematic sample methodologies provide an additional control on the sample allocation, beyond the control provided from the stratification. This makes it less likely to select a "bad sample." This should reduce the variability of the estimates as compared to a comparable nonsystematic selection procedure. The problem with systematic samples is that variance estimators are biased. This paper presents a bootstrap variance estimator, which can have less bias than standard methodologies, such as half-sample replication. The results will be demonstrated with a simulation study based on an important National Center for Education Statistics survey, the Schools and Staffing Survey. An appendix demonstrates the consistency of the proposed estimators. (Contains 8 tables and 12 references.) (Author/SLD)

In September 1998, the National Center for Education Statistics (NCES) established a 1-year task force to review the NCES's role concerning lifelong learning. The eight-member task force established a working definition of lifelong learning ("a process or system through which individuals are able and willing to learn at all stages of life, from preschool through old age") and conducted the following activities: (1) summarized and prioritized policy issues concerning lifelong learning; (2) synthesized exiting data to address monitoring and policy needs; (3) identified and prioritized gaps in existing data; and (4) developed recommendations on data collection strategies. The recommendations focused on the following lifelong learning issue areas: the adult population; learning attitudes and skills of adults; labor market demand for adult learning; participation levels and patterns; goals, incentives, and disincentives; investments in adult learning; adult learning providers; instructional delivery and new technologies; informal learning; services and accommodations for adults; outcomes and effectiveness; and the government's role in adult learning. The task force concluded that adult learning is an important area of education that should have a coherent data collection and reporting system within NCES and that NCES should take the following steps to develop such a system: (1) develop a compendium report summarizing existing information on lifelong learning; and (2) modify existing survey instruments that collect relevant information. (MN)

A study explored how rural women in the village of Srefultoli, Bangladesh describe, from their own point of view, their experiences with nonformal education (NFE). Feminist research has shown that existing NFE programs in developing countries give women traditional knowledge of family planning, nutrition, and health care, but they do not deal with the need to increase women's awareness of their oppression and exploitation. The study examined whether current NFE programs in this village in Bangladesh give women new knowledge about their current situation in society and in the family and whether these women are conscious of their strategic and practical needs. A social mapping methodology was used to pattern these women's perspectives, and the map serves as a conventional symbol for the "way things are," or perceptions of truth. Findings suggest that rural Bangladeshi women are aware of their socio-economic and political subordination and that they use multiple strategies to resist patriarchal domination. Most of the women see participation in NFE programs as an empowering process. Findings also suggest that rural women wanted NFE programs to provide increased opportunities for income generation, for better access to education, for training to help them become economically independent, and for the conscientization of men, so these other objectives may flourish. These women also had specific suggestions regarding the class time, course design, and courses for men that would enhance female participation and would introduce some changes in men's and women's relationships. Conclusions were that certain more participatory NFE programs may bring meaningful changes in women's lives, and that rural women themselves can best identify their own needs. (Contains 5 figures and 43 references.) (Author/BT)

Anticipated demographic changes in the United States adult population in the decade between the National Adult Literacy Survey (NALS) of 1992 and the National Assessment of Adult Literacy (NAAL), which is scheduled for 2002, were reviewed. Next, the implications of those changes for the NALS and NAAL were analyzed. The analysis focused on births, deaths, and international migration. Next, a demographic model of adult literacy proficiency was described that links changes in literacy proficiency to compositional and maturational changes in the adult population. Composition components were defined as changes over time in membership in the target population, whereas maturational components were defined as changes over time in the literacy proficiencies of given individuals who are in the target population at both time points. The model predicted increases over the decade in the mean literacy proficiency of both the foreign- and native-born adult populations and decreases in the percentages of each that are performing at the lowest level. (Eight implications of the model for the NAAL methodology were discussed, including implications related to the following issues: parallel population coverage in the NALS and NAAL assessments; adequate representation of minority and immigrant groups; and working with populations having limited English proficiency.) (MN)

The New Approaches to Lifelong Learning (NALL) project is a Canada-wide 5-year research initiative during which more than 70 academic and community members are working collaboratively within a framework of informal learning to address the following issues: informal computer-based learning, recognition of prior learning, informal learning in a variety of social locations, learning within marginalized or disadvantaged cultures, and learning about school-to-work transitions. The NALL project's primary objective is to identify major social barriers to integrating informal learning with formal/nonformal learning and certification and to support new program initiatives to overcome such barriers. The NALL project's focus is on the informal and nonformal learning practices of people involved with the Growing Jobs for Living Project (GJOBS) in the Quinte bioregion, located on the north shore of Lake Ontario in Canada. These learning practices are related to the principles and practices of environmental adult education, feminist adult education, and transformative learning. The global and ideational contexts of some of the major socio-environmental changes and problems that have affected the Quinte bioregion and been a catalyst for GJOBS were examined. The methods used to study the informal learning practices of GJOBS participants were reviewed. The major outcomes of the study were discussed from the standpoint of their relationship to the broader field of adult education. (Contains 25 references.) (MN)

An on-going study is examining a method proposed by S. Goetz and D. Debertin for estimating demand for educational attainment at the county level by using secondary data. The method produces compact information showing the relationship between employment, occupation, and educational attainment, information not available at the county level. The method offers a way to obtain such information without conducting a costly and time-consuming survey. The information obtained allows local government to estimate the quality-type of workers (as reflected by their educational attainment) required by particular types of occupations demanded by private businesses in the county, and to assess the training needs of its workforce. A case study of Taylor County, West Virginia, was conducted using 1990 census data and data from the Current Population Survey and Occupational Employment Statistics Survey for 1997. Four different methods were shown to be feasible for generating the pattern of occupational distribution of employer demand for workers with various education levels. Two main assumptions of the Goetz-Debertin method were evaluated: a fixed proportion of types of education required per individual occupation, regardless of sector, and a fixed proportion of types of occupation required per individual sector. (Contains 17 data tables and figures.) (SV)

A study examined challenges associated with investigating the role and occupational identity of learning practitioners who operate within the organizational as opposed to the educational institution context. These challenges include the following: (1) naming this group of organizational practitioners; (2) distribution of learning responsibilities; (3) diversity of theoretical bases for practice; (4) diverse models of organizational practice; (5) reported shifts in prerequisite competences of practice; and (6) new understanding of learning. The study shows that a new type of learning practitioner is emerging in organizational context. Research challenges arising at least in part from the shifting nature of this comparatively unregulated field of practice exist and need to be addressed. Such research could be seen as contributing a better understanding of the learning that is occurring in organizations and in institutions contributing to work force capability building as a basis for the ongoing development of professionals who work in this area. (Contains 33 references.) (KC)

A study examined the profiles of vocational education and training (VET) clients in disadvantaged areas of Sydney and Melbourne, Australia, building on earlier analysis of VET participation in those cities. The study examined the characteristics of VET clients coming from postal codes identified as disadvantaged to determine, if possible, to what extent high levels of participation are accompanied by comparable achievement, judged by appropriate measures such as level of course and module outcomes. Information was gathered from earlier studies and by mapping VET client data for postal codes in Sydney and Melbourne. The study confirmed earlier findings that the most disadvantaged areas of both cities have the highest numbers of TAFE (Technical and Further Education) participants. The relationship between disadvantage and participation is apparent mainly at the extremes of the most disadvantaged and most advantaged postal codes. A different map could be created by mapping another criterion of disadvantage, such as economic resource, because some lower income people have higher education levels. The study supports the contention that VET policy should give greater attention to addressing equity issues in disadvantaged urban regions because this is where large numbers of people do participate in VET. (Contains 16 references, 6 tables, and 5 figures.) (KC)

Indexes of socioeconomic status (SES) are widely used in school and higher education because of the known relationship of low educational participation and achievement by socioeconomically disadvantaged groups. Because an index of SES can predict which areas and social groups need to be "targeted" to improve educational participation, this methodology assumes that disadvantage is concentrated in particular localities. Some analysis of "area" disadvantage is an essential step in determining what policies can be effective in countering the effects of socioeconomic disadvantage on post-school participation, especially vocational and technical (VET) education. However, SES is one among several useful dimensions, including the characteristics of "equity groups" and the particular strategies that are known to be effective in addressing VET needs. An approach can be taken where SEIFA indexes (Socio-Economic Indexes for Areas) are used to identify those localities in Australia where large areas of disadvantaged people live. The rationale for using such indexes springs not only from their widespread use in public policy but from the evident need to address the conceptual weakness of equity policy in the VET system. The SEIFA indexes predict areas with low achievement in education generally, but these areas are those with higher levels of VET participation and achievement. Economic factors may be more useful than educational and occupational measures to identify these potential VET clients. Further research will summarize an analysis of the application of SEIFA indexes to VET participation in disadvantaged urban regions. (Contains 15 references and 2 tables.) (KC)

The relationship between life history, gender, and the discourse of modernization was examined from the perspective of a researcher with extensive experience performing evaluations about modernization within human services in Denmark. Three stories about site-based management in two human service institutionsa youth center and a boarding school for autistic childrenwere presented to illuminate fragments of the complex condition that modernization represents. One story, which was told by a female superintendent, reflects feelings of skepticism and professional pride and discusses modernization in terms of development. A second story, which was told by a male superintendent, stresses the challenges and criticism encountered during modernization. In analyzing the stories, the author explored the following themes: (1) the discourse of modernization; (2) pre-existing or interpretive narratives; (3) the notion of the dialectical, liberated, and dispersed subject and the various theoretical forms in which life histories emerge; (4) the concepts of subject and surface and their relationship to the discourse of life history and modernization; (5) fluent and fixed icons of life history; (6) lost virginity and floating space as images of modernization; and (7) rituals of passage, storytelling, imagination, and the researcher's voice. (Fourteen endnotes and 27 references are included.) (MN)

The Schools and Staffing Survey (SASS) has made it possible to develop comprehensive profiles of the nation's public and private schools and teachers and principals and to study issues of teacher supply and demand. The planned switch to a 5-year cycle of data collection provides a good time to re-examine the survey's design. This paper focuses on an aspect of design, survey depth. It addresses questions of the level at which estimates should be provided, from whom data should be collected, and how much data can reasonably be collected from each respondent. The paper starts with a description of the context within which choices can be made about survey depth. It examines the purposes of the SASS, identifies emerging policy issues, and describes users and uses of the data. This exploration of survey depth leads to a conclusion that, given the current focus on measuring quality at the classroom level, the SASS needs to reach more extensively into the classroom. This means gathering more information on teacher training and professional development. State- and affiliation-level estimates should be continued, as should the current oversampling of schools with large American Indian/Alaskan Native population, but no additional types of respondents should be added. The increased response burden will fall mainly on teachers, and modifications to keep the burden on teachers realistic may be required. (SLD)

This paper addresses issues related to the possible inclusion of items in the Schools and Staffing Survey (SASS) to measure instructional practices and teaching effectiveness. The SASS is a national survey of schools and teachers conducted by the National Center for Education Statistics. This report explores the purposes that can be served by measuring instructional practices on a national scale, and then examines how teaching effectiveness is conceptualized. Another section describes some strengths and limitations of observational studies of classroom instruction and makes a similar assessment of survey studies. Some attention is given to curricular reforms and how their impact might be assessed. The last sections of the paper suggest specific ways in which instructional practices and curricular content might be measured through the SASS, including the selection of specific school subjects and grade levels for attention. A number of large survey studies have already provided valuable information about curriculum and instruction in U.S. classrooms, and the SASS could join them. There does not appear to be any federal program in which instructional practices and opportunities to learn will be monitored in the future and the SASS, with its large sample of teachers, seems to be an excellent vehicle for the measurement of both curriculum and instructional practices. The SASS also seems ideally suited to monitor the classroom consequences of reforms such as curriculum standards. (Contains 50 references.) (SLD)