Search

The purpose of the study was to report normative AEL CSIQ ("AEL Continuous School Improvement Questionnaire") data for the total of 132 schools who had completed it by 2002. The normative data were developed and reported by type (level) of school, locale type (Johnson) codes, and schools nominated to be high performing learning communities. This latter group of schools requires description. Within the total of 132 schools, there was a special subgroup of schools who were nominated by either AEL or Tennessee Department of Education staff as being high-performing schools and professional learning communities. These special schools, which are referred to as the "Known" schools, were viewed as possessing positive characteristics relative to continuous learning and improvement by both the students in the school and by the adult professional staff. These special schools were nominated to possess the characteristics of high-performing learning communities--there was no guarantee that they were, in fact, high-performing learning communities. Indeed, one of the chief purposes of the norming study for the AEL CSIQ was to study the normative data from the set of "Known" schools (Meehan, Wiersma, Cowley, Craig, Orletsky, & Childers, 2002, p. 7). This report includes the following sections: Introduction; Methods; Findings and Discussion; and Conclusions and Recommendations.

This paper provides a summary of a comprehensive evaluation of the Kentucky statewide Extended School Services (ESS) program. The Extended School Services (ESS) program was established in 1990 as part of the Kentucky Education Reform Act (KERA). Designed specifically to address the needs of Kentucky's at-risk student population, ESS is an aggressive, proactive program for addressing academic problems before they become ingrained (Nesselrodt & Schaffer, 2000b). The ESS program extends the school day, week, or year for students at risk of academic failure, providing them with additional instructional time to help them meet academic goals. Rather than being an "add-on" or "stand-alone" program, ESS is designed to be an integral part of each school's regular academic program, thus ensuring that students receive instructional assistance in core content subjects in which they are performing poorly.

A cross validation study was conducted using data from white, rural seventh graders to investigate the stability of the construct validity of a model of eight conditions that are purported to support adolescent academic aspirations. A factor analytic study examined GEAR-UP survey data from 4,365 students across two consecutive years. In contrast to the model proposed by the University of Maine, the results of this study supported a four-factor model that persisted across years and gender. These factors are Self- Efficacy, Teacher-Centric, Leadership, and Like School. While the factors were consistent, the weightings did vary by gender and by year with girls rating the Teacher- Centric factor as most important and boys rating Self-Efficacy first across time periods.

A review of the critical literature provides a brief history of systematic observation of classroom behaviors, long valued as an important data collection method in educational research. Milestones in systematic observation of classrooms are traced back to 1914 and the development and use of the Special Strategies Observation System (SSOS) through a series of four phases is described. AEL revisions to SSOS Demographic Information, the SSOS Eight-Minute Observation Page, QAIT Assessment Instrument and the Classroom Environment and Resources Checklist are described. Extensive appendices document the revisions and include instruments, checklists, and forms.

The purpose of this report is to provide a formative evaluation of the McKelvey Foundation Program to Distribute Scholarships to Entrepreneurial Rural Students in the States of Pennsylvania, New York, and West Virginia. This evaluation provides databased recommendations for making program adjustments to improve delivery of services and/or other suggestions for improving program efficiency and effectiveness. (Author)

The Extended School Services (ESS) program was established in 1990 as part of the Kentucky Education Reform Act. The program extends the school day, week, or year for students at risk of academic failure, providing them with additional instructional time to help them meet academic goals. An evaluation of ESS in 2001 utilized statewide surveys and site visits to 18 elementary, middle, and high schools. Findings indicated that ESS participation was fairly equal across schools, but varied at individual schools. Fewer females participated at the middle and secondary levels. Program participants were characterized by lower socioeconomic status. Generally, students were referred to ESS because of poor academic performance. Students' goals were heavily influenced by their teachers, yet they adopted the goals as their own. ESS classrooms engaged in student-led activities. Students received individualized instruction as needed and had the opportunity to have concepts not mastered retaught to them. All stakeholders agreed that the program helped students increase their academic achievement. Parents and students reported improved study skills and increased motivation to learn. Major program strengths included early targeting of students, dedicated staff, student transportation, collaboration between teachers and coordinators, flexible scheduling, low teacher-student ratios, and individualized instruction. Improvements could be made in areas of student transportation, staff development, expanded services in terms of hours and/or subjects, and a reduction of the student-teacher ratio, all of which require increased funding. Twelve recommendations are discussed. (TD)

This study examined differences at the classroom level between Kentucky schools with minimum versus large gaps in academic achievement between particular groups of students. Data were gathered via observations of 213 classrooms at 18 elementary, middle, and high schools. Although all the schools were identified as high-performing in terms of overall academic index scores, nine had minimum achievement gaps and nine had large gaps between socioeconomic or racial groups. Classroom time was used more efficiently and effectively in minimum-gap schools. Teachers in large-gap schools spent more time on administrative routines in their classrooms. Teachers in minimum-gap schools provided more minutes of instruction, more time for student-led activities, and a more appropriate pace. Textbooks and multi-racial materials were used more in minimum-gap schools. The climate in minimum-gap school classrooms was more cheerful, inviting, open, and risk-free, and had less distracting external noises and interruptions. Teachers in minimum-gap schools communicated high expectations to their students, conducted formal or informal assessments of their students, and also provided immediate and corrective feedback to students. The quality of instruction was higher in minimum-gap school classrooms, both overall and for many subscale items. Rural and small-town schools comprised seven of the nine minimum-gap schools and two of the nine large-gap schools. (Contains 29 references) (TD)

A study examined rural students' academic aspirations and expectations, their perceptions of their parents' aspirations and expectations, and their parents' actual aspirations and expectations. Surveys were completed by 3,733 seventh grade-students in 54 schools in 17 West Virginia counties. Parent response rates were lower, with approximately two thirds of mothers and slightly less than half of fathers responding. Forty four of the schools were rural. Results indicated that rural parents expected and were expected by their children to play a large role in providing information about continued education. Parents placed a much higher value on the role of the guidance counselor than did students. Female students aspired to a bachelor's degree, and males aspired to an associate's degree. Students felt that their parents' goals exceeded their own and that both parents held these higher aspirations. Parents did report higher aspirations for their children than their children held, but the gap was greater than the students perceived. Both parents and students reported similar perceptions of students' academic effort, with at least half reporting effort equivalent to peers and a fourth professing greater effort. Parents reported little contact with schools about courses and grades needed to get into college. Parents and students had reasonably accurate expectations concerning college costs, but low-income parents were doubtful they could afford it. There was a disconnect between academic expectations and educational aspirations that could lead to disappointment and the failure to achieve those aspirations. (TD)

Targets of educational reform in the United States have included classroom teachers, students, and school structure. Increasingly, nonstructural aspects of schooling are being examined as avenues to educational improvement. The purpose of this study was to investigate differences among professional staff members' perceptions of being in a continuous mode of learning and improvement in low- and high-performing schools and in high-performing learning communities. Data were collected and analyzed from questionnaires sent to the faculty members of 45 low-performing schools in Alabama and 48 high-performing schools in Kentucky. Results show that in low-performing schools across both states, professional staff members identify the area of school/family/community connections as being most in need of learning and improvement, and that high-performing schools are not necessarily high-performing learning communities. Because staff members in high-performing schools always scored higher in questionnaire scaled responses than staff members in low-performing schools, it is concluded that measuring a faculty's commitment to continuous learning and improvement is one effective way to assess the reculturing of a school's professional staff. Recommendations include repeating this study with schools in other states, and doing longitudinal studies, among other designs. (RT)

The Extended School Services (ESS) program was established in 1990 in Kentucky. The program extends the school day, week, or year for students at risk of academic failure. All Kentucky school districts receive funding for ESS implementation. This paper reports on a comprehensive evaluation of the ESS program during the 2001-02 school year. Evaluation instruments consisted of statewide surveys of district and school ESS coordinators, and onsite visits to a sample of 18 schools with ESS programs. The surveys focused on community, school, and ESS descriptions; selection and retention; consolidated plan; children's perceptions; relationship to regular school programs; role of parents; and staffing and evaluation. Based on the findings from the surveys, the ESS program was viewed positively by district and school coordinators in terms of helping to address the needs of students at risk academically. Also, the coordinators agreed that ESS funds should remain a separate categorical fund. Further, the coordinators stated that staffing issues, professional development, and transportation were the most needed improvements. Additional findings are included in the report. Appended are eight graphs to supplement the findings from the surveys. (WFA)

The Extended School Services (ESS) program was established in 1990 as part of the Kentucky Education Reform Act. Designed specifically to address the needs of Kentuckys at-risk student population, ESS is an aggressive, proactive program for addressing academic problems before they become ingrained. This report summarizes findings from 24 site visits to selected schools across Kentucky that were implementing a variety of ESS programs. A total of 32 conclusions were generated from the evaluation, divided into nine topical areas: student demographics, adherence to intended goals, classroom instruction, student outcomes, program strengths, barriers to maximum success, program fidelity, patterns of implementation, and overall. Based on the findings and conclusions of this comprehensive evaluation of the statewide Kentucky Extended School Services Program, a number of specific recommendations are offered for staffs review and reflection. These 12 recommendations focus on transportation, professional development, communication and goal setting, and staffing. Nine appendixes contain evaluation checklists, surveys, questionnaires, and forms. (Contains 27 references and 17 tables.) (GCP)

A faculty's commitment to continuous learning and improvement is a critical dimension in defining schools as high-performing learning communities. When planning an improvement effort, a school's staff needs a conceptual framework that outlines the dimensions of school improvement. The AEL Continuous School Improvement Questionnaire (CSIQ) is a self-report inventory that asks professional staff to rate their school on six major dimensions: learning culture; connections among school, family, and community; shared leadership; shared goals for learning; purposeful student assessment; and effective teaching. Extensive pilot and field testing have shown the CSIQ to be a reliable and valid measure of faculty commitment to continuous learning and improvement. This paper reports normative CSIQ data from 3,821 staff in 132 schools in Kentucky, Tennessee, Virginia, and West Virginia that took part in field testing. Within the sample was a subgroup of 11 schools identified as continuously improving or high-performing schools. These schools were labeled "Known" schools. Schools were categorized by grade level and by Johnson Code indicating rural/urban locale. CSIQ scores were highly reliable. School's grade-level had a slight to modest effect, with elementary schools and those containing elementary grades having higher total and subscale scores. CSIQ scores had no relationship to extent of rurality. Educators in "Known" schools almost always scored higher than their counterparts in schools of the same type. (Contains 15 statistical data tables.) (SV)

As part of its school improvement effort, AEL, a regional education laboratory, developed the Continuous School Improvement Questionnaire (AEL CSIQ). Staff from the AEL Quest schools program drafted a 65-item questionnaire to help measure and assess the efforts of the project team in their work with the 18 schools in the Quest network. These items embodied the six major concepts of the Quest school improvement program. After review and refinement, 42-item and 117-item versions of the scale were developed and combined for pilot testing. Pilot and field tests provided much information about the AEL CSIQ and its measurement. With data from more than 3,800 educators in 132 schools, the statistics generated have high stability and precision. The testing of the AEL CSIQ in the field fostered its development into an excellent and usable 6-item inventory. The CSIQ was reduced from 147 items with unequal numbers of items in subscales to 10 items in each subscale. The AEL CSIQ was found to be very sound technically. Internal consistency reliability was sound for all subscales, as was stability reliability. The instrument showed satisfactory concurrent validity with another school climate instrument. The items had face validity based on content, and the factor analysis provided strong support for the construct validity of the entire inventory as reflected by the subscales. (Contains 12 tables, 1 figure, and 5 references.) (SLD)

West Virginia received two grants from the U.S. Department of Education to encourage disadvantaged youth to have high expectations, stay in school, and take academically rigorous courses to prepare for college. A total of 17 counties were served by these grants. As part of the program, surveys were completed by 3,716 students in 54 middle and junior high schools to gather baseline data on seventh-grade students' and parents' aspirations for students' postsecondary education. Of the students surveyed, 1,218 were classified as nonrural and 2,498 as rural. Findings indicate that families of nonrural students had a greater propensity toward and history of postsecondary education. Nonrural students participated more in clubs, while rural students participated more in sports. Nonrural students seemed more confident of their academic abilities than rural students. Nonrural students more often indicated that a parent helped them with homework, while rural students more often indicated that a classmate or friend helped them. Rural students were more likely to get information related to postsecondary education from a principal or guidance counselor and to discuss college entrance requirements, indicating that rural students were making plans for postsecondary education. However, students often failed to follow through on these ambitions. (Contains 16 references, 10 tables, and 2 figures.) (TD)

Collaboration between and among educational organizations is much discussed and often required by funding agencies, but measuring such collaboration is discussed much less. Collaboration has been characterized as a continuum of interdependence between partners that ranges from cooperation to coordination to collaboration. Seven features have been identified to determine where an interorganizational relationship would be plotted on the continuum. This study focused on the twin factors of the extent of independence-interdependence of the collaboration of clients with an educational R & D organization and the level of satisfaction with the collaboration. Data were collected annually for 4 years via a mailed survey to about 500 clients of the R & D organization. Satisfaction with the collaboration was collected from about 230 clients for 2 of those years. The internal factors of collaboration, such as leadership role and adequate support from the organizations involved, have greater effect than more external factors, such as the professional role of the educators or the years on the job. The important implications of this study are found more in the measurement related to collaboration than in factors that affect those measures. (Contains 10 references and 6 tables.) (Author/RT)

Nonprofit organizations seeking to improve the education system have found it necessary and desirable to form collaborative partnerships with other organizations, such as community and/or faith-based groups, and school districts. Such partnerships can reduce the costs of research and service delivery, limit the likelihood of service duplication, and provide much-needed access to additional resources and staff who possess expertise not found in any one organization. From FY 1996 through FY 1999, AEL, Inc., a nonprofit research and development organization, surveyed a number of clients to examine their perceptions of collaboration on various projects and the perceived impact of the products and services resulting from those collaborations. Survey instruments included the Collaboration Continuum Scale, the AEL Impact Questionnaire, and the Satisfaction with Collaboration Scale. Results were analyzed statistically. Results show that satisfaction was high across and within factors that affect collaboration. Level of collaboration was perceived to be slightly more interdependent than independent as measured using Intriligator's continuum framework. Finally, as clients became more satisfied with collaboration and saw collaboration as increasingly interdependent, they also perceived more impact of AEL products and services. (Contains 12 references and 12 tables.) (RT)

In 1999, Fairmont State College (West Virginia) received a Gaining Early Awareness and Readiness for Undergraduate Programs (GEAR UP) grant for a 5-year program designed to encourage college aspirations and preparation among disadvantaged rural youth. During the first 2 years, parents and students in nine counties were surveyed to determine their aspirations for postsecondary education. Data summaries from these surveys were distributed to school administrators involved with the program. This paper reports on a survey of those administrators to determine how the data summaries were used, possible effects of the data, and suggestions for improvement. Findings indicate that those administrators who used the summaries were satisfied with them. Of the seven rating characteristics,"overall quality,""comprehensiveness," and "presentation/layout" were rated highest, followed by "relevance,""met my needs," and "usability.""Promptness" was rated lowest, although that rating was well above the midpoint on the rating scale. Summary findings were used as input for specific programs, needs assessment or program planning, and less specific programs. It is too early for the data summaries to have affected students in the program. The main influence of the summaries on parents was in making them more aware of the GEAR UP program. An appendix presents a sample data summary. (Contains 16 references.) (TD)

This study was conducted to explore the characteristics of and interrelationships between teacher efficacy and professional learning community in 19 schools participating in the Quest project at AEL, Inc. The Quest project is an applied regional research project that assists schools with educational reform efforts. The School Professional Staff as Learning Community instrument developed by S. Hoard (1997) and an AEL-developed Teaching Questionnaire containing 2 scales (perceptions of external and internal efficacy) based on work by T. Guskey were administered to staff in the 19 schools, with 624 responses received. Based on the Cronbach alpha reliability estimates, it was concluded that all three scales had satisfactory reliability. Elementary school teachers had a higher level of internal efficacy and a higher sense of a professional learning community at their schools than did the high school teachers. High school teachers had a higher level of perceived external efficacy regarding learning than did the elementary school teachers. The results support the 1994 finding of T. Guskey and P. Passaro that the external and internal scales measure two separate constructs of teacher efficacy that have an inverse relationship. Internal and external measures of teacher efficacy were not significantly related to perceptions of the school as a learning community. Teachers' years of experience had no bearing on their perceptions of internal or external efficacy or their perceptions of the school as a learning community. Implications for the design or implementation of Quest programs are discussed. (Contains 4 figures, 6 tables, and 12 references.) (SLD)

Parent and student surveys were conducted in nine primarily rural West Virginia counties to gather baseline information on incoming seventh-grade students' and parents' awareness of and aspirations for postsecondary education. The surveys are administered each year as part of Fairmont State College's GEAR UP (Gaining Early Awareness and Readiness for Undergraduate Programs) grant. This paper summarizes findings from first-year surveys, February 2000. Surveys were completed by over 2,200 students at 29 middle and junior high schools and by 1,840 mothers and 1,464 fathers. Findings indicate that parents frequently helped their children with homework but had limited school involvement. Parents were generally satisfied with their child's education. Overall, students seemed fairly confident of their academic ability and saw themselves as good, hardworking students. Students were willing to admit that they needed academic help with some classes. Most students recognized the benefits of postsecondary education. Nearly three-fourths believed that further education is necessary to get a satisfying job, that they would attend college, and that their parents wanted them to go to college. However, students had limited awareness of postsecondary institutions. Students regarded parents as the most important source of educational information, but parents actually had limited information. Parents and students were familiar with financial aid sources but were disinclined to believe that students would receive scholarships. Parents seemed to have a realistic picture of college expenses. County-specific findings and recommendations are listed. (SV)

The issue of treatment fidelity has been a concern in the field of evaluation research. Nearly three decades ago, Cook and Campbell (1975) outlined four types of validity that may influence treatment outcomes. Defining these validity measures has since prompted researchers to closely examine potential threats within the context of program evaluation. Related to Cook and Campbell's concept of construct validity, Sechrest and associates (1979) identified two complications that could further impact evaluation research. These complications involve the strength and integrity of treatment, and their associated implications on construct validity. The purpose of this study was to assess the delivery, receipt, and adherence of the CRI treatment implemented in the four Kanawha County schools. In creating a model for assessing treatment implementation, Lichstein, Riedel, and Grieve (1994) posit that there are three components in their model. Delivery assessment inspects how the treatment was delivered and the factors that enhanced or hindered delivery of the treatment. Receipt assessment examines whether the intended audience was able to understand the information that was provided to them. Adherence assessment examines whether the clients? receipt of the treatment will effect them; enactment looks at whether clients adhere to the directives and change behaviors. This study was designed to provide a viable test of the evaluation of an educational intervention employing a set of multiple components and multiple methods. These various components and methods for assessing the integrity of the intervention were suggested by Lichstein, Riedel, and Grove.

The Interdisciplinary Teamed Instruction (ITI) project investigated the effects of interdisciplinary, teamed instruction on secondary school teaching and learning. It examined the effectiveness of a professional development model that facilitated the development, implementation, and evaluation of ITI. Through summer institutes and onsite workshops, project staff provided training, resources, and ongoing technical assistance to teams of teachers and administrators. Teams designed and tested integrated curricula that incorporated learning activities and alternative assessments; conducted collaborative action research to determine the effectiveness of units; and redesigned units using feedback from key stakeholders. Three summer institutes were held to help teacher-administrator teams build capacity for collaboration and interdisciplinary teamed instruction. One institute helped teams design integrated units, plan learning activities and assessments, and present their units. Researchers evaluated the institute's processes, outcomes, and products via questionnaires, observations, interviews, reflective assignments, feedback forms, oral evaluation feedback, and ratings of participants' presentations. Overall, the summer institute affected teacher self-efficacy, decision making, impact, and autonomy in scheduling. Its greatest impact was on collaboration, cooperative learning, assessment rubrics, multiple intelligences, and team building. Participants were very positive about the institute, which strengthened their belief in and commitment to ITI concepts and practices. (Contains 14 references.) (SM)

Since 1991-92, student writing portfolios have been part of Kentucky's high-stakes performance-based assessment. A study of schools with continuously improving or continuously declining writing portfolio scores led to identification of 36 indicators of successful writing programs. These indicators were used to develop a comprehensive needs assessment instrument, to be used in a school self-evaluation involving all stakeholders. This paper describes the pilot test and field test of the instrument, the School Study of Writing Instruction. The pilot test was carried out in a small K-6 school in western Kentucky and was deemed successful due to faculty participation and sense of ownership and the steering committee's customizing of the instrument. The results generated 18 recommendations and 22 proposed revisions. The field test was conducted in 11 primarily rural and small-town schools. The revised instrument involved a process in four steps: interviewing administrators, students, and teachers using questions related to the 36 quality indicators; writing a report profiling the school; rating the school on the indicators; and setting priorities for improvement. Five schools were assigned an external facilitator. Overall, conclusions about the self-study process were positive. Most schools completed the study to a satisfactory point and believed the effort to be worthwhile. The time required was a concern, but facilitator assistance was highly beneficial. The instrument was generally valid and reliable, but may be improved with various adjustments. Appendices contain statistical data tables and a time and task summary. (Contains 30 references.) (SV)

The Even Start Family Literacy Program is a national program designed to break the cycle of poverty and illiteracy by working with low-income families to improve adult literacy, parenting skills, and developmental and preschool readiness of young children in the family. The Monongalia County (West Virginia) Even Start Program is unusual in that home visits are the primary service delivery mechanism. A program evaluation during the 1997-98 program year determined the outcomes of the adult and parenting education components and evaluated the program's parenting education literacy materials. Thirteen adults were tested for adult education outcomes, and 32 were tested for parenting education outcomes. Results indicate that the program had some impact on parents' reading comprehension, moderate impact on parents' number operations, very large impact on parents' problem solving, and large impact on parents' total mathematics. The program had more impact in the mathematics area than in reading comprehension. The program had no impact on the adults' parenting efficacy, little impact on parental responsibility, some impact on parents' internal locus of control, and some impact on parents' self-esteem. The parenting education materials developed by program staff were found to be interesting, understandable, informative, and utilitarian, but poor in terms of attractiveness. Recommendations included implementing changes to increase reading comprehension, determining the reason for the low parenting efficacy outcomes, and improving the appearance of the parenting education materials. (Contains 22 references.) (TD)

The Monongalia County (West Virginia) Even Start Program is a comprehensive, rural, home-based family literacy program that provides families with adult literacy education, parenting education, and early childhood education. Although all Even Start programs conduct some home visits, the Monongalia County program is unique in that home visits are the primary service delivery mechanism. Following an overview of the national Even Start program and the Monongalia County program, this report describes an evaluation of program impact on the vocabulary development of child participants. The sample consisted of 32 children with an average age of 4 years, 4 months, and included 5 children for whom English was a second language (ESL) and 10 children with an active individualized education plan (IEP). Even Start staff administered the Peabody Picture Vocabulary Test-III (PPVT-III) to the children on a pretest-posttest basis during the 1998-99 program year. Most children had 7-8 months of program participation. From pretest to posttest, PPVT-III scores increased significantly for the full group and the subgroups. The effect size of the point gain was "moderate" for the full group, "large" for the ESL group, and "very large" for the IEP group. The PPVT-III average age-equivalent score increased 13 months during an average treatment period of 7 months. Therefore, the children in the sample nearly doubled the vocabulary growth expected of them. (Contains 20 references.) (SV)

The Even Start Family Literacy Program is a national program that addresses the literacy and educational needs of adults in eligible families, teaches effective parenting skills, and addresses the preschool readiness needs of young children in the family. The Monongalia County (West Virginia) Even Start Program was unusual in that it took place in a rural area and used home visits as the primary service delivery mechanism. An independent, third-party evaluation of the Monongalia County program collected data from program records, interviews, observations, and seven participant outcome measures over a 2-year period. An innovation configuration matrix (ICM) was constructed and used to describe and evaluate eight program components: identification and recruitment, home visits, early childhood education, parenting education, adult education and literacy, collaboration, evaluation, and staff development. Findings indicate that the ICM is an accurate and useful portrayal of the program, and that the program had a large impact on adults' reading and mathematics literacy and parenting outcomes; tremendous impact on children's school readiness; and some impact on children's auditory comprehension, expressive communication, and total language development. Recommendations are offered concerning program continuation, program improvements, evaluation utilization, and new evaluations. (Contains nine references, seven data tables, and the ICM.) (SV)

The Interdisciplinary Teamed Instruction (ITI) Project was a 2-year project aimed at determining the effects of ITI on teaching and learning and at validating the effectiveness of a professional development model to facilitate development, implementation, and evaluation of ITI. Through summer institutes and onsite workshops, project staff provided training, resources, and ongoing technical assistance to teacher/administrator teams from four rural Virginia secondary schools. Teams designed and tested integrated curricula that incorporated learning activities and alternative assessment forms; conducted collaborative action research to determine the effectiveness of units; and redesigned units using feedback from students, teachers, parents, and supervisors. Researchers evaluated the 1996 ITI Summer Institute. Evaluation included questionnaires on teacher empowerment, schools as learning communities, school effectiveness, and teachers' sense of control; observations; participant interviews, reflection, and feedback; oral evaluation feedback; and rating of participants' videotaped presentations. Results indicated that the Institute met its intended outcomes. The greatest impact on participants related to collaboration, cooperative learning, assessment rubrics, multiple intelligences, and team building. Participants were very positive in evaluating the Institute, which strengthened their belief in and commitment to ITI concepts and practices. The quantitative instrumentation utilized may not have fully captured and documented Institute effects. Appendices contain a meeting agenda; three questionnaires on teachers, school as community, school products and services; a control questionnaire; a feedback form; interview questions; a team reflection log; presentation scoring rubric; and an evaluation standards checklist. (Contains 20 references.) (SM)

The Monongalia County (West Virgina) Even Start Program begun in 1993 to address family literacy and to prepare children for entrance into school, consists of three interrelated approaches: adult education, parenting education, and early childhood education. In June 1996, the program was serving 62 families (mostly low-income) with an average of 2.2 children. The unique feature of the Monongalia program is that it is mainly a rural, home-based program--home visits are the primary service delivery mechanism. This detailed evaluation of the program covers the years 1994-96. The introduction includes the history of the national Even Start program, including its emphasis on families, services to eligible families, and the first and second national evaluations; site specifics, approach, and interagency collaboration of the Monongalia program; objectives of the evaluation; and audience. The section on methodology addresses national evaluation population characteristics, the Monongalia sample study, data collection techniques (adult and child), and the data analysis process. Program component findings cover program model, recruitment, home visits, early childhood education, adult education, parenting education, interagency collaboration, evaluation, and staff development. Among the participants' educational and developmental outcomes, the program was found to have a tremendous impact on school readiness; large impact on adult reading literacy, mathematics literacy, and parenting; and some impact on children's auditory comprehension, expressive communication, and total language development. Recommendations relate to program continuation, improvements, and evaluations. Contains 20 references. Appendices include 1995-1996 program statistics, the home visit report sheet, the innovation configuration matrix (program evaluation model), a staff self-assessment report; and the completed program evaluation standards checklist. (SAS)

A new instrument was designed by S. Hord (1996) to assess globally the maturity of a school's professional staff as a learning community. The Appalachia Educational Laboratory agreed to field test this instrument to determine its reliability and validity and to draw conclusions about its use in educational improvement. The instrument consisted of 17 descriptors of a professional learning community grouped into 5 major dimensions or areas. Faculties of 21 schools (6 elementary, 6 middle and junior high schools, and 9 high schools) participated, for a total of 690 teachers. Some analyses used smaller samples. All five internal consistency reliabilities for the dimension items were in the mid 0.80s, and concurrent validities for the 114 high school teachers scores was 0.75. Factor analysis revealed that a unitary factor of all 17 items accounted for 54 percent of the variance. Results show that the instrument did differentiate the faculties in terms of their maturity as learning communities, and that these differences were evident across elementary, middle/junior high, and high school levels. The instrument appears useful as a screening, filtering, or measuring device to assess the maturity of a school's professional staff. (Contains 16 tables and 22 references.) (SLD)

An electronic survey of a listserv discussion group, the Interdisciplinary Teamed Instruction (ITI) group, was conducted to learn more about the group and to explore surveying electronically. A 10-question survey was posted electronically, as were respondents' replies. Three reminders were posted over the 3-week reply period. The number of subscribers was estimated at 250 in the United States and 6 foreign countries. Twenty-three completed surveys were returned in the first 24 hours, and only 26 more were completed in the next 3 weeks, for a final response rate of 23.6%. Results suggest that the listserv is an active community of learners. They also demonstrate that defining the real population becomes a technological problem. If one were to hypothesize that an active electronic community would respond better than other groups to a survey, these results would not support the hypothesis, although respondents to the initial effort provided very rapid responses. The major implications of the research approach are: (1) to calculate an accurate response rate, it is crucial to determine the exact number of listserv subscribers before first posting the survey; and (2) more efforts should be put into an electronic survey before its first posting, such as announcements. The survey text is attached. (Contains 11 references.) (Author/SLD)

This report is a formative evaluation of the use of an online listserv discussion group as part of a continuing project with schools exploring or implementing interdisciplinary teamed instruction (ITI). A survey was developed to identify member characteristics, what services were liked and used most, and what services were desired. Analysis of 59 completed surveys (out of 250 listserv subscribers) indicated respondents included educators in 29 states and 3 foreign countries with the majority (56 percent) being teachers. On a general level, the results supported the inclusion of cyberspace in the concept of professional community. Most respondents were local education agency staff who reported they were satisfied with the listserv services and used it frequently. Respondents also indicated strong interest in online education in ITI and the practical aspects of curriculum integration. The survey is appended. (JLS)

As one of the federally-funded regional educational laboratories, the Appalachia Educational Laboratory (AEL) is committed to assessing the impact of its products and services on a variety of client subgroups. These include educators and administrators at the elementary, secondary, and higher levels. To address the issue of measuring impact, the AEL identified seven impact categories: (1) overall satisfaction; (2) knowledge and skills; (3) information sharing; (4) use in planning; (5) use in implementation; (6) indirect benefits; and (7) secondary clients' use in implementation. Over a 3-year period, the AEL measured impact with a client questionnaire consisting, in the final version, of 5-item subscales that were administered to 672 individuals in primary client subgroups. Scores from this normative group were converted to percentiles, and mean scores were plotted as profiles for subgroups. Results then reflected specific professional roles and the impacts of the AEL on each. An appendix presents the AEL questionnaire development checklist. (Contains 3 tables, 10 figures, and 12 references.) (SLD)

In 1990, the Youth Risk Behavior Survey (YRBS) was completed by a random sample of 1,448 West Virginia public-school students in grades 9-12. The sample was 51 percent male and 89 percent white. About 71 percent of subjects were aged 15-17; 39 percent were in the ninth grade. The YRBS covered behaviors producing vehicle-related or other injuries, drug and alcohol use, sexual behaviors, tobacco use, dietary behaviors, and physical activity. A 6-point scale to measure level of rurality was developed, based on the number of enrolled students per school attendance area in square miles. Half of the subjects attended extremely rural or very rural schools. One-way analysis of variance resulted in significantly higher risk scores on the entire YRBS for males compared to females, for 12th graders compared to lower grades, and for subjects aged 16-18 compared to younger subjects. Both when subjects were clustered as three risk groups (high, average, and low) and when they were clustered as two groups (high and average), discriminant function analysis indicated that rurality was the most consistent discriminator of the groups, with higher risk being associated with less rural classification. Gender was the next most potent discriminator of risk groups, but results were affected by a few extreme scores. (SV)

The study attempted a comprehensive evaluation of the process and product of a school-based job placement model operating in three Pennsylvania area vocational schools (AVTS). Random samples of AVTS and sending school seniors were post-tested on employability skills. Students, parents, school personnel, employers, Bureau of Employment Security (BES) personnel, and community contacts were surveyed. Study objectives were to assess job placement specialists' performance in: (1) preparing students with job searching skills and knowledge; (2) job matching (finding jobs satisfying students' interests and training); and (3) establishing and maintaining cooperative working relations with BES. The study concluded that a formalized placement service can increase the probability of a student obtaining employment over those schools who do not have such a service. Further, students utilize such a service if it is available, and, along with their parents, employers, and others, believe that the school should provide such services. Finally, better cooperation between the school and BES can lead to effective job placement. Findings are supported with statistical data throughout. Selected recommendations addressing the pilot job placement projects deal with the continuation and improvement, where appropriate, of the projects' successful practices. Project-related materials and survey instruments are appended. (BP)

The study investigated the interaction of personal and environmental factors influencing seventh graders' selection of career exploration courses for increased exploration in the eighth grade. A stratified random sample (496 students) was drawn from the Pittsburgh Public School's Occupational, Vocational, and Technical Exploratory Program. Personal and environmental factors included: (1) achievement subscores for industrial arts/home economics/business education; (2) factors of age, race, breadwinner's socioeconomic status, and home school; (3) vocational attitude maturity; (4) motivation; (5) occupational values of interest and satisfaction, salary, prestige, and security; (6) attitude scores for industrial arts/home economics/business education; and (7) student self-reports on teacher/program/peer influence on course choices. The dependent variables were 12 career exploration courses in rank order of popularity. Computations included Pearson product moment correlation, multiple regression analysis, and Behrens-Fisher t-tests. Some conclusions were that exploratory business education courses can successfully be implemented at the middle school and that it is possible to design learning activities which appeal to a wide range of student backgrounds. Sex, race, and occupational value-security were the most frequently appearing and stable influencing variables. Career exploration course selection by preadolescents is only moderately explainable by personal/environmental analysis. (Author/EA)

To determine whether the content of the Appalachia Educational Laboratory's (AEL) Regional Exchange, which is published as a regional insert to a quarterly magazine entitled the Educational R&D Report, is geared to its target audiences, a content analysis was conducted on five of the AEL inserts. The research questions formulated for the content analysis were: (1) What categories of and how many educators receive the AEL Regional Exchange? (2) What percent of the articles are geared to those categories of educators? (3) What proportion of the insert's total column lines is targeted to those categories of educators? and (4) What is the average number of column lines dedicated to each category? The mailing list of the Educational R&D Report was analyzed to determine the educator categories used in the content analysis. A content analysis form was then designed and the articles in each insert coded for the educator categories at which they were targeted. Data were tallied and analyzed. The results are discussed in detail and presented in four tables corresponding to the four questions addressed by the study. A set of recommendations for changes in the AEL Regional Exchange and a copy of the content analysis form are included. (JL)

The Orthopedic Homebound Program was meant to prevent the institutionalization of orthopedically impaired, mentally retarded children and to involve parents in their training. The program was proposed and managed by the Shawnee Hills Community Mental Health/Mental Retardation Center in rural West Virginia. Each child's developmental needs were assessed by an interdisciplinary team of psychologists, social workers and therapists to determine an individualized plan of behavioral objectives to be carried out by home trainers and parents. Progress was measured in part by the TARC Assessment Inventory for Severely Handicapped Children. Interviews of parents assessed their interaction and expectations for progress, as well as their evaluation of the program. The parent/home environment was evaluated by home workers. Major program support components included: (1) quarterly inhouse reports, (2) a monthly newsletter and (3) a handbook for parents, (4) regional and national conferences, and (5) a concluding seminar for parents and program personnel to discuss process and product effectiveness. Data tables, references and 13 appendices of program material, publication samples and evaluation instruments are provided. (CM)

This report documents the implementation and evaluation of a one-week competency-based workshop on designing learning systems for coordinators and curriculum development specialists of the West Virginia Department of Education, Division of Instructional Learning Systems. The main objectives of the workshop were: (1) the development of an innovative learning system model; (2) the training of staff to become state-wide change agents toward model implementation; and (3) the subsequent performance of instructional behaviors demonstrated by the workshop learders. The material is presented in the following format: (1) introduction; (2) planning phase; (3) participant characteristics; (4) workshop evaluation; (5) conclusions; (6) recommendations; (7) references; (8) appendices (workshop instructional materials, data collection instruments). Recommendations drawn from the experience were that: (1) the elements of the systems model developed in the workshop should be used by the curriculum leaders as a diffusion strategy; (2) follow-up data on participants' on-the-job behaviors regarding model implementation should be gathered; (3) the use of the competency-based workshops should be considered seriously by other state department of education bureaus and divisions adopting innovations for their staff; and (4) the use of full- or small-group followup sessions for staff members requesting or requiring aid in the implementation of the learning systems model should be considered by the Director of the Division of Instructional Learning Systems. (MJB)

An evaluation is presented of the Stallings Classroom Management Staff Development Demonstration Project, designed to increase student achievement in basic skills through the use of research-based, systematic change in teachers' classroom management and organizational techniques. One objective of the evaluation was to utilize techniques and to administer a set of instruments to assess their utility for future evaluations. Process evaluation methods focused on teachers' feelings about the demonstration project, their concerns about the innovations, and evaluative comments made by involved teachers at the project's conclusion. Product evaluation focused on the pre-/post-test changes in teachers' perceived responsibility for student achievement, ratings on the teachers' levels of use of the innovation, and the teachers' pre-/post-test intervention changes in "correct implementation" of specific classroom teaching behaviors. Following an executive summary of the project, chapter 2 of this report outlines the purpose and objectives of the staff development program. Chapter 3 describes the evaluation design, the data collection instruments, and the data collection and analysis procedures of the evaluation. Process and product evaluation results are presented in the fourth and fifth chapters. Data displays and narrative copy are used to clarify evaluation results. The sixth chapter presents conclusions and recommendations based on the evaluation study. (JD)

Besides spotlighting education and influencing agencies' intervention in the educational processes, the recent reform movement has focused attention on all educational actors and called for increased accountability for teachers and administrative staff, including principals. While researchers disagree on the degree of instructional leadership required of principals, even the harshest critics of school effectiveness research conclude that staff development is a vital dimension of effective schools. This study focuses on staff development aspects of building principals' instructional leadership and addresses (1) whether training session content could cut across all school levels; (2) whether this content could be taught to principals; (3) whether principals would use the content in training sessions; and (4) how well they could teach it. (The expectation was that all of this was possible.) In fact, all the principals in two school districts in two different school years were taught to be the change facilitators of classroom organization and management content, using materials developed by the University of Texas (Austin). When tested, principals showed significant gains in their knowledge of classroom management principles and proficiency in facilitating use of new materials. Teachers also showed significant gains in their classroom management knowledge. Included are numerous figures and tables, a summary, 18 references, and 2 appendices describing workshop training activities and the project's phases. (MLH)

Data gathered from a review of the literature, state department of education personnel, and directories of school districts are used to report on small school districts in Kentucky, Tennessee, Virginia and West Virginia. Enrollment size of school districts, expenditures per pupil, transportation costs per pupil, and student density were computed for the study area. Only student density proved a useful index to rurality. The review of the literature notes that educational research has shown an urban bias and that problems and strengths of rural schools has been researched outside of the educational research community. Other discussion notes that further research should seek to develop data on family income and building-level enrollment. Statistical information is presented in 47 tables and 93 references are appended to the review of the literature. (SKW)