20,404 results on '"Field Tests"'
Search Results
2. School-Based Agricultural Education Teachers' Experiences during a Year-Long Field Test of the CASE Mechanical Systems in Agriculture (MSA) Curriculum
- Author
-
Wells, Trent, Hainline, Mark S., Smalley, Scott W., and Chumbley, Steven
- Abstract
Science, technology, engineering, and mathematics (STEM) applications have become increasingly more commonplace in school-based agricultural education (SBAE) settings in the past few decades. In recent years, Curriculum for Agricultural Science Education (CASE) programming has provided a practical outlet for STEM-focused, inquiry-based teaching and learning activities. The CASE Mechanical Systems in Agriculture (MSA) course was recently field-tested nationally with several SBAE teachers. Framed within Rogers' (2003) diffusion of innovations theory, we sought to study six teachers' experiences when implementing the CASE MSA curriculum throughout the 2018-2019 academic year. Using qualitative research methods, we conducted multiple one-on-one interviews with each SBAE teacher at different parts of their respective academic years. Data were coded in accordance with Merriam's (2009) recommendations. Four dominant themes emerged: (1) the journey toward innovation; (2) learning as you go; (3) logistical and implementation challenges; and (4) students' needs and preferences. Several prominent sub-themes emerged as well. Our findings highlight that although challenges existed, the CASE MSA curriculum was suitable for enhancing the rigor and relevance of these teachers' agricultural mechanics curricula. We recommend CASE MSA curriculum stakeholders collaborate to continuously improve its design and flexibility.
- Published
- 2021
3. Development of a Tool to Assess Inference-Making and Reasoning in Biology
- Author
-
Cromley, Jennifer G., Dai, Ting, Fechter, Tia, Nelson, Frank E., Van Boekel, Martin, and Du, Yang
- Abstract
Making inferences and reasoning with new scientific information is critical for successful performance in biology coursework. Thus, identifying students who are weak in these skills could allow the early provision of additional support and course placement recommendations to help students develop their reasoning abilities, leading to better performance and less attrition within biology courses. Researchers across universities partnered to develop a measure to assess students' inference-making abilities in biology. We describe the development of the inference-making and reasoning in biology assessment (IMRB). The IMRB is a 15-item multiple-choice assessment that uses short paragraphs of content--from the most-used textbook--taught at the end of a semester of survey biology courses designed for science majors. Based on our research, when the IMRB is conducted at the beginning of a semester, it measures deductive reasoning with new biology information, is fair across various student groups, and is reliable. The IMRB can be used with or without SAT or ACT scores to place students into regular undergraduate introductory biology courses, to predict grades in such courses, and/or to identify students who may need extra support or remediation in reasoning with new biology information. The IMRB is available free of charge to interested faculty and researchers.
- Published
- 2021
- Full Text
- View/download PDF
4. The Student-Centered Assessment Network: Testing Change Ideas in Real Time
- Author
-
Nellie Mae Education Foundation and Bayerl, Katie
- Abstract
This report summarizes the practices and findings of the pilot year of Student-Centered Assessment Network (SCAN), which used real-time testing to determine effective and engaging classroom assessment practices for students. Data gathered through quality assessment can be used by teachers to support their students' needs, as well as by students themselves to take ownership of their learning. This report covers some of the major shifts in classroom practice teachers saw as a result of participation in SCAN, as well as some of the challenges in shifting towards more student-centered assessment practices. The end of the report delves into some of the adjustments suggested to refine future versions of the network.
- Published
- 2020
5. Scale Evaluation and Eligibility Determination of a Field-Test Version of the Assessment, Evaluation, and Programming System--Third Edition
- Author
-
Toland, Michael D., Grisham, Jennifer, Waddell, Misti, Crawford, Rebecca, and Dueber, David M.
- Abstract
Rasch and classification analyses on a field-test version of the third edition of the Assessment, Evaluation, and Programming System (AEPS-3), a curriculum-based assessment used to assess young children birth to age 6 years, were conducted. First, an evaluation of the psychometric properties of data from each developmental area of an AEPS-3 field-test version was conducted. Next, cutoff scores at 6-month age intervals were created and then the validity of the cutoff scores was evaluated. Results using Rasch modeling indicated acceptable model fit statistics with reasonable reliability estimates within each developmental area. Classification results showed cutoff scores accurately classified a high percentage of eligible children. Findings suggest that scores from a field-test version of the AEPS-3 are reliable within developmental areas. To the extent allowed by state criteria, early childhood interventionists could possibly use a new field-test version of the AEPS-3 to determine or corroborate eligibility for special education services.
- Published
- 2022
- Full Text
- View/download PDF
6. Cultural Adaptations for Disaster Response for Children in Puerto Rico after Hurricane María
- Author
-
Baggerly, Jennifer, Ceballos, Peggy, Rodríguez, Mónica, and Reyes, Ana G.
- Abstract
Adaptations for specific cultural populations are needed to respond to children affected by natural disasters. After Hurricane María, we provided disaster response training to mental health professionals and Disaster Response Play Therapy services to children in Puerto Rico. In this article, we describe culturally specific, field-tested preparation and procedures to promote Puerto Rican children's mental health after disasters. We recommend cultural adaptations for training mental health professionals and providing children's disaster response in Puerto Rico.
- Published
- 2022
- Full Text
- View/download PDF
7. Defining Preliminary Research for Digital Game-Based Learning Evaluation: Best Practices
- Author
-
Pauline-Graf, Denise and Mandel, Susan Elaine
- Abstract
The surge of learners being immersed in computer game contexts for learning has instigated dialogue about the contextually appropriate collection of reliable and valid data to inform education-based decisions. The purpose of this article is to develop educational practitioners' understanding of preliminary research work, and to inform educational researchers about design and reporting of preliminary research work, in the context of reported preliminary studies on Digital Game-Based Learning Interventions (DGBLIs). First a checklist of processes for the reporting of preliminary studies is provided. Second, a summary is offered of the characteristics of each type of preliminary study including the description, objectives, and methodology. Third, an example from peer-reviewed literature is identified of each type of preliminary study relevant to DGBLIs and conducted within the past five years. Evident from the examples selected, educational researchers and practitioners are best advised to recognize the characteristics of preliminary studies -- pilot work, feasibility study, pilot study, pilot trial, and field test -- to better inform DGBLIs before embarking on a full-scale study, and to meet the need of educational practitioners for concrete evidence about DGBLIs.
- Published
- 2019
8. Elementary Mathematics Student Assessment: Measuring the Performance of Grade 3, 4, and 5 Students in Number (Whole Numbers and Fractions), Operations, and Algebraic Thinking in Fall 2015. Research Report No. 2018-24
- Author
-
Florida State University, Learning Systems Institute (LSI), Schoen, Robert C., Anderson, Daniel, Riddell, Claire M., and Bauduin, Charity
- Abstract
This report provides a description of the development process, field testing, and psychometric properties of the fall 2015 grades 3-5 Elementary Mathematics Student Assessment (EMSA), a student mathematics test designed to be administered in a whole-group setting to students in grades 3, 4, and 5. The test was administered to 2,614 participating grade 3, 4, and 5 students in 266 classrooms located in 10 public school districts in Florida during fall 2015. Focused on number (including whole number and fractions), operations, and algebraic thinking, the student assessment was designed to serve as a baseline measure of student achievement for use as a covariate in a randomized controlled trial evaluating the impact of a teacher professional development program on student learning. [This report was prepared by the Florida Center for Research in Science, Technology, Engineering, and Mathematics (FCR-STEM) within the Learning Systems Institute.]
- Published
- 2018
9. Geogebra Applets Design and Development for Junior High School Students to Learn Quadrilateral Mathematics Concepts
- Author
-
Nisiyatussani, Ayuningtyas, Vidya, Fathurrohman, Maman, and Anriani, Nurul
- Abstract
This design and development research was motivated by the rapid expansion and use of GeoGebra by mathematics educators (teachers and lecturers) in Indonesia. One of GeoGebra features is GeoGebra Applet that can be used, modified, and/or developed by educators for dynamic and interactive mathematics teaching and learning. At the time of research project, there is no GeoGebra Applets closely linked and aligned to the Indonesia national curriculum. The availability would be benefit for mathematics teaching and learning aligned to this curriculum. This research proceeds through seven steps of the Need, Capability, and Analysis (NCA) Model of Design and Development: 1) User Need Analysis; 2) Researcher as Developer Capability; 3) GeoGebra Applets Design; 4) GeoGebra Applets Development; 5) Experts Judgements; 6) Field testing in its Natural Setting Environment; and 7) the Prototype. The field testing was conducted with 8th grade students in a junior high school. The field testing shows that the developed Quadrilateral GeoGebra Applets can work as expected in its purposed natural setting environment.
- Published
- 2018
10. Elementary Mathematics Student Assessment: Measuring the Performance of Grade K, 1, and 2 Students in Counting, Word Problems, and Computation in Fall 2015. Research Report No. 2017-20
- Author
-
Florida State University, Learning Systems Institute (LSI), Schoen, Robert C., Anderson, Daniel, Champagne, Zachary, and Bauduin, Charity
- Abstract
This report provides a description of the development process, field testing, and psychometric properties of a student mathematics test designed to assess grades K, 1, and 2 student abilities. The test was administered to 4,486 participating grade K, 1, and 2 students in 67 schools located in 10 public school districts in Florida during fall 2015. Focused on counting, word problems, and computation, the student assessment was designed to serve as a baseline measure of student achievement for use as a covariate in a randomized controlled trial evaluating the impact of a teacher professional development program called Cognitively Guided Instruction (CGI) on student learning.
- Published
- 2017
11. Elementary Mathematics Student Assessment: Measuring the Performance of Grade K, 1, and 2 Students in Number, Operations, and Equality in Spring 2016. Research Report No. 2017-22
- Author
-
Florida State University, Learning Systems Institute (LSI), Schoen, Robert C., Anderson, Daniel, and Bauduin, Charity
- Abstract
This report provides a description of the development process, field testing, and psychometric properties of a student mathematics test designed to assess grades K, 1, and 2 student abilities. The test was administered to 4,535 participating grade K, 1, and 2 students in 66 schools located in 9 public school districts in Florida during spring 2016. Focused on number, operations, and equality, the student assessment was designed to serve as a baseline measure of student achievement in a randomized controlled trial evaluating the impact of a teacher professional development program called Cognitively Guided Instruction (CGI) on student learning.
- Published
- 2017
12. Developing of Environmental Education Textbook Based on Local Potencies
- Author
-
Ilma, Silfia and Wijarini, Fitri
- Abstract
Environmental education subject aims to form students who have the character to maintain the environment. One effort to achieve the objectives of the Environmental education subject is the local Environmental Education Textbook Based on Local Potencies. This research was aimed to produce textbook of environment-based education subject Environmental education based on local potencies. This research and development using Borg & Gall model (1983). This study uses only five stages of the Borg & Gall model, namely (1) need analysis, (2) planning, (3) develop preliminary form of product, (4) preliminary field testing, and (5) main product revision, due to time constraints and cost. The results of the research and development that has been done is a textbook of environmental education based on local potencies that is suitable for use in semester 5 college level.
- Published
- 2017
13. Operational Study 4: Accessibility of New Items/Functionality. Component 3 Report
- Author
-
Partnership for Assessment of Readiness for College and Careers (PARCC), Steedle, Jeffrey, and LaSalle, Amy
- Abstract
Partnership for Assessment of Readiness for College and Careers (PARCC) Operational Study 4 Component 3 was designed to compare performance on PARCC mathematics field-test items for grade 3 taken with and without a drawing tool. For the 2016 testing window, five field-test items were selected to have the directions edited to allow students to provide a drawing as part of responses and directions on how to include a drawing. For those students, using the drawing tool was optional, and a drawing was not required for earning full credit. The standard variant of these items did not include the directions or the tool option. Both variants were included in the embedded online field-test forms to determine if there was a significant difference in performance. If no significant differences were found, then there would be evidence to allow the inclusion of the drawing response interaction on existing items. All mathematics items available to be embedded into the Spring 2016 forms were reviewed and approved by PARCC. No new items were created for this study, only variant items with modified direction lines. This report summarizes the quantitative and qualitative evidence bearing on the comparability of the study items administered with and without a drawing tool.
- Published
- 2016
14. The Benefits of Fixed Item Parameter Calibration for Parameter Accuracy in Small Sample Situations in Large-Scale Assessments
- Author
-
König, Christoph, Khorramdel, Lale, Yamamoto, Kentaro, and Frey, Andreas
- Abstract
Large-scale assessments such as the Programme for International Student Assessment (PISA) have field trials where new survey features are tested for utility in the main survey. Because of resource constraints, there is a trade-off between how much of the sample can be used to test new survey features and how much can be used for the initial item response theory (IRT) scaling. Utilizing real assessment data of the PISA 2015 Science assessment, this article demonstrates that using fixed item parameter calibration (FIPC) in the field trial yields stable item parameter estimates in the initial IRT scaling for samples as small as n = 250 per country. Moreover, the results indicate that for the recovery of the county-specific latent trait distributions, the estimates of the trend items (i.e., the information introduced into the calibration) are crucial. Thus, concerning the country-level sample size of n = 1,950 currently used in the PISA field trial, FIPC is useful for increasing the number of survey features that can be examined during the field trial without the need to increase the total sample size. This enables international large-scale assessments such as PISA to keep up with state-of-the-art developments regarding assessment frameworks, psychometric models, and delivery platform capabilities.
- Published
- 2021
- Full Text
- View/download PDF
15. Redesigning Design: Field Testing a Revised Design Rubric Based on iNACOL Quality Course Standards
- Author
-
Adelstein, David and Barbour, Michael K.
- Abstract
Designers have a limited selection of K-12 online course creation standards to choose from that are not blocked behind proprietary or pay walls. For numerous institutions and states, the use of the iNACOL "National Standards for Quality Online Courses" is becoming a widely used resource. This article presents the final phase in a three-part study to test the validity and reliability of the iNACOL standards specifically to online course design. Phase three was a field test of the revised rubric based on the iNACOL standards against current K-12 online courses. While the results show a strong exact match percentage, there is more work to be done with the revised rubric. [Note: The volume number (32) displayed on the PDF is incorrect. The correct volume number is v31.]
- Published
- 2016
16. Development of Accessible Laboratory Experiments for Students with Visual Impairments
- Author
-
Kroes, KC, Lefler, Daniel, Schmitt, Aaron, and Supalo, Cary A.
- Abstract
The hands-on laboratory experiments are frequently what spark students' interest in science. Students who are blind or have low vision (BLV) typically do not get the same experience while participating in hands-on activities due to accessibility. Over the course of approximately nine months, common chemistry laboratory experiments were adapted and field tested for use in a residential school for the blind. These adaptations most commonly used a SciVoice Talking LabQuest and associated sensors, as well as other tactile methods.
- Published
- 2016
17. Inquiry-Oriented Learning Material to Increased General Physics Competence Achievement
- Author
-
Sinuraya, Jurubahasa
- Abstract
This study aims to produce inquiry-oriented general physics learning material to improve student learning outcome. Development steps of learning materials were adapted from the design model of Dick and Carey. Stages of development consists of three phases: planning, development, and formative evaluation and revision. Implementation of formative evaluation involves three expert reviewers, they are subject matter experts, general physics lecturer, and media experts; one-to-one evaluation involved three students, evaluation of small groups involve 15 students, and a field test (effectiveness test) involve 30 students. The structure of the contents of the learning material refers to the concept of scientific investigations which generally consists of: stating the problem in a question form, formulating the hypothesis, literature, testing the hypothesis, analyzing the result and writing the conclusion. The result obtained from subject experts, general physics lecturer and media expert are in a good categories with their respective ideals for each are 80.2%, 82.7% and 84.5%, an increase competency achievement of pretest and postest on field tests in the medium category
, and the results of student responses to the instructional materials belonging to the positive response with a score of 73.5%. These results illustrate that the inquiry-oriented general physics learning material has a good criteria and suitable used as learning materials. - Published
- 2016
18. Culturally Responsive Assessment of Physical Science Skills and Abilities: Development, Field Testing, Implementation, and Results
- Author
-
Alfaiz, Fahad S., Pease, Randy, and Maker, C. June
- Abstract
During the Cultivating Diverse Talent in STEM (CDTIS) Project, a team of scientists, teachers, and a researcher developed a performance-based assessment of high school students' creative problem-solving skills and ability to apply physical science principles in practical ways. It was one of six measures to identify exceptionally talented students. Students identified using conventional methods (M1), with an average grade point average (GPA) of 3.93, had an average rating of 2.95 on a 5-point scale on the mechanical-technical assessment. The M2 students, who were from schools with high percentages of Hispanic, American Indian, and low socioeconomic status (SES) students, had an average GPA of 3.07 and an average rating of 3.27, demonstrating that this assessment can be an important way to change the cultural and economic balance of students identified as exceptionally talented in Science, Technology, Engineering, and Mathematics (STEM). Other researchers are encouraged to examine the validity of the mechanical-technical assessment to identify exceptionally talented students in different groups.
- Published
- 2020
- Full Text
- View/download PDF
19. Smarter Balanced 'Tests of the Test' Successful: Field Test Provides Clear Path Forward
- Author
-
Smarter Balanced Assessment Consortium and Doorey, Nancy
- Abstract
Between March and June of 2014, the Smarter Balanced Assessment Consortium conducted a field test of its new online assessment system. Thirteen participating states provided the results of surveys given to students and adults involved in the Field Test. Overall, more than 70% of test coordinators in each of seven states indicated that the Field Test had gone either as well as or better than expected, and most students (an average of 67% across 5 states) found the test interface "easy" or "very easy" to use. Several important lessons emerged to inform final preparations this fall and winter. In addition, the surveys indicate that states need to continue their work to help teachers align instruction with the increased rigor that college and career readiness standards require.
- Published
- 2014
20. TAP High School Symposium: Lessons Learned from Principals and Teachers
- Author
-
National Institute for Excellence in Teaching (NIET) and Barnett, Joshua H.
- Abstract
Since the 1999-2000 school year, TAP: The System for Teacher and Student Advancement (TAP) has been implemented in hundreds of schools across the nation and demonstrated an ability to raise student achievement, improve the quality of instruction and increase the ability of high-need schools to recruit, retain and support effective teachers. The TAP system has been implemented in schools across nearly 20 states in urban, suburban, rural, and tribal districts, as well as at the elementary, middle, junior, and high school levels. Throughout TAP's fifteen year history, much information has been learned with regard to the most effective and efficient method to ensure high fidelity implementation. As the National Institute for Excellence in Teaching (NIET) continues to expand their partnerships with schools, this document specifically examines their lessons learned for implementation procedures at the high school level. In the fall 2013, NIET held a High School Summit meeting, including principals and teachers from across the nation among our partnering schools, who have experience implementing the TAP system in high school environments. At this Summit, a series of conversations and breakout sessions were used to identify common issues and the effective solutions for implementing TAP in a high school. Among the topics discussed, the participants organized their conversations around three implementation themes and offered eleven solutions. Each of these topics is detailed throughout this document. The first theme that emerged from the discussion centered on cluster groups, where the Summit attendees identified four common challenges and offered a wealth of key solutions to address those issues. Additionally, the Summit attendees discussed the role of field testing at the high school level, noting the challenges of identifying whether the same strategy should be used across content areas or if different strategies should be used; determining which benchmarks are selected; differentiating across grade levels and various content levels within content areas (i.e. algebra/ geometry); and using student work to monitor student achievement gains. The attendees also identified and discussed multiple solutions to each of these challenges. Finally, the group discussed the role of Follow-Up at the high school level to address the challenges regarding scheduling; pedagogy and content; and perceptions. The group also identified and discussed multiple solutions to each of these challenges. This report details the findings from this High School Summit, including the most common issues and the multiple solutions for each issue. Following the themes discussed by the participants, the report is separated into key sections and explores why these issues are different at the high school level and captures the experiences of the experts involved in the Summit. [Contributors to this report included: Gary E. Stark, Jason Culbertson, Keith Wilson, Jennifer Oliver, Monique Wild, and Jessica Alexander.]
- Published
- 2014
21. Considerations for Consortia as States Transition Away from AA-MAS. NCEO Brief. Number 7
- Author
-
National Center on Educational Outcomes
- Abstract
States with an alternate assessment based on modified achievement standards (AA-MAS) that received a flexibility waiver from some of the requirements of No Child Left Behind are required to phase out their use of this assessment. And, on August 23, 2013, the U.S. Department of Education published a proposed rollback of regulation that allowed the AA-MAS. Not all states developed an AA-MAS. Sixteen states offered this optional assessment between 2011 and 2013. Many, but not all of these states, also belonged to one of the two Race to the Top Assessment (RTTA) consortia that are developing new general assessments--the Partnership for Assessment of Readiness for College and Careers (PARCC) and the Smarter Balanced Assessment Consortium (Smarter Balanced). This Brief addresses the need for consortia to assist their member states in moving away from the AA-MAS. It focuses on key considerations for consortia as they address the inclusion of low-performing students in all member states, informed by the lessons learned from the AA-MAS. Topics highlighted in this brief are: including low-performing students in field test designs, customizing non-summative assessments to include low-performing students, and collecting and examining data on this group of students. [Contributors to the writing of this Brief were Laurene Christensen, Sheryl Lazarus, Vitaliy Shyyan, and Martha Thurlow.]
- Published
- 2014
22. Developing Mathematics Problems Based on PISA Level of Change and Relationships Content
- Author
-
Ahyan, Shahibul, Zulkardi, and Darmawijoyo
- Abstract
This research aims to produce mathematics problems based on PISA level with valid and practical content of change and relationships and has potential effect for Junior High School students. A development research method developed by Akker, Gravemeijer, McKenney and Nieveen is used this research. This development research consists of three stages; analysis, design, and evaluation. In the first stage, the researcher analyzed students, algebra material in school-based curricula (KTSP) and mathematics problems of PISA 2003 of change and relationships content. The second stage, the researcher designed 13 problems with content of change and relationships. The last, the researcher used formative evaluation design developed by Tessmer which includes self evaluation, one-to-one, expert review, small group, and field test. The data collect by walk through (step of expert review) to get valid problems in aspect of content, construct, and language; interview (steps of one-to-one, small group, and field test) to know problems practicality; questionnaire (step of field test) to know problems' potential effects. The result of this research indicated that 12 mathematical problems based on PISA level of change and relationships content that developed have validity, practically, and potential effects for Junior High School students.
- Published
- 2014
23. Accountability for Student Learning: Slow and Steady Progress or Persistent Resistance?
- Author
-
Morse, Andrew Q.
- Abstract
The purpose of this study was to explore the present status of efforts to assess student-learning outcomes within the bachelor's degree granting institutions of the campuses in one system of public higher education. Further, the purpose of this study was also to understand what challenges and criticisms academic leaders report about the call to provide learning outcome evidence. Themes and findings of the study suggest that accountability for student learning is a persistent accountability challenge within higher education institutions.
- Published
- 2014
24. Embedded Field Test Item Statistics: Can They Be Trusted for Estimating Student Proficiency?
- Author
-
Steedle, Jeffrey T. and Morrison, Kristin M.
- Abstract
Assessment items are commonly field tested prior to operational use to observe statistical item properties such as difficulty. Item parameter estimates from field testing may be used to assign scores via pre-equating or computer adaptive designs. This study examined differences between item difficulty estimates based on field test and operational data and the relationship of such differences to item position changes and student proficiency estimates. Item position effects were observed for 20 assessments, with items in later positions tending to be more difficult. Moreover, field test estimates of item difficulty were biased slightly upward, which may indicate examinee knowledge of which items were being field tested. Nevertheless, errors in field test item difficulty estimates had negligible impacts on student proficiency estimates for most assessments. Caution is still warranted when using field test statistics for scoring, and testing programs should conduct investigations to determine whether the effects on scoring are inconsequential.
- Published
- 2019
- Full Text
- View/download PDF
25. Developing and Evaluating an Eighth Grade Curriculum Unit That Links Foundational Chemistry to Biological Growth: Changing the Research-Based Curriculum
- Author
-
Kruse, Rebecca, Howes, Elaine V., and Carlson, Janet
- Abstract
Much of modern biology has become increasingly chemical in character. Not surprisingly, students often have trouble understanding key ideas in biology because they lack foundational chemistry ideas. AAAS and BSCS are collaborating to develop and study a curriculum unit that supports students' ability to explain a variety of biological processes involving growth in chemical terms. The unit provides conceptual coherence between chemical processes in nonliving and living systems through the core idea of atom rearrangement and conservation during chemical reactions, which is critical for understanding how growth occurs while conserving matter. An initial draft of the unit was pilot tested at two schools in 2011. The results of the pilot test were used to revise the unit. In the spring of 2012, the revised unit and teacher materials was field tested. In this paper we will describe the iterative development process and the research that supports it. We will describe the Year 2 curriculum and, specifically, highlight how the curriculum enacts its four key design principles. Selected findings will be discussed that informed subsequent revisions during the final year of the project. (Contains 8 tables.)
- Published
- 2013
26. Developing and Evaluating an Eighth Grade Curriculum Unit That Links Foundational Chemistry to Biological Growth: Designing Professional Development to Support Teaching
- Author
-
Kruse, Rebecca, Howes, Elaine V., and Carlson, Janet
- Abstract
AAAS and BSCS are collaborating to develop and study a curriculum unit that supports students' ability to explain a variety of biological processes such as growth in chemical terms. The unit provides conceptual coherence between chemical processes in nonliving and living systems through the core idea of atom rearrangement and conservation during chemical reactions, which is critical for understanding how growth occurs while conserving matter. Abundant evidence in the literature suggests that many middle school teachers lack adequate science preparation, but even teachers with excellent science content knowledge often lack pedagogical skills necessary to effectively use research-based curricular materials. Thus, we consider teacher support materials and professional development (PD) to be a critical component of this intervention. We will describe the design of teacher support materials and professional development to support teachers' implementation of the curriculum through developing their knowledge and skills in the areas of Content Coherence and Pedagogical Support for Student Learning. In the context of face-to-face and online interactions with the curriculum unit, teachers learn about science content, student ideas, and strategies for teaching the content and curriculum effectively. This paper reports on the iterative development of the suite of teacher support materials and PD, highlights of key features, findings of our Year 2 field test, and implications of those findings for future revisions. (Contains 7 tables and 3 figures.)
- Published
- 2013
27. The Validity of the Major Field Test in Psychology as a Programme Assessment Tool
- Author
-
Gallagher, Shawn P. and Cook, Shaun P.
- Abstract
The Major Field Test in Psychology (MFT) is a standardised test designed to assess subject mastery at the conclusion of an undergraduate career. Eighty-one graduating majors completed the MFT and 56 of them also took a multiple-choice exam of questions drawn randomly from an introductory psychology test bank. Like the MFT, the constructed exam was divided into four subscales. A second sample of 29 novice majors also completed the MFT. For the advanced majors, total and subscale scores from the two tests were highly correlated (all p<0.01). Advanced majors scored above national norms and were significantly better than the novices. However, few courses could be linked to performance in the associated MFT subscales but general academic indices like grade point average were highly predictive. The novice MFT scores were similar to the norms for advanced majors. We conclude that the MFT measures broad, basic understanding, not advanced mastery.
- Published
- 2013
28. Improving the Quality of Career and Technical Alternative Teacher Preparation: An Induction Model of Professional Development and Support. Appendices
- Author
-
National Research Center for Career and Technical Education, Southern Regional Education Board, Bottoms, Gene, Egelson, Paula, Sass, Heather, and Uhn, John
- Abstract
This report presents the appendices to the report of the National Research Center for Career and Technical Education's (NRCCTE's) five-year collaboration with the Southern Regional Education Board (SREB) to develop an induction model for new career and technical education (CTE) teachers pursuing an alternative route to certification that increases their competence, self-efficacy, and retention. CTE teachers who enter the profession through alternative routes are more likely to feel confident about their knowledge of their career field but less confident about their ability to convey that knowledge to students. Many alternatively certified CTE teachers express concern regarding classroom management, student motivation, and planning instruction for special needs students. As this report outlines, research indicates that alternatively certified teachers need professional development in planning, instructional methods, assessment, and how to support struggling students. Alternatively certified CTE teachers also need feedback about their work, strategies for managing added demands on their time and energy, and resources for planning and teaching. In response to these needs, this field-tested induction model builds the capacity of beginning CTE teachers to offer instruction that is both intellectually demanding and standards-focused and thus more likely to improve CTE students' academic achievement. The model also builds CTE teachers' capacity to design instruction that is actively engaging using strategies like project-based learning and cooperative learning. Students who are actively engaged intellectually and emotionally in their high school courses are more likely to stay in school, acquire their high school diplomas in four years, and enter postsecondary institutions without the need for remediation. [For the full report, see ED574498. For the executive summary, see ED574499.]
- Published
- 2013
29. Improving the Quality of Career and Technical Alternative Teacher Preparation: An Induction Model of Professional Development and Support
- Author
-
National Research Center for Career and Technical Education, Southern Regional Education Board, Bottoms, Gene, Egelson, Paula, Sass, Heather, and Uhn, John
- Abstract
This report presents the results of the National Research Center for Career and Technical Education's (NRCCTE's) five-year collaboration with the Southern Regional Education Board (SREB) to develop an induction model for new career and technical education (CTE) teachers pursuing an alternative route to certification that increases their competence, self-efficacy, and retention. CTE teachers who enter the profession through alternative routes are more likely to feel confident about their knowledge of their career field but less confident about their ability to convey that knowledge to students. Many alternatively certified CTE teachers express concern regarding classroom management, student motivation, and planning instruction for special needs students. As this report outlines, research indicates that alternatively certified teachers need professional development in planning, instructional methods, assessment, and how to support struggling students. Alternatively certified CTE teachers also need feedback about their work, strategies for managing added demands on their time and energy, and resources for planning and teaching. In response to these needs, this field-tested induction model builds the capacity of beginning CTE teachers to offer instruction that is both intellectually demanding and standards-focused and thus more likely to improve CTE students' academic achievement. The model also builds CTE teachers' capacity to design instruction that is actively engaging using strategies like project-based learning and cooperative learning. Students who are actively engaged intellectually and emotionally in their high school courses are more likely to stay in school, acquire their high school diplomas in four years, and enter postsecondary institutions without the need for remediation. The appendices listed in the table of contents for this report are presented in a separate document. [For the executive summary, see ED574499. For the appendices, see ED574500.]
- Published
- 2013
30. Education Longitudinal Study of 2002 (ELS:2002/12) Third Follow-up Field Test Report. Working Paper Series. NCES 2012-03
- Author
-
National Center for Education Statistics (ED), Ingels, Steven J., Pratt, Daniel J., and Jewell, Donna M.
- Abstract
This report describes the methodologies and results of the third follow-up Education Longitudinal Study of 2002 (ELS:2002/12) field test which was conducted in the summer of 2011. The field test report is divided into six chapters: (1) Introduction; (2) Field Test Survey Design and Preparation; (3) Data Collection Procedures and Results; (4) Field Test Questionnaire Timing and Data Quality; (5) Survey Control Systems and Data Processing; and (6) Summary of Recommendations for the Full-scale Study. The following are appended: (1) Technical Review Panel Summary; (2) ELS:2002 Data Collection Materials; (3) Third Follow-up Field Test Questionnaire; (4) ELS:2002 Cognitive Testing Reports, 2010 and 2011; (5) ELS:2002 Scale Reliability Analyses; and (6) ELS:2002 Third Follow-up Field Test Codebooks.
- Published
- 2012
31. Analyzing Educational Testing Service Graduate Major Field Test Results
- Author
-
Thornton, Barry and Arbogast, Gordon
- Abstract
The Educational Testing Service (ETS) created the Graduate Major Field Test in Business (GMFT-B) for MBA students. This test is administered to all MBA classes at Jacksonville University for the purpose of measuring student academic achievement and growth, as well as to assess educational outcomes. The test is given in the capstone course, Business Strategy and Policy, typically taken by students at the end of their MBA program. It provides valuable feedback to each student as to their standing with regard to students across the nation. A total of six years of data (2006 to 2011) was collected for analysis of these test scores. For the first half of this time period, Jacksonville University was in candidacy for accreditation by the Association for the Advancement of Colleges and Schools of Business (AACSB), and for the time period 2009-2011 it was an AACSB accredited institution. Executive MBA, evening MBA (also known as flex MBA) and day-time (accelerated) MBA classes took the exam over that time period. It was determined that the executive MBA student grades were significantly better than their counterparts in the evening (flex) and day-time (accelerated) MBA students at Jacksonville University. It was also found that there was no significant difference in student performance during the first half (AACSB candidacy) and second half (AACSB accreditation) of the time period under investigation.
- Published
- 2012
32. The Inclusive Classroom Profile (ICP) Preliminary Findings of Demonstration Study in North Carolina
- Author
-
University of North Carolina at Chapel Hill, FPG Child Development Institute, Soukakou, E., Winton, P., and West, T.
- Abstract
The Inclusive Classroom Profile (ICP) was developed in response to a lack of validated instruments designed specifically to measure the quality of inclusive practices, and it is based on research evidence on the effectiveness of specialized instructional strategies for meeting the individual needs of children in inclusive settings (Odom, 2004; National Professional Development Center on Inclusion, 2011; Buysse and Hollingsworth, 2009). The ICP has been used in a pilot study in the United Kingdom with promising results for the measure's reliability and validity (Soukakou, in press). The purpose of this study was to conduct the first USA demonstration study leading to validation. The measure was field tested in 51 inclusive settings in North Carolina. The following are the preliminary results from this demonstration study: (1) The measure has acceptable inter-rater agreement, is internally consistent, and shows a good factor structure; (2) Correlations with another measure of global classroom quality (ECERS-R) provided initial evidence for construct validity; and (3) Assessors reported the measure to be feasible and easy to use. (Contains 1 footnote.) [Additional funding for this paper was provided by the NC Rated License Assessment Project, the Division of Child Development and Early Education; and the NC Department of Public Instruction, Exceptional Children.]
- Published
- 2012
33. Conducting Rigorous Research in Multilingual Context: A Randomized Field Trial of the Good Behavior Game
- Author
-
Society for Research on Educational Effectiveness (SREE), Kurki, Anja, Wang, Wei, Poduska, Jeanne, Gomez, Mary Jane, Li, Yibing, and Brown, C. Hendricks
- Abstract
This paper describes the design and preliminary results regarding the testing of the Good Behavior Game in schools that vary in their proportion of Latinos and the use of bilingual instruction. The authors discuss how working with multicultural schools/school district has influenced the design, intervention and training, data collection, and analytic approaches of a randomized field trial testing the impact of GBG under various models of professional development for teachers. They will also present preliminary impact results from the first cohort (from school year 2010-2011). (Contains 4 tables.) [This work is jointly supported by IES grant R305A090446 (PI Poduska), NIDA grant R01 DA030452 (PI Poduska), NIMH grant R01MH040859 (PI Brown), and NIDA grant P30DA027828 (PI Brown).]
- Published
- 2012
34. Progress Report Year 4: National Center for Research on Evaluation, Standards, and Student Testing (CRESST). The Development and Impact of POWERSOURCE[C]. CRESST Report 795
- Author
-
National Center for Research on Evaluation, Standards, and Student Testing and Baker, Eva L.
- Abstract
The POWERSOURCE[C] intervention is intended as a generalizable and powerful formative assessment strategy that can be integrated with any on-going mathematics curriculum to improve teachers' knowledge and practice and, in turn, student learning. Combining theory and research in cognition, assessment and learning (for both adults and students) with design elements to support the transformation of practice within existing constraints, POWERSOURCE[C] includes both a system of learning-based assessments and an infrastructure to support teachers' use of those assessments to improve student learning. The current study focuses on middle school mathematics, starting in grade 6, and on helping to assure that students possess key understandings they need for success in Algebra I. Such a focus is motivated by ample research showing the frequency and price of failure for subsequent academic performance, including high school graduation, college entry and preparation (e.g., Brown & Niemi, 2007). The author and her colleagues' primary research objectives are based on their hypotheses that as a result of POWERSOURCE[C], teachers will become more proficient in their subject matter knowledge, more skilled in their formative use of assessment, and better focus their instruction on key ideas, and, as a result, will be more effective in helping students to improve their understanding, as shown by measures of student learning. Ultimately, they expect the improvements in student understanding to drive better performance on No Child Left Behind (NCLB) mandated state tests, transfer measures, and future coursework. This report summarizes some of the key trends found across this qualitative data set. The findings provided some useful information and insights as the implementation continues to move into other schools or districts. (Contains 7 figures and 9 tables.)
- Published
- 2011
35. Improving Secondary Career and Technical Education through Professional Development: Alternative Certification and Use of Technical Assessment Data
- Author
-
National Research Center for Career and Technical Education
- Abstract
Secondary career and technical education (CTE) is a field in transition. It is moving from a primary focus on preparing students for entry-level employment to preparing them for continuing education and training as well as employment. The rapid pace of change in technology and the global economy has created a demand for workers who are able to learn and adapt, and CTE must prepare its students to meet these demands. The National Research Center for Career and Technical Education (NRCCTE) is responding to these developments with a number of projects, some of which are being conducted by its own staff and others that are being directed by institutions that are partners in the NRCCTE consortium. Two of the projects are developing professional development models for improving the skills of secondary CTE teachers. The Southern Regional Education Board (SREB) is developing and testing an induction model for alternatively certified teachers; that is, those who have not completed a traditional teacher education program. NOCTI (formerly the National Occupational Competency Testing Institute) is applying its expertise to a professional development model designed to improve the ability of secondary-level CTE teachers and administrators to interpret data from technical skill assessments to improve instruction. An analysis of the field test data provided clear indicators as to changes needed in program materials to meet the needs of alternative route teachers. Many learning activities were revised to fit the audience in order to provide more time for reflection or to clarify content. Field test participants identified key elements of the modules that they felt would be necessary for new teachers prior to entering the classroom, including: (a) the use of rubrics, (b) formative and summative assessment, (c) how to use a table of specifications to align their instructional goals and assessments to technical standards and 21st century skills, (d) getting to know students, (e) engaging students in developing classroom rules and procedures, and (f) classroom management scenarios. Data suggested that three strategies used by program developers were particularly effective in supporting participant learning: (a) use of examples in participants' content areas, (b) use of "floating" one-on-one and small group coaching during cooperative learning segments, and (c) facilitated small group discussion in the afternoon or evening to structure reflection. Field testing on the model will continue through 2012, when the fully-developed model will be ready for rigorous experimental testing. Appendices include: (1) Concept Paper for Instructional Planning Module; (2) Concept Paper for Instructional Strategies Module; (3) Concept Paper for Classroom Assessment Module; (4) Concept Paper for Classroom Management Module; (5) Flowcharts of Module Units and Lessons; (6) Teacher Self-Efficacy Scale Results; (7) Focus Group and Instructor Debrief Protocols; (8) Results of Year 1 Five-State Survey, Summary of Survey Responses; and (9) Material Review Protocol. (Contains 16 tables, 9 figures and 5 footnotes.)
- Published
- 2011
36. Measuring Outcomes for Young Children and Their Families. Outcome Indicators for Everyday Kids, Everyday Lives: A Vision for Pennsylvania's Children
- Author
-
Orelena Hawks Puckett Institute, Pennsylvania Developmental Disabilities Council (PADDC), Trivette, Carol M., and Dunst, Carl J.
- Abstract
This monograph includes the final report for a project funded by the Pennsylvania Developmental Disabilities Council for "Measuring Outcomes for Children" (2008 RFP). The goal of the project was to "develop and demonstrate the effectiveness of an instrument designed to measure life outcomes of children with disabilities being serving by multiple child serving systems." The instrument and scales that were developed include indicators for measuring the life circumstances of young children and their families that were identified as valued outcomes in the "Monitoring Outcomes for Children project" which was the predecessor to the project described in this monograph. This monograph includes a description of the activities that were used to develop outcome indicators and the scales that were developed to measure valued outcomes identified by parents, practitioners, and others in the Monitoring Outcomes for Children project. The monograph also includes descriptions of lessons learned and recommendations for next steps for measuring and improving the life circumstances of young children and their families. The following attachments are included: (1) Everyday Kids, Everyday Lives Survey; (2) Everyday Kids, Everyday Lives Scale Survey; (3) Everyday Kids, Everyday Lives Field-Test Evaluation; (4) Parent and Child Everyday Experiences Scale; (5) Everyday Kids, Everyday Lives Scale; (6) Family and Child Early Experiences Scales; (7) Code Manual for the Measuring Outcomes for Children Scale Items; and (8) Excel Data Files for the Measuring Outcomes for Children Scale Items. The following are appended: (1) Everyday Kids, Everyday Lives Major Themes and Codes; (2) Everyday Kids, Everyday Lives Delphi Study Items and Corresponding Everyday Kids, Everyday Lives Themes and Codes; (3) Everyday Kids, Everyday Lives Modified Delphi Process Survey Results; (4) Everyday Kids, Everyday Lives Field-Test Social Validity Results; (5) Psychometric Properties of the Parent and Child Everyday Experiences Scale; (6) Confirmatory Factor Analyses of the Parent and Child Everyday Experiences Scale Items; and (7) Profiles of the Parents' and Children's Everyday Experiences According to Selected Parent, Child and Residence Variables.
- Published
- 2011
37. Multilevel Assessments of Science Standards
- Author
-
Society for Research on Educational Effectiveness (SREE), Quellmalz, Edys S., Timms, Michael J., and Silberglitt, Matt D.
- Abstract
The Multilevel Assessment of Science Standards (MASS) project is creating a new generation of technology-enhanced formative assessments that bring the best formative assessment practices into classrooms to transform what, how, when, and where science learning is assessed. The project is investigating the feasibility, utility, technical quality, and effectiveness of formative assessments, summative assessments and the Learning Management System (LMS) developed in the SimScientists program. The MASS project is funded by IES and has the following goals: (1) Use systematic design principles to create formative assessments with technical quality to be used during (embedded) and at the end of (benchmark) science curriculum units; (2) Use systematic assessment principles to create a coherent, multilevel state science assessment system by aligning (1) the items within the embedded assessments and the items within the benchmark assessments with the student, task, and evidence models used to design them (horizontal alignment), and (2) the designs of the embedded and benchmark assessments and items with state science standards and relevant items on the state science test (vertical articulation); (3) Study the relationship of the formative assessments and activities to student learning; (4) Study the validity of the use of data from the embedded and benchmark assessments for interpreting student performance on the targeted science standards; and (5) Describe the components of the formative assessments and their implementation so that they can serve as scalable models. The pilot test involved 5,867 middle school students in Spring 2010. This population is from a range of small to large schools and districts, including rural, urban, and suburban districts, a variety of ethnic and socioeconomic backgrounds, and includes English learners and students with disabilities. In the field test, approximately 800 middle school students are participating in Spring 2011; an additional 500-800 middle-school students will participate in Fall 2011. These populations are from a large school district, a variety of ethnic and socioeconomic backgrounds, and include English learners and students with disabilities. The simulation-based assessments studied in this project could contribute to the coherence, comprehensiveness, and continuity of a state science assessment system. Comprehensiveness would be improved by using simulation-based unit assessments to add measurements of science standards for integrated system knowledge and active inquiry practices. Continuity would be improved by the multiple measures unit benchmark assessments could add to state science assessment reports. Coherence could be forged by a nested set of simulation-based assessments in the form of curriculum-embedded modules for formative uses, unit benchmark assessments for summative proficiency, and use of the unit benchmark data or tasks in district or state science testing. (Contains 4 tables.)
- Published
- 2011
38. High School Longitudinal Study of 2009 (HSLS:09) Base-Year Field Test Report. Working Paper Series. NCES 2011-01
- Author
-
National Center for Education Statistics (ED), Ingels, Steven J., Herget, Deborah, Pratt, Daniel J., Dever, Jill, Copello, Elizabeth, and Leinwand, Steve
- Abstract
This report examines the results of the field test for the base year of the High School Longitudinal Study of 2009 (HSLS:09). The general purposes of the field test were, in anticipation of the base-year full-scale effort, to test instruments, forms, and procedures; to experiment with different approaches to questionnaire content and survey methodology; and to evaluate the overall study design. The HSLS:09 field test faced a number of challenges. In varying degrees, these challenges will require further strategies and efforts be applied in the main study. Specific recommendations are summarized in this report, challenge by challenge. Appendices include: (1) HSLS:09 Sampling Plan for Main Study and Field Test; (2) HSLS:09 Field Test Codebook; (3) HSLS:09 Assessment Pilot Report; (4) HSLS:09 Technical Review Panel Participants and Meeting Minutes; (5) HSLS:09 Field Test Letters, Permission Forms, and Scripts; (6) HSLS:09 Mathematics Assessment Specifications:Final Working Version; (7) HSLS:09 Field Test Classical Item Statistics; (8) HSLS:09 Field Test Item Parameter Estimates; (9) HSLS:09 Student Instrument Scale Reliability Analyses; and (10) Questionnaires. (Contains 42 tables, 10 figures and 9 footnotes.)
- Published
- 2010
39. The Seeds to Success Modified Field Test: Findings from the Impact and Implementation Studies
- Author
-
Mathematica Policy Research, Inc., Boller, Kimberly, Del Grosso, Patricia, Blair, Randall, Jolly, Yumiko, Fortson, Ken, Paulsell, Diane, Lundquist, Eric, Hallgren, Kristin, and Kovac, Martha
- Abstract
In 2006, the Bill & Melinda Gates Foundation launched the Early Learning Initiative (ELI) to improve the school readiness of Washington State's children through three main strategies: (1) development of high-quality, community-wide early learning initiatives in two communities; (2) enhancement of statewide systems that support early learning; and (3) support for implementation of promising practices. The foundation joined with other private funders and state officials to form Thrive by Five (Thrive) Washington to energize development and support of high-quality early learning opportunities for all children in the state. In tandem with the formation of Thrive, the Gates Foundation sought two communities with a high level of need for early learning services and the capacity to develop and implement high-quality, community-wide early learning initiatives. The Gates Foundation selected East Yakima, a neighborhood in the central Washington community of Yakima, and White Center, an unincorporated area just outside Seattle. Thrive has worked with an intermediary agency in each community to develop and implement the initiative. Educational Service District 105, through its Ready by Five project, serves as the intermediary in East Yakima. In White Center, Puget Sound Educational Service District (PSESD) serves as the intermediary for the White Center Early Learning Initiative (WCELI). Thrive, the two communities, and other stakeholders worked closely with the Washington State Department of Early Learning (DEL) to develop Seeds to Success, a child care quality rating and improvement system (QRIS). In spring 2009, Thrive, Ready by Five, and Child Care Resources (PSESD's partner in White Center) began a six-month field test of a streamlined version of the DEL QRIS, referred to as the Seeds to Success Modified Field Test (Seeds). This report provides a detailed description of Seeds. Chapter II provides an overview of the impact and implementation studies that comprised the evaluation; specifically, the authors describe the study designs, data collection processes, and analysis methods. In Chapter III, they present findings from the impact evaluation. In Chapter IV, they discuss findings from the implementation study. Chapter V presents lessons learned about implementing the Seeds model. Appendix A contains additional details about the evaluation methodology. Appendices include: (1) Methodological and Technical Details; and (2) Distribution of Mean ERS and Arnett CIS Scores. (Contains 3 boxes, 36 tables, 10 figures and 41 footnotes.) [This report was submitted to the Bill & Melinda Gates Foundation.]
- Published
- 2010
40. Building a Learning Agenda around Disconnected Youth
- Author
-
MDRC, Bloom, Dan, Thompson, Saskia Levy, and Ivry, Rob
- Abstract
In December 2007, the Bill & Melinda Gates Foundation gave MDRC a grant to conduct reconnaissance on promising strategies to reengage disconnected young people and improve their long-term outcomes. The primary objective of the grant was to identify key leverage points for future investment by government and foundations. MDRC consulted with researchers and policy experts, reviewed the results of completed and ongoing evaluations of youth programs, visited a number of innovative youth programs and cities with strong youth strategies, and hosted a meeting of youth practitioners. The goal of the paper's recommendations is to develop a menu of approaches for the heterogeneous population of disconnected youth--analogous in some ways to the multiple pathways that are being developed for high school students. The recommendations fall into two broad categories: building knowledge about mature, existing programs (to better understand whether they work, for whom, and why) and investment in developing and/or scaling up new programs that address areas of unmet need, such as efforts to restructure General Educational Development (GED) preparation programs so that they are more tightly linked with postsecondary programs, both occupational and academic; new "leg-up" strategies for older youth with very low basic skill levels, for whom a GED may not be a realistic goal; and new strategies to engage young people who are more profoundly disconnected and unlikely to volunteer for youth programs. (Contains 3 tables, 2 boxes and 7 footnotes.)
- Published
- 2010
41. Technical Report and Data File User's Manual: For the 2003 National Assessment of Adult Literacy. NCES 2009-476
- Author
-
National Center for Education Statistics (ED), Baldi, Stephane, Kutner, Mark, Greenberg, Elizabeth, Jin, Ying, Baer, Justin, Moore, Elizabeth, Dunleavy, Eric, Berlin, Martha, Mohadjer, Leyla, Binzer, Greg, Krenzke, Thomas, Hogan, Jacqueline, Amsbary, Michelle, Forsyth, Barbara, Clark, Lyn, Annis, Terri, Bernstein, Jar, and White, Sheida
- Abstract
The 2003 National Assessment of Adult Literacy (NAAL) assessed the English literacy skills of a nationally representative sample of more than 19,000 U.S. adults (age 16 and older) residing in households and correctional institutions. NAAL is the first national assessment of adult literacy since the 1992 National Adult Literacy Survey (NALS). The NAAL and NALS produced direct estimates of Prose, Document, and Quantitative literacy, each reported on a 0 to 500 scale and on four performance levels: Below Basic, Basic, Intermediate, and Proficient based on this scale. This report provides information on the 2003 NAAL development and implementation, including development of survey instruments; field test and main assessment sample design; assessment items and scoring rubrics; data collection process and quality of the data; weighting, scaling, treatment of missing data, and data analysis. In addition, the NAAL technical report includes detail information on the development of NAAL innovative additions--Fluency Addition to NAAL (FAN) and Adult Literacy Supplement Assessment (ALSA). Other sections of the report contain information about health literacy cognitive items, data collection and analysis in the US correctional institutions and a user's manual on how to use statistical programs for analyzing NAAL data. The NAAL Technical Report and Data File User's Manual is a key resource for secondary data analysts and for those who want to understand how NAAL was developed and implemented. Chapters include: (1) The 2003 National Assessment of Adult Literacy: An Overview (Mark Kutner and Elizabeth Moore); (2) Development of the Survey Instruments (Elizabeth Greenberg); (3) Field Test (Michelle Amsbary, Martha Berlin, Thomas Krenzke, Leyla Mohadjer, Lyn Clark, and Jacqueline Hogan); (4) Field-Test Scoring and Analysis of Field-Test Data (Elizabeth Greenberg); (5) Field Test of the Fluency Addition to NAAL (Michelle Amsbary); (6) Field Test of Adult Literacy Supplemental Assessment (Michelle Amsbary and Barbara Forsyth); (7) Sample Design (Leyla Mohadjer and Thomas Krenzke); (8) Main Household Study Data Collection and Quality Control (Michelle Amsbary, Martha Berlin, Thomas Krenzke, and Leyla Mohadjer); (9) Correctional Institution Study Data Collection and Quality Control (Michelle Amsbary, Terri Annis, and Martha Berlin); (10) Reducing the Risk of Data Disclosure (Thomas Krenzke, Sylvia Dohrmann, and Laura Alvarez-Rojas); (11) Response Rates and Nonresponse Bias Analysis (Thomas Krenzke and Leyla Mohadjer); (12) Weighting and Variance Estimation (Thomas Krenzke and Leyla Mohadjer); (13) Scoring (Justin Baer, Jared Bernstein, and Michelle Amsbary); (14) Item Analysis, Scaling, and Estimates of Subpopulation Proficiencies (Stephane Baldi and Justin Baer); (15) The Literacy of Adults without Cognitive Data (Ying Jin and Stephane Baldi); (16) Variable Construction and File Development (Ying Jin); and (17) The NAAL Health Literacy Component (Ying Jin and Eric Dunleavy). Appended are: (1) Derived Variables; (2) Instructions On Using AM Software to Analyze the 2003 NAAL Data; (3) Electronic Codebook For Windows User's Manual; (4) NAAL Item Descriptions and Parameters; (5) P-Values Of NAAL 2003 Items; (6) NAAL Household Background Questionnaire; and (7) NAAL Prison Background Questionnaire. (Contains 135 tables, 19 figures, and 1 exhibit.)
- Published
- 2009
42. Student Use of and Perspectives Regarding Podcasted Lectures: Survey Pilot
- Author
-
Preuss, Michael
- Abstract
As part of a podcasting initiative at a community college, a student survey was developed. This survey was based upon the work of Janossy (2007) and Evans (2008) and addressed student use of, opinions of and preferences regarding podcasted lectures. The development, field test and field test outcomes of the survey instrument are described. (Contains 7 tables.)
- Published
- 2009
43. Formative Assessment for College Readiness: Measuring Skill and Growth in Five Key Cognitive Strategies Associated with Postsecondary Success
- Author
-
Conley, David, Lombardi, Allison, Seburn, Mary, and McGaughy, Charis
- Abstract
This study reports the preliminary results from a field test of the College-readiness Performance Assessment System (C-PAS), a large-scale, 6th-12th grade criterion-referenced assessment system that utilizes classroom-embedded performance tasks to measure student progress toward the development of key cognitive skills associated with success in college. A sample of 1,795 students completed C-PAS performance tasks in English and mathematics at 13 New York City high schools in grades 9-12 during Fall 2007. The performance tasks were derived from construct maps and "task shells" designed to elicit the key cognitive strategies. Teachers administered the tasks to students and scored the tasks using standardized scoring guides. Preliminary analyses using Item Response Theory (IRT) yielded evidence that C-PAS measures the acquisition of college readiness cognitive thinking skills in both math and English. The study is significant because it suggests that cognitive strategies important to college readiness can be measured discretely and within separate subject areas. Additionally, the study suggests that complex performance assessments can be utilized to systematically contribute useful information on student performance to help improve student learning. This is important given the current search for ways to address some of the limitations of current large-scale testing methods and systems. (Contains 10 tables and 7 figures.)
- Published
- 2009
44. National Assessment Governing Board and Voluntary National Tests: A Tale of Tribulations without Trials
- Author
-
National Assessment Governing Board and Guerra, Michael
- Abstract
As the National Assessment Governing Board marks its 20th anniversary, it looks back at its stewardship of the National Assessment of Educational Progress (NAEP) with understandable pride. NAEP existed for 20 years before the Board was established to provide an independent governance structure for the assessment. During its first two decades, NAEP tested only national and regional samples of students. Its reports were generally considered technically excellent. But while they may have been groundbreaking in the assessment field, they were little noticed and played virtually no role in shaping educational policy or practice. NAEP received little political heat in exchange for casting modest educational light--a tacit trade making NAEP non-controversial and inconsequential. For the past 20 years, the Board has been second-guessed often, but never criticized for politically motivated decisions. The Governing Board's political purity explains why the Board was eventually given the job of developing the Voluntary National Tests (VNT). This paper takes a look at the history of the National Assessment Governing Board and the Voluntary National Tests. Appendices include: (1) Archived Information for Voluntary National Tests; (2) Sec. 447. Prohibition on Federally Sponsored Testing; and (3) Tables about Draft Scenarios for the Proposed Voluntary National Test and (4) Implementation and Other Issues.
- Published
- 2009
45. Exploring the Relationship between Student Engagement and Common Business Knowledge: A Pilot Study
- Author
-
Ward, Chris, Yates, Dan, and Song, Joon
- Abstract
This study examined the extent to which student engagement is associated with a traditional assessment of student knowledge. In this study, ETS Business Major Field Test (MFT) scores were compared to student's self-reported survey responses to specific questions on the National Survey of Student Engagement (NSSE). Areas of the NSSE survey such as Level of Academic Challenge, Active and Collaborative Learning, Enriching Educational Experience, Higher-Order Thinking, Integration of Diversity into Coursework were included in the study. Grade Point Average (GPA) was also compared to MFT scores and NSSE items. While the sample size was small from one institution (41 students), a number of measures of student engagement were showing signs of linkages to higher MFT scores.
- Published
- 2009
46. Development of a Diagnostic Tool Aimed at Pinpointing Undergraduate Students' Knowledge about Sound and Its Implementation in Simple Acoustic Apparatuses' Analysis
- Author
-
Volfson, Alexander, Eshach, Haim, and Ben-Abu, Yuval
- Abstract
The present study describes the development and field testing of a simple apparatus-based diagnostic instrument (SABDI) that examines undergraduate students' understanding of the underlying physics principles that explain how simple acoustic apparatuses work. SABDI comprises 13 items. Based on previous research studies and the history of science, the 38 distractors of SABDI were designed to reflect the following four categories of possible misconceptions: (a) direct and (b) incorrect emergent views of sound; (c) explanations based on a device's salient feature; and (d) wrong applications of physics knowledge. Field testing carried out on 159 participants showed the salient features category to be the most popular. SABDI is a valid and reliable tool that could be used to determine students' thinking regarding acoustic devices.
- Published
- 2018
- Full Text
- View/download PDF
47. Item and Testlet Position Effects in Computer-Based Alternate Assessments for Students with Disabilities
- Author
-
Bulut, Okan, Lei, Ming, and Guo, Qi
- Abstract
Item positions in educational assessments are often randomized across students to prevent cheating. However, if altering item positions results in any significant impact on students' performance, it may threaten the validity of test scores. Two widely used approaches for detecting position effects -- logistic regression and hierarchical generalized linear modelling -- are often inconvenient for researchers and practitioners due to some technical and practical limitations. Therefore, this study introduced a structural equation modeling (SEM) approach for examining item and testlet position effects. The SEM approach was demonstrated using data from a computer-based alternate assessment designed for students with cognitive disabilities from three grade bands (3-5, 6-8, and high school). Item and testlet position effects were investigated in the field-test (FT) items that were received by each student at different positions. Results indicated that the difficulty of some FT items in grade bands 3-5 and 6-8 differed depending on the positions of the items on the test. Also, the overall difficulty of the field-test task in grade bands 6-8 increased as students responded to the field-test task in later positions. The SEM approach provides a flexible method for examining different types of position effects.
- Published
- 2018
- Full Text
- View/download PDF
48. Design of an Assessment to Probe Teachers' Content Knowledge for Teaching: An Example from Energy in High School Physics
- Author
-
Etkina, Eugenia, Gitomer, Drew, Iaconangelo, Charles, Phelps, Geoffrey, Seeley, Lane, and Vokos, Stamatis
- Abstract
Research into teacher learning and practice over the last three decades shows that the teachers of a specific subject need to possess knowledge that is different from the knowledge of other content experts. Yet this specialized version of content knowledge that teachers need to plan instruction, respond to student ideas, and assess student understanding in real time is a theoretically elusive construct. It is crucial for the fields of precollege teacher preparation, teacher professional education, and postsecondary faculty professional development to (a) clarify the construct that underlies this specialized content knowledge, (b) operationalize it in some domain, (c) measure it in both static contexts and as it is enacted in the classroom, and (d) correlate its presence with "richness" of classroom instruction and its effect on student learning. This paper documents a piece of a multiyear, multi-institutional effort to investigate points (a)-(d) in the domain of energy in the first high school physics course. In particular, we describe the framework that we developed to clarify content knowledge for teaching in the context of high school energy learning. We then outline the process through which we developed, tested, and refined a "paper-and-pencil" assessment administered on a computer and discuss the substantive and psychometric features of several items based on a field test of the final form of the assessment. We choose to discuss these items for a dual purpose: to illustrate the application of our general framework and to present performance findings from a sample of 362 practicing high school teachers of physics.
- Published
- 2018
- Full Text
- View/download PDF
49. Verbal Techniques. The Montessori Observer. Volume 29, Number 3
- Author
-
International Montessori Society
- Abstract
"The Montessori Observer" is mailed four times each year, in March, May, September and November, to Society members throughout the world. The purpose is to provide news and information about the Society's work in Montessori education, and to extend awareness of Montessori principles throughout the world. This issue contains a feature article, "Verbal Techniques," by Lee Havis. Other articles included in this issue are: (1) Baity Society for Special Needs: Montessori Teaching in Cairo, Egypt; (2) IMS Technology Field-testing Continues in Idaho; (3) Montessori Workshops; (4) Moment of Peril: Talking Over Disorder (Wendy Lieberman); (5) Reflections: Tia's Testing (Valerie James); and (6) Marketplace: Positions and Placements.
- Published
- 2008
50. Identifying the Multiple Intelligences of Your Students
- Author
-
McClellan, Joyce A. and Conti, Gary J.
- Abstract
One way of addressing individual differences among adult learners is to identify the Multiple Intelligences of the learner. Multiple Intelligences refers to the concept developed by Howard Gardner that challenges the traditional view of intelligence and explains the presence of nine different Multiple Intelligences. The purpose of this study was to develop a valid and reliable instrument for identifying these Multiple Intelligences. Items were developed by field testing with 168 college students, and responses from 874 community college students were factor analyzed to develop a 27-item indicator to identify Multiple Intelligences preferences of adult learners. (Contains 1 figure.)
- Published
- 2008
Catalog
Discovery Service for Jio Institute Digital Library
For full access to our library's resources, please sign in.