Search

The article is a diagnostic of how science ‘thinks’ as a mode of reasoning and space of action about inclusive education. Attention is given to the comparative reason generated in the Organization for Economic, Co-operation and Development’s (OECD)’s Programme for International Student Assessment (PISA). The infrastructure or architecture of the assessment is an exemplar of a science designed to provide practical knowledge for producing inclusive educational systems. Examined is the grid of calculative practices as producing patterns of recognition and expectations of experience that compare and differentiate nations, societies, and people. The calculations, however, are not merely descriptive. They embody desires as normative inscriptions of who students are, should be, and the dangerous populations threatening the imagined future. The calculations are likened to the seventeenth century projection machines of the magic lanterns, phantasmagrams of an inclusive education that distributes differences that exclude and abject. The comparing of people appears as non-polemical benchmarks, competences, literacies, and well-being organised as ‘highways’ of data to activate in policy and professional practices. The study of PISA is not a critique of science, per se; but of the historical impracticality of an infrastructure that reinscribes inequalities as its method to correct social wrongs. [ABSTRACT FROM AUTHOR]

The objective of the present study was to investigate item-, examinee-, and country-level correlates of rapid guessing (RG) in the context of the 2018 PISA science assessment. Analyzing data from 267,148 examinees across 71 countries showed that over 50% of examinees engaged in RG on an average proportion of one in 10 items. Descriptive differences were noted between countries on the mean number of RG responses per examinee with discrepancies as large as 500%. Country-level differences in the odds of engaging in RG were associated with mean performance and regional membership. Furthermore, based on a two-level cross-classified hierarchical linear model, both item- and examinee-level correlates were found to moderate the likelihood of RG. Specifically, the inclusion of items with multimedia content was associated with a decrease in RG. A number of demographic and attitudinal examinee-level variables were also significant moderators, including sex, linguistic background, SES, and self-rated reading comprehension, motivation mastery, and fear of failure. The findings from this study imply that select subgroup comparisons within and across nations may be biased by differential test-taking effort. To mitigate RG in international assessments, future test developers may look to leverage technology-enhanced items. [ABSTRACT FROM AUTHOR]

Math anxiety is a common affective disorder in students that is characterized by intrusive thoughts that disrupt critical cognitive resources required for math problem-solving. Consistent associations between math anxiety and math achievement have been observed across countries and age groups, placing math anxiety among other important correlates of math achievement, such as socioeconomic status and magnitude representation ability. However, studies examining math anxiety's relation to achievement have largely focused on the effect of students' own math anxiety (individual effect), while little is known regarding the effect of math anxiety in students' educational context (contextual effect). Using three international studies of achievement (n = 1,175,515), we estimated both the individual and contextual effects of math anxiety across the globe. Results suggest that while there are consistent individual effects in virtually all countries examined, the contextual effects are varied, with only approximately half of the countries exhibiting a contextual effect. Additionally, we reveal that teacher confidence in teaching math is associated with a reduction of the individual effect, and country's level of uncertainty avoidance is related to a lessening of the contextual effect. Finally, we uncovered multiple predictors of math anxiety; notably, student perception of teacher competence was negative related with math anxiety, and parental homework involvement was positively related with math anxiety. Taken together, these results suggest that there are significant between-country differences in how math anxiety may be related with math achievement and suggest that education and cultural contexts as important considerations in understanding math anxiety's effects on achievement. [ABSTRACT FROM AUTHOR]

• The ECI measures infant and toddlers' growth in expressive communication. • ECI growth trajectories are unknown for children in English-speaking countries outside of the U.S. • We compared ECI growth trajectories between U.S. children and those in Australia. • Growth patterns were similar, conforming to developmental theory and prior reports of infant-toddler growth trajectories. Increasingly, measurement of child outcomes is becoming an international priority. However, the psychometric properties of standardized measures are rarely explored for populations beyond the country in which the measures were developed. The Early Communication Indicator (ECI) is a measure of infant-toddler expressive communication developed in the U.S. and designed for use by service providers to inform intervention decision making and outcome measurement. Because the ECI measures growth in prelinguistic and spoken language and can be administered in any language, it has implications for international applications. An intervention study with children from disadvantaged backgrounds in Australia that used the ECI provided the opportunity to explore growth in expressive communication for this population. Children in Australia evidenced significant growth on individual key skills and composite growth trajectories of the ECI. These trajectories conformed to a theoretical continuum of growth in expressive language, and trajectories were similar in onset timing, pattern of growth, and 36-month outcomes to children in the U.S. Implications for the ECI's use for data-driven decision making by early childhood service providers internationally are discussed, as well as limitations and future research directions. [ABSTRACT FROM AUTHOR]

Over time international large-scale assessments have grown in terms of number of studies, cycles, and participating countries, many of which are a heterogeneous mix of economies, languages, cultures, and geography. This heterogeneity has meaningful consequences for comparably measuring both achievement and non-achievement constructs, such as social and emotional skills. In this paper we propose one way to directly incorporate country-specific differences into the methods used to construct background scales. We use research that demonstrates data quality issues in international assessment and the degree to which these issues can impact inferences. Our proposed solution incorporates innovations that have been developed for achievement measures but have not been applied to background scales. We demonstrate this possible solution with PISA 2012 data. [ABSTRACT FROM AUTHOR]

Demand for science, technology, engineering, and mathematics (STEM) professionals is on the rise worldwide. To effectively meet this demand, many governments and private organizations have revamped STEM education and promoted training to enhance math and science skills among students and workers. Education and training programs typically focus on increasing individuals' math and science knowledge. However, data from laboratory studies and large-scale international assessments suggest that fear or apprehension about math, math anxiety, should also be considered when trying to increase math achievement and, in turn, STEM career success. This article reviews findings that shed light on antecedents of math anxiety, the bidirectional math anxiety-performance relation, underlying mechanisms, and promising routes to mitigating the negative relation between math anxiety and math performance. [ABSTRACT FROM AUTHOR]

Purpose - This moderated discussion appraises popular misconceptions, benefits and limitations of International Large Scale Assessment (ILSA) programs, clarifying how ILSA results could be more appropriately interpreted and used in public policy contexts in the United States (U.S.) and elsewhere in the world. Methodology/Approach - To bring key issues, points-of-view and recommendations on the theme to light, the method employed is a "moderated policy discussion". Nine commentaries were invited to represent voices of leading ILSA scholars/researchers and measurement experts, juxtaposed against views of prominent leaders of U.S. education systems that participate in ILSA programs. The discussion is excerpted from a recent blog published by Education Week. It is moderated with introductory remarks from the guest-editor and concluding recommendations from an ILSA researcher who did not participate in the original blog. References and author biographies are presented at the end of the article. Findings - Together, the commentaries address historical, methodological, socio-political and policy issues surrounding ILSA programs vis-à-vis goals of education and larger societal concerns. Authors offer recommendations for improving the international studies themselves and ILSA reports for educators and the public to facilitate greater understanding of their purposes, meaning, and policy implications. Originality/Value -When assessment policies are implemented from the top down, as is often the case with ILSA program participation, educators and leaders in school systems tend to be left out of the conversation. This article is intended to foster a productive two-way dialogue among key ILSA actors that can serve as a stepping stone to more concerted policy actions within and across national education systems. [ABSTRACT FROM AUTHOR]

In this article, we investigate the creation of comparable score scales across countries in international assessments. We examine potential improvements to current score scale calibration procedures used in international large-scale assessments. Our approach seeks to improve fairness in scoring international large-scale assessments, which often ignore item misfit in score scale calibrations. We also seek to obtain improved model-data fit estimates when calibrating international score scales. To this end, we examine the use of two alternative score scale calibration procedures: (a) a language-based score scale and (b) a more parsimonious international scale wherein a large proportion of international parameters are used with a subset of country-based parameters for items that misfit in the international scale. In our analyses, we used data from all 40 countries participating in the Progress in International Reading Literacy Study. Our findings revealed that current score scale calibration procedures yield large numbers of misfitting items (higher than 25% for some countries). Our proposed approach diminished the effects of proportion of item misfit on score scale calibrations and also yielded enhanced model-data fit estimates. These results lead to enhancing confidence in measurements obtained from international large-scale assessments. [ABSTRACT FROM PUBLISHER]

Colombian results related to international assessments (TIMSS, PISA and SERCE) generate a critical reflection about our educational quality, regardless scores since 20 years ago (TIMSS, 1995). Understanding relevance of these assessments demands understanding the concept of "quality of education" based on the UNESCO proposal, particularly those components of equality, relevance and adequacy to any process linked to student learning, which aims to solve the four major challenges facing education - overcoming poverty, reduce inequalities, fight corruption, and strengthen democracy. But this proposal is not detached from learning, something critical to the educational process. Howard Gardner mainly focused to a comprehensive formation of the human being involved in several social processes. Neither the proposed disclaims emotional intelligence as an essential component of the teacher-student relationship, which provides something significant to the knowledge acquisition and human development. Finally, it is very important to analyze particularly how those results are produced in international assessments, showing its usefulness in order to understand the educational processes. The final reflection highlights school/reality linkage related to student learning and student/teacher relationship where a possibility to improve the educational quality is based on the educational system (standards) proposed. [ABSTRACT FROM AUTHOR]

In this study, we investigated differential item functioning (DIF) and its sources using a latent class (LC) modeling approach. Potential sources of LC DIF related to instruction and teacher-related variables were investigated using substantive and three statistical approaches: descriptive discriminant function, multinomial logistic regression, and multilevel multinomial logistic regression analyses. Results revealed that differential response patterns, as indicated by identification of LCs, were most strongly associated with student achievement levels and teacher-related variables rather than manifest characteristics such as gender, test language, and country, which are the focus of typical measurement comparability research. Findings from this study have important implications for measurement comparability and validity research. Evidence of within-group heterogeneity in the test data structure suggests that the identification of DIF and its sources may not apply to all examinees in the group and that measurement incomparability may be greater among groups that are not defined by manifest variables such as gender and ethnicity. Results suggest that alternative variables that may be more closely related to the investigated construct should be examined when conducting measurement comparability research. [ABSTRACT FROM PUBLISHER]

This study illustrates how a diagnostic testing model can be used to make detailed comparisons between student populations participating in international assessments. The performance of Turkish students on the TIMSS-R mathematics test was reanalyzed with a diagnostic testing model called the Rule Space Model. First, mathematical and cognitive skills (‘attributes’) measured by the test were determined. One hundred sixty-two items were coded in terms of their attribute involvement, creating an incidence matrix—the Q-matrix. Using the Q-matrix and the student response data, each student’s attribute mastery profile was determined. Mean attribute mastery levels of Turkish students were computed and compared to those of their American peers. It was shown that Turkish students were weak in algebra and probability/statistics. They also demonstrated poor profiles in skills such as applying rules in algebra, approximation/estimation, solving open-ended problems, recognizing patterns and relationships, and quantitative reading. [ABSTRACT FROM AUTHOR]

Identifying the sources of differential item functioning (DIF) in international assessments is very challenging, because such sources are often nebulous and intertwined. Even though researchers frequently focus on test translation and content area, few actually go beyond these factors to investigate other cultural sources of DIF. This article introduces the multiple-variable matching method using logistic regression analysis to identify sources of DIF. A case study demonstrates how this methodology identified Extra Lesson Hours After School (ELHAS) as a potential source of DIF between Taiwan and the United States in the Third International Mathematics and Science Study (TIMSS) 1999. DIF is not a fixed character of any test item, nor is a cultural factor an inherent source of DIF. The legitimacy of a source of DIF relies on the specific context and purpose for the cross-country comparison. [ABSTRACT FROM AUTHOR]

In this digital ITEMS module, Dr. Robert [Bob] Mislevy and Dr. Maria Elena Oliveri introduce and illustrate a sociocognitive perspective on educational measurement, which focuses on a variety of design and implementation considerations for creating fair and valid assessments for learners from diverse populations with diverse sociocultural experiences. The first part of the module, narrated by Dr. Mislevy, contains a general overview section, a description of the sociocognitive framing of assessment issues, and a section on implications for assessment around key concepts such as reliability, validity, and fairness. The second part of the module, narrated by Dr. Oliveri, contains a section on frameworks for fairness investigations and principled assessment design as well as brief vignette‐based illustrations of the principles using a prototype activity to support collaboration and communication skills in the workplace. The module is designed to provide a relatively high‐level, conceptual, and nonstatistical overview and is intended for interdisciplinary team members who need to create fair and equitable learning and assessment systems for diverse populations. [ABSTRACT FROM AUTHOR]