Search

This study aimed to describe and compare cognitive activation practices designed to enhance students' mathematical reasoning in Saudi Arabia (KSA), the United States (USA), the United Arab Emirates (UAE), and Singapore. It also explored the impact of these practices on students' mathematical literacy development in each country. Using a causal-comparative methodology, the research examined student scores from the PISA 2022 mathematical literacy assessments, alongside responses to specific questionnaire items on teachers' cognitive activation practices aimed in fostering mathematical reasoning. The findings revealed statistically significant differences in the implementation of cognitive activation practices across the countries studied. Saudi Arabia demonstrated significantly lower levels of these practices compared to the other nations, while the UAE exhibited significantly higher levels. No statistically significant differences were observed between the USA and Singapore in this regard. Moreover, the results indicated that all cognitive activation practices had a predictive effect on the variance in mathematical literacy among students in each country\. Notably, certain practices exhibited a negative impact on mathematical literacy in some countries while contributing positively in others. [ABSTRACT FROM AUTHOR]

Abstract Understanding the long-standing educational inequities associated with socioeconomic status remains significant for transforming educational policies and practices. To better understand entanglements among socioeconomic status and students’ performance in mathematics, we examined different home factors (including language of the test) that influence Lebanese learners’ performance in TIMSS. Exploring TIMSS data can assist us in identifying areas and groups of students who require additional assistance in order to address inequities. The purpose of this study is to investigate how language and other home factors influence Lebanese students’ mathematics performance in TIMSS. Mathematics is taught in a foreign language (English or French) in Lebanon, according to Language of Learning and Teaching policy (LoLT) that dates to 1926. Using TIMSS data and hierarchical linear modeling (HLM), we looked at how students performed in mathematics based on the language of the test and how often they spoke it at home. Other home factors such as parents’ education level, number of books owned, and parents’ involvement were also examined. Results show that not speaking the language of the test at home and other SES-related factors had different but mostly significant contribution to students’ mathematics scores. Lebanese students’ overall low performance suggests the time is ripe for a reformed Lebanese curricula that responds to the needs of learners and of society, taking into consideration students’ cultural capital and language of instruction.

Copyright of Revista Caribeña de Investigación Educativa is the property of RECIE and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

The article is a diagnostic of how science ‘thinks’ as a mode of reasoning and space of action about inclusive education. Attention is given to the comparative reason generated in the Organization for Economic, Co-operation and Development’s (OECD)’s Programme for International Student Assessment (PISA). The infrastructure or architecture of the assessment is an exemplar of a science designed to provide practical knowledge for producing inclusive educational systems. Examined is the grid of calculative practices as producing patterns of recognition and expectations of experience that compare and differentiate nations, societies, and people. The calculations, however, are not merely descriptive. They embody desires as normative inscriptions of who students are, should be, and the dangerous populations threatening the imagined future. The calculations are likened to the seventeenth century projection machines of the magic lanterns, phantasmagrams of an inclusive education that distributes differences that exclude and abject. The comparing of people appears as non-polemical benchmarks, competences, literacies, and well-being organised as ‘highways’ of data to activate in policy and professional practices. The study of PISA is not a critique of science, per se; but of the historical impracticality of an infrastructure that reinscribes inequalities as its method to correct social wrongs. [ABSTRACT FROM AUTHOR]

Understanding the long-standing educational inequities associated with socioeconomic status remains significant for transforming educational policies and practices. To better understand entanglements among socioeconomic status and students' performance in mathematics, we examined different home factors (including language of the test) that influence Lebanese learners' performance in TIMSS. Exploring TIMSS data can assist us in identifying areas and groups of students who require additional assistance in order to address inequities. The purpose of this study is to investigate how language and other home factors influence Lebanese students' mathematics performance in TIMSS. Mathematics is taught in a foreign language (English or French) in Lebanon, according to Language of Learning and Teaching policy (LoLT) that dates to 1926. Using TIMSS data and hierarchical linear modeling (HLM), we looked at how students performed in mathematics based on the language of the test and how often they spoke it at home. Other home factors such as parents' education level, number of books owned, and parents' involvement were also examined. Results show that not speaking the language of the test at home and other SES-related factors had different but mostly significant contribution to students' mathematics scores. Lebanese students' overall low performance suggests the time is ripe for a reformed Lebanese curricula that responds to the needs of learners and of society, taking into consideration students' cultural capital and language of instruction. [ABSTRACT FROM AUTHOR]

Copyright of Education Policy Analysis Archives / Archivos Analíticos de Políticas Educativas / Arquivos Analíticos de Políticas Educativas is the property of Educational Policy Analysis Archives & Education Review and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

The objective of the present study was to investigate item-, examinee-, and country-level correlates of rapid guessing (RG) in the context of the 2018 PISA science assessment. Analyzing data from 267,148 examinees across 71 countries showed that over 50% of examinees engaged in RG on an average proportion of one in 10 items. Descriptive differences were noted between countries on the mean number of RG responses per examinee with discrepancies as large as 500%. Country-level differences in the odds of engaging in RG were associated with mean performance and regional membership. Furthermore, based on a two-level cross-classified hierarchical linear model, both item- and examinee-level correlates were found to moderate the likelihood of RG. Specifically, the inclusion of items with multimedia content was associated with a decrease in RG. A number of demographic and attitudinal examinee-level variables were also significant moderators, including sex, linguistic background, SES, and self-rated reading comprehension, motivation mastery, and fear of failure. The findings from this study imply that select subgroup comparisons within and across nations may be biased by differential test-taking effort. To mitigate RG in international assessments, future test developers may look to leverage technology-enhanced items. [ABSTRACT FROM AUTHOR]

Math anxiety is a common affective disorder in students that is characterized by intrusive thoughts that disrupt critical cognitive resources required for math problem-solving. Consistent associations between math anxiety and math achievement have been observed across countries and age groups, placing math anxiety among other important correlates of math achievement, such as socioeconomic status and magnitude representation ability. However, studies examining math anxiety's relation to achievement have largely focused on the effect of students' own math anxiety (individual effect), while little is known regarding the effect of math anxiety in students' educational context (contextual effect). Using three international studies of achievement (n = 1,175,515), we estimated both the individual and contextual effects of math anxiety across the globe. Results suggest that while there are consistent individual effects in virtually all countries examined, the contextual effects are varied, with only approximately half of the countries exhibiting a contextual effect. Additionally, we reveal that teacher confidence in teaching math is associated with a reduction of the individual effect, and country's level of uncertainty avoidance is related to a lessening of the contextual effect. Finally, we uncovered multiple predictors of math anxiety; notably, student perception of teacher competence was negative related with math anxiety, and parental homework involvement was positively related with math anxiety. Taken together, these results suggest that there are significant between-country differences in how math anxiety may be related with math achievement and suggest that education and cultural contexts as important considerations in understanding math anxiety's effects on achievement. [ABSTRACT FROM AUTHOR]

Abstract The development of international surveys on children’s learning like PISA, PIRLS and TIMSS—delivering comparable achievement measures across educational systems—has revealed large cross-country variability in average performance and in the degree of inequality across social groups. A key question is whether and how institutional differences affect the level and distribution of educational outcomes. In this contribution, we discuss the difference-in-differences strategies employed in the existing literature to evaluate the effect of early tracking on learning inequalities exploiting international assessments administered at different age/grades. In their seminal paper, Hanushek and Woessmann (Econ J 116:C63–C76, 2006) analyze with two-step estimation the effect of early tracking on overall inequalities, measured by test scores’ variability indexes. Later work of other scholars in the economics and sociology of education focuses instead on inequalities among children of different family background, using individual-level models on pooled data from different countries and assessments. In this contribution, we show that individual pooled difference-in-differences models are quite restrictive and that in essence they estimate the effect of tracking by double differentiating the estimated cross-sectional family background regression coefficients between tracking regimes and learning assessments. Starting from a simple learning growth model, we show that if test scores at different surveys are not measured on the same scale, as occurs for international learning assessments, pooled individual models may deliver severely biased results. Instead, the scaling problem does not affect the two-step approach. For this reason, we suggest using two-step estimation also to analyze family-background achievement inequalities. Against this background, using PIRLS-2006 and PISA-2012 we conduct two-step analyses, finding new evidence that early tracking fosters both overall inequalities and family background differentials in reading literacy.

A scale that does not measure the same structure in different cultures should not be used in the cross-cultural comparison-based studies. The validity of the scales should be tested primarily for the countries. In this study, as the evidence of validity of the Programme for International Student Assessment (PISA) 2015 science self-efficacy scale, the measurement invariance has been tested for different cultures. As determined in scientific studies, general self-efficacy is focused on more and this should be examined in academic self-efficacy known to be to student learning and achievement. In the analysis of the data, the measurement model is initially tested. Then, the measurement invariance is tested with a series of analysis. As a result, the scale confirms a unidimensional structure for each country. However, it has provided partial measurement invariance. This indicates that item-based analysis should be conducted.

This study uses a quantitative content analysis of learning competences – as described and prescribed in 21st century frameworks – and those competences evaluated by international assessments to explore the nexus between recommendation and reality. In drawing insights from the theoretical underpinnings of human capital theory we argue, with respect to creativity, that (i) there is a degree of alignment in the prescription and assessment of creativity as a learning competence and (ii) there is a divergence in the way the competence is discussed, which may account for the lack of acknowledgement as a key skill in preparing students for employment in the knowledge-based economy. These findings suggest a discrepancy between recommendation and reality in that the international frameworks consistently place creativity in the top five highest priority learning competences being prescribed while one of the two international assessments examined places it in the top five highest priority learning competences being assessed. Based on the discourse examined in the documents, we assert that schools need to adjust how and when creativity is discussed, ensuring it is included in every subject. This will ensure students link creativity and innovation in every subject area and, subsequently, every industry in the knowledge-based economy. By making this shift, schools will help students ensure long-term employability as the knowledge-based economy transforms into the intelligent economy.

Save for the 2009 Programme for International Student Assessment (PISA) shock, Ireland has recorded strong average scores in mathematics, science, and reading on national and international large-scale assessments. Despite this, percentages of high achievers in mathematics and science in these assessments have remained stubbornly lower than those of some countries with average performance similar to that of Ireland. Reflecting the multifaceted benefits to individuals and society of knowledge and skills in science, technology, engineering, and mathematics (STEM), increasing prioritisation in educational policy in Ireland of high achievers in mathematics and science over the past decade in particular is not unexpected, albeit this was not always the case. This paper offers a chronology of Irish educational policy documents since 1995 illuminating why, when, and how high achievement in mathematics and science has emerged as a key component and driver of educational policy reform. [ABSTRACT FROM AUTHOR]

The development of international surveys on children's learning like PISA, PIRLS and TIMSS—delivering comparable achievement measures across educational systems—has revealed large cross-country variability in average performance and in the degree of inequality across social groups. A key question is whether and how institutional differences affect the level and distribution of educational outcomes. In this contribution, we discuss the difference-in-differences strategies employed in the existing literature to evaluate the effect of early tracking on learning inequalities exploiting international assessments administered at different age/grades. In their seminal paper, Hanushek and Woessmann (Econ J 116:C63–C76, 2006) analyze with two-step estimation the effect of early tracking on overall inequalities, measured by test scores' variability indexes. Later work of other scholars in the economics and sociology of education focuses instead on inequalities among children of different family background, using individual-level models on pooled data from different countries and assessments. In this contribution, we show that individual pooled difference-in-differences models are quite restrictive and that in essence they estimate the effect of tracking by double differentiating the estimated cross-sectional family background regression coefficients between tracking regimes and learning assessments. Starting from a simple learning growth model, we show that if test scores at different surveys are not measured on the same scale, as occurs for international learning assessments, pooled individual models may deliver severely biased results. Instead, the scaling problem does not affect the two-step approach. For this reason, we suggest using two-step estimation also to analyze family-background achievement inequalities. Against this background, using PIRLS-2006 and PISA-2012 we conduct two-step analyses, finding new evidence that early tracking fosters both overall inequalities and family background differentials in reading literacy. [ABSTRACT FROM AUTHOR]

This paper introduces intsvy, an R package for working with international assessment data (e.g., PISA, TIMSS, PIRLS). The package includes functions for importing data, performing data analysis, and visualizing results. The paper describes the underlying methodology and provides real data examples. Tools for importing data allow useRs to select variables from student, home, school, and teacher survey instruments as well as for specific countries. Data analysis functions take into account the complex sample design (with replicate weights) and rotated test forms (with plausible values of achievement scores) in the calculation of point estimates and standard errors of means, standard deviations, regression coefficients, correlation coefficients, and frequency tables. Visualization tools present data aggregates in standardized graphical form.

• The ECI measures infant and toddlers' growth in expressive communication. • ECI growth trajectories are unknown for children in English-speaking countries outside of the U.S. • We compared ECI growth trajectories between U.S. children and those in Australia. • Growth patterns were similar, conforming to developmental theory and prior reports of infant-toddler growth trajectories. Increasingly, measurement of child outcomes is becoming an international priority. However, the psychometric properties of standardized measures are rarely explored for populations beyond the country in which the measures were developed. The Early Communication Indicator (ECI) is a measure of infant-toddler expressive communication developed in the U.S. and designed for use by service providers to inform intervention decision making and outcome measurement. Because the ECI measures growth in prelinguistic and spoken language and can be administered in any language, it has implications for international applications. An intervention study with children from disadvantaged backgrounds in Australia that used the ECI provided the opportunity to explore growth in expressive communication for this population. Children in Australia evidenced significant growth on individual key skills and composite growth trajectories of the ECI. These trajectories conformed to a theoretical continuum of growth in expressive language, and trajectories were similar in onset timing, pattern of growth, and 36-month outcomes to children in the U.S. Implications for the ECI's use for data-driven decision making by early childhood service providers internationally are discussed, as well as limitations and future research directions. [ABSTRACT FROM AUTHOR]

In this paper, we first examine the challenges of score comparability associated with the use of assessments that are exported. By exported assessments, we mean assessments that are developed for domestic use and are then administered in other countries in either the same or a different language. Second, we provide suggestions to better support their valid and fair use. We illustrate these issues in the context of higher education assessments that are designed to serve different purposes—inform admissions decisions and assess student learning outcomes within one country (e.g., the United States) that are later used in other countries. In higher education, the use of exported assessments is on the rise due to increases in globalization, student mobility, and cross‐national comparisons of student achievement. An increase in the use of exported assessments leads to more diverse test‐taker populations and requires special attention due to possible sources of construct‐irrelevant variance, which may threaten the score‐based inferences made for various populations. Irrelevant sources of variance may emerge due to differences in opportunity to learn, curricular exposure, and lack of familiarity with the cultural references used in the assessments that are exported. Report Number: ETS RR–18‐35 [ABSTRACT FROM AUTHOR]

Abstract: The identification of the causal effects of educational policies is the top priority in recent education economics literature. As a result, a shift can be observed in the strategies of empirical studies. They have moved from the use of standard multivariate statistical methods, which identify correlations or associations between variables only, to more complex econometric strategies, which can help to identify causal relationships. However, exogenous variations in databases have to be identified in order to apply causal inference techniques. This is a far from straightforward task. For this reason, this paper provides an extensive and comprehensive overview of the literature using quasi‐experimental techniques applied to three well‐known international large‐scale comparative assessments, such as PISA, PIRLS or TIMSS, over the period 2004–2016. In particular, we review empirical studies employing instrumental variables, regression discontinuity designs, difference in differences and propensity score matching to the above databases. Additionally, we provide a detailed summary of estimation strategies, issues treated and profitability in terms of the quality of publications to encourage further potential evaluations. The paper concludes with some operational recommendations for prospective researchers in the field. [ABSTRACT FROM AUTHOR]

This research paper tries to look into the issues of achievement gaps in education in Gulf countries. The paper sets forth an idea that the international projects such as Programme for International Student Assessment (PISA), Trends in International Mathematics and Science (TIMSS), and Progress in Reading Literacy Study (PIRLS) could serve not only as means of setting and checking quality standards in education but also as mediators in closing achievement gaps and in helping increase the accountability of schools toward the wider public. The paper also makes an attempt to look at the relative standing of Arab Gulf countries in the international projects such as TIMSS, PISA, and PIRLS. The research assumes that the large-scale international data might appear useful for benchmarking the progress made in any individual country. The paper suggests that the arrows of influence move and operate in both directions, implying that while setting global standards, international projects base their judgments on identified local challenges in education systems of individual countries. Most importantly, they could be used in influencing national policies to make education systems more transparent and comparable to international standards. First, the paper states that in order to carry out the benchmarking process efficiently and obtain meaningful results for policy making on an international level, assessment procedures such as testing and questionnaire reporting should be conducted on a local level prior to moving to international level. The paper draws experiences from high-performing countries such as Chinese Taipei, South Korea, and Japan. These countries go through intensive local testing practices at early schooling stages before moving on to participating in TIMSS, PISA, and PIRLS projects. Consequently, school children, schools, school districts, and entire countries show high academic readiness and performance at the fourth and eighth grades on an international level. Second, the paper hypothesizes that discrepancies between teaching methodologies of items covered by individual country curricula, a variety of approaches toward explanation and delivery of various concepts to students, and teacherspecific implementation of theory and practice balance during classes could all potentially contribute to wide gaps and discrepancies between TIMSS, PISA, and PIRLS scores of separate countries no matter how similar their national curricula might look. Strategies for narrowing down examination and questionnaire issues to the items covered in the curricula of all countries have been offered on several occasions; however, this procedure causes oversimplification of teaching and items, and leads to considerable lowering of standards. Therefore, this issue has been presenting a substantial dilemma in TIMSS, PISA, and PIRLS projects' success across countries. [ABSTRACT FROM AUTHOR]

Over time international large-scale assessments have grown in terms of number of studies, cycles, and participating countries, many of which are a heterogeneous mix of economies, languages, cultures, and geography. This heterogeneity has meaningful consequences for comparably measuring both achievement and non-achievement constructs, such as social and emotional skills. In this paper we propose one way to directly incorporate country-specific differences into the methods used to construct background scales. We use research that demonstrates data quality issues in international assessment and the degree to which these issues can impact inferences. Our proposed solution incorporates innovations that have been developed for achievement measures but have not been applied to background scales. We demonstrate this possible solution with PISA 2012 data. [ABSTRACT FROM AUTHOR]

Demand for science, technology, engineering, and mathematics (STEM) professionals is on the rise worldwide. To effectively meet this demand, many governments and private organizations have revamped STEM education and promoted training to enhance math and science skills among students and workers. Education and training programs typically focus on increasing individuals' math and science knowledge. However, data from laboratory studies and large-scale international assessments suggest that fear or apprehension about math, math anxiety, should also be considered when trying to increase math achievement and, in turn, STEM career success. This article reviews findings that shed light on antecedents of math anxiety, the bidirectional math anxiety-performance relation, underlying mechanisms, and promising routes to mitigating the negative relation between math anxiety and math performance. [ABSTRACT FROM AUTHOR]

This article traces the evolution of the French policy PISA debate from 2001 to 2014 by analysing the results of two original qualitative researches. Theoretically, this debate is the outcome of specific policy configurations, which predetermine its scope, content and effectiveness. These configurations are themselves described through their political, institutional, professional and cognitive dimensions. Thus, three configurations are highlighted, all of which favour different public discourses: the Programme for International Student Assessment (PISA) is seen as a topic only interesting to insiders (2001–2004), then as a politicised ranking (2005–2010) and, finally, as a tool purely for communication (2010–2014). The article shows that the major effect of PISA on the French education policy debate was to confirm and legitimate pre-existing opinions and policy stances, and that PISA was progressively scooped out from its statistical content. [ABSTRACT FROM AUTHOR]

PISA is an extremely influential large-scale assessment, and its ‘policy lessons’ are being incorporated in a range of nations all over the world. In this paper I argue that not only is PISA influencing policies and practices, but also that ‘seeing like PISA’ is becoming a widespread phenomenon. Globally, education administration is now characterized by an intense focus on output measurement, a highly competitive environment heightened by national and international rankings, and an economic and instrumentalist approach to education and education reform. Using James Scott’s account of 18th Century German forestry practices as a parable, this paper suggests that ‘seeing like PISA’ could have far reaching and damaging effects. The paper proposes the following: first, understanding PISA as a ‘project of legibility’ enhances our appreciation of its purposes and possibilities. Second, PISA is much more than a ‘representation’ of existing conditions, but is creating new conditions – in other words, it is not descriptive but performative; and, finally, ‘seeing like PISA’ is bringing about deep-rooted changes, and it is likely that the effects will be very long-term. Some of these effects may only manifest themselves in the next fifteen or twenty years; and, by then, the possibilities of redressing some of the ill effects may be very limited. [ABSTRACT FROM AUTHOR]

This paper applies the causal mediation framework proposed by Kosuke Imai and colleagues (Imai, Keele, & Tingley, 2010) to educational research by examining the causal mediating role of early literacy activities in parental education influences on reading performance. The paper argues that the study of causal mediation is particularly relevant in educational settings because learning outcomes result from complex interactions involving multiple actors. The analyses use retrospective longitudinal data from the Progress in International Reading Literacy Study (PIRLS) 2011. The value of international assessment data for producing evidence of causation is discussed. The paper aims to contribute to the study of causation with observational data from educational studies. [ABSTRACT FROM AUTHOR]

Purpose - This moderated discussion appraises popular misconceptions, benefits and limitations of International Large Scale Assessment (ILSA) programs, clarifying how ILSA results could be more appropriately interpreted and used in public policy contexts in the United States (U.S.) and elsewhere in the world. Methodology/Approach - To bring key issues, points-of-view and recommendations on the theme to light, the method employed is a "moderated policy discussion". Nine commentaries were invited to represent voices of leading ILSA scholars/researchers and measurement experts, juxtaposed against views of prominent leaders of U.S. education systems that participate in ILSA programs. The discussion is excerpted from a recent blog published by Education Week. It is moderated with introductory remarks from the guest-editor and concluding recommendations from an ILSA researcher who did not participate in the original blog. References and author biographies are presented at the end of the article. Findings - Together, the commentaries address historical, methodological, socio-political and policy issues surrounding ILSA programs vis-à-vis goals of education and larger societal concerns. Authors offer recommendations for improving the international studies themselves and ILSA reports for educators and the public to facilitate greater understanding of their purposes, meaning, and policy implications. Originality/Value -When assessment policies are implemented from the top down, as is often the case with ILSA program participation, educators and leaders in school systems tend to be left out of the conversation. This article is intended to foster a productive two-way dialogue among key ILSA actors that can serve as a stepping stone to more concerted policy actions within and across national education systems. [ABSTRACT FROM AUTHOR]

Purpose – Although high-quality comparative data from international assessments are now more widely available, to what extent is that data being used to trigger, inform, and direct educational change in non-Organization for Economic Co-Operation and Development (OECD) countries? The purpose of this paper is to develop a theoretical framework to guide a case analysis of Trinidad and Tobago's system response to international assessment data. Design/methodology/approach – This is a single-nation explanatory case study using data from policy documents and elite interviews. Findings are generated through inductive thematic analysis. Findings – The four emerging themes were: first, weaknesses in the national evaluation system; second, policy-making practices not attuned to data; third, lack of collaboration and stakeholder involvement; and fourth, challenges in accessing and using data. Findings suggested that data rarely acted alone to trigger system change. Critical to initiating and sustaining effective data use for system reform were policy-making contexts and mental maps of system leaders, which in this context acted as barriers. Respondents believed that greater strategic leadership from politicians and technocrats could ensure data-informed systemic change. Research limitations/implications – The study focuses upon data use and data-driven decision making for whole system reform within a single country context. However, it advances theory that might be applied to other non-OECD cases. Originality/value – The findings contribute to the refinement of a conceptual model explaining data-driven system reform applicable to non-OECD contexts. The role of system leaders when using international assessment data is clarified. [ABSTRACT FROM AUTHOR]

In this article, we investigate the creation of comparable score scales across countries in international assessments. We examine potential improvements to current score scale calibration procedures used in international large-scale assessments. Our approach seeks to improve fairness in scoring international large-scale assessments, which often ignore item misfit in score scale calibrations. We also seek to obtain improved model-data fit estimates when calibrating international score scales. To this end, we examine the use of two alternative score scale calibration procedures: (a) a language-based score scale and (b) a more parsimonious international scale wherein a large proportion of international parameters are used with a subset of country-based parameters for items that misfit in the international scale. In our analyses, we used data from all 40 countries participating in the Progress in International Reading Literacy Study. Our findings revealed that current score scale calibration procedures yield large numbers of misfitting items (higher than 25% for some countries). Our proposed approach diminished the effects of proportion of item misfit on score scale calibrations and also yielded enhanced model-data fit estimates. These results lead to enhancing confidence in measurements obtained from international large-scale assessments. [ABSTRACT FROM PUBLISHER]

Colombian results related to international assessments (TIMSS, PISA and SERCE) generate a critical reflection about our educational quality, regardless scores since 20 years ago (TIMSS, 1995). Understanding relevance of these assessments demands understanding the concept of "quality of education" based on the UNESCO proposal, particularly those components of equality, relevance and adequacy to any process linked to student learning, which aims to solve the four major challenges facing education - overcoming poverty, reduce inequalities, fight corruption, and strengthen democracy. But this proposal is not detached from learning, something critical to the educational process. Howard Gardner mainly focused to a comprehensive formation of the human being involved in several social processes. Neither the proposed disclaims emotional intelligence as an essential component of the teacher-student relationship, which provides something significant to the knowledge acquisition and human development. Finally, it is very important to analyze particularly how those results are produced in international assessments, showing its usefulness in order to understand the educational processes. The final reflection highlights school/reality linkage related to student learning and student/teacher relationship where a possibility to improve the educational quality is based on the educational system (standards) proposed. [ABSTRACT FROM AUTHOR]

In this study, we investigated differential item functioning (DIF) and its sources using a latent class (LC) modeling approach. Potential sources of LC DIF related to instruction and teacher-related variables were investigated using substantive and three statistical approaches: descriptive discriminant function, multinomial logistic regression, and multilevel multinomial logistic regression analyses. Results revealed that differential response patterns, as indicated by identification of LCs, were most strongly associated with student achievement levels and teacher-related variables rather than manifest characteristics such as gender, test language, and country, which are the focus of typical measurement comparability research. Findings from this study have important implications for measurement comparability and validity research. Evidence of within-group heterogeneity in the test data structure suggests that the identification of DIF and its sources may not apply to all examinees in the group and that measurement incomparability may be greater among groups that are not defined by manifest variables such as gender and ethnicity. Results suggest that alternative variables that may be more closely related to the investigated construct should be examined when conducting measurement comparability research. [ABSTRACT FROM PUBLISHER]

This study illustrates how a diagnostic testing model can be used to make detailed comparisons between student populations participating in international assessments. The performance of Turkish students on the TIMSS-R mathematics test was reanalyzed with a diagnostic testing model called the Rule Space Model. First, mathematical and cognitive skills (‘attributes’) measured by the test were determined. One hundred sixty-two items were coded in terms of their attribute involvement, creating an incidence matrix—the Q-matrix. Using the Q-matrix and the student response data, each student’s attribute mastery profile was determined. Mean attribute mastery levels of Turkish students were computed and compared to those of their American peers. It was shown that Turkish students were weak in algebra and probability/statistics. They also demonstrated poor profiles in skills such as applying rules in algebra, approximation/estimation, solving open-ended problems, recognizing patterns and relationships, and quantitative reading. [ABSTRACT FROM AUTHOR]