Search

This paper explores participation trends in interventions that promote self-evaluation exercises on the effective use of digital technologies in schools. We use a unique dataset consisting of 83,185 respondents from 924 Spanish schools that used SELFIE, a tool based on self-reflection questionnaires that capture different dimensions of school's digital capacity. We benefit from a natural experiment situation caused by the parallel use of SELFIE by two groups of schools. The first group was externally selected as part of a representative sample of Spanish schools. Conversely, the second group voluntarily decided to use SELFIE as a diagnostic tool for a subsequent self-evaluation exercise. Moreover, a subset of schools were located in regions where authorities embedded SELFIE in broader digitalisation programmes. By comparing these groups, it is shown that schools that decide to participate in SELFIE voluntarily are those with a lower initial digitalisation level. It is also found that the promotion of the use of SELFIE as part of public interventions can increase participation but mainly attracts digitally advanced schools. In conclusion, policy interventions aiming to develop the digital capacity of schools need to plan how to reach those schools that need it more in order to be more equitable.

International Large-Scale Assessments (LSA) allow comparisons of education systems' effectiveness in promoting student learning in specific domains, such as reading, mathematics, and science. However, it has been argued that students' scores in International LSAs mostly reflect general cognitive ability (g). This study examines the extent to which students' scores in reading, mathematics, science, and a Raven's Progressive Matrices test reflect general ability g and domain-specific abilities with data from 3,472 Polish students who participated in the OECD's 2009 Programme for International Student Assessment (PISA) and who were retested with the same PISA instruments, but with a different item set, in 2010. Variance in students' responses to test items is explained better by with a bifactor Item Response Theory (IRT) model than by the multidimensional IRT model routinely used to scale PISA and other LSAs. The bifactor IRT model assumes that non-g factors (reading, math, science, and Raven's test) are uncorrelated with g and with each other. The bifactor model generates specific ability factors with more theoretically credible relationships with criterion variables than the multidimensional standard model. Further analyses of the bifactor model indicate that the domain-specific factors are not reliable enough to be interpreted meaningfully. They lie somewhere between unreliable measures of domain-specific abilities and nuisance factors reflecting measurement error. The finding that PISA achievement scores reflect mostly g, which may arise because PISA aims to test broad abilities in a variety of contexts or may be a general characteristic of LSAs and national achievement tests.

This study contributes to the literature by examining the evolution of socio-economic disparities in literacy skills between age 15 and 27. It uses combined cross-sectional data from the Programme for International Student Assessment and the Programme for the International Assessment of Adult Competencies in 20 countries and adopts a synthetic cohort approach. In the article we examine if there are differences in the evolution of socio-economic disparities in literacy between the teenage years and young adulthood among high-achieving students (90th percentile) and among low-achieving students (10th percentile) and compare differences across education systems. We used parental education as an indicator of socio-economic status. Findings indicate that on average, disparities in literacy skills observed at age 15 widen by 15% of a standard deviation by age 27. Such increase is determined by a more pronounced growth in achievement among individuals with tertiary educated parents than among individuals whose parents did not complete tertiary education. The increase in socio-economic gaps between age 15 and 27 is more pronounced among low-achieving students. The group that improves the most between age 15 and 27 is low-achieving individuals with tertiary educated parents. We find that socio-economic differences in educational attainment, NEET status and use of skills at home and in the workplace explain a large part of socio-economic disparities in literacy achievement among young adults. Although countries with above-average disparities at age 15 tend to also display above-average disparities at age 27, we identify differences across countries.

Background: Studies demonstrate that occupational optimism can boost adolescents' academic attainment and perseverance in education. To contribute to this literature, we consider two hypotheses. The first posits that bilingual immigrants are remarkably resourceful and determined. Thus, they are more occupationally ambitious than their peers. The second proposes that immigrant students engage in "strategic adaptation" by specializing in science, viewed as a level playing field. Methods: To assess these hypotheses at two points of time, we analyze data from 19 societies that participated in the Programme for International Student Assessment (PISA) in 2006 and 2015. Our primary method is path analysis with balanced replicate weights (BRR) undertaken separately for each country's data. Results: We find that, in many countries, bilingual immigrants expect to enter higher status occupations than non-immigrants. However, immigrants who do not speak another language are also optimistic, so linguistic resources cannot explain occupational ambition. Furthermore, immigrants accord science more instrumental value and enjoy it more at school, which accounts, across societies, for up to 12% of the variation in vocational optimism indicated by the expected occupational status, and up to 41% in plans to pursue a career in science professions. Conclusion: Our results align with the "strategic adaptation" argument that many young immigrants might seek to specialize in science as a pragmatic tactic to ensure high occupational attainment.

This article presents the pseudo-equivalent group approach and discusses how it can enhance the quality of linking in the presence of nonequivalent groups. The pseudo-equivalent group approach allows to achieve pseudo-equivalence using propensity score reweighting techniques. We use it to perform linking to establish scale concordance between two assessments. The article presents Monte-Carlo simulations and a real data application based on data from the Survey of Adult Skills (PIAAC) and the Programme for International Student Assessment (PISA). Monte-Carlo simulations suggest that the pseudo-equivalent group design is particularly useful whenever there is a large overlap across the two groups with respect to balancing variables and when the correlation between such variables and ability is medium or high. The example based on PISA and PIAAC data indicates that the approach can provide reasonable accurate linking that can be used for group-level comparisons.

We examine the relevance of task exposure specificity in the development of self-efficacy evaluations among 15-year-old students in Organisation for Economic Co-operation and Development (OECD) countries. We use data from the 2012 edition of the Programme for International Student Assessment (PISA) to assess if students' self-efficacy evaluations with respect to a set of mathematics tasks are associated with how frequently they encountered those same tasks or tasks requiring them to apply similar processes or procedures. We find evidence of task exposure specificity in the development of self-efficacy evaluations toward both applied and pure mathematics problems. The relationship between task exposure and self-efficacy evaluations is not moderated by students' socioeconomic status (SES) or anxiety toward mathematics. Exposure to easy items appears to be more strongly associated with the development of self-efficacy among high achieving students and exposure to pure mathematics problems is more strongly associated with the development of self-efficacy among girls.

This study examines the influence of family income on student achievement in mathematics utilising data from the parents' questionnaire for nine countries participating in the OECD's 2012 PISA study. It finds non-trivial effects for family income that were consistently larger than, or comparable to, the effects of more commonly used measures of socio-economic background: parents' education and occupational status. Its effects ranged from weak (Hong Kong and Macau) to moderate (Belgium, Chile and Portugal) and were often stronger than the effects of parents' education and occupational status. There was no evidence that the effects of family income were mediated through cultural resources, wealth (indexed by household possessions) or educational resources in any country. This suggests that the effects of family income reflect largely unmeasured aspects of families rather than resources in the home. In Belgium and Portugal, much of the effect of family income could be attributed to prior knowledge and skills in mathematics.

For assessment scales applied to different groups (e.g., students from different states; patients in different countries), multigroup differential item functioning (MG‐DIF) needs to be evaluated in order to ensure that respondents with the same trait level but from different groups have equal response probabilities on a particular item. The current study compares two approaches for DIF detection: a multiple‐group item response theory (MG‐IRT) model and a generalized linear mixed model (GLMM). In the MG‐IRT model approach, item parameters are constrained to be equal across groups and DIF is evaluated for each item in each group. In the GLMM, groups are treated as random, and item difficulties are modeled as correlated random effects with a joint multivariate normal distribution. Its nested structure allows the estimation of item difficulty variances and covariances at the group level. We use an excerpt from the PISA 2015 reading domain as an exemplary empirical investigation, and conduct a simulation study to compare the performance of the two approaches. Results from the empirical investigation show that the detection of countries with DIF is similar in both approaches. Results from the simulation study confirm this finding and indicate slight advantages of the MG‐IRT model approach. [ABSTRACT FROM AUTHOR]

Large-scale international assessments rely on indicators of the resources that students report having in their homes to capture the financial capital of their families. The scaling methodology currently used to develop the Programme for International Student Assessment (PISA) background indices is designed to maximize within-country comparability over time. However, questions remain on the extent to which alternative scaling methodologies can ensure greater cross-country comparability. Establishing indicators of household resources that are comparable both across countries and over time is fundamental to assessing cross-country differences in socioeconomic inequalities in academic achievement. We use multigroup confirmatory factor analysis for categorical variables to examine the measurement equivalence of the household resource factors across the Organisation for Economic Co-Operation and Development countries that participated in the PISA 2000 and 2012 cycles. We construct a series of alternative measures that use students' reports on the availability of household resources and compare them to the index of home possessions that is generally used in analyses of PISA data. Our findings show that alternative scaling methodologies can be useful in developing cross-country indicators of socioeconomic status.

Digital competence is crucial for living, working and participating in current societies. Despite its huge importance, objective measurement tools for it are scarce due to its developmental difficulty. Self-assessment of digital competence seems a promising proxy of objective tests, and it additionally offers the possibility for surveying otherwise unmeasurable constructs such as attitudes and beliefs. However, self-assessment tools are burdened with validity problems, most notably response biases such as overly positive descriptions, overclaiming or careless and insufficient effort responding. In this paper, we investigate how these problems can be mitigated by using the overclaiming technique, a technique that identifies and corrects the bias variance in self-assessments. Our main result was that the use of the overclaiming technique can lead to higher reliability and validity of digital competence self-assessment tools, especially for short scales. Moreover, it allows for correcting additional spurious variance in comparison with careless responding indexes, which allows the use of both these techniques in parallel to increase the quality of data. Our results are important in providing advances in enhanced information on digital competence that can result in better lifelong learning decisions when used at the individual level and in better policy-making decisions when used at the aggregate level. [ABSTRACT FROM AUTHOR]

A response model that is able to detect guessing behaviors and produce unbiased estimates in low-stake conditions using timing information is proposed. The model is a special case of the grade of membership model in which responses are modeled as partial members of a class that is affected by motivation and a class that responds only according to the level of ability. Monte Carlo simulations were conducted to compare the proposed model with an approach that ignored guessing and an approach based on item filtering. In each simulated condition, the proposed model outperformed the other approaches by showing the lowest level of bias and the highest precision of item and persons estimates. Finally, the model was estimated using real life data from Programme for the International Assessment of Adult Competencies research (PIAAC). The results showed slight but expected corrections for the levels of proficiency in all countries.

In the social sciences, estimating causal effects is particularly difficult. Gold standards are set by randomized experiments in many cases expensive, unenforceable for ethical and practical reasons. Recent research has drawn attention to techniques that under some conditions, could estimate causal effects on non-experimental observable data. One technique is the instrumental-variables (IVs) approach. This approach is used to determine variation that is exogenous in treatment and to estimate causal inferences. This paper begins by explaining the logic of IVs and then reviews the literature on the use of the IVs approach in the educational context. The most common types of IVs and the guidelines for selecting appropriate variables are explained. The statistical background of IVs estimation is described, which is followed by a discussion of the assumptions that underlie statistical procedures. Finally, empirical examples that use data from the Polish extension of the Programme for International Student Assessment are presented to estimate the effects on student learning outcomes of having at least one neighborhood friend in the classroom.

This article combines statistical and applied research perspective showing problems that might arise when measurement error in multilevel compositional effects analysis is ignored. This article focuses on data where independent variables are constructed measures. Simulation studies are conducted evaluating methods that could overcome the measurement error problems. These methods are a multilevel regression model with reliability correction, a multilevel model with plausible values, and a multilevel latent regression model (doubly latent model). While the latter one performs best, all models have their advantages and disadvantages. Examples using data from the Trends in International Mathematics and Science Study are shown to present the behavior of the tested models in real-data situations and indicate the consequences of ignoring the problem of reliability.

The analysis, based on a Polish national sample from 2006, aims to cast light on the regional variation in the effect of education on incomes. Building on the conceptual framework developed in the theory of human capital we investigate to what extent pay-offs for human capital differ across detailed administrative districts in Poland. By incorporating contextual characteristics, we examine how micro- and macro-level factors shape labour market outcomes. Our findings provide further support for the hypothesis that there is much regional variation in the influence of education on incomes, which suggests that there are better and worse places for the development of a meritocratic distribution of incomes. It appears that education pays more highly in more economically-developed regions, marked by a higher rate of occupational activity.

Using data from the Program for International Student Assessment 2006 surveys for 50 countries, this paper explores gender segregation of adolescent science career plans. We ask whether, in different cultures, bridging the male-female gap in science self-concept could reduce gender disparities in students' career preferences. Bringing together the theory of gender essentialism and the biased self-concept thesis, we interpret the cross-national variation in the relationship between self-concept and occupational plans. To this end, we fit a series of random intercept regression models to country-specific and pooled data. In all countries, science-oriented girls prefer employment in biology, agriculture, or health (BAH), whereas boys favor careers in computing, engineering, or mathematics (CEM). Almost everywhere, boys have more confidence in their science ability than girls, even after science performance is taken into account. In advanced industrial countries the male-female gap in science self-concept is larger than the corresponding gap in developing or transforming societies. The male-female segregation of preferences for science careers is also stronger in advanced industrial countries. Nevertheless, nowhere are gender disparities in science self-assessment related to the gender segregation in preferences for BAH and CEM careers. We discuss the significance of these cross-national patterns for science educators and educational policy makers. (Contains 9 tables, 2 figures and 15 footnotes.)

Using data from 24 countries, which participated in the 2006 Programme for International Student Assessment (PISA), we examine the relationship between parental science employment and students' career expectations. In contrast to prior PISA-based studies, we find that the link between parental employment and adolescent plans to work in science is non-trivial and merits attention. In this context, we consider three versions of the gender socialisation hypothesis. The strong variant posits that girls' plans are shaped solely by their mothers' career pathways while boys model their expectations exclusively on fathers' occupations. The weaker version of this hypothesis expects children to be influenced more by the same-sex than by the opposite-sex parent. Finally, the third possibility is that, as egalitarian ideologies prevail, parents inspire adolescent occupational plans regardless of gender. These hypotheses are assessed separately for student career plans related to biology, agriculture and health (BAH) in contrast to computing, engineering and mathematics (CEM), because the involvement in these fields of science is known to be segregated by gender. Using two-level multinomial logit modelling, we find some support for the weak version of the gender socialisation hypothesis. Although within-family transfers of preferences for science careers vary considerably across countries, we note certain regularities. In many nations, relevant paternal employment enhances sons' interest in science careers regardless of their field. In contrast, maternal employment inspires daughters in fewer countries and this influence tends to be limited to careers in BAH. We discuss the possible implications of these findings for science educators. (Contains 6 tables, 3 figures, and 9 notes.)