Search

This paper explores the measurement capabilities of the Trends in International Mathematics and Science Study (TIMSS) and the Program for International Student Assessment (PISA) in assessing school factors that influence student performance. We specifically focus on the 2015 assessments of the science performance of eighth graders in Sweden and Norway. This was the latest year when the two assessments were conducted in the same year and science was the major subject area in the PISA assessment, which was essential for maximizing the assessments’ comparability. Using multilevel models, the study identifies common and unique factors across the assessments and investigates the factors that influence student performance at different proficiency levels. The findings highlight the importance of school-level factors, which are significant in both assessments. Moreover, both assessments provide information on overlapping sets of factors that have varying influence on the performance of students with different proficiency levels. Overall, there are limited common factors between TIMSS and PISA. School factors vary between low-performing and high-performing schools, with differing significance in Norway and Sweden. The results indicate that TIMSS and PISA assessments offer complementary information, particularly for low-performing schools. Our findings suggest that different school types may benefit or suffer from distinct school factors. The findings are relevant for both educational professionals and policy-makers.

The overall aim of this study is to examine the association between Swedish students’ attitudes towards mathematics, mathematics achievement as measured by the Trends in Mathematics and Science Study (TIMSS), socioeconomic status (SES), and educational background variables. A further aim is to investigate whether students’ attitudes towards mathematics have a mediating role between their mathematics achievement and their background. Several indicators of students’ SES and background, taken from both the TIMSS 2015 database and from Swedish official registers, were used. The overall results show that there were differences in attitudes towards mathematics in relation to the different SES and educational background measures. There are also associations between students’ SES and both TIMSS mathematics achievement and their attitudes towards mathematics. The students’ attitudes towards mathematics only had a small mediation role between the students’ backgrounds and TIMSS mathematics achievement. Finally, although the mediation models had a better fit when including other information, the mediation effect was lower. Practical implications of the obtained results are discussed.

Högskoleprovet är ett urvalsinstrument i antagningen till universitets- och högskolestudier i Sverige. Tanken med provet är att det ska fungera som ett alternativ för den som är behörig till högre utbildning men som inte har tillräckligt konkurrenskraftiga gymnasiebetyg, och därigenom fungera somen andra chans till högre utbildning när det är konkurrens om studieplatserna. Provet är frivilligt och öppet för alla, och det finns ingen gräns för hur många gånger provet kan tas. Ett provresultat är giltigt i fem åroch om en person har tagit provet flera gånger räknas det bästa resultatet. Provets popularitet varierar, och påverkas av vad som händer inom gymnasieskolan, hur det ser ut på arbetsmarknaden och söktrycket till högre utbildning. Detta gör att antal provtagare och vilka som tar provet skiljer sig åt mellan provtillfällen. Syftet med denna rapport är att beskriva provtagargrupperna våren och hösten 2019 med avseende på sammansättning och resultat. Resultaten presenteras för provtagarna, även uppdelat utifrån kön, ålder och utbildning. I rapporten beskrivs också normeringen av provresultaten och utfallet av normeringen, samt i vilken mån provtagare väljer att ta provet flera gånger och presterar vid upprepade tillfällen., The SweSAT, or Swedish Scholastic Assessment Test (in Swedish:Högskoleprovet) is a test used in admissions to higher education in Sweden. The test serves as an alternative to high school GPA to give students who donot have a competitive high school GPA, or simply want to maximize their possibility to compete for attractive study positions, a second chance. The testis optional, open for all and can be taken unlimited number of times. This means that the number of test takers and composition of test takers varies between administrations; the popularity of the test to some degree depends on external circumstances such as the situation on the job market, school reforms, etc, that will affect the interest in higher education studies. The aim of this report is to describe the population of test-takers in the spring and autumn of 2019 when it comes to composition and test scores. Results are presented for test-takers of different gender, age and previous education. Additionally, the norming and equating procedure and the outcome of the standardization of the test is described. Test taker performance and repeated test taking is also examined and discussed.

Multilevel modeling is an important tool for analyzing large-scale assessment data. However, the standard multilevel modeling will typically give biased results for such complex survey data. This bias can be eliminated by introducing design weights which must be used carefully as they can affect the results. The aim of this paper is to examine different approaches and to give recommendations concerning handling design weights in multilevel models when analyzing large-scale assessments such as TIMSS (The Trends in International Mathematics and Science Study). To achieve the goal of the paper, we examined real data from two countries and included a simulation study. The analyses in the empirical study showed that using no weights or only level 1 weights sometimes could lead to misleading conclusions. The simulation study only showed small differences in estimation of the weighted and unweighted models when informative design weights were used. The use of unscaled or not rescaled weights however caused significant differences in some parameter estimates.

Plausible values are typically used in large–scale assessment studies, in particular in the Trends in International Mathematics and Science Study and the Programme for International Student Assessment. Despite its large spread there are still some questions regarding the use of plausible values and how such use affects statistical analyses. The aim of this paper is to demonstrate the role of plausible values in large–scale assessment surveys when multilevel modelling is used. Different user strategies concerning plausible values for multilevel models as well as means and variances are examined. The results show that some commonly used user strategies give incorrect results while others give reasonable estimates but incorrect standard errors. These findings are important for anyone wishing to make secondary analyses of large–scale assessment data, especially those interested in using multilevel models to analyze the data., Originally included in thesis in manuscript form

This thesis contributes to the area of research based on large-scale educational assessments, focusing on the application of multilevel models. The role of sampling weights, plausible values (response variable imputed multiple times) and imputation methods are demonstrated by simulations and applications to TIMSS (Trends in International Mathematics and Science Study) and PISA (Programme for International Student Assessment) data. The large-scale assessments use multistage sampling design, which means that the units such as schools, classrooms, or students at some or all stages are selected with unequal probabilities. In order to make valid estimates and inferences sampling weights should be used. Thus, in the first paper, we examine different approaches and give recommendations concerning handling sampling weights in multilevel models when analyzing large-scale assessments. Due to limitations in time and the number of students, the complex surveys use matrix sampling of items. This means that a response variable, i.e. students' performance, contains a large amount of information that is missing by design. Therefore, in order to estimate students' proficiency, TIMSS and PISA use the plausible values approach, which results in a set of five plausible values – proficiencies, computed for each student. In the second paper, different user strategies concerning plausible values for multilevel models as well as means and variances are examined with both real and simulated data. Missing information that is present because of the matrix sampling design for instance like the one used in PISA, can be arranged into a non-monotone missing data pattern, where all variables are incomplete and highly positively correlated. In the third paper, we compare a few imputation methods: a single imputation from a conditional distribution (with and without weights) and multiple imputation, for data with a non-monotone missing pattern (with no complete variables) and high positive correlati

International studies like TIMSS (Trends in International Mathematics and Science Study) usually highlight students’ study result changes between two assessments. The aim of this study is to identify factors that contribute to the explanation why some schools is effective but others are less effective in terms of the students’ academic achievement in mathematics on TIMSS. We focus on the school efficiency in Sweden and Norway at the time points 2003, 2007 and 2011. In the core subject mathematics, Sweden’s result declined in TIMSS between 2003 and 2011 while Norway’s result inclined. Since Sweden and Norway have similar educational systems it is of interest to examine the incline in Norway as opposed to the decline in Sweden between 2003 and 2011. We used multilevel analysis in order to separate the effect of school level variables from the effect of student’s home environment and to take care of the sampling design used in TIMSS. The results show that Norway and Sweden exhibit different trends. In Norway it was possible to identify school level factors which affect the students’ achievements. In Sweden this could not be obtained. Note, in both countries at all three timepoints most of the student home background factors were significant suggesting that the students’ background play a large role in mathematics achievement.

In a Nordic perspective, the finish students overall achieve the highest score on PISA (Programme for International Student Assessment), while the Swedish students exhibit declining results. The results of the Swedish students have drawn attention to the quality of education and the role of the educational professionals and the efficiency of the school. It is therefore of vital importance to investigate whether these results can be related to school level factors in a Nordic perspective. However, TIMSS (Trends in International Mathematics and Science Study) and PISA exhibit similarities as well as differences as they target different subjects. In addition, the results on TIMSS and PISA differ between countries. The aim of this study is to investigate whether school level factors can contribute to the explanation of the results for the Nordic countries participating in PISA 2009 and, if so, identify factors that can be influenced in order to enhance students’ achievement. We focus on school effectiveness in relation to PISA, since all Nordic countries participate in PISA. However, the results are contrasted to results from TIMSS for Sweden and Norway. In order to separate the effect of school level variables from the effect of student’s home environment and to take care of the sampling design used in TIMSS and PISA, multilevel analysis was used. The results show that only a few school level factors were significant, and only in Sweden and Finland. Furthermore, school level factors in Sweden and Norway on PISA differ from school level factors based on TIMSS data.

Decreasing performance among students in Sweden on international comparative studies and increasing segregation of schools, has led to a debate concerning strategies for improving student performance. The aim of this study is to analyse the between school variance and to identify factors associated with student performance in PISA in mathematics at different performance levels in the Nordic countries. In order to separate the effect of school-level variables, from the effect of student’s background factors and to take care of the multistage sampling design used in PISA, multilevel analysis was used. The results show that no evidence regarding the relationship between the average student performance in mathematics and the between-school variance was found which, is in contrast to previous studies conducted on science performance in PISA. Regarding school-level factors, our results overall have shown that few school-level factors (having a positive or a negative effect) seemed to be associated with performance. School-level factors associated with performance have mainly been identified among low- and medium-performing schools, and to a less extent among students at high-performing schools (only in Sweden and Denmark). This is a result which is in line with other studies showing the educational system’s incapacity to provide support for high-performing students and to enhance their learning., What can we learn from large-scale assessments about promoting student success?

Missing information is common in real data studies. When missingness is large, it should not be ignored and, instead, a missing data imputation method should be considered. The choice of the imputation method depends on the type or the pattern of missing information, as well as the nature of data. For instance, observations in large-scale educational assessments are incomplete by missing some components and based on usually positively correlated results within students. In all types of analysis of such data, the correlation has to be considered in a reliable calculation of properties of estimates. The aim of this paper is to compare a single imputation from a conditional distribution (with or without weights) and multiple imputation for data with a non-monotone missing pattern and high positive correlation between variables. For this purpose, such estimates as mean and variance are compared. The simulation results showed that expectation and variance are estimated more reliably when the imputation from a conditional distribution (without and with weights) or a complete-data set are used, compared to multiple imputation.

Missing information is common in real data studies. When missingness is large, it should not be ignored and, instead, a missing data imputation method should be considered. The choice of the imputation method depends on the type or the pattern of missing information, as well as the nature of data. For instance, observations in large-scale educational assessments are incomplete by missing some components and based on usually positively correlated results within students. In all types of analysis of such data, the correlation has to be considered in a reliable calculation of properties of estimates. The aim of this paper is to compare a single imputation from a conditional distribution (with or without weights) and multiple imputation for data with a non-monotone missing pattern and high positive correlation between variables. For this purpose, such estimates as mean and variance are compared. The simulation results showed that expectation and variance are estimated more reliably when the imputation from a conditional distribution (without and with weights) or a complete-data set are used, compared to multiple imputation.