A Partial Simulation Study of Phantom Effects in Multilevel Analysis of School Effects: The Case of School Socioeconomic Composition

Hierarchical linear modeling (HLM) is often used to estimate the effects of socioeconomic status (SES) on academic achievement at different levels of an educational system. However, if a prior academic achievement measure is missing in a HLM model, biased estimates may occur on the effects of student SES and school SES. Phantom effects describe the phenomenon in which the effects of student SES and school SES disappear once prior academic achievement is added to the model. In the present analysis, partial simulation (i.e., simulated data are used together with real-world data) was employed to examine the phantom effects of student SES and school SES on science achievement, using the national sample of the United States from the 2015 Programme for International Student Assessment. The results showed that the phantom effects of student SES and school SES are rather real. The stronger the correlation between prior science achievement and (present) science achievement, the greater the chance that the phantom effects occur.