Review of the Use of Standardized Achievement Tests for Accountability Purposes in Education: The Colombia and Chile Cases

Accountability in education was implemented to improve poor learning outcomes by documenting and monitoring learning achievement results. In this process, external standardized achievement tests have played a central role, being the mechanism most frequently used to measure learning outcomes. However, several decades after its initial implementation, most students are still below learning standards, especially in developing countries. Additionally, several issues have been documented when using standardized achievement tests for accountability purposes. This study contributes to the discussion regarding why implementing accountability systems has not been enough to ensure that all students meet learning objectives. Particularly, this research explored technical, ethical, and political issues derived from implementing accountability systems in K12 in Colombia and Chile that incorporate standardized achievement tests to understand how the issues affect the quality of the accountability systems and limit their expected benefits. Specifically, this paper addressed the following questions: how do, if any, technical and political issues related to the use of standardized achievement tests affect the quality of the accountability systems? How do political decisions exacerbate technical issues and affect the quality of accountability systems? Can technical and political issues also be considered ethical? If so, how can all of them be overcome? This study built on theories and contributions about accountability in the education sector through the 2014 edition of the Standards for Educational and Psychological Testing developed by the American Educational Research Association (AERA), the American Psychological Association (APA), and the National Council on Measurement in Education (NCME), as well as a literature review regarding pros and cons of implementing an accountability system in the three dimensions analyzed: technical, political and ethical. A multiple case study with a theory-guided qualitative content analysis approach were also used; and the documentation of the current accountability systems in Chile and Colombia, and the most recent change due to political decisions were mainly reviewed to answer the research questions. Regarding the first one, seven issues that negatively affect the Colombian and Chilean accountability systems were identified; these limit their potential benefits. Issue 1: There is no clear connection between the intended inferences and how test results are reported. Issue 2: There are test reports with incomplete descriptions. Issue 3: Measurement errors are omitted when reporting some test results. Issue 4: The unit of intended inference for test results is not defined clearly in the regulatory documentation. Issue 5: There are omissions of support when describing the way in which test results should be used given the intended inferences, uses and implications. Issue 6: There is no clear connection between documentation, indicating how each one responds or builds on the others. Issue 7: Current standardized achievement tests measure the national curriculum partially, but it is not stated explicitly. Finally, the way in which these uses affect the quality of the accountability systems analyzed is described. For the second research question, some mixed evidence was found. For Colombia, the most recent political decision generated additional technical pressure, exacerbating technical issues, while Chile could have reduced the technical pressure. For the third research question, the political and technical issues identified can be considered ethical given the evidence analyzed, and they can be overcome by following the international standards in testing, as the accountability systems already declare. However, there is enough information for a better implementation of accountability systems in both Chile and Colombia, so a discussion about the implications for the field and practice is reported. [The dissertation citations contained here are published with the permission of ProQuest LLC. Further reproduction is prohibited without permission. Copies of dissertations may be obtained by Telephone (800) 1-800-521-0600. Web page: http://www.proquest.com/en-US/products/dissertations/individuals.shtml.]