Search

Teacher judgment accuracy is assumed to be positively related to student achievement; however, the empirical evidence for this assumption is inconclusive. Using two accuracy indicators measured at different levels (class/teacher- and student-level), we examined the theoretically hypothesized effects of judgment accuracy on German language achievement over a three-year period, and tested whether teaching quality mediates these effects. We applied multilevel mediation models and small sample methods to data from 35 language teachers and 646 students from 42 classes. While no mediating effects were found, the student-level indicator positively predicted achievement, suggesting that student-level accuracy measures should receive more attention.

Teacher judgment accuracy is assumed to be positively related to student achievement; however, the empirical evidence for this assumption is inconclusive. Using two accuracy indicators measured at different levels (class/teacher- and student-level), we examined the theoretically hypothesized effects of judgment accuracy on German language achievement over a three-year period, and tested whether teaching quality mediates these effects. We applied multilevel mediation models and small sample methods to data from 35 language teachers and 646 students from 42 classes. While no mediating effects were found, the student-level indicator positively predicted achievement, suggesting that student-level accuracy measures should receive more attention.

Beitrag präsentiert auf der Tagung der Gesellschaft für Empirische Bildungsforschung (GEBF), Bamberg.

Beitrag präsentiert auf der Tagung der Gesellschaft für Empirische Bildungsforschung (GEBF), Bamberg.

Posterpräsentation bei der Sektionstagung empirische Bildungsforschung, Mainz (D).

Posterpräsentation bei der 19. European Association of Research on Learning and Instruction (EARLI), Göteborg (SE).

Folio – die Zeitschrift für Lehrpersonen in der Berufsbildung, Jg 145, Nr. 5 (2021), S. 24-27

Teachers’ accuracy in judging students’ achievement is often assumed to be a general ability of teachers. Based on this assumption, teachers should be at least consistent in their accuracy across different content domains within a school subject. Yet, this assumption has rarely been investigated empirically so far. Data from 54 mathematics teachers (N = 1170 students) and 55 language teachers (N = 1255 students) were analysed using a Bayesian multivariate multilevel modelling approach. Results indicate that latent accuracy measures across content domains indeed are substantially correlated within both investigated subjects, but may still be considered to represent different dimensions.

Folio – die Zeitschrift für Lehrpersonen in der Berufsbildung, Jg 145, Nr. 5 (2021), S. 24-27

Posterpräsentation bei der Sektionstagung empirische Bildungsforschung, Mainz (D).

Teachers’ accuracy in judging students’ achievement is often assumed to be a general ability of teachers. Based on this assumption, teachers should be at least consistent in their accuracy across different content domains within a school subject. Yet, this assumption has rarely been investigated empirically so far. Data from 54 mathematics teachers (N = 1170 students) and 55 language teachers (N = 1255 students) were analysed using a Bayesian multivariate multilevel modelling approach. Results indicate that latent accuracy measures across content domains indeed are substantially correlated within both investigated subjects, but may still be considered to represent different dimensions.

Posterpräsentation bei der 19. European Association of Research on Learning and Instruction (EARLI), Göteborg (SE).

In L. Grützmacher & A. Naumann (Chairs), Relationship between heterogeneity and students’ learning gains. EARLI Special Interest Group (SIG) 18, Freiburg (D), September 2020, [Conference postponed]

In L. Grützmacher & A. Naumann (Chairs), Relationship between heterogeneity and students’ learning gains. EARLI Special Interest Group (SIG) 18, Freiburg (D) , September 2020, [Conference postponed].

EARLI SIG 18, Symposium: Valid Use and Issues of Educational Assessments to Improve Classroom Learning and Teaching, Freiburg (D), September 2020, [Conference postponed]

Treffen der Schweizerischen Gesellschaft für Lehrerinnen- und Lehrerbildung (SGL), AG Entwicklung & Lernen, 5. März 2020, St. Gallen.

Zeitschrift für Pädagogik, Beiheft 66 (Ausgabe 1, April 2020), S. 179-196, One major goal of research on educational effectiveness is to detect the effects of teaching and learning. Reliably detecting the effects of teaching and learning requires the identification and adequate measurement of (a) the relevant classroom processes and (b) outcomes on the student and the classroom level and also (c) modeling the link between both. The present paper aims to identify and discuss current conceptual and methodological challenges in regard to making inferences on the effectiveness of teaching and learning. We give a brief overview of current practices, discuss key quality criteria with respect to these three aspects, and identify areas in need of further development. (Verlag)

In C. Köhler & A. Naumann (Chairs), Herausforderungen bei der Messung der Wirkungen von Unterricht. 8. Tagung der Gesellschaft für Empirische Bildungsforschung (GEBF), Potsdam (D), März 2020, [Konferenz abgesagt]

Gesellschaft für Empirische Bildungsforschung (GEBF), Potsdam (D)

Treffen der Schweizerischen Gesellschaft für Lehrerinnen- und Lehrerbildung (SGL), AG Entwicklung & Lernen, 5. März 2020, St. Gallen.

Zeitschrift für Pädagogik, Beiheft 66 (Ausgabe 1, April 2020), S. 179-196, One major goal of research on educational effectiveness is to detect the effects of teaching and learning. Reliably detecting the effects of teaching and learning requires the identification and adequate measurement of (a) the relevant classroom processes and (b) outcomes on the student and the classroom level and also (c) modeling the link between both. The present paper aims to identify and discuss current conceptual and methodological challenges in regard to making inferences on the effectiveness of teaching and learning. We give a brief overview of current practices, discuss key quality criteria with respect to these three aspects, and identify areas in need of further development. (Verlag)

In L. Grützmacher & A. Naumann (Chairs), Relationship between heterogeneity and students’ learning gains. EARLI Special Interest Group (SIG) 18, Freiburg (D), September 2020, [Conference postponed]

EARLI SIG 18, Symposium: Valid Use and Issues of Educational Assessments to Improve Classroom Learning and Teaching, Freiburg (D), September 2020, [Conference postponed]

In L. Grützmacher & A. Naumann (Chairs), Relationship between heterogeneity and students’ learning gains. EARLI Special Interest Group (SIG) 18, Freiburg (D) , September 2020, [Conference postponed].

In C. Köhler & A. Naumann (Chairs), Herausforderungen bei der Messung der Wirkungen von Unterricht. 8. Tagung der Gesellschaft für Empirische Bildungsforschung (GEBF), Potsdam (D), März 2020, [Konferenz abgesagt]

Gesellschaft für Empirische Bildungsforschung (GEBF), Potsdam (D)

Journal of Educational Psychology. Advance online publication., Teacher-assigned grades provide important information that is used by universities and colleges to make admission decisions and by employers to make hiring decisions. Besides grades, the results of standardized achievement tests are frequently used for student selection and allocation. However, correlations between the two achievement measures are far from perfect, and researchers have argued that this discrepancy can be at least partially attributed to norm-referenced grading, which is based on the composition of performance in a class. In this study, we investigated the results of a curricular intensification reform, which introduced changes in the distribution of student performance by making enrollment in certain courses mandatory. We investigated whether the reform resulted in changes in the relationship between standardized achievement-test results and teacher-assigned grades. We analyzed cohort control design data from two large representative samples of students from two German states (Baden-Württemberg: N = 5,574; Thuringia: N = 2,202) before and after the reform. Results indicated that students who received a certain grade before the reform (e.g., a grade of A, B, C, or D) differed in their standardized test performance from students who received the same grade after the reform. Furthermore, in math, course-level-specific reform effects on the association between grades and standardized test performance were found to vary between groups of students receiving good and poor grades. Implications for educational policy and school reforms and suggestions for grading are discussed.

IDeA-Retraite, Langenselbold, September 2019

Podiumsveranstaltung „Leistungsmessung zwischen Bildungsmonitoring und Lernförderung“, Bern, 18. Januar 2019

Podiumsveranstaltung „Leistungsmessung zwischen Bildungsmonitoring und Lernförderung“, Bern, 18. Januar 2019

IDeA-Retraite, Langenselbold, September 2019

Tagung Fachdidaktiken im Dialog der Sprachkulturen. April 5-6, 2019, Lausanne, Switzerland.

Beitrag präsentiert an der 7. Tagung der Gesellschaft für Empirische Bildungsforschung (GEBF), 25. - 27. Februar 2019, Köln (D)

Beitrag präsentiert am Kongress der Schweizerischen Gesellschaft für Bildungsforschung (SGBF), 26. - 29. Juni 2019, Basel

Paper presented at the Symposium: Application of multilevel IRT models for collecting valitity evidence in educational assessment, National Council on Measurement in Education (NCME), Toronto (CAN).

Beitrag präsentiert bei der European Association of Research on Learning and Instruction (EARLI), Aachen (D), Instructional sensitivity relates to the extent to which a test or a single item reflects effects of classroom instruction (Polikoff, 2010). Substantiating instructional sensitivity is important when students’ test scores are used for evaluating teaching effectiveness: If instruments are not instructionally sensitive, valid inferences on teaching effectiveness based on test scores are precluded (Popham et al., 2014). While researchers have asserted that widely-used statistical sensitivity measures do not per se allow for valid inferences on instruction (e.g., van der Linden, 1981), it is unclear how to incorporate instructional measures into the evaluation of instructional sensitivity. Thus, we aim at investigating the extent to which instructional sensitivity measures are related to measures of (a) teaching quality and (b) content of instruction. We expect instructionally sensitive items to become easier with higher teaching quality and implementation of content in class. The sample for our analyses comprises 824 fifth-graders in 48 classrooms participating in a math assessment at two measurement occasions throughout the school year 2016/2017. Teaching quality was assessed via student questionnaires. In addition, teachers provided information on the implementation of teaching objectives. All analyses are carried out using the longitudinal multilevel IRT model to measure instructional sensitivity (Naumann, Hartig, & Hochweber, 2017). The results indicate (a) that the instructional measures partially explain statistical item sensitivity, (b) that it makes a difference which kind of measure is used to evaluate instructional sensitivity, and (c) that the relationship of instructional measures and items’ sensitivity depends on the item under investigation.