Search

Computer-based analysis of preservice teachers' written reflections could enable educational scholars to design personalized and scalable intervention measures to support reflective writing. Algorithms and technologies in the domain of research related to artificial intelligence have been found to be useful in many tasks related to reflective writing analytics such as classification of text segments. However, mostly shallow learning algorithms have been employed so far. This study explores to what extent deep learning approaches can improve classification performance for segments of written reflections. To do so, a pretrained language model (BERT) was utilized to classify segments of preservice physics teachers' written reflections according to elements in a reflection-supporting model. Since BERT has been found to advance performance in many tasks, it was hypothesized to enhance classification performance for written reflections as well. We also compared the performance of BERT with other deep learning architectures and examined conditions for best performance. We found that BERT outperformed the other deep learning architectures and previously reported performances with shallow learning algorithms for classification of segments of reflective writing. BERT starts to outperform the other models when trained on about 20 to 30% of the training data. Furthermore, attribution analyses for inputs yielded insights into important features for BERT's classification decisions. Our study indicates that pretrained language models such as BERT can boost performance for language-related tasks in educational contexts such as classification.

Science education researchers typically face a trade-off between more quantitatively oriented confirmatory testing of hypotheses, or more qualitatively oriented exploration of novel hypotheses. More recently, open-ended, constructed response items were used to combine both approaches and advance assessment of complex science-related skills and competencies. For example, research in assessing science teachers' noticing and attention to classroom events benefitted from more open-ended response formats because teachers can present their own accounts. Then, open-ended responses are typically analyzed with some form of content analysis. However, language is noisy, ambiguous, and unsegmented and thus open-ended, constructed responses are complex to analyze. Uncovering patterns in these responses would benefit from more principled and systematic analysis tools. Consequently, computer-based methods with the help of machine learning and natural language processing were argued to be promising means to enhance assessment of noticing skills with constructed response formats. In particular, pretrained language models recently advanced the study of linguistic phenomena and thus could well advance assessment of complex constructs through constructed response items. This study examines potentials and challenges of a pretrained language model-based clustering approach to assess preservice physics teachers' attention to classroom events as elicited through open-ended written descriptions. It was examined to what extent the clustering approach could identify meaningful patterns in the constructed responses, and in what ways textual organization of the responses could be analyzed with the clusters. Preservice physics teachers (N = 75) were instructed to describe a standardized, video-recorded teaching situation in physics. The clustering approach was used to group related sentences. Results indicate that the pretrained language model-based clustering approach yields well-interpretable, specific, and robust clusters, which could be mapped to physics-specific and more general contents. Furthermore, the clusters facilitate advanced analysis of the textual organization of the constructed responses. Hence, we argue that machine learning and natural language processing provide science education researchers means to combine exploratory capabilities of qualitative research methods with the systematicity of quantitative methods.

Effective professional development programs (PDPs) rely on well-defined goals. However, recent studies on PDPs have not explored the goals from a multi-stakeholder perspective. This study identifies the most important learning goals of PDPs at science research institutions as perceived by four groups of stakeholders, namely teachers, education researchers, government representatives, and research scientists. Altogether, over 100 stakeholders from 42 countries involved in PDPs at science research institutions in Europe and North America participated in a three-round Delphi study. In the first round, the stakeholders provided their opinions on what they thought the learning goals of PDPs should be through an open-ended questionnaire. In the second and third rounds, the stakeholders assessed the importance of the learning goals that emerged from the first round by rating and ranking them, respectively. The outcome of the study is a hierarchical list of the ten most important learning goals of PDPs at particle physics laboratories. The stakeholders identified enhancing teachers' knowledge of scientific concepts and models and enhancing their knowledge of the curricula as the most important learning goals. Furthermore, the results show strong agreement between all the stakeholder groups regarding the defined learning goals. Indeed, all groups ranked the learning goals by their perceived importance almost identically. These outcomes could help policymakers establish more specific policies for PDPs. Additionally, they provide PDP practitioners at science research institutions with a solid base for future research and planning endeavors.

Reflecting in written form on one's teaching enactments has been considered a facilitator for teachers' professional growth in university-based preservice teacher education. Writing a structured reflection can be facilitated through external feedback. However, researchers noted that feedback in preservice teacher education often relies on holistic, rather than more content-based, analytic feedback because educators oftentimes lack resources (e.g., time) to provide more analytic feedback. To overcome this impediment to feedback for written reflection, advances in computer technology can be of use. Hence, this study sought to utilize techniques of natural language processing and machine learning to train a computer-based classifier that classifies preservice physics teachers' written reflections on their teaching enactments in a German university teacher education program. To do so, a reflection model was adapted to physics education. It was then tested to what extent the computer-based classifier could accurately classify the elements of the reflection model in segments of preservice physics teachers' written reflections. Multinomial logistic regression using word count as a predictor was found to yield acceptable average human-computer agreement (F1-score on held-out test dataset of 0.56) so that it might fuel further development towards an automated feedback tool that supplements existing holistic feedback for written reflections with data-based, analytic feedback.

The process of reflection is assumed to be important for developing professional knowledge through practical experience in science teaching. However, this claim requires more evidence, based on a clear definition of reflection. The main goal of the present study is to explore how reflection skills influence the development of professional knowledge gained through teaching experience. Before and after a five-month field experience, we have measured pre-service physics teachers' professional knowledge and reflection skills (N = 94; 133 cases pre and post from four German universities). We also collected data for learning opportunities during the field experience (e.g. the number of taught lessons). The present study uses a novel standardized digital simulation of collaborative oral reflections to measure reflection skills (performance assessment), as a way of increasing validity compared to self-reports. The data have been analyzed using path analysis. The main results show that reflection skills before a field experience impact the development of content knowledge ([beta] = 0.231*) and pedagogical knowledge ([beta] = 0.354**) during the field experience. Regarding the learning opportunities during the field experiences, we develop the following evidence-based post-hoc hypothesis: the more pre-service science teachers are enculturated into a community of practical teachers, the less (academic) content knowledge and pedagogical content knowledge they acquire during a field experience. Consequences for science teacher education will be discussed.

Professional knowledge is an important source of science teachers' actions in the classroom (e.g., personal professional content knowledge [pedagogical content knowledge, PCK] is the source of enacted PCK in the refined consensus model [RCM] for PCK). However, the evidence for this claim is ambiguous at best. This study applied a cross-lagged panel design to examine the relationship between professional knowledge and actions in one particular instructional situation: explaining physics. Pre- and post a field experience (one semester), 47 preservice physics teachers from four different universities were tested for their content knowledge (CK), PCK, pedagogical knowledge (PK), and action-related skills in explaining physics. The study showed that joint professional knowledge (the weighted sum of CK, PCK, and PK scores) at the beginning of the field experience impacted the development of explaining skills during the field experience ([beta]= 0.38**). We interpret this as a particular relationship between professional knowledge and science teachers' action-related skills (enacted PCK): professional knowledge is necessary for the development of explaining skills. That is evidence that personal PCK affects enacted PCK. In addition, field experiences are often supposed to bridge the theory-practice gap by transforming professional knowledge into instructional practice. Our results suggest that for field experiences to be effective, preservice teachers should start with profound professional knowledge.

Pre-service physics teachers often do not recognise the relevance for their future career in their university content knowledge courses. A lower perceived relevance can, however, have a negative effect on their motivation and on their academic success. Several intervention studies have been undertaken with the goal to increase this perceived relevance. A previous study shows that conceptual physics problems used in university physics courses are perceived by pre-service physics teachers as more relevant for their future career than regular, quantitative problems. It is however not clear, what the students' meaning of the construct 'relevance' is: what makes a problem more relevant to them than another problem? To answer this question, N = 7 pre-service teachers were interviewed using the repertory grid technique, based on the personal construct theory. Nine physics problems were discussed with regards to their perceived relevance and with regards to problem properties that distinguish these problems from each other. We are able to identify six problem properties that have a positive influence on the perceived relevance. Physics problems that are based on these properties should therefore potentially have a higher perceived relevance, which can have a positive effect on the motivation of the pre-service teachers who solve these problems.

The structure and definition of professional knowledge is a continuing focus of science education research. In 2012, a pedagogical content knowledge (PCK) summit was held and it suggested a model of professional knowledge and skill including PCK, which was later often called the Consensus Model (Gess-Newsome, 2015. A model of teacher professional knowledge and skill including PCK: Results of the thinking from the PCK summit. In A. Berry, P. J. Friedrichsen, & J. Loughran (Eds.), "Teaching and learning in science series. Re-examining pedagogical content knowledge in science education" (1st ed., pp. 28-42). New York, NY: Routledge). The Consensus Model proposes a potential powerful framework for the relations among teachers' different professional knowledge bases, but to date it has neither been investigated empirically nor systematically. In this study, we investigated the relationships suggested by the Consensus Model among different aspects of teachers' knowledge and skill. A sample of 35 physics teachers and their classes participated in the investigation; both teachers and their students in these classes took paper-and-pencil tests. Furthermore, a lesson taught by each of the teachers was videotaped and analysed. The video analysis focused on the interconnectedness of the content structure of the lesson as representation of the in-class actions of the teachers. The interconnectedness is understood as a direct result of the application of professional knowledge of the teachers to their teaching. The teachers' knowledge showed no significant influence on the interconnectedness of the lesson content structure. However, the results confirmed the influence of interconnectedness and certain aspects of professional knowledge on students' outcomes. Therefore, interconnectedness of content structure could be verified as one indicator of teachers' instructional quality.

Over the last decades, the percentage of the age group choosing to pursue university studies has increased significantly across the world. At the same time, there are university teachers who believe that the standards have fallen. There is little research on whether students nowadays demonstrate knowledge or abilities similar to that of the preceding cohorts. However, in times of educational expansion, empirical evidence on student test performance is extremely helpful in evaluating how well educational systems cope with the increasing numbers of students. In this study, we compared a sample of 2322 physics freshmen from 2013 with another sample of 2718 physics freshmen from 1978 at universities in Germany with regard to their physics knowledge based on their results in the same entrance test. Previous results on mathematics knowledge and abilities in the same sample of students indicated that there was no severe decline in their average achievement. This paper compares the physics knowledge of the same two samples of students. Contrary to their mathematics results, their physics results showed a substantial decrease in physics knowledge as measured by the test.

Teachers' professional knowledge is assumed to be a key variable for effective teaching. As teacher education has the goal to enhance professional knowledge of current and future teachers, this knowledge should be described and assessed. Nevertheless, only a limited number of studies quantitatively measures physics teachers' professional knowledge. The study reported in this paper was part of a bigger project with the broader goal of understanding teacher professional knowledge. We designed a test instrument to assess the professional knowledge of physics teachers (N = 186) in the dimensions of "content knowledge" (CK), "pedagogical content knowledge" (PCK), and "pedagogical knowledge" (PK). A model describing the relationships between these three dimensions of professional knowledge was created to inform the design of the tests used to measure CK, PCK, and PK. In this paper, we describe the model with particular emphasis on the PCK part, and the subsequent PCK test development and its implementation in detail. We report different approaches to evaluate the PCK test, including the description of content validity, the examination of the internal structure of professional knowledge, and the analysis of construct validity by testing teachers across different school subjects, teachers from different school types, pre-service teachers, and physicists. Our findings demonstrate that our PCK test results could distinguish physics teachers from the other groups tested. The PCK test results could not be explained by teachers' CK or PK, cognitive abilities, computational skills, or science knowledge.