Search

Teaching is a complex profession and feedback on teacher practices is needed for teachers' development. Many instruments are available to measure teacher practices, but little is known about their quality. This systematic review aimed to gain insight into the quality of instruments available to measure teacher practices. A systematic review based on ERIC, PsychINFO, and Web of Science databases (2000-2016) was conducted. In total 96 journal articles were included, describing 127 measurement instruments. The instruments were mainly self-evaluation questionnaires, focussing on activities during teaching. Most evidence was provided for the validity and impact of the instruments. Evidence for utility was generally low. Questionnaire data gathered from students seems to best meet the quality requirements. It is discussed to evaluate teachers with different measurement instruments to provide a rich perspective of their practices.

Laboratory e‐learning support tools can assist students' learning while preparing for laboratory classes. To successfully work in such virtual experimental environments (VEEs) outside class, students require self‐regulated learning (SRL) skills. A deeper understanding of the continuous reciprocal interactions between SRL, satisfaction, and online engagement is needed to develop more effective online learning experiences. This study therefore aimed to explore the interconnection between students' satisfaction with, effort/importance and engagement in an exemplary VEE, and to relate this to their perceived SRL and learning outcomes. Based on surveys in 79 university students, SRL was related to VEE engagement, effort/importance, and satisfaction. VEE engagement and satisfaction were not related to learning outcomes, while SRL and effort were. Students with different SRL also tended to interact differently with the VEE and experienced differing degrees of procedural and feedback support by the e‐environment. We conclude that, for optimal learning experience and outcomes, students' effort regulation and SRL need to be supported while interacting with the VEE, preferably by interventions that integrate personalized and adaptive features. This study has implications for designing and optimizing VEEs and indicates that future research should focus on VEEs taking students' SRL and effort regulation into account to support individual learners effectively. [ABSTRACT FROM AUTHOR]

The combination of two methodological resources-natural user interface and multimodal learning analytics-is creating opportunities for educational researchers to empirically evaluate theoretical models accounting for the emergence of concepts from situated sensorimotor activity. Seventy-six participants (9-14 years old) solved tablet-based presymbolic manipulation tasks designed to foster grounded meanings for the mathematical concept of proportional equivalence. Data gathered in task-based semi-structured clinical interviews included action logging, eye-gaze tracking, and videography. Analysis of these data indicates that successful task performance coincided with spontaneous emergence of stable dynamical gaze path patterns soon followed by multimodal articulation of strategy. Significantly, gaze patterns included unmanipulated, non-salient screen locations. We present cumulative evidence that these gaze patterns served as "attentional anchors" mediating participants' problem solving. By way of further contextualizing our claim, we also present case studies from the various experimental conditions. We interpret the findings as enabling us to revisit, support, refine, and perhaps elaborate on seminal claims from Piaget's theory of genetic epistemology and in particular his insistence on the role of situated motor-action coordination in the process of reflective abstraction.

Eye tracking research has been conducted for decades to gain understanding of visual diagnosis such as in radiology. For educational purposes, it is important to identify visual search patterns that are related to high perceptual performance and to identify effective teaching strategies. This review of eye-tracking literature in the radiology domain aims to identify visual search patterns associated with high perceptual performance. Databases PubMed, EMBASE, ERIC, PsycINFO, Scopus and Web of Science were searched using "visual perception" OR "eye tracking" AND "radiology" and synonyms. Two authors independently screened search results and included eye tracking studies concerning visual skills in radiology published between January 1, 1994 and July 31, 2015. Two authors independently assessed study quality with the Medical Education Research Study Quality Instrument, and extracted study data with respect to design, participant and task characteristics, and variables. A thematic analysis was conducted to extract and arrange study results, and a textual narrative synthesis was applied for data integration and interpretation. The search resulted in 22 relevant full-text articles. Thematic analysis resulted in six themes that informed the relation between visual search and level of expertise: (1) time on task, (2) eye movement characteristics of experts, (3) differences in visual attention, (4) visual search patterns, (5) search patterns in cross sectional stack imaging, and (6) teaching visual search strategies. Expert search was found to be characterized by a global-focal search pattern, which represents an initial global impression, followed by a detailed, focal search-to-find mode. Specific task-related search patterns, like drilling through CT scans and systematic search in chest X-rays, were found to be related to high expert levels. One study investigated teaching of visual search strategies, and did not find a significant effect on perceptual performance. Eye tracking literature in radiology indicates several search patterns are related to high levels of expertise, but teaching novices to search as an expert may not be effective. Experimental research is needed to find out which search strategies can improve image perception in learners.

Formula scoring (FS) is the use of a don't know option (DKO) with subtraction of points for wrong answers. Its effect on construct validity and reliability of progress test scores, is subject of discussion. Choosing a DKO may not only be affected by knowledge level, but also by risk taking tendency, and may thus introduce construct-irrelevant variance into the knowledge measurement. On the other hand, FS may result in more reliable test scores. To evaluate the impact of FS on construct validity and reliability of progress test scores, a progress test for radiology residents was divided into two tests of 100 parallel items (A and B). Each test had a FS and a number-right (NR) version, A-FS, B-FS, A-NR, and B-NR. Participants (337) were randomly divided into two groups. One group took test A-FS followed by B-NR, and the second group test B-FS followed by A-NR. Evidence for impaired construct validity was sought in a hierarchical regression analysis by investigating how much of the participants' FS-score variance was explained by the DKO-score, compared to the contribution of the knowledge level (NR-score), while controlling for Group, Gender, and Training length. Cronbach's alpha was used to estimate NR and FS-score reliability per year group. NR score was found to explain 27 % of the variance of FS [F(1,332) = 219.2, p < 0.0005], DKO-score, and the interaction of DKO and Gender were found to explain 8 % [F(2,330) = 41.5, p < 0.0005], and the interaction of DKO and NR 1.6 % [F(1,329) = 16.6, p < 0.0005], supporting our hypothesis that FS introduces construct-irrelevant variance into the knowledge measurement. However, NR-scores showed considerably lower reliabilities than FS-scores (mean year-test group Cronbach's alphas were 0.62 and 0.74, respectively). Decisions about FS with progress tests should be a careful trade-off between systematic and random measurement error.

The knowledge and skills that are required for radiological image interpretation are not well documented, even though medical imaging is gaining importance. This study aims to develop a comprehensive framework of knowledge and skills, required for two-dimensional and multiplanar image interpretation in radiology. A mixed-method study approach was applied. First, a literature search was performed to identify knowledge and skills that are important for image interpretation. Three databases, PubMed, PsycINFO and Embase, were searched for studies using synonyms of "image interpretation skills" or "visual expertise" combined with synonyms of "radiology." Empirical or review studies concerning knowledge and skills for medical image interpretation were included and relevant knowledge and skill items were extracted. Second, a preliminary framework was built and discussed with nine selective experts in individual semi-structured interviews. The expert team consisted of four radiologists, one radiology resident, two education scientists, one cognitive psychologist and one neuropsychologist. The framework was optimised based on the experts comments. Finally, the framework was applied to empirical data, derived from verbal protocols of ten clerks interpreting two-dimensional and multiplanar radiological images. In consensus meetings adjustments were made to resolve discrepancies of the framework with the verbal protocol data. We designed a framework with three main components of image interpretation: perception, analysis and synthesis. The literature study provided four knowledge and twelve skill items. As a result of the expert interviews, one skill item was added and formulations of existing items were adjusted. The think-aloud experiment showed that all knowledge items and three of the skill items were applied within all three main components of the image interpretation process. The remaining framework items were apparent only within one of the main components. After combining two knowledge items, we finally identified three knowledge items and thirteen skills, essential for image interpretation by trainees. The framework can serve as a guideline for education and assessment of two- and three-dimensional image interpretation. Further validation of the framework in larger study groups with different levels of expertise is needed.

There is a need for valid methods to assess the readiness for clinical practice of medical graduates. This study evaluates the validity of Utrecht Hamburg Trainee Responsibility for Unfamiliar Situations Test (UHTRUST), an authentic simulation procedure to assess whether medical trainees are ready to be entrusted with unfamiliar clinical tasks near the highest level of Miller's pyramid. This assessment, in which candidates were judged by clinicians, nurses and standardized patients, addresses the question: can this trainee be trusted with unfamiliar clinical tasks? The aim of this paper is to provide a validity argument for this assessment procedure. We collected data from various sources during preparation and administration of a UHTRUST-assessment. In total, 60 candidates (30 from the Netherlands and 30 from Germany) participated. To provide a validity argument for the UHTRUST-assessment, we followed Kane's argument-based approach for validation. All available data were used to design a coherent and plausible argument. Considerable data was collected during the development of the assessment procedure. In addition, a generalizability study was conducted to evaluate the reliability of the scores given by assessors and to determine the proportion of variance accounted by candidates and assessors. It was found that most of Kane's validity assumptions were defendable with accurate and often parallel lines of backing. UHTRUST can be used to compare the readiness for clinical practice of medical graduates. Further exploration of the procedures for entrustment decisions is recommended.

Several studies concluded that deep reflection is infrequently reached in student portfolios. An explanation for these disappointing conclusions might be that motivation for portfolio reflection determines the quality of reflection. This study aimed to examine the relationship between motivation for using digital portfolios and reflection. Participants were 156 eleventh-grade students in secondary education, whose motivation for composing a digital portfolio was measured by the motivation part of the Motivated Strategies for Learning Questionnaire. Portfolios of 37 of the 156 students were examined in terms of the amount and nature of reflection by means of a coding scheme based on Mezirow's model of transformative learning. On average, one-fifth (19.5%) of the paragraphs in a portfolio contained reflection, and paragraphs with deep reflection were hardly found (0.8%). It was concluded that motivation for composing a portfolio was fair, but not related to the amount and nature of reflection. This exploratory study gives rise to further research into factors that might influence the quality of portfolio reflection. (Contains 5 tables.)

Students in competence-based senior secondary vocational education are expected to actively construct personal knowledge during school-based learning and workplace learning. This study introduces the construct "personal professional theory" (PPT) to monitor and analyse students' personal knowledge development. It aims to investigate the development of the content and nature of students' PPTs and their perceptions of learning processes in vocational education during school-based learning and workplace learning. One group of students (n=20) in the domain of Information and Communication Technologies (ICT) is followed during one course of 40 weeks in which they alternately participate in school-based learning and workplace learning. A longitudinal design (t=4) including semi-structured concept maps is used to reveal the content and nature of students' PPTs (n=240). Semi-structured interviews (n=10) and log files (n=39) are used to explore students' perceptions of their learning process. No significant differences are found between the four measurements concerning the content and nature of students' PPTs. However, the actual content of students' PPTs shows a relatively unsteady and changeable pattern. Additionally, students perceive workplace learning as an essential process for the development of their PPT. It is concluded that the development of students' PPTs involves knowledge compilation as well as thematic turbulence. (Contains 5 tables and 2 figures.)

Students in competence-based vocational education are expected to actively construct a personal professional theory, in which they integrate different types of knowledge and beliefs. Students' personal professional theories are seen as an important learning outcome of competence-based vocational education. However, it is unknown how personal professional theories can be measured. This study focused on measuring the content and nature of students' personal professional theories using a multi-method triangulation approach, in which 16 students in the domain of Social Work constructed a concept map, an interview and a self-report. The results show that the relatively structured methods (i.e., interviews and concept maps) reveal more insight into students' personal professional theories than less structured methods (i.e., self-reports). It is concluded that both structure as well as adequate prompts are important in the process of explicating personal professional theories. (Contains 6 tables, 2 figures and 1 footnote.)

Competence-based vocational education is based on a constructivist learning paradigm, where the development of students' personal professional knowledge is emphasised. However, there is a lack of insight into how students construct their own professional knowledge and what the content and nature of personal professional knowledge is. This article elaborates the concept Personal Professional Theory (PPT): a personal knowledge base in which professional knowledge and beliefs are internalised. PPTs are built upon a combination of declarative and procedural knowledge and are stored in the long-term memory. Furthermore, knowledge in PPTs refers to compiled knowledge which can be specified and applied to different professional situations. These PPTs develop through an interrelated process of internalisation and socialisation, in which students grow into the existing body of shared knowledge and collective norms, values and beliefs of professionals of an occupational domain. It is concluded that the elaboration of PPTs can be used to stimulate and monitor personal professional knowledge development in vocational education, but further research into the content and nature of PPTs is required.

Developing and using a design for teacher portfolio assessment is a complex process including several components: the domain to be assessed (the teacher competences), the content standards or criteria, the portfolio format, the completion of the format (by teachers) with content, and the scoring of the portfolios (by raters). For a portfolio assessment to be valid, these components should be systematically linked. Using a chain model of the assessment process two links were verified: the link between content standards and portfolio format, and the link between content standards and raters' scoring. Eight experts evaluated an initial portfolio assessment design. They greatly supported the design, including the link between standards and format. Subsequently, using the format 18 teachers developed a portfolio. Six raters scored these portfolios according to eight content standards. Their scoring was substantially based on the content standards. Implications of the results and suggestions for research are discussed. (Contains 2 figures and 2 tables.)