Search

National assessments yield a description of the proficiency level in a domain while accounting for differences between tasks. For instance, in writing assessments the level of proficiency is typically evaluated with a variety of topics and multiple tasks. This enables generalizations from specific tasks to a domain. In (quasi-)experimental research, however, writing skills are often evaluated with a single task. Yet, conclusions about the effectiveness of the treatment are formulated on the level of the domain, which is, euphemistically put, quite a stretch. Although conclusions drawn about the effect of the treatment are specific to the task administered, they are often generalized to the domain without any form of reservation. This raises the question whether we can use the results of national assessments about differences between tasks in the analyses of experimental studies. In this paper, we demonstrate how the information of a baseline data set can be used as a kind of control condition in the analysis of an experimental study. ispartof: Journal of Writing Research vol:14 issue:3 pages:447-469 status: published more...

In Dutch L1 classrooms, style in non-fictional genres is typically taught by means of normative exercises in which students are tasked to identify stylistic lapses. Not much is known about the effectiveness of such exercises when teaching style. Unknown factors include what kinds of stylistic shortcomings are found in Dutch students’ writing, and how the occurrence of certain stylistic lapses relates to writing quality. The current study empirically explores these scarcely investigated issues. Teachers rated 125 argumentative texts written by tenth-grade pre-university students by means of comparative judgement. Additionally, these texts were manually analyzed to investigate the occurrence of stylistic lapses, taking into account stylistic lapses that are common in text books (‘standard category’) and other types of style related errors (‘other category’). Multilevel regression analyses revealed that only one of the stylistic lapses from the standard category negatively influenced text quality as evaluated by teachers, namely the use of detached phrases. In the other category, only mistakes in question marks negatively predicted text quality. A final model including those two predictors explained 11.1% of the variance in text quality. The article discusses the implications of these findings for non-fictional style education, suggesting that it might need to be refocused. more...

In Dutch L1 classrooms, style in non-fictional genres is typically taught by means of normative exercises in which students are tasked to identify stylistic lapses. Not much is known about the effectiveness of such exercises when teaching style. Unknown factors include what kinds of stylistic shortcomings are found in Dutch students' writing, and how the occurrence of certain stylistic lapses relates to writing quality. The current study empirically explores these scarcely investigated issues. Teachers rated 125 argumentative texts written by tenth-grade pre-university students by means of comparative judgement. Additionally, these texts were manually analyzed to investigate the occurrence of stylistic lapses, taking into account stylistic lapses that are common in text books ('standard category') and other types of style related errors ('other category'). Multilevel regression analyses revealed that only one of the stylistic lapses from the standard category negatively influenced text quality as evaluated by teachers, namely the use of detached phrases. In the other category, only mistakes in question marks negatively predicted text quality. A final model including those two predictors explained 11.1% of the variance in text quality. The article discusses the implications of these findings for non-fictional style education, suggesting that it might need to be refocused. more...

Janneke Stuulen, Renske Bouwer en Huub van den Bergh onderzochten hoe leerlingen peerfeedback gaven met verschillende beoordelingsmethoden, en hoe zij deze peerfeedback gebruikten voor het reviseren van hun eigen tekst. De ene groep leerlingen gaf feedback aan de hand van paarsgewijs vergelijken, de andere groep gaf feedback met behulp van een analytisch beoordelingsformulier. De resultaten laten zien dat leerlingen uit de conditie met het beoordelingsformulier meer feedback gaven op inhoudelijke aspecten dan leerlingen uit de conditie paarsgewijs vergelijken, en dat zij deze feedback ook vaker verwerkten. Er is echter geen effect van peerfeedbackconditie op tekstkwaliteit: in beide condities was de tekstkwaliteit na het herschrijven in dezelfde mate verbeterd. more...

Janneke Stuulen, Renske Bouwer en Huub van den Bergh onderzochten hoe leerlingen peerfeedback gaven met verschillende beoordelingsmethoden, en hoe zij deze peerfeedback gebruikten voor het reviseren van hun eigen tekst. De ene groep leerlingen gaf feedback aan de hand van paarsgewijs vergelijken, de andere groep gaf feedback met behulp van een analytisch beoordelingsformulier. De resultaten laten zien dat leerlingen uit de conditie met het beoordelingsformulier meer feedback gaven op inhoudelijke aspecten dan leerlingen uit de conditie paarsgewijs vergelijken, en dat zij deze feedback ook vaker verwerkten. Er is echter geen effect van peerfeedbackconditie op tekstkwaliteit: in beide condities was de tekstkwaliteit na het herschrijven in dezelfde mate verbeterd. more...

Background: Understanding consumers’ interpretation of allergy information is crucial for effective food safety policies. We evaluated consumer understanding of allergy information on foods in controlled, experimental studies. Method: Using 18 packaged foods, we evaluated consumer understanding of information about allergens in two experiments: First, a comparison of foods with no stated allergen versus allergen as a stated ingredient versus a precautionary allergen label (PAL); second, a comparison of three common variants of PAL. In each experiment, consumers with and without self-reported food allergy were asked to estimate the risk of allergic reaction and to rate the comprehensibility of the allergen information. In the second experiment, consumers were also asked which form of PAL they preferred. Results: Risk of reaction was assessed as high and low for foods with the allergen stated as ingredient, or without any mention of allergen. However, risk assessment for PAL varied and was judged as higher by non-allergic than allergic participants (82% vs. 58%, p more...

Het gaat niet goed met het tekstbegrip van de Nederlandse middelbare scholier. Eén waarschijnlijke verklaring hiervoor is dat het Nederlandse leesonderwijs vanuit wetenschappelijk oogpunt nogal verouderd is. more...

Het gaat niet goed met het tekstbegrip van de Nederlandse middelbare scholier. Eén waarschijnlijke verklaring hiervoor is dat het Nederlandse leesonderwijs vanuit wetenschappelijk oogpunt nogal verouderd is. more...

De didactisch-inhoudelijke aanpak van leestaken in leergangen is nogal verouderd. Welke alternatieven zijn er?

Observerend leren heeft vrij breed ingang gevonden in het schrijfonderwijs in het schoolvak Nederlands. In het leesonderwijs gebeurt het via modellen op expert-niveau: een docent doet leerlingen voor hoe hij een tekst al lezend begrijpt. Modellen op peerniveau, via het leesgedrag van leerlingen, lijkt nog sporadisch te gebeuren. In onze studie hebben we bij vwo-leerlingen onderzocht in hoeverre het proces achter diep tekstbegrip effectief kan worden onderwezen via oogbewegingsfilmpjes (EMME’s) van lezende leerlingen, teneinde het tekstbegrip te bevorderen. De deelnemers aan de EMME-lessenserie geven achteraf meer van diep begrip blijk dan de leerlingen die zelf teksten lazen en vragen maakten. more...

Observerend leren heeft vrij breed ingang gevonden in het schrijfonderwijs in het schoolvak Nederlands. In het leesonderwijs gebeurt het via modellen op expert-niveau: een docent doet leerlingen voor hoe hij een tekst al lezend begrijpt. Modellen op peerniveau, via het leesgedrag van leerlingen, lijkt nog sporadisch te gebeuren. In onze studie hebben we bij vwo-leerlingen onderzocht in hoeverre het proces achter diep tekstbegrip effectief kan worden onderwezen via oogbewegingsfilmpjes (EMME’s) van lezende leerlingen, teneinde het tekstbegrip te bevorderen. De deelnemers aan de EMME-lessenserie geven achteraf meer van diep begrip blijk dan de leerlingen die zelf teksten lazen en vragen maakten. more...

De didactisch-inhoudelijke aanpak van leestaken in leergangen is nogal verouderd. Welke alternatieven zijn er?

In this study, the authors tested the effects of Tekster [Texter], a comprehensive strategy-focused writing instruction program, using a switching replication design with three measurement occassions. The program was implemented by fourth, fifth, and sixth grade teachers (N = 76) in 60 general education classrooms in the Netherlands. Students (n = 688) and teachers (n = 31) in Group 1 worked with Tekster during the first 8-week period, between the first and second measurement occasion. Students (n = 732) and teachers (n = 45) in Group 2 implemented Tekster during the second 8-week period, between the second and third measurement occasion. The intervention led to statistically significant improvements in the quality of students’ writing. The effect size for the full sample was 0.32 and 0.40 for students who students who completed all 16 Tekster lessons. Gains shown by students in Group 1 were maintained after 8 weeks. Because writing quality was assessed in 3 genres, the findings are generalizable across students, classes, and writing tasks. Taken together, the results of this study demonstrate that a strategy-focused writing instruction program, such as Tekster, can be an effective way to improve upper-elementary students’ written language skills. (PsycINFO Database Record (c) 2018 APA, all rights reserved) more...

In the present study, aspects of the measurement of writing are disentangled in order to investigate the validity of inferences made on the basis of writing performance and to describe implications for the assessment of writing. To include genre as a facet in the measurement, we obtained writing scores of 12 texts in four different genres for each participating student. Results indicate that across raters, tasks and genres, only 10% of the variance in writing scores is related to individual writing skill. In order to draw conclusions about writing proficiency, students should therefore write at least three different texts in each of four genres rated by at least two raters. Moreover, when writing scores are obtained through highly similar tasks, generalization across genres is not warranted. Inferences based on text quality scores should, in this case, be limited to genre-specific writing. These findings replicate the large task variance in writing assessment as consistently found in earlier research and emphasize the effect of genre on the generalizability of writing scores. This research has important implications for writing research and writing education, in which writing proficiency is quite often assessed by only one task rated by one rater. more...

Rationale, aims and objectives We examined the effect of two interventions on both the reliability and validity of regulatory judgments: adjusting the regulatory instrument and attending a consensus meeting. Method We adjusted the regulatory instrument. With a randomized controlled trial (RCT) we examined the effect of the adjustments we made to the instrument. In the consensus meeting inspectors discussed cases and had to reach consensus about the order of the cases. We used a before and after case study to assess the effect of the consensus meeting. We compared the judgments assigned in the RCT with the unadjusted instrument with the judgments assigned with the unadjusted instrument after the consensus meeting. Moreover we explored the effect of increasing the number of inspectors per regulatory visit based on the estimates of the two interventions. Results The consensus meeting improved the agreement between inspectors; the variance between inspectors was smallest (0.03) and the reliability coefficient was highest (0.59). Validity is assessed by examining the relation between the assigned judgments and the corporate standard and expressed by a correlation coefficient. This coefficient was highest after the consensus meeting (0.48). Adjustment of the instrument did not increase reliability and validity coefficients. Conclusions Participating in a consensus meeting improved reliability and validity. Increasing the number of inspectors resulted in both higher reliability and validity values. Organizing consensus meetings and increasing the number of inspectors per regulatory visit seem to be valuable interventions for improving regulatory judgments. more...

The present study tests the effect of ability pairing in two instructional methods in L2 collaborative revision. Two continuous indices determine a pair: individual proficiency level, distance in proficiency between pair members (heterogeneity), and the interaction between both indices. Instructional methods tested are modelling and practising. Results show that the effect of pair composition depends on instructional strategies. In the Practising condition less proficient learners profit most from a heterogeneous ability pair, whereas more proficient learners are best paired homogeneously. In the Modelling condition no effect of pair composition factors was observed. This result illustrates that Modelling is a powerful instructional method for complex learning tasks like collaborative revision in L2 as it overrides some of the grouping effects which can be found in more traditional learning conditions. more...

This paper is a longitudinal investigation of adjective use by children aged 1;8−2;8, speaking Dutch, German, French, Hebrew, and Turkish, and by their caregivers. Each adjective token in transcripts of spontaneous speech was coded for semantic class. The development of adjective use in each semantic class was analysed by means of a multilevel logistic regression. The results show that toddlers and their parents use adjectives more often as the child grows older. However, this holds only for semantic classes denoting concrete concepts, such as physical properties, colour, and size. Adjectives denoting more abstract properties are barely used by children and parents throughout the first year of adjective acquisition. The correlations between adjective frequencies in child speech and child-directed speech are very strong at the beginning, but decrease with time as the child develops independent adjective use. The composition of early adjective lexicons is very similar in the five languages under study. more...

The paper examines the role of contrast in the acquisition of adjectives. It presents the results of research conducted on seven languages.

DOEL Onderzoek naar de variatie in oordelen tussen inspecteurs van de Inspectie voor de Gezondheidszorg (IGZ) in het systeem van gefaseerd toezicht op de verpleeghuiszorg en naar de relatie tussen het type oordeel en de aan- of afwezigheid van onderbouwing daarbij. OPZET Descriptief, kwantitatief, retrospectief. METHODE In totaal werden 4914 oordelen met bijbehorende onderbouwingen van 26 inspecteurs uit 182 toezichtrapporten geanalyseerd. De oordelen waren in 2005 en 2006 gegeven op 25 criteria voor verantwoorde zorg in verpleeghuizen. Om de inspecteurs en hun oordelen over verschillende instellingen te kunnen vergelijken, werd met covariantieanalyse statistisch gecorrigeerd voor instellingskenmerken. RESULTATEN Er waren statistisch significante verschillen in beoordeling tussen de inspecteurs. Na correctie voor instellingskenmerken bleven voor 14 van de 25 criteria significante verschillen bestaan. Tevens bleek de aanwezigheid van een onderbouwing bij een oordeel zowel afhankelijk te zijn van de inspecteur als van het gegeven oordeel. CONCLUSIE Beoordelaarsverschillen tussen inspecteurs spelen een rol in de tweede fase van het gefaseerd toezicht op de verpleeghuizen. De IGZ zal resultaten van dit onderzoek gebruiken om haar toezicht verder te ontwikkelen. more...

Three groups of typically-developing 6th grade students (total N = 62) each completed strategy-focused writing training. Using a combined lagged-group and cross-panel design we assessed the effectiveness of a sequence of four different instructional components: observation of and group reflection on a mastery model, direct (declarative) instruction, peer feedback and solo practice. Cumulative effects on written product and writing process were assessed at baseline and after each component. Findings supported the effectiveness of strategy-focused intervention: All three groups showed gains, relative to controls, in the quality of their written products assessed by both holistic and text-analytic measures, and a more structured and goal-focused planning processes. These effects were associated almost exclusively with the modelling and reflection component. Improved performance was sustained through other instructional components but there was no strong evidence that they provided additional benefit. This finding was replicated in all three groups, and across two different text-types. more...

Rationale, aims and objectives We examined the effect of two interventions on both the reliability and validity of regulatory judgments: adjusting the regulatory instrument and attending a consensus meeting. Method We adjusted the regulatory instrument. With a randomized controlled trial (RCT) we examined the effect of the adjustments we made to the instrument. In the consensus meeting inspectors discussed cases and had to reach consensus about the order of the cases. We used a before and after case study to assess the effect of the consensus meeting. We compared the judgments assigned in the RCT with the unadjusted instrument with the judgments assigned with the unadjusted instrument after the consensus meeting. Moreover we explored the effect of increasing the number of inspectors per regulatory visit based on the estimates of the two interventions. Results The consensus meeting improved the agreement between inspectors; the variance between inspectors was smallest (0.03) and the reliability coefficient was highest (0.59). Validity is assessed by examining the relation between the assigned judgments and the corporate standard and expressed by a correlation coefficient. This coefficient was highest after the consensus meeting (0.48). Adjustment of the instrument did not increase reliability and validity coefficients. Conclusions Participating in a consensus meeting improved reliability and validity. Increasing the number of inspectors resulted in both higher reliability and validity values. Organizing consensus meetings and increasing the number of inspectors per regulatory visit seem to be valuable interventions for improving regulatory judgments more...

Many primary school students in the Netherlands struggle with reading comprehension, especially when it comes to higher-order comprehension skills, such as summarizing. Scientific literature has suggested various ways to foster text comprehension, for example by providing explicit instruction about text structures (e.g., cause-effect, chronology), as this might foster the construction of a solid mental representation of the text: a coherent situation model (e.g., see Meyer & Ray, 2011). A text structure is the underlying skeleton of a text that organizes all the information. Text structure can be defined as the organization of ideas, the relationship among the ideas and the vocabulary used to convey meaning to the reader” (Pyle et al., 2017:1; see also Armbruster, 2004). What if we would teach children in primary school about text structures? Our meta-analysis showed that explicit instruction about narrative and informative text structures yields positive immediate effects on students' reading comprehension, and that effects vary across outcome measures: recall tasks (g = 0.37), summarization (g = 0.57), reading comprehension tests (g = 0.25), and text structure knowledge tests (g = 0.38) for students in the upper elementary grades. The moderator analysis revealed that content-related features such as a focus on paragraph-level structure, active construction of graphic organizers, and rule-based summarization techniques moderated the effectiveness, but these effects also varied with outcome measure. The mixed-methods study examined the state-of-affairs analysis of the current Dutch primary school reading curriculum for grades 4 and 5. The data are based on a materials analysis of reading comprehension lessons (n = 80) in eight textbooks for grades 4 and 5, supplemented with semi-structured teacher interviews (n = 29) and lesson observations (n = 11). Main findings were (1) a lack of alignment between lesson goals, theory, and assignments, (2) a strong focus on practicing strategies, (3) limited declarative knowledge about strategies and text structure, (4) little opportunities for selfregulated strategy application, and (5) strong emphasis on individual question answering. Therefore, a lesson series on text structure was developed in a design-based research in close collaboration with four teachers. Based on data from lesson artefacts, logbooks, panel interviews, and lesson observations, this study revealed several difficulties related to the design principles and their implementation in practice, and identified several areas in which teachers' pedagogical content knowledge about reading comprehension could be strenghthened. This lesson series was then tested for its effectiveness among 201 students in a switching-panels replication design with three measurement occasions. There were positive effects on writing skills for all students, but the effects on reading, summarizing, and metacognitive knowledge varied between intervention groups. This variability in effects raised questions about the exact lesson implementation. In the final study, we closely monitored three teachers and their students in another implementation of the intervention. Implementation fidelity scores varied across teachers and lesson phases, with the lowest scores for collaborative learning (51%) and the highest for explicit instruction (91%).This study generated useful insights for the professionalization of teachers and reinforcement of teaching materials for reading comprehension instruction. more...

Many primary school students in the Netherlands struggle with reading comprehension, especially when it comes to higher-order comprehension skills, such as summarizing. Scientific literature has suggested various ways to foster text comprehension, for example by providing explicit instruction about text structures (e.g., cause-effect, chronology), as this might foster the construction of a solid mental representation of the text: a coherent situation model (e.g., see Meyer & Ray, 2011). A text structure is the underlying skeleton of a text that organizes all the information. Text structure can be defined as the organization of ideas, the relationship among the ideas and the vocabulary used to convey meaning to the reader” (Pyle et al., 2017:1; see also Armbruster, 2004). What if we would teach children in primary school about text structures? Our meta-analysis showed that explicit instruction about narrative and informative text structures yields positive immediate effects on students' reading comprehension, and that effects vary across outcome measures: recall tasks (g = 0.37), summarization (g = 0.57), reading comprehension tests (g = 0.25), and text structure knowledge tests (g = 0.38) for students in the upper elementary grades. The moderator analysis revealed that content-related features such as a focus on paragraph-level structure, active construction of graphic organizers, and rule-based summarization techniques moderated the effectiveness, but these effects also varied with outcome measure. The mixed-methods study examined the state-of-affairs analysis of the current Dutch primary school reading curriculum for grades 4 and 5. The data are based on a materials analysis of reading comprehension lessons (n = 80) in eight textbooks for grades 4 and 5, supplemented with semi-structured teacher interviews (n = 29) and lesson observations (n = 11). Main findings were (1) a lack of alignment between lesson goals, theory, and assignments, (2) a strong focus on practicing strategies, (3) limited declarative knowledge about strategies and text structure, (4) little opportunities for selfregulated strategy application, and (5) strong emphasis on individual question answering. Therefore, a lesson series on text structure was developed in a design-based research in close collaboration with four teachers. Based on data from lesson artefacts, logbooks, panel interviews, and lesson observations, this study revealed several difficulties related to the design principles and their implementation in practice, and identified several areas in which teachers' pedagogical content knowledge about reading comprehension could be strenghthened. This lesson series was then tested for its effectiveness among 201 students in a switching-panels replication design with three measurement occasions. There were positive effects on writing skills for all students, but the effects on reading, summarizing, and metacognitive knowledge varied between intervention groups. This variability in effects raised questions about the exact lesson implementation. In the final study, we closely monitored three teachers and their students in another implementation of the intervention. Implementation fidelity scores varied across teachers and lesson phases, with the lowest scores for collaborative learning (51%) and the highest for explicit instruction (91%).This study generated useful insights for the professionalization of teachers and reinforcement of teaching materials for reading comprehension instruction. more...

There are currently many concerns about the declining reading comprehension of Dutch pupils. According to critics, this is partly because teachers often use ‘texts-with-questions’ in reading comprehension lessons. Indeed, these assignments do not teach pupils to read independently. Teachers should use realistic, purposeful reading assignments and reflect with pupils on how to approach these assignments. In the first part of this book, reading comprehension is analysed in pre-university pupils dealing with ‘texts-with-questions’. These relatively skilled readers read superficially; pre-reading and answering questions about a text are, mostly, separate thinking processes. For a correct answer it does not matter if pupils pre-read texts intensively. It does, however, matter if they can consult texts when answering questions. As such, pupils seem to practice how to answer comprehension questions rather than how to study texts independently. In the second part, an alternative lesson series on ‘deep comprehension’ is designed, based on recent international research. This series focuses on reading independently, realistically and purposefully; pre-university pupils reflect extensively on approaching challenging reading tasks. The lesson series is effective, provided that teachers actively manage reflective discussions. Then, pre-university pupils afterwards show a better understanding of texts and self-reportedly have altered their reading behaviour. The dissertation concludes with eleven recommendations to make reading comprehension lessons for the school subject Dutch more meaningful, up to date and reflective. more...

There are currently many concerns about the declining reading comprehension of Dutch pupils. According to critics, this is partly because teachers often use ‘texts-with-questions’ in reading comprehension lessons. Indeed, these assignments do not teach pupils to read independently. Teachers should use realistic, purposeful reading assignments and reflect with pupils on how to approach these assignments. In the first part of this book, reading comprehension is analysed in pre-university pupils dealing with ‘texts-with-questions’. These relatively skilled readers read superficially; pre-reading and answering questions about a text are, mostly, separate thinking processes. For a correct answer it does not matter if pupils pre-read texts intensively. It does, however, matter if they can consult texts when answering questions. As such, pupils seem to practice how to answer comprehension questions rather than how to study texts independently. In the second part, an alternative lesson series on ‘deep comprehension’ is designed, based on recent international research. This series focuses on reading independently, realistically and purposefully; pre-university pupils reflect extensively on approaching challenging reading tasks. The lesson series is effective, provided that teachers actively manage reflective discussions. Then, pre-university pupils afterwards show a better understanding of texts and self-reportedly have altered their reading behaviour. The dissertation concludes with eleven recommendations to make reading comprehension lessons for the school subject Dutch more meaningful, up to date and reflective. more...

Het schrijven van een synthesetekst - een tekst waarin informatie uit verschillende bronnen geïntegreerd wordt - maakt deel uit van het curriculum in het Nederlandse vwo-onderwijs. Deze studie bestaat uit een nationale peiling naar de synthesevaardigheid in de drie hoogste leerjaren van het vwo. Het doel van deze studie was om drie aspecten van syntheseschrijven in kaart te brengen: tekstkwaliteit, schrijfproces en leerlingperspectief op schrijven. Een representatieve steekproef van 658 leerlingen nam deel; elke leerling schreef vier teksten. Teksten werden beoordeeld met behulp van tekstschalen met benchmarks; het schrijfproces werd geregistreerd met keystroke logging; en de perspectieven van de leerlingen op schrijven werden gemeten met een vragenlijst. Via multilevel analyses gingen we het effect van leerjaar, geslacht en tekstgenre (argumentatieve/ informatieve synthese) na op tekstkwaliteit en schrijfproces, en het effect van leerjaar en geslacht op de perspectieven. Deze nationale peiling is een beschrijvend onderzoek dat inzicht biedt in de huidige stand van zaken omtrent syntheseschrijven: hoe presteren leerlingen op synthesetaken?, hoe schrijven ze syntheseteksten?, en wat zijn hun perspectieven op het schrijven van een synthesetekst? Bovendien dient deze studie ook als baseline voor toekomstig onderzoek. Abstract: Writing a synthesis text - a text that integrates information from different sources - forms part of the educational curriculum of Dutch secondary education. Representative information on students' synthesis writing skills is currently missing. Therefore, we carried out a national survey on synthesis writing in the three highest grades of preuniversity education in the Netherlands. The aim was to map synthesis writing on three aspects: text quality, writing process and students' perspectives on writing. A large and representative sample of 658 students participated. Each participant wrote several synthesis texts. Text quality was rated with benchmark texts: writing processes were registered with keystroke logging software and a questionnaire measured students' perspectives on writing. Multilevel analyses were used to identify the effect of grade. gender and genre (argumentative/ informative synthesis) on text quality and writing process. and the effect of grade and gender on perspectives. This national survey is a descriptive study providing information on the current state of synthesis writing of Dutch students: how well do students perform on synthesis tasks?, how do they write synthesis tasks?, and what are their perspectives on synthesis writing? Moreover, this study serves as a baseline for future research. more...

There are many concerns in the Netherlands about the level of writing skills of students in higher general secondary education, Dutch havo. That is why five studies were conducted in order to answer our research question: is a writing course that combines several evidence based learning activities effective in the upper havo levels? The studies were intended to tie in with the results of previous research in writing didactics within varied target groups in primary and secondary education in the area of strategy instruction, peer interaction and observational learning. It appears that learning activities that reduce the cognitive load of the working memory are especially effective in writing classes, e.g. through dividing the process of learning to write into different steps and through separating learning to write from writing itself. Based on previous research and the opinion of experienced teachers of Dutch, learning needs that are regularly found in the upper havo levels were taken into account: a need for communication, a need for concrete examples and a need for help with maintaining an overview of learning goals. In view of the advantages that digital and online writing courses offer over writing classes with pen and paper, the design of the writing course included the use of educational software. Results show that digital writing education in which upper level havo students learn strategies in steps by frequently exchanging reader reactions with classmates and by observing and evaluating recordings of the approaches of others has a positive effect on text quality. more...

The Dutch Inspectorate for Education established that the quality of elementary students’ writing is below the desired level, and that the teaching of writing must be improved. The aim of this PhD research was therefore to improve writing education in upper elementary grades by developing an effective teaching program for writing, including professional development for teachers, and tools for the assessment of writing. Building on the findings of previous research, we developed and tested Tekster [Texter], a strategy-focused comprehensive program for teaching writing in grade 4 to 6. Tekster combines several evidence-based practices, such as strategy instruction, goal setting, text structure instruction and observational learning into one general approach to address both the focus and mode of writing instruction. Tekster was developed in close collaboration with elementary teachers. The effectiveness of Tekster was tested in two large-scale intervention studies with 144 teachers and 2766 students from 52 schools. Students’ writing performance was assessed with multiple tasks in multiple genres in order to draw inferences about students’ underlying writing proficiency. The quality of their texts was rated by multiple raters using a benchmark rating scale. To ensure that the intervention was implemented with fidelity, teachers were provided with a manual and training. In the first study students’ writing performance improved significantly after following the Tekster program, with an effect size of 0.40, while generalizing over students, teachers, and tasks. The improvement was established in all grades and maintained two months after the intervention. In the second study, in which we added a professional development program, the effect size was even higher, i.e., 0.55. This study specifically showed that teachers became more positive and felt more efficacious about the teaching of writing after working with the program. They were highly satisfied with Tekster and the professional development activities. Moreover, it was shown that teachers were able to transfer their newly acquired knowledge and skills to colleagues. In two subsequent studies we specifically investigated the effectiveness of the writing strategy. This strategy divides the writing process into a prewriting, writing, and post-writing phase. We found that explicitly teaching students a writing strategy enhanced their metacognitive knowledge about writing and promoted the use of prewriting activities, which improved the quality of their texts. Further, we examined whether teachers are able to adequately provide feedback to texts of varying quality. Results show that the type and amount of feedback depends more on the style of the teacher than on the actual performance of the student, and that additional training for teachers is necessary to optimize the effectiveness of feedback. Therefore feedback formed a substantial part of the professional development program of Tekster. To summarize, this PhD research demonstrates that a comprehensive writing program, such as Tekster, is a promising approach to improve writing education in elementary grades. The program appeared to be effective for students and teachers. All in all, this research provides valuable clues on how the gap between research and classroom practice can be bridged to improve writing education. more...

The Dutch Inspectorate for Education established that the quality of elementary students’ writing is below the desired level, and that the teaching of writing must be improved. The aim of this PhD research was therefore to improve writing education in upper elementary grades by developing an effective teaching program for writing, including professional development for teachers, and tools for the assessment of writing. Building on the findings of previous research, we developed and tested Tekster [Texter], a strategy-focused comprehensive program for teaching writing in grade 4 to 6. Tekster combines several evidence-based practices, such as strategy instruction, goal setting, text structure instruction and observational learning into one general approach to address both the focus and mode of writing instruction. Tekster was developed in close collaboration with elementary teachers. The effectiveness of Tekster was tested in two large-scale intervention studies with 144 teachers and 2766 students from 52 schools. Students’ writing performance was assessed with multiple tasks in multiple genres in order to draw inferences about students’ underlying writing proficiency. The quality of their texts was rated by multiple raters using a benchmark rating scale. To ensure that the intervention was implemented with fidelity, teachers were provided with a manual and training. In the first study students’ writing performance improved significantly after following the Tekster program, with an effect size of 0.40, while generalizing over students, teachers, and tasks. The improvement was established in all grades and maintained two months after the intervention. In the second study, in which we added a professional development program, the effect size was even higher, i.e., 0.55. This study specifically showed that teachers became more positive and felt more efficacious about the teaching of writing after working with the program. They were highly satisfied with Tekster and the professional development activities. Moreover, it was shown that teachers were able to transfer their newly acquired knowledge and skills to colleagues. In two subsequent studies we specifically investigated the effectiveness of the writing strategy. This strategy divides the writing process into a prewriting, writing, and post-writing phase. We found that explicitly teaching students a writing strategy enhanced their metacognitive knowledge about writing and promoted the use of prewriting activities, which improved the quality of their texts. Further, we examined whether teachers are able to adequately provide feedback to texts of varying quality. Results show that the type and amount of feedback depends more on the style of the teacher than on the actual performance of the student, and that additional training for teachers is necessary to optimize the effectiveness of feedback. Therefore feedback formed a substantial part of the professional development program of Tekster. To summarize, this PhD research demonstrates that a comprehensive writing program, such as Tekster, is a promising approach to improve writing education in elementary grades. The program appeared to be effective for students and teachers. All in all, this research provides valuable clues on how the gap between research and classroom practice can be bridged to improve writing education. more...

The results of this study increased insight into factors that explain differences in the judgments of IGZ inspectors. This study shows that IGZ inspectors systematically differ in the regulatory judgments they assign to similar health care institutions. This study also shows that IGZ inspectors systematically tend to assign judgments that are too positive compared with the IGZ corporate standards. In this study it was examined whether the reliability and validity of regulatory judgments varied between two types of instruments: a lightly structured instrument (LSI) used for the regulation of hospital care was compared with the highly structured instrument (HSI) used for the regulation of nursing home care. The results indicated problems with the reliability and validity of the judgments assigned with the HSI; however, reliability and validity could not be calculated with the LSI. The results showed that using an HSI is preferable to using an LSI. To analyze the interventions professionals carry out to improve reliability, a systematic meta-analytic review of the research literature was performed. Three types of interventions could be defined: improving the diagnostic instrument, training the professional, and a combination of both. On average, although all types of interventions are effective, improving the diagnostic instrument seems to be the most effective; especially in the case of highly technical instruments, improvement has proven to be very effective. Because instrumental variables constitute a major source of error, improving the instrument is an important approach. However, this review offers solid arguments that can complement the literature and practice, with a focus on training the user of the instrument. An experimental study was performed to determine what kind of intervention would be effective for improving the reliability and validity of the regulatory judgments of IGZ inspectors. A case study was set up to examine the effect of participating in a consensus meeting and the effect of improving the regulatory instrument. The results showed that when an HSI was used, participating in a consensus meeting improved both the reliability and the validity of the regulatory judgments. Adjusting this instrument influenced but did not improve the reliability and validity of the judgments. This means that changing the instrument without training the inspectors in the use of the adjusted instrument does not improve the reliability and validity of the judgments. These outcomes emphasize the importance of the human factor in explaining variance between inspectors, and highlight the significance of training inspectors in the use of regulatory instruments. The effect of increasing the number of inspectors per case was calculated. This increased the reliability and validity of the regulatory judgments. more...

The results of this study increased insight into factors that explain differences in the judgments of IGZ inspectors. This study shows that IGZ inspectors systematically differ in the regulatory judgments they assign to similar health care institutions. This study also shows that IGZ inspectors systematically tend to assign judgments that are too positive compared with the IGZ corporate standards. In this study it was examined whether the reliability and validity of regulatory judgments varied between two types of instruments: a lightly structured instrument (LSI) used for the regulation of hospital care was compared with the highly structured instrument (HSI) used for the regulation of nursing home care. The results indicated problems with the reliability and validity of the judgments assigned with the HSI; however, reliability and validity could not be calculated with the LSI. The results showed that using an HSI is preferable to using an LSI. To analyze the interventions professionals carry out to improve reliability, a systematic meta-analytic review of the research literature was performed. Three types of interventions could be defined: improving the diagnostic instrument, training the professional, and a combination of both. On average, although all types of interventions are effective, improving the diagnostic instrument seems to be the most effective; especially in the case of highly technical instruments, improvement has proven to be very effective. Because instrumental variables constitute a major source of error, improving the instrument is an important approach. However, this review offers solid arguments that can complement the literature and practice, with a focus on training the user of the instrument. An experimental study was performed to determine what kind of intervention would be effective for improving the reliability and validity of the regulatory judgments of IGZ inspectors. A case study was set up to examine the effect of participating in a consensus meeting and the effect of improving the regulatory instrument. The results showed that when an HSI was used, participating in a consensus meeting improved both the reliability and the validity of the regulatory judgments. Adjusting this instrument influenced but did not improve the reliability and validity of the judgments. This means that changing the instrument without training the inspectors in the use of the adjusted instrument does not improve the reliability and validity of the judgments. These outcomes emphasize the importance of the human factor in explaining variance between inspectors, and highlight the significance of training inspectors in the use of regulatory instruments. The effect of increasing the number of inspectors per case was calculated. This increased the reliability and validity of the regulatory judgments. more...

The primary aim of this study is to provide information on how effective institutes for Dutch as a foreign language organise their education. As many teachers still seem to be left to fend for themselves in their efforts to improve their teaching, examples of good practice are provided through six case study descriptions of effective institutes. These descriptions are set into the context of the theoretical approaches concerning Dutch studies and the effectiveness of the institutes. It is hoped that this thesis will give these teachers clues as to how to improve their teaching of Dutch as a foreign language. This curriculum study is based upon five sources used to a different extent during its three phases, namely: an extensive questionnaire filled in at the institutes labelled as effective in teaching Dutch as a foreign language; interviews with teachers; interviews with students; observations of classroom practice, and publications/websites concerning the institutes. Great attention has been paid to the triangulation of the data, i.e. the extent to which information originating from these sources leads to a specific perception of the given area, making the description more complete and objective. The choice of the effective institutes is based upon the results of the lowest level of the Certificate of Dutch as a Foreign Language (CNaVT; A2-level CEFR) examination as the students, typically without any existing knowledge of Dutch, reach this level within the first year of their studies. To describe the video/audio recordings of an “as-normal-as possible” teaching week, an adapted version of the taxonomy of the Dutch Inspectorate for Education is used. In the descriptions short transcriptions of these recordings are provided in order to give as accurate as possible a picture of the lessons that were delivered. In the last chapter the future trends are also described. The pillars of this study are: the didactic preconditions, organisational setting, content of the curriculum, and didactic approach of the teacher. Regardless of the history, size, sort, or specialisation of the institutes, the key to their effectiveness seems to be the combination and the complementary character of these pillars. It was concluded that there are many ways in which an institute can be effective. It can be seen e.g. that the selection of students prior to their study of Dutch plays an important role in five of the institutes and three of the institutes have stringent selection criteria during the study period as well. However, there is still one effective institute for the teaching of Dutch as a foreign language where no explicit selection procedures are applied. The general conclusion reached is, then, that what matters and should be greatly stimulated is the quality of the teachers. Even though they do have their reservations e.g. with regard to making/applying institutional policies, they make a great effort to improve the effectiveness of their teaching. At the end of the day the teachers are the ones in contact with the students and the ones who are (or should be) making sense of the teaching. That is why it is essential to have good and motivated teachers who are able to compensate for the missing or insufficiently realised elements of the teaching programme. more...

The primary aim of this study is to provide information on how effective institutes for Dutch as a foreign language organise their education. As many teachers still seem to be left to fend for themselves in their efforts to improve their teaching, examples of good practice are provided through six case study descriptions of effective institutes. These descriptions are set into the context of the theoretical approaches concerning Dutch studies and the effectiveness of the institutes. It is hoped that this thesis will give these teachers clues as to how to improve their teaching of Dutch as a foreign language. This curriculum study is based upon five sources used to a different extent during its three phases, namely: an extensive questionnaire filled in at the institutes labelled as effective in teaching Dutch as a foreign language; interviews with teachers; interviews with students; observations of classroom practice, and publications/websites concerning the institutes. Great attention has been paid to the triangulation of the data, i.e. the extent to which information originating from these sources leads to a specific perception of the given area, making the description more complete and objective. The choice of the effective institutes is based upon the results of the lowest level of the Certificate of Dutch as a Foreign Language (CNaVT; A2-level CEFR) examination as the students, typically without any existing knowledge of Dutch, reach this level within the first year of their studies. To describe the video/audio recordings of an “as-normal-as possible” teaching week, an adapted version of the taxonomy of the Dutch Inspectorate for Education is used. In the descriptions short transcriptions of these recordings are provided in order to give as accurate as possible a picture of the lessons that were delivered. In the last chapter the future trends are also described. The pillars of this study are: the didactic preconditions, organisational setting, content of the curriculum, and didactic approach of the teacher. Regardless of the history, size, sort, or specialisation of the institutes, the key to their effectiveness seems to be the combination and the complementary character of these pillars. It was concluded that there are many ways in which an institute can be effective. It can be seen e.g. that the selection of students prior to their study of Dutch plays an important role in five of the institutes and three of the institutes have stringent selection criteria during the study period as well. However, there is still one effective institute for the teaching of Dutch as a foreign language where no explicit selection procedures are applied. The general conclusion reached is, then, that what matters and should be greatly stimulated is the quality of the teachers. Even though they do have their reservations e.g. with regard to making/applying institutional policies, they make a great effort to improve the effectiveness of their teaching. At the end of the day the teachers are the ones in contact with the students and the ones who are (or should be) making sense of the teaching. That is why it is essential to have good and motivated teachers who are able to compensate for the missing or insufficiently realised elements of the teaching programme. more...

This chapter aims to give an overview of relevant questions related to the measurement of first language written text quality. Referring to the relevance of standardized assessment in current educational research, various formats of measurement are introduced. Focusing on the field of writing research, the question of text quality is depicted as depending on the particular discipline and theoretical grounding. Using different text assessment studies as examples, advantages and disadvantages of numerous coding procedures (e.g. holistic, analytical, mixed models) will be discussed. Possible applications of the different methods are examined in the light of the corresponding research contexts and questions (e.g. large-scale assessment vs. individual learning support), methodical issues of measurement and cost-benefit aspects. Also, desiderata and possibilities for integrating information from formative assessment into educational settings, that is, instructional support, are considered, indicating ideas for future theory and praxis in writing research, assessment and teaching. more...

Standardized surveys are used in many contexts to measure people’s opinions and attitudes. Although it is widely assumed that survey answers represent the ‘true values’ of the concepts measured, a large body of research has shown that seemingly irrelevant question characteristics influence how respondents report their attitudes. The research presented in this dissertation revolved around on one of these characteristics: whether the question is worded positively (This is an interesting book. Yes/No.) or negatively (This is an uninteresting book. Yes/No.). A first aim of the current dissertation was to investigate whether the choice for a positive or a negative wording systematically affects survey answers. Results show that there is an effect of question polarity on survey answers: irrespective of the word pair used, the answering scale offered, and the question asked, respondents are more likely to answer no or disagree to negative questions than to answer yes or agree to equivalent positive ones. Hence, respondents express their opinions more positively when the question is worded negatively (Chapter 2, 4, 5). Even though the choice for a positive or a negative wording was shown to affect survey answers systematically, we also observed variation between word pairs in the size of the effect. A second aim of the current dissertation was to explain this variation using a semantic classification of antonyms in absolute versus relative types (Kennedy & McNally 2005). Unfortunately, we were unable to explain variation between word pairs using this typology (Chapter 3). The third and last aim of the current dissertation was to explain cognitively why respondents, in general, answer positive and negative attitude questions differently. To do so, we used eye-tracking as a method to measure the temporal aspects of the question-answering process for positive versus negative questions. In order to relate the observed time differences to cognitive processes of question answering (Tourangeau, Rips & Rasinski 2000), we used a combination of theoretical insights about the question-answering process, experimental designs in which that knowledge was used, and newly designed eye-tracking measures that are relevant for these specific purposes. With this combination of strategies we were able to demonstrate that positive and negative questions cause similar information to be retrieved from memory and a similar attitude to be formed. Therefore, contrastive questions measure the same underlying attitude, and hence, are equally valid. Wording effects arise only late in the question-answering process, when respondents translate their own opinion into the response options to the question. This is due to the fact that respondents assign a meaning to the response options that is not absolute, but relative to the polarity of the evaluative term used in the question. Hence, although response options like yes and no are straight opposites, the meaning of yes in answer to a positive question is not the same as the meaning of no in answer to a negative question. This dissertation concludes with a discussion of the implications of these results for survey theory and suvey practice, as well as the methodological implications about the use of eye-tracking for measuring cognitive processes more...

Standardized surveys are used in many contexts to measure people’s opinions and attitudes. Although it is widely assumed that survey answers represent the ‘true values’ of the concepts measured, a large body of research has shown that seemingly irrelevant question characteristics influence how respondents report their attitudes. The research presented in this dissertation revolved around on one of these characteristics: whether the question is worded positively (This is an interesting book. Yes/No.) or negatively (This is an uninteresting book. Yes/No.). A first aim of the current dissertation was to investigate whether the choice for a positive or a negative wording systematically affects survey answers. Results show that there is an effect of question polarity on survey answers: irrespective of the word pair used, the answering scale offered, and the question asked, respondents are more likely to answer no or disagree to negative questions than to answer yes or agree to equivalent positive ones. Hence, respondents express their opinions more positively when the question is worded negatively (Chapter 2, 4, 5). Even though the choice for a positive or a negative wording was shown to affect survey answers systematically, we also observed variation between word pairs in the size of the effect. A second aim of the current dissertation was to explain this variation using a semantic classification of antonyms in absolute versus relative types (Kennedy & McNally 2005). Unfortunately, we were unable to explain variation between word pairs using this typology (Chapter 3). The third and last aim of the current dissertation was to explain cognitively why respondents, in general, answer positive and negative attitude questions differently. To do so, we used eye-tracking as a method to measure the temporal aspects of the question-answering process for positive versus negative questions. In order to relate the observed time differences to cognitive processes of question answering (Tourangeau, Rips & Rasinski 2000), we used a combination of theoretical insights about the question-answering process, experimental designs in which that knowledge was used, and newly designed eye-tracking measures that are relevant for these specific purposes. With this combination of strategies we were able to demonstrate that positive and negative questions cause similar information to be retrieved from memory and a similar attitude to be formed. Therefore, contrastive questions measure the same underlying attitude, and hence, are equally valid. Wording effects arise only late in the question-answering process, when respondents translate their own opinion into the response options to the question. This is due to the fact that respondents assign a meaning to the response options that is not absolute, but relative to the polarity of the evaluative term used in the question. Hence, although response options like yes and no are straight opposites, the meaning of yes in answer to a positive question is not the same as the meaning of no in answer to a negative question. This dissertation concludes with a discussion of the implications of these results for survey theory and suvey practice, as well as the methodological implications about the use of eye-tracking for measuring cognitive processes more...

This thesis is about writing proficiency among students of secondary education. Due to globalization, the ability to express oneself in a language other than the first language (L1) is increasingly becoming a condition for educational success. In The Netherlands, this ‘other’ or second language (L2) is usually English. Although secondary school students are already quite able to express themselves in English, their L2 essays are often of lower quality than L1 essays, in terms of language use, but also in terms of organization. The research reported in this thesis was aimed at explaining this quality difference by comparing L1 and L2 relations between essay quality and writing processes. Analyses of writing processes involved cognitive activities such as reading the assignment, process planning, content planning, evaluating and revising. Results show that, in general, cognitive activities are relevant to essay quality at different stages of task execution during L1 and L2 writing. This means that writers need to distribute their attention differently across task execution during L1 and L2 writing. However, additional analyses show that if students’ general language proficiency levels are included in the analyses, this L1/L2 contrast disappears. For students with high general L2 proficiency, the demands of L2 writing in terms of how often cognitive activities are applied during certain stages of task execution are similar to the demands of L1 writing. more...

This thesis is about writing proficiency among students of secondary education. Due to globalization, the ability to express oneself in a language other than the first language (L1) is increasingly becoming a condition for educational success. In The Netherlands, this ‘other’ or second language (L2) is usually English. Although secondary school students are already quite able to express themselves in English, their L2 essays are often of lower quality than L1 essays, in terms of language use, but also in terms of organization. The research reported in this thesis was aimed at explaining this quality difference by comparing L1 and L2 relations between essay quality and writing processes. Analyses of writing processes involved cognitive activities such as reading the assignment, process planning, content planning, evaluating and revising. Results show that, in general, cognitive activities are relevant to essay quality at different stages of task execution during L1 and L2 writing. This means that writers need to distribute their attention differently across task execution during L1 and L2 writing. However, additional analyses show that if students’ general language proficiency levels are included in the analyses, this L1/L2 contrast disappears. For students with high general L2 proficiency, the demands of L2 writing in terms of how often cognitive activities are applied during certain stages of task execution are similar to the demands of L1 writing. more...