Search

Meta-analyses suggest that peer feedback is particularly useful for learning, but additional research is needed to understand the large variation in effect sizes that is regularly seen. We examined the relationship between amount of provided and received feedback with growth in writing and reading performance in English as a Foreign Language (EFL) learners across multiple assignments. Only provided comments, and in particular comments on meaning-level issues, were consistently associated with growth in writing performance. These types of comments were also associated with growth in reading performance, but less consistently so. Comments provided at the surface level were either uncorrelated or negatively correlated with growth in writing performance; follow up analyses suggest that the nature of the comments provided in response to training can explain the changes in effectiveness over assignments. Amount of comments received was never associated with growth in writing or reviewing skills. These findings highlight the value of peer feedback assignment focused on meaning-level issues, even for EFL learners. Further, these findings add to the growing literature that suggest that extensive opportunities to provide meaning-level feedback, especially when it is detailed, is critical for student learning. [ABSTRACT FROM AUTHOR]

Improved English Language Arts instruction within classrooms and schools that typically serve low‐income, English Language Learners, and Students with Special Needs is important for ensuring all students receive rigorous and inclusive instruction. From the case of a Networked Improvement Community focused on improving instructional practices through student‐centered routines, this research explores the role of humanizing pedagogies in enabling effective student‐centered instructional routines by interviewing 16 teachers who were identified as particularly successful in working with Students of Color. Two major themes and six subthemes were identified from these interviews focused on building relationships as a foundation for humanizing pedagogies and then leveraging those relationships to create a positive classroom culture that fosters increased student engagement and academic risk‐taking with challenging student‐centered routines. These strong teacher–student relationships and student‐centered routines led to improved academic outcomes. [ABSTRACT FROM AUTHOR]

As the population of emergent bilingual students in the United States continues to grow, it has become increasingly important to ensure that content area instruction is linguistically and culturally inclusive and accessible. Longitudinal survey and state mathematics assessment data were used to examine how the use of students' home language in the English-medium mathematics classroom related to growth in student achievement and attitudes towards mathematics learning. Surveys from 1,274 students attending four middle schools in a Spanish-dominant community reported on teacher and student use of Spanish as well as attitudes towards learning strategies and mathematics. Structural equation modelling revealed that teachers' use of Spanish predicted students' use of Spanish, which was associated with growth in comfort participating in the mathematics classroom. Increased comfort, moreover, was related to growth in mathematics self-efficacy and, in turn, increased interest in mathematics as well as growth on the state mathematics assessment. [ABSTRACT FROM AUTHOR]

Background: For peer assessment, reliability (i.e., consistency in ratings across peers) and validity (i.e., consistency of peer ratings with instructors or experts) are frequently examined in the research literature to address a central concern of instructors and students. Although the average levels are generally promising, both reliability and validity can vary substantially from context to context. Meta‐analyses have identified a few moderators that are related to peer assessment reliability/validity, but they have lacked statistical power to systematically investigate many moderators or disentangle correlated moderators. Objectives: The current study fills this gap by addressing what variables influence peer assessment reliability/validity using a large‐scale, cross‐context dataset from a shared online peer assessment platform. Methods: Using multi‐level structural equation models, we examined three categories of variables: (1) variables related to the context of peer assessment; (2) variables related to the peer assessment task itself; and (3) variables related to rating rubrics of peer assessment. Results and Conclusions: We found that the extent to which assessment documents varied in quality on the given rubric played a central role in mediating the effect from different predictors to peer assessment reliability/validity. Other variables that are significantly associated with reliability and validity included: Education Level, Language, Discipline, Average Ability of Peer Raters, Draft Number, Assignment Number, Class Size, Average Number of Raters, and Length of Rubric Description. The results provide information to guide practitioners on how to improve reliability and validity of peer assessments. Lay Description: What is already known about this topic: The reliability and validity of peer assessment have been found to be, on average, at or above acceptable levels.Inter‐rater reliabilities, in particular, were reported to be medium to high in different online peer assessment studies in both K‐12 and higher education settings.Correlations between peer ratings and expert ratings were reported to be medium to high in different studies. What this paper adds: Document Quality Variability (DQV) is the degree to which submitted documents varied in quality on the given rubric dimension versus being all of roughly similar quality.DQV strongly predicts peer assessment reliability and validity.Class Size, Average Number of Raters, and Length of Rubric Descriptions are associated with reliability directly.Average Number of Raters and Assignment Number predict validity directly.Average Ability of Peer Raters, Draft Number, and Assignment Number predict validity indirectly through DQV. Implications for practice and/or policy: To increase IRR and validity of ratings, instructors can include more students with different proficiency levels in the subject area in peer assessment.Instructors should avoid using rubric scales that will mainly have students cluster at the top (or bottom) of the scale.The appropriate number of peer raters usually lies within 3‐to‐5 raters per document.Instructors should use peer assessment activities primarily for first drafts rather than for second or later drafts of an assignment. [ABSTRACT FROM AUTHOR]

Scientific progress on creativity research depends on having properly operationalized measures. In psychological research on creativity, it is common to operationalize creativity as the combination of novelty and appropriateness. However, the operationalization of appropriateness varies widely across researchers, studies, and domains (e.g. technical goodness, significance, elegance, usefulness, and feasibility). We argue that a core distinction between impact (how useful an idea is for solving the problem) and feasibility (how easy it is to realize the idea) underlies the variation. We further claim that this distinction is both possible to capture reliably in practice and psychologically significant. To test these claims, 318 ideas from 5 real-world social innovation problems (e.g. improving accessibility in elections) were rated for novelty, impact, and feasibility by a set of six experts selected for each of the 5 challenges. We find that all three constructs can be measured reliably and are statistically separable. Further, we show that distinguishing impact and feasibility reveals theoretically meaningful patterns of relationships with key psychological processes of creativity – analogy and conceptual combination – that would be difficult if impact and feasibility were conflated. These results demonstrate the theoretical importance of separating appropriateness into impact and feasibility for the psychology of creativity. [ABSTRACT FROM AUTHOR]

Recent research highlights the concept of feedback literacy, focusing on students' active engagement in feedback processes, including giving and receiving peer feedback. However, most studies rely on self-assessments through surveys and interviews, with few examining actual feedback behaviors. This study explores the behavioral aspects of peer feedback literacy among 844 high school students using an online system for peer feedback and revisions. Multiple measures related to provided ratings and comments as well as use in revisions were extracted using systematic coding. Factor analysis and structural equation modeling identified two distinct elements of high-quality feedback provision—providing features and rating validity—and confirmed a moderate correlation between the ability to provide quality feedback and the use of feedback. These skills were weakly correlated with writing ability. This research contributes to the emerging literature of peer feedback literacy and supports the development of more effective peer feedback training approaches. • Identified two key aspects of high-quality provided feedback: features and rating validity. • Feedback providing skill was moderately correlated with feedback receiving skill. • Providing and receiving skills were weakly correlated with writing ability. [ABSTRACT FROM AUTHOR]

Although peer review has been widely used for formative assessment in writing instruction, there remain concerns about whether assessors are at a sufficient writing performance level that would allow them to identify major problems in the reviewed work and provide helpful feedback to improve draft quality. Little empirical research has examined how assessor writing performance specifically influences problem identification accuracy and helpfulness of feedback, nor has it acknowledged different grain sizes of assessor performance. Assessor writing performance at different grain sizes (i.e., performance at the levels of genre, dimension of a genre, and specific problem topic) was assessed alongside problem identification accuracy and feedback helpfulness in 234 high school students who participated in an anonymous multipeer review in a secondary writing course in the United States. A correlation analysis showed that assessor performance levels on specific problem topics were meaningfully separable, thereby allowing for consideration of the effects of assessor performance at genre, dimension, and topic levels. Multiple regression results indicated that assessor writing performance was unrelated to problem identification accuracy at any grain size. Therefore, scaffolds in the reviewing process appear sufficient to support problem identification accuracy. However, assessor writing performance, particularly on specific dimensions and specific topics, consistently predicted helpfulness of feedback, even though lower performing assessors rarely produce incorrect advice. Theoretical and practical implications of the findings are discussed. Educational Impact and Implications Statement: Assessor writing performance is conceptualized in terms of different grain sizes: overall writing performance, genre writing performance, writing performance on a cluster of dimensions, dimensional writing performance, and topic writing performance. An assessor who is at a given writing performance level at a large grain size may often be at varying performance levels at more fine-grain sizes. Assessors at both high and low writing performance levels, at any grain size, are equally likely to identify problems accurately. Assessors with high performance on specific dimensions and topics are more likely to provide helpful feedback on those dimensions and topics. [ABSTRACT FROM AUTHOR]

Community cultural wealth (CCW) is shown to play a role in the STEM persistence of Black students. White teachers often view their Black students through a deficit-based lens and may struggle to recognize and access the CCW of their nonwhite students. Using observational analysis of two White teachers in the US south who consistently maintained high engagement among Black students in conceptually-rich mathematics classrooms, we present a framework that identifies pedagogical practices that leverage and cultivate the cultural wealth of Black students. This framework can serve as a foundation for all teachers to utilize assets-based instruction in their mathematics classrooms. • We identified actionable asset-based practices to utilize in mathematics instruction. • Community cultural wealth translates to pedagogical actions for mathematics teaching. • Teachers leveraged social, aspirational, linguistic, and navigational capital. • Teachers leveraged student assets within racially/ethnically diverse classrooms. [ABSTRACT FROM AUTHOR]

Peer review is regularly found to be a powerful and efficient technique for assessment and feedback, but many students are inexperienced and sometimes struggle to provide meaningful feedback. It is considered best practice to provide students with some training on how to be a good reviewer, but few classes can afford to devote much time to such training, and the assumption is that review quality will improve with experience. This study directly examines what kinds of experiences during peer feedback activities improve reviewing quality. In particular, organized by theories of norm-setting and practice-based learning, it examines the relationship of the amount of provided and received feedback on one assignment to improvements in the quality of feedback on the next assignment. Data on peer feedback experiences and behaviors across multiple assignments were taken from students across two introductory level undergraduate courses (N = 360). Multi-regression analyses reveal that the number and length of feedback provided predicted growth in helpfulness rates, and both improvements in domain performance and the reviewer's preference for length explains the effects on review helpfulness. Further, compared with high-performing students, low-performing students show more remarkable growth in helpfulness from providing feedback. [ABSTRACT FROM AUTHOR]

Digital badges have long been assumed to possess motivational qualities that could encourage learners to engage with learning content. However, prior studies have found the effects of badges to be complex, differing by learner, type of badge, and potentially other factors. Qualitative reports suggest that individuals' perceptions of digital badges may play a role in moderating badges' effects: badges are only motivating when they are perceived as relevant and desirable. In the current study, we examine longitudinal episodes of data from 2,410 middle and high school users of a badged online programming curriculum to test whether there is evidence that learners' perceptions of badges' relevance predict changes in engagement over time; and whether that relationship is equitable with respect to age, sex, and ethnicity. We also investigate whether a reciprocal relationship may exist in which engagement predicts relative increases in learners' perceptions of badges as relevant to them. Learners' positive perceptions of badges' relevance predicted rank-order increases in engagement over time. Further, this relationship did not vary by age, sex, or minoritized racial/ethnic status. In addition, higher engagement also predicted upward shifts in perceived badge relevance. These results suggest that learners' subjective evaluations of digital badges are closely related to changes in their engagement with program activities, that badged engagement neither widens nor closes educational equity gaps, and that learners' regard for badges and engagement in program activities may be mutually reinforcing. Educational Impact and Implications Statement: Digital badges have been making inroads into classrooms and out-of-school time programs for nearly a decade, but their real-world effectiveness as motivational tools vary widely. In this study, we modeled individuals' opinions of badges, and found that learners who perceive a program's badges as sensible, valuable, and desirable become increasingly engaged with that program activities over time. This effect was equally true across age, sex, or race/ethnicity subgroups. Learners who are more engaged also tended to find the badges increasingly relevant over time, creating a "virtuous cycle" of increasing program engagement and perceived badge relevance over the course of the program. Thus, educational programs may be able to use digital badges to sustain and deepen student engagement, provided they create badges that participants find sensible, valuable, and desirable. [ABSTRACT FROM AUTHOR]

While gender parity has been achieved in overall science degree earning, large gaps still exist within many science disciplines. Further, studies addressing gender inequity in science often ignore a large source of undergraduate science degree earners: those who enroll in science courses intending to pursue careers in health or medicine. This study examines pathways toward or away from science degrees in N = 4,345 men and women enrolled in early science courses at a large undergraduate research university. Importantly, to understand shifts in students' academic intentions and how pathways to science may be differential by gender, this study analyzed students' incoming major and career intentions, estimates of incoming academic abilities, and relative performance in science and various non‐science courses. Results show that while men and women initially intending to pursue a science major graduate with science degrees in equal numbers, the plurality of science degree earners are students entering college intending health or medical careers. Further, from those subgroups, a significantly larger proportion of men end up in science, while a significantly larger proportion of women end up outside of STEM completely. Understanding disciplinary differences in gender barriers to science participation can help inform interventions that specifically target those populations. [ABSTRACT FROM AUTHOR]

While pointing out important features of cognitive science, Núñez et al. (2019) also argue prematurely for the end of cognitive science. I discuss problematic analytic features in the application of hierarchical cluster analysis to journal citation data. On the conceptual side, I argue that the research programs framework of Lakatos may not be so wisely applied to cognitive science. Further, the diversity of structure in cognitive science departments may represent a rational, strategic adaptation by an interdisciplinary department to cognitive and other resource challenges rather than the sign of low progress on a discipline. [ABSTRACT FROM AUTHOR]

Dialogic peer feedback has been recommended and increasingly used in English as a foreign language writing context, yet the specific effects of peer-to-peer written dialogue about feedback remain under-researched. Using a quasi-experimental design, this empirical study investigated the effects of the presence/absence of written dialogue between the peer feedback provider and receiver on students' provision of feedback, adoption of received feedback, improvements in writing quality, and attitudes towards feedback dialogue across a four-week writing program. The study drew on several data sources including feedback texts, revision texts, a questionnaire survey and monthly reflection journals of forty-one students. Results showed that students involved in the written dialogue demonstrated a generally positive attitude towards it and outperformed those without such a process by generating more accurate adoption, focusing comments to a greater extent on higher-order dimensions of writing (unity, support and coherence), and implementing better revisions. The dialogic process enhanced students' feedback literacy and engagement by improving their understanding of feedback, their linguistic and subject knowledge, and their agentic clarification and negotiation of revisions. [ABSTRACT FROM AUTHOR]

We examined the influence of accountability on the consistency of peer ratings and quality of peer feedback by comparing three conditions: only rating accountability, only feedback accountability, or both rating and feedback accountability. From a large undergraduate course, 287 students' peer ratings and peer feedback were coded for rating consistency, comment helpfulness, amount of feedback, and feedback features. Because only 30% of the students accurately perceived their assigned condition, data were analyzed according to the perceived condition. Students who believed their reviewing grade would be influenced by the helpfulness of their feedback not only provided more feedback, but also more criticism, solutions, and localized comments. These students also provided more consistent ratings than those who thought their reviewing grade would be influenced by the consistency of their ratings. These findings indicate that constructing helpful comments could have a broad influence on peer assessment and consistent ratings are grounded in commenting. [ABSTRACT FROM AUTHOR]

Curricula, assessments and teacher professional development programmes wield a powerful influence on teaching and learning enactment. Together with the interpretation of those using them, these products mediate the flow of ideas from research to practice. In most countries, those curricula, assessments and professional development programmes that become widely used are created by educational designers. Given their crucial function, it is surprising that the role of educational designers is rarely recognised in the educational research literature, studied empirically or supported in practice. This article argues that educational research stands to (better) support practice at scale when it is attuned to the needs of educational designers. First, mechanisms for knowledge production and use are discussed, including the linkage role played by educational designers in the educational infrastructure of most countries. Then, the importance of understanding and bolstering the linkage between research findings and the work of designers is discussed. Arguments are given for research to better support those who design for scale, along with sample research questions posed by educational designers. In these, a distinction is made between the knowledge designers crave to shape their products (curricula, assessments, teacher professional development) and the knowledge they need to shape the processes through which those products come to fruition. This article closes with a call for educational researchers to explicitly focus their knowledge creation and dissemination efforts towards research consumers with the largest direct effects on teaching and learning: educational designers. [ABSTRACT FROM AUTHOR]

Open Learner Models are used in modern e-learning to show system users the content of their learner models. This approach is known to prompt reflection, facilitate planning and navigation. Open Learner Models may show different levels of detail of the underlying learner model, and may structure the information differently. However, a trade-off exists between useful information and the complexity of the information. This paper investigates whether offering richer information is assessed positively by learners and results in more effective support for learning tasks. An interview pre-study revealed which information within the complex learner model is of interest. A controlled user study examined six alternative visualisation prototypes of varying complexity and resulted in the implementation of one of the designs. A second controlled study involved students interacting with variations of the visualisation while searching for suitable learning material, and revealed the value of the design alternative and its variations. The work contributes to developing complex open learner models by stressing the need to balance complexity and support. It also suggests that the expressiveness of open learner models can be improved with visual elements that strategically summarise the complex information being displayed in detail. [ABSTRACT FROM AUTHOR]

Science learning is most often examined within formal education contexts, even though students spend more of their lives outside of the classroom. Students may interact with scientific phenomena during these out-of-school experiences, providing additional opportunities for learning to take place. Prior studies have found that optional science experiences have positive effects on science knowledge and attitudes. However, these studies do not always account for initial differences between students who are able to participate in many optional experiences and those who cannot. Moreover, many studies focus on high-quality science programs, which may not be representative of the average out-of-school experience. Using a longitudinal dataset of 3,700 6th and 8th grade students in urban and suburban schools from two regions in the United States, the current study investigates the effects of typically occurring optional science experiences during the school year on the development of science knowledge and attitudes. Using propensity score matching, we matched participating and nonparticipating students for characteristics that drive self-selection into these experiences, then analyzed separately for effects of school-related versus home-related science learning experiences. Stable patterns across analytic models reveal growth in science attitudes with both school- and home-related science experiences, but a greater relative decline in science knowledge with school-related science experiences. Thus, typically occurring optional science experiences can influence students' attitudes and knowledge, but the effects can vary by the type of experience that students attend. [ABSTRACT FROM AUTHOR]

Abstract: Introduction: Adults can represent numerical information in nonsymbolic and symbolic formats and flexibly switch between the two. While some studies suggest a strong link between the two number representation systems (e.g., Piazza, Izard, Pinel, Le Bihan, & Dehaene, 2004 Neuron, 44(3), 547), other studies show evidence against the strong‐link hypothesis (e.g., Lyons, Ansari, & Beilock, 2012 Journal of Experimental Psychology: General, 141(4), 635). This inconsistency could arise from the relation between task demands and the closeness of the link between the two number systems. Methods: We used a passive viewing task and event‐related potentials (ERP) to examine the temporal dynamics of the implicit integration between the nonsymbolic and symbolic systems. We focused on two ERP components over posterior scalp sites that were found to be sensitive to numerical distances and ratio differences in both numerical formats: a negative component that peaks around 170 ms poststimulus (N1) and a positive component that peaks around 200 ms poststimulus (P2p). We examined adults' (n = 55) ERPs when they were passively viewing simultaneously presented dot quantities and Arabic numerals (i.e., nonsymbolic and symbolic numerical information) in the double‐digit range. For each stimulus, the nonsymbolic and symbolic content either matched or mismatched in number. We also asked each participant to estimate dot quantities in a separate behavioral task and observed that they tended to underestimate the actual dot quantities, suggesting a need to adjust the match between nonsymbolic and symbolic information to reflect the perceived quantity of the nonsymbolic information. Results: Using this adjustment, participants showed greater N1 and P2p amplitudes when perceived dot quantities matched Arabic numerals than when there was a mismatch. However, no differences were found between the unadjusted match and mismatch conditions. Conclusion: Our findings suggest that adults rapidly integrate nonsymbolic and symbolic formats of double‐digit numbers, but evidence of such integration is best observed when the perceived (rather than veridical) dot quantity is considered. [ABSTRACT FROM AUTHOR]

Abstract: Educational robotics programs offer an engaging opportunity to potentially teach core computer science concepts and practices in K–12 classrooms. Here, we test the effects of units with different programming content within a virtual robotics context on both learning gains and motivational changes in middle school (6th–8th grade) robotics classrooms. Significant learning gains were found overall, particularly for groups introduced to content involving program flow, the structural logic of program execution. Relative gains for these groups were particularly high on items that require the transfer of knowledge to dissimilar contexts. Reaching units that included program flow content was also associated with greater maintenance of programming interest when compared with other units. Therefore, our results suggest that explicit instruction in the structural logic of programming may develop deeper transferrable programming knowledge and prevent declines in some motivational factors. [ABSTRACT FROM AUTHOR]

Consistent with movements integrating performance evaluation and improvement, the current study examines an assumption that more peer raters will produce better results or the so-called maxima strategy . This study examines the maxima strategy from both the agreement and performance perspectives with the intent of examining the role of feedback information in reliability and performance through an optimal number of peer raters per student ratee. It was found that the maxima strategy works consistently from agreement perspectives, whereas the relationship between the maxima strategy and student performance improvement follows an inverted U-shaped function. Accordingly, we recommend that the number of peer raters needs to be decided according to the optimal balance between reliability and performance, which maximizes ratees’ performances without sacrificing evaluation reliability. [ABSTRACT FROM AUTHOR]

Summary: Four principles of cognitive science were used to make systematic revisions in middle school science instructional modules from two kinds of curriculum: one popular textbook series and one popular hands‐on series (two modules each). Schools were randomly assigned to 1 of the 3 arms (cognitive science modifications with professional development, active control with professional development, or business‐as‐usual). Two cohorts of students were followed in each arm for each setting. There were significant benefits of the cognitive science intervention, but the nature of effects varied for the two settings and curricula. For the text‐based curriculum, positive effects of cognitive science modifications were concentrated in classrooms with lower proportions of underrepresented minority students. For the hands‐on curriculum, there were positive effects that were not linked to school composition. Participation in the active control did not significantly improve student learning. Implications for policy and research are discussed. [ABSTRACT FROM AUTHOR]

The National Research Council framework for science education and the Next Generation Science Standards have developed a need for additional research and development of curricula that is both technologically model-based and includes engineering practices. This is especially the case for biology education. This paper describes a quasi-experimental design study to test the effectiveness of a model-based curriculum focused on the concepts of natural selection and population ecology that makes use of Excel modeling tools (Modeling Instruction in Biology with Excel, MBI-E). The curriculum revolves around the bio-engineering practice of controlling an invasive species. The study takes place in the Midwest within ten high schools teaching a regular-level introductory biology class. A post-test was designed that targeted a number of common misconceptions in both concept areas as well as representational usage. The results of a post-test demonstrate that the MBI-E students significantly outperformed the traditional classes in both natural selection and population ecology concepts, thus overcoming a number of misconceptions. In addition, implementing students made use of more multiple representations as well as demonstrating greater fascination for science. [ABSTRACT FROM AUTHOR]

The impact of online peer feedback on learning outcomes has been well-established in previous meta-analyses. However, considerable variations remain unexplained, emphasizing the need to better understand the underlying reasons for heterogeneity. In this study, a large-scale and cross-context dataset, comprising 20,879 students enrolled in 505 assignments within 243 courses at 76 different institutions, was employed. Meta-regression, multi-level modeling, and ANCOVA were used to examine the effect size of each peer feedback experience and explore the associated heterogeneity of effects across courses and assignments within courses. Results revealed that learning benefits are more closely associated with providing feedback rather than receiving it, and are influenced more by the length of peer feedback rather than its helpfulness. Furthermore, heterogeneity exists at the assignment level rather than the course level, and only two peer feedback experiences (i.e., provided length and received length) exhibit particularly large variations in effect size. Particularly novel to our study, we found that learning benefits of both peer feedback experiences diminished as knowledge transfers further away. Similarly, providing feedback demonstrates robust learning compared to receiving feedback, primarily in the form of smaller downward trends and higher learning gains at more further transfer level. Instructors are recommended to design logically-structured consecutive assignments within a course and to provide more guidance to students on giving detailed feedback. • Tested the relative benefits of providing and receiving peer feedback in a large naturalistic dataset. • Learning was more connected to providing rather than receiving, and with comment length rather than perceived helpfulness. • Large effect-size heterogeneity was predominantly at the assignment rather than course level. • Developed a novel method for examining knowledge transfer levels. • Effect sizes varied as function of relative level of knowledge transfer distance. [ABSTRACT FROM AUTHOR]

• Peer feedback quality can be conceptualized into five dimensions: feedback features, content, impact, accuracy and amount. • Three feedback quality measures were tested and found to be robustly reliable, semi-stable, and separable in a large datasets. • All three peer feedback quality measures showed growth in response to reviewing experiences, but only feedback helpfulness in particular showed effects of all four. Although peer review is a widely-used pedagogical technique, its value depends upon the quality of the reviews that students produce, and much research remains to be done to systematically study the nature, causes, and consequences of variation in peer review quality. We propose a new framework that conceptualizes five larger dimensions of peer review quality and then present a study that investigated three specific peer review quality constructs in a large dataset and further explored how these constructs change through different types of self-regulation peer reviewing experiences. Peer review data across multiple assignments were analyzed from 2,092 undergraduate students enrolled in one of three offerings of a biology course at a large public research university in the United States. Peer review quality was measured in terms of comment amount, comment accuracy, and rating accuracy; the measures of reviewing experience focused upon self-regulated learning factors such as practice, feedback, others' modeling, and relative performance. Meta-correlation (for testing reliability, separability, and stability) and meta -regression (as a time-series analysis for testing the relationship of changes across assignments in reviewing quality with experiences as reviewer and reviewee) are used to establish the robustness of effects and meaningful variation of effects across course offerings and assignments. Results showed that there were three meaningful review quality constructs (i.e., were measured reliably, separable, and semi-stable over time). Further, all three showed changes in response to previous reviewer and reviewee experiences, but only feedback helpfulness, in particular, showed effects of all four examined types of self-regluation experiences (practice, feedback, others' modeling, and relative performance). The findings suggest that instructors can improve review quality by providing comment prompt scaffolds that lead to longer comments as well as by matching authors with similarly performing reviewers. [ABSTRACT FROM AUTHOR]

• New theoretical analysis of the learning opportunities in peer feedback. • Providing explanations is directly connected to learning. • Providing suggestions is connected to revisions which is connected to learning. • Receiving explanations is also connected to learning through revisions. To deepen understanding of learning through peer feedback, the current study investigated the relationships between different peer feedback activities (organized into constructive vs active activities) and learning (i.e., transfer to new tasks), examining the nature of activities within provided feedback, received feedback, and revisions in response to feedback. Across five US high schools, 367 students in Advanced Placement classes participated, implementing common assignments and peer assessment rubrics. Provided/received comments and revisions in one assignment, and writing improvements observed in a second assignment were exhaustively coded and subjected to hierarchical model regression analyses. Results showed that constructive activities (providing explanations and making revisions after receiving explanations or providing suggestions) were consistently associated with learning, whereas passive (e.g., receiving feedback without making revisions) or active activities (e.g., implementing specific suggestions) were not. Further, the effects of received feedback on learning were mediated by the number of revisions. Theoretical and practical implications of the findings are discussed. [ABSTRACT FROM AUTHOR]

Number of raters is theoretically central to peer assessment reliability and validity, yet rarely studied. Further, requiring each student to assess more peers' documents both increases the number of evaluations per document but also assessor workload, which can decline performance. Moreover, task complexity is likely a moderating factor, influencing both workload and validity. This study examined whether changing the number of required peer assessments per student / number of raters per document affected peer assessment reliability and validity for tasks at different levels of task complexity. 181 students completed and provided peer assessments for tasks at three levels of task complexity: low complexity (dictation), medium complexity (oral imitation), and high complexity (writing). Adequate validity of peer assessments was observed for all three task complexities at low reviewing loads. However, the impacts of increasing reviewing load varied by reliability vs. validity outcomes and by task complexity. • Peer assessment validity and reliability is influenced by both assessment task complexity and reviewer load. • Effects on reliability and validity were largely separate and sometimes in opposite directions. • Higher complexity assessments tasks had lower validity. • Increasing reviewer load can decline or improve single-rater reliability depending upon task complexity. • Increasing reviewer load had large negative effects on validity for high complexity task. [ABSTRACT FROM AUTHOR]

Learners encounter science in a wide variety of contexts beyond the science classroom which collectively could be quite influential on student attitudes and abilities. But relatively little is known about the relative influence of different forms of informal science experiences, especially for the kinds of experiences that students typically access. We conduct factor and regression analyses on data collected from a large number of diverse public-school attending 6th and 8th graders drawn from two regions in the USA. Students completed a science reasoning measure and surveys of attitudes, previously completed informal science learning experiences, and demographic factors. Factor analyses identify four dimensions of informal science learning participation (in home, semiformal, nature, and museums). Regression analyses find a relative specificity of effects, with particular outcomes associated with a subset of the forms of informal science participation, highlighting the importance of controlling for correlated factors. There were also a few differences by grade level, with different experiences influencing the development of competency beliefs in science in early vs. late middle schoolers. [ABSTRACT FROM PUBLISHER]

Although feedback is often seen as a critical component of the learning process, many open questions about how specific feedback features contribute to the effectiveness of feedback remain—especially in regards to peer feedback of writing. Nelson and Schunn (2009) identified several important features of peer feedback in their nature of feedback model. In the current study, we test an updated theoretical model that includes a broader set of features and considers not only students' likelihood of implementing a comment but also the quality of their revisions. To empirically test the updated theoretical model, we analyze over 7,500 comments from 351 reviewers to 189 authors. Each comment was coded for the presence of praise, a problem description, a suggested solution, localization, focus (i.e., low prose, high prose, substance), implementation, and revision quality. To account for the cross-classified nesting of data, we used a 2-level, cross-classified, hierarchical logistic regression model. Only 2 feedback features increased students' likelihood of implementation (i.e., overall praise and localization), while several feedback features reduced students' likelihood of implementation (i.e., mitigating praise, solutions, and high-prose comments). Overlapping feedback features affected students' ability to revise and in opposing directions from their effect on likelihood of implementation: Revisions were less likely to improve the quality of their paper when implementing comments that included a specific location in the text, but they were more likely to improve the quality of their paper when implementing comments that focused on high-prose and substance issues. Implications of these findings are discussed. [ABSTRACT FROM AUTHOR]

ABSTRACT Amid calls for integrating science, technology, engineering, and mathematics (iSTEM) in K-12 education, there is a pressing need to uncover productive methods of integration. Prior research has shown that increasing contextual linkages between science and mathematics is associated with student problem solving and conceptual understanding. However, few studies explicitly test the benefits of specific instructional mechanisms for fostering such linkages. We test the effect of students developing a modeled process mathematical equation of a scientific phenomenon. Links between mathematical variables and processes within the equation and fundamental entities and processes of the scientific phenomenon are embedded within the equation. These connections are made explicit as students participate in model development. Pre-post gains are tested in students from diverse high school classrooms studying inheritance. Students taught using this instructional approach are contrasted against students in matched classrooms implementing more traditional instruction (Study 1) or prior traditional instruction from the same teachers (Study 2). Students given modeled process instruction improved more in their ability to solve complex mathematical problems compared to traditionally instructed students. These modeled process students also show increased conceptual understanding of mathematically modeled processes. The observed effects are not due to differences in instructional time or teacher effects. [ABSTRACT FROM AUTHOR]

Peer feedback activities are characterized by peers performing both roles of feedback provider and receiver, and peers are found to benefit from both roles. However, in current feedback literacy studies, the emphasis has been overwhelmingly given to students' understandings, capacities and dispositions in receiving feedback, and existing student feedback literacy scales measure only the receiving side of feedback literacy. Students learn from providing and what they provide fundamentally shapes what others receive. This study conceptualizes peer feedback literacy as students' capacities and attitudes in providing and receiving peer feedback, and developed and validated a peer feedback literacy scale on the basis of responses from 474 Chinese university students. Exploratory and confirmatory factor analyses were conducted, and four factors emerged: feedback-related knowledge and abilities, cooperative learning ability, appreciation of peer feedback, and willingness to participate. The resulting scale showed good psychometric properties and revealed some surprising associations with student major, year and amount of prior experience. [ABSTRACT FROM AUTHOR]

Theories of creative conceptual combination hypothesize that, to generate highly creative concepts, one should attempt to combine source concepts that are very different from each other. While lab studies show a robust link between far combinations and increased novelty of concepts, empirical evidence that far combinations lead to more creative concepts (i.e., both more novel and of higher quality) is mixed. Drawing on models of the creative process, we frame conceptual combination as a divergent process, and hypothesize that iteration is necessary to convert far combinations into creative concepts. We trace conceptual genealogies of many hundreds of concepts proposed for a dozen different problems on a large-scale Web-based innovation platform, and model the effects of combination distance on creative outcomes of concepts. The results are consistent with our predictions: (1) direct effects of far combinations have a mean zero effect, and (2) indirect effects of far combinations (i.e., building on concepts that themselves build on far combinations) have more consistently positive effects. This pattern of effects is robust across problems on the platform. These findings lend clarity to theories of creative conceptual combination, and highlight the importance of iteration for generating creative concepts. [ABSTRACT FROM AUTHOR]

ABSTRACT Understanding the features of science learning experiences that organize and motivate children at early ages can help educators and researchers find ways to ignite interest to support future passion and learning in the sciences at a time when children's motivation is declining. Using a sample of 252 fifth- and sixth-grade students, we systematically explore differences in children's motivations toward science experiences across context (formal, informal, neutral), manner of interaction (consuming new knowledge, analyzing, action), and topic (e.g., biology, earth science, physics). Motivations toward science were most influenced by topic. Responses were generally consistent across context and manner of interaction. Implications for science education, as well as measurement and assessment methodology, are discussed. [ABSTRACT FROM AUTHOR]

A large-scale online teacher resource exchange is studied to examine the ways in which metadata influence teachers’ selection of resources. A hierarchical linear modeling approach was used to tease apart the simultaneous effects of resource features and author features. From a decision heuristics theoretical perspective, teachers appear to rely on complex heuristics that integrate many dimensions when determining whether to download a resource. Most surprisingly, numbers of ratings more strongly predict downloads than do mean rating levels, such that multiple negative ratings appear to attract more downloads than do few positive ratings. Implications for system design are discussed. [ABSTRACT FROM AUTHOR]

Computer science proficiency continues to grow in importance, while the number of students entering computer science-related fields declines. Many rich programming environments have been created to motivate student interest and expertise in computer science. In the current study, we investigated whether a recently created environment, Robot Virtual Worlds (RVWs), can be used to teach computer science principles within a robotics context by examining its use in high-school classrooms. We also investigated whether the lack of physicality in these environments impacts student learning by comparing classrooms that used either virtual or physical robots for the RVW curriculum. Results suggest that the RVW environment leads to significant gains in computer science knowledge, that virtual robots lead to faster learning, and that physical robots may have some influence on algorithmic thinking. We discuss the implications of physicality in these programming environments for learning computer science. [ABSTRACT FROM PUBLISHER]

Abstract: Prior studies of mind wandering find the default network active during mind wandering, but these studies have yielded mixed results concerning the role of cognitive control brain regions during mind wandering. Mind wandering often interferes with reading comprehension, and prior neuroimaging studies of discourse comprehension and strategic reading comprehension have shown that there are at least two networks of brain regions that support strategic discourse comprehension: a domain-general control network and a network of regions supporting coherence-building comprehension processes. The present study was designed to further examine the neural correlates of mind wandering by examining mind wandering during strategic reading comprehension. Participants provided ratings of mind wandering frequency that were used to investigate interactions between the strategy being performed and brain regions whose activation was modulated by wind wandering. The results support prior findings showing that cognitive control regions are at times more active during mind wandering than during a task with low control demands, such as rereading. This result provides an initial examination of the neural correlates of mind wandering during discourse comprehension and shows that the processes being engaged by the primary task need to be considered when studying mind wandering. The results also replicate, in a different learning domain, prior findings of key brain areas associated with different reading strategies. [Copyright &y& Elsevier]

Abstract: Creative teamwork in multidisciplinary teams is a topic of interest to cognitive psychologists on the one hand, and to both social and organizational psychologists on the other. However, the interconnections between cognitive and social layers have been rarely explored. Drawing on mental models and dissonance theories, the current study takes a central variable studied by cognitive psychologists—analogy—and examines its relationship to a central variable examined by social psychologists—conflict. In an observational, field study, over 11h of audio–video data from conversations of the Mars Exploration Rover scientists were coded for different types of analogy and micro-conflicts that reveal the character of underlying psychological mechanisms. Two different types of time-lagged logistic models applied to these data revealed asymmetric patterns of associations between analogy and conflict. Within-domain analogies, but not within-discipline or outside-discipline analogies, preceded science and work process conflicts, suggesting that in multidisciplinary teams, representational gaps in very close domains will be more likely to spark conflict. But analogies also occurred in reaction to conflict: Process and negative conflicts, but not task conflicts, preceded within-discipline analogies, but not to within-domain or outside-discipline analogies. This study demonstrates ways in which cognition can be bidirectionally tied to social processes and discourse. [Copyright &y& Elsevier]

Abstract: The internal representation of numbers generated during calculation has received little attention. Much of the mathematics learning literature focuses on symbolic retrieval of math facts; in contrast, we critically test the hypothesis that internally generated numbers are represented analogically, using an approximate number system. In an fMRI study, the spontaneous processing of arithmetical expressions was tested. Participants passively viewed a sequence of double-digit addition expressions that summed to the same number. Adaptation was found in number-related regions in a fronto–parietal network. Following adaptation, arrays of dots were introduced, differing in their numerical distance from the sum of the addition expressions. Activation in voxels that showed adaptation to a repeated sum was also sensitive to the distance of the dot quantity from the sum. We conclude that participants exhibited adaptation to an internally generated number, that adapted representations were analogic in nature, and that these analogic representations may undergird arithmetic calculation. [Copyright &y& Elsevier]

Summary Understanding the best way to present information has been an issue of interest to many researchers. Regardless of the content of the information, purely physical elements such as spatial organization may also influence performance. Across two studies with 111 undergraduates (78 in Study 1 and 33 in Study 2), we compared spatially distributed (i.e. when information sources are presented side-by-side) against stacked displays (i.e. when information sources are sitting on top of one another with only the top source fully visible). A distributed display time advantage was consistently observed. As potential explanations, a memory strategy selection hypothesis was examined along with other procedural alternative explanations. Copyright © 2011 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

Abstract: Neuroimaging studies of text comprehension conducted thus far have shed little light on the brain mechanisms underlying strategic learning from text. Thus, the present study was designed to answer the question of what brain areas are active during performance of complex reading strategies. Reading comprehension strategies are designed to improve a reader''s comprehension of a text. For example, self-explanation is a complex reading strategy that enhances existing comprehension processes. It was hypothesized that reading strategies would involve areas of the brain that are normally involved in reading comprehension along with areas that are involved in strategic control processes because the readers are intentionally using a complex reading strategy. Subjects were asked to reread, paraphrase, and self-explain three different texts in a block design fMRI study. Activation was found in both executive control and comprehension areas, and furthermore, learning from text was associated with activation in the anterior prefrontal cortex (aPFC). The authors speculate that the aPFC may play a role in coordinating the internal and external modes of thought that are necessary for integrating new knowledge from texts with prior knowledge. [Copyright &y& Elsevier]

An empirically sensitive formulation of the norms of transformative criticism must recognize that even public and shared standards of evaluation can be implemented in ways that unintentionally perpetuate and reproduce forms of social bias that are epistemically detrimental. Helen Longino's theory can explain and redress such social bias by treating peer evaluations as hypotheses based on data and by requiring a kind of perspectival diversity that bears, not on the content of the community's knowledge claims, but on the beliefs and norms of the culture of the knowledge community itself. To illustrate how socializing cognition can bias evaluations, we focus on peer-review practices, with some discussion of peer-review practices in philosophy. Data include responses to surveys by editors from general philosophy journals, as well as analyses of reviews and editorial decisions for the 2007 Cognitive Science Society Conference. [ABSTRACT FROM AUTHOR]

Learning requires applying limited working memory and attentional resources to intrinsic, germane, and extraneous aspects of the learning task. To reduce the especially undesirable extraneous load aspects of learning environments, cognitive load theorists suggest that spatially integrated learning materials should be used instead of spatially separated materials, thereby reducing the split-attention effect (Sweller & Chandler, 1994). Recent work, however, has suggested a new distinction between two common formats of spatially separated displays: spatially distributed versus spatially stacked (Jang & Schunn, 2010). Moreover, a distinction between instructions and learning task materials has rarely been made. Across two studies with 106 college students (56 in Study 1 and 50 in Study 2), we compared spatially distributed (multiple sources of information are placed side by side) versus spatially stacked (only one at the top is visible) instructions, without changing the learning task materials, on both task performance and learning. With materials more typical of practice, Study 1 showed that the distributed-display instructions led learners to more efficient learning; learners finished the task faster and scored higher in the overall learning test. With materials more tightly controlled for spatial format per se, Study 2 replicated the effect and found that the benefit of the distributed instructions appeared to be associated with changes in cognitive load. Implications for educational practice are discussed. [ABSTRACT FROM AUTHOR]

With only two to five slots of visual working memory (VWM), humans are able to quickly solve complex visual problems to near optimal solutions. To explain the paradox between tightly constrained VWM and impressively complex human visual problem-solving ability, we propose several principles for dynamic VWM allocation. In particular, we propose that complex visual information is represented in a temporal manner using only a few slots of VWM that include global and local visual chunks. We built a model of human traveling salesman problem solving based on these principles of VWM allocation and tested the model with eye-movement data. Exactly as the model predicted, human eye movements during traveling salesman problem solving have precise quantitative regularities with regard to both the general statistical pattern of attentional fixations and how they vary across individuals with different VWM capacities. Even though VWM capacity is very limited, eye movements dynamically allocate VWM resources to both local and global information, enabling attention to fine details without loss of the big picture. [ABSTRACT FROM AUTHOR]

The psychology of science typically lacks integration between cognitive and social variables. We present a new framework of team innovation in multidisciplinary science and engineering groups that ties factors from both literatures together. We focus on the effects of a particularly challenging social factor, knowledge diversity, which has a history of mixed effects on creativity, most likely because those effects are mediated and moderated by cognitive and additional social variables. In addition, we highlight the distinction between team innovative processes that are primarily divergent versus convergent; we propose that the social and cognitive implications are different for each, providing a possible explanation for knowledge diversity's mixed results on team outcomes. Social variables mapped out include formal roles, communication norms, sufficient participation and information sharing, and task conflict; cognitive variables include analogy, information search, and evaluation. This framework provides a roadmap for research that aims to harness the power of multidisciplinary teams. [ABSTRACT FROM AUTHOR]

The focus of this research was an evaluation of the impact of teacher professional development (PD) on student achievement during implementation of a reform curriculum. The PD consisted of five four-hour workshop sessions distributed over the time teachers were implementing the reform curriculum in their classrooms. The research was conducted in a mid-size, urban school district over the span of two years. Three groups of teachers were contrasted: teachers who continued to use the established curriculum (N = 5), teachers who implemented the reform curriculum without participating in the PD sessions (N = 5), and teachers who implemented the reform curriculum while participating in the PD sessions (N = 13). Teachers who participated in the PD had approximately a one standard deviation advantage in their students' achievement over those who did not. We collected evidence of particular features of the PD that explained the differences in student achievement. The features included: distributing the workshops throughout the implementation; engaging teachers in an active learning process situated in the curriculum; and facilitating a collaborative community of teacher professionals. This study led us to believe that not only are the individual features of the PD important, but the combination of all three together is particularly powerful. [ABSTRACT FROM AUTHOR]

This study examines the use of engineering design to facilitate science reasoning in high-needs, urban classrooms. The Design for Science unit utilizes scaffolds consistent with reform science instruction to assist students in constructing a design solution to satisfy a need from their everyday lives. This provides a meaningful context in which students could reason scientifically. Eighth grade students from two urban schools participated in the unit. Both schools contained large percentages of racial/ethnic minority and economically disadvantaged students. Students demonstrated statistically significant improvement on a paper-and-pencil, multiple-choice pre and post assessment. The results compare favorably with both a high-quality inquiry science unit and a traditional textbook curriculum. Implications for the use of design-based curricula as a viable alternative for teaching science reasoning in high-needs, urban settings are discussed. [ABSTRACT FROM AUTHOR]

Although theories of mental simulations have used different formulations of the premises of ‘thought experiments’, they can be fitted under a minimalist hypothesis stating that mental simulations are run under situations of uncertainty to turn that uncertainty into approximate answers. Three basic assumptions of mental simulations were tested by using naturalistic data from engineering design. Results from the design protocols showed (1) initial representations in mental simulation had higher than base-rate uncertainty, (2) uncertainty in mental simulations were lowered after simulation runs, (3) resulting representations had more approximations than base-rate or initial representations. Further, the reference to external representational systems (sketches and prototypes) was examined. It was found that prototypes had fewer technical/functional simulations compared to sketches or unsupported cognition. Although prototypes were associated with more approximation than unsupported cognition, the different external representation categories did not differ in information uncertainty. The results support the minimalist hypothesis of mental simulations. Copyright © 2008 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

Across disciplines and educational levels, peer feedback has emerged as a broadly useful pedagogical strategy. However, the value of peer feedback depends upon students being willing to provide each other substantial feedback. We develop a novel application of game theory to study whether students' experiences with interaction inequality in peer feedback predict lower participation in future peer feedback assignments. Two kinds of inequality are explored: inequality in amount and inequality in the helpfulness of received vs. providing feedback. We examine data from students (N = 732) enrolled in the three different courses (varying in discipline and level) using the same online peer review platform. We use advanced multiple regression models to test four theoretically-derived hypotheses regarding the overall effects of experienced inequality, three hypotheses involving contextual moderators of that effect. The results confirmed the hypothesized relationship of experiencing inequality in the number and quality of feedback provided vs. received with the subsequent depth of participation in peer reviewing. The results also generally confirmed the three predicted moderators of the relationship: stronger observed relationships with inequality experiences for weaker students, in later assignments, and in more advanced coursework. This study is unique in applying and extending game theory to computerized peer review and in its approach for studying the relationship of specific prior experiences with future peer review participation. The findings provide new practical insights into what is essential to manage in peer review processes. • Novel application of Game Theory to study peer reviewing behaviors. • Experienced inequality is associated with lower peer feedback participation. • Inequality moderated by task performance, course level, and amount of experience. [ABSTRACT FROM AUTHOR]

Abstract: Human visual/spatial problem solving often requires both global and local information to be processed. But the relationship between those two kinds of information and the way in which they interact with one another during problem solving has not been thoroughly discussed. In the particular setting of solving the traveling salesman problem (TSP), we investigated into the relative roles of global and local information processing. An experiment was conducted to measure the importance of global information and the possible constraints of global information processing on search. A model was built to simulate human TSP performance and was used to investigate further the relationship between global information processing and local information processing. Our model was compared with the human data we collected and with other models of human TSP solving. [Copyright &y& Elsevier]

Abstract: This paper describes how SWoRD (scaffolded writing and rewriting in the discipline), a web-based reciprocal peer review system, supports writing practice, particularly for large content courses in which writing is considered critical but not feasibly included. To help students gain content knowledge as well as writing and reviewing skills, SWoRD supports the whole cycle of writing, reviews, back-reviews, and rewriting by scaffolding the journal publication process as its authentic practice model. In addition, SWoRD includes algorithms that compute individual reviewer’s review accuracy, which is in turn used to support the various drawbacks of reciprocal peer reviews (e.g., variation in motivation or ability of reviewers). Finally, this paper describes an empirical evaluation showing that the SWoRD approach is effective in improving writing quality in content classes. [Copyright &y& Elsevier]

Analogy was studied in real-world engineering design, using the in vivo method. Analogizing was found to occur frequently, entailing a roughly equal amount of within- and between-domain analogies. In partial support for theories of unconscious plagiarism (Brown & Murphy, 1989; Marsh, Landau, & Hicks, 1996) and Ward's (1994) path-of-least-resistance model, it was found that the reference to exemplars (in the form of prototypes) significantly reduced the number of between-domain analogies between source and target, as compared with using sketches or no external representational systems. Analogy served three functions in relation to novel design concepts: identifying problems, solving problems, and explaining concepts. Problem identifying analogies were mainly within domain, explanatory analogies were mainly between domain, and problem-solving analogies were a mixture of within- and between-domain analogies. [ABSTRACT FROM AUTHOR]