Search

Although peer review has been widely used for formative assessment in writing instruction, there remain concerns about whether assessors are at a sufficient writing performance level that would allow them to identify major problems in the reviewed work and provide helpful feedback to improve draft quality. Little empirical research has examined how assessor writing performance specifically influences problem identification accuracy and helpfulness of feedback, nor has it acknowledged different grain sizes of assessor performance. Assessor writing performance at different grain sizes (i.e., performance at the levels of genre, dimension of a genre, and specific problem topic) was assessed alongside problem identification accuracy and feedback helpfulness in 234 high school students who participated in an anonymous multipeer review in a secondary writing course in the United States. A correlation analysis showed that assessor performance levels on specific problem topics were meaningfully separable, thereby allowing for consideration of the effects of assessor performance at genre, dimension, and topic levels. Multiple regression results indicated that assessor writing performance was unrelated to problem identification accuracy at any grain size. Therefore, scaffolds in the reviewing process appear sufficient to support problem identification accuracy. However, assessor writing performance, particularly on specific dimensions and specific topics, consistently predicted helpfulness of feedback, even though lower performing assessors rarely produce incorrect advice. Theoretical and practical implications of the findings are discussed.

Background: Motivational factors are one active area of research that aims to increase the inclusion of women in physics. One of these factors that has only recently gained traction in physics is intelligence mindset (i.e., the belief that intelligence is either innate and unchangeable or can be developed). We studied 781 students in calculus-based Physics 1 to investigate if their mindset views were separable into more nuanced dimensions, if they varied by gender/sex and over time, and if they predicted course grade. Confirmatory factor analysis was used to divide mindset survey questions along two dimensions: myself versus others and growth versus ability aspects of mindset. Paired and unpaired t-tests were used to compare mindset factors over time and between genders, respectively. Multiple regression analysis was used to find which mindset factors were the best predictors of course grade. Results: This study shows that intelligence mindset can be divided into four factors: My Ability, My Growth, Others' Ability, and Others' Growth. Further, it reveals that gender differences are more pronounced in the "My" categories than the "Others'" categories. At the start of the course, there are no gender differences in any mindset component, except for My Ability. However, gender differences develop in each component from the start to the end of the course, and in the My Ability category, the gender differences increase over time. Finally, we find that My Ability is the only mindset factor that predicts course grade. Conclusion: These results allow for a more nuanced view of intelligence mindset than has been suggested in previous interview and survey-based work. By looking at the differences in mindset factors over time, we see that learning environments affect women's and men's intelligence mindsets differently. The largest gender difference is in My Ability, the factor that best predicts course grade. This finding has implications for developing future mindset interventions and opens new opportunities to eliminate classroom inequities.

Physics is a science, technology, engineering, and mathematics discipline in which women are severely underrepresented. Prior work has identified motivation-based explanations for low participation and retention rates of women in physics. Among various motivational factors, intelligence mindsets (i.e., having fixed or growth mindsets) have been rarely examined in the context of physics. Because physics is commonly associated with requiring brilliance to be successful, many students are likely to hold fixed mindset views for physics, which can be especially detrimental for students from underrepresented groups. We examined physics mindset views of 755 engineering and physical-science majors enrolled in calculus-based Physics 1 to understand separable aspects of such views, how these views varied by gender or sex (i.e., for female versus male students), and whether they predicted physics course grade. Multidimensional scaling analyses revealed four different mindset views that are relatively independently held beliefs. Multiple regression analyses showed that physics course grade is most closely associated with whether students deny or accept a fixed mindset view about themselves in particular. One particular view also had the largest gender difference. Therefore, understanding why students hold different mindset views and designing appropriate interventions for physics courses are important areas to consider in efforts seeking to improve outcomes in physics for diverse student groups.

Peer review is regularly found to be a powerful and efficient technique for assessment and feedback, but many students are inexperienced and sometimes struggle to provide meaningful feedback. It is considered best practice to provide students with some training on how to be a good reviewer, but few classes can afford to devote much time to such training, and the assumption is that review quality will improve with experience. This study directly examines what kinds of experiences during peer feedback activities improve reviewing quality. In particular, organized by theories of norm-setting and practice-based learning, it examines the relationship of the amount of provided and received feedback on one assignment to improvements in the quality of feedback on the next assignment. Data on peer feedback experiences and behaviors across multiple assignments were taken from students across two introductory level undergraduate courses (N = 360). Multi-regression analyses reveal that the number and length of feedback provided predicted growth in helpfulness rates, and both improvements in domain performance and the reviewer's preference for length explains the effects on review helpfulness. Further, compared with high-performing students, low-performing students show more remarkable growth in helpfulness from providing feedback.

Digital badges have long been assumed to possess motivational qualities that could encourage learners to engage with learning content. However, prior studies have found the effects of badges to be complex, differing by learner, type of badge, and potentially other factors. Qualitative reports suggest that individuals' perceptions of digital badges may play a role in moderating badges' effects: badges are only motivating when they are perceived as relevant and desirable. In the current study, we examine longitudinal episodes of data from 2,410 middle and high school users of a badged online programming curriculum to test whether there is evidence that learners' perceptions of badges' relevance predict changes in engagement over time; and whether that relationship is equitable with respect to age, sex, and ethnicity. We also investigate whether a reciprocal relationship may exist in which engagement predicts relative increases in learners' perceptions of badges as relevant to them. Learners' positive perceptions of badges' relevance predicted rank-order increases in engagement over time. Further, this relationship did not vary by age, sex, or minoritized racial/ethnic status. In addition, higher engagement also predicted upward shifts in perceived badge relevance. These results suggest that learners' subjective evaluations of digital badges are closely related to changes in their engagement with program activities, that badged engagement neither widens nor closes educational equity gaps, and that learners' regard for badges and engagement in program activities may be mutually reinforcing.

Self-efficacy is an aspect of students' motivation that has been shown to play a critical role in students' engagement, participation, and retention in academic careers in science, technology, engineering, and mathematics (STEM). Since women are underrepresented in STEM domains such as physics, we studied female and male students' self-efficacy and its relation to learning outcomes in physics that can be useful for creating equitable and inclusive learning environments. In a longitudinal study, we surveyed approximately 1400 students in calculus-based physics 2 courses to investigate students' motivational beliefs in physics using a validated survey. We examined female and male students' self-efficacy scores and the extent to which self-efficacy related to learning outcomes (students' grades and conceptual post-test scores), especially the significant gender difference in conceptual post-test scores. To reveal the unique contribution of self-efficacy on outcomes, we controlled for several other variables including Physics 1 grades, SAT math scores, and conceptual pretest scores in physics. We found that the gender differences in conceptual post-test performance were mediated by the model variables. In particular, initial self-efficacy differences showed a direct effect on outcomes even when controlling for students' prior physics knowledge and skill differences, and self-efficacy also had the strongest total gender effect on conceptual learning. Given these findings, future work should focus on better understanding the drivers of these self-efficacy differences including the role that societal stereotypes and biases play in these in order to mitigate these differences.

Background: Large achievement and motivation gaps exist in science between students from higher and lower socioeconomic status (SES) backgrounds. Middle and high school are an important time to address these disparities, as science motivation typically declines for all students at this time, leading to particularly low science interest and achievement for lower SES students on average when the gaps are left unaddressed. Students' control over their free time also increases at this time, providing opportunities for optional science experiences that may improve science attitudes and skills to combat these achievement and motivation gaps. Using a longitudinal dataset of 2252 middle and high school students from two regions in the USA, we investigate (1) disparities between higher and lower SES students in participation in optional summer science experiences and post-summer science attitudes and skills; (2) whether the child and family characteristics that predict participation in home-related, nature-related, and STEM camp experiences in the summer differ for higher and lower SES students; and (3) how participation in these types of optional summer science experiences contribute to post-summer science attitudes and skills when controlling for self-selection biases. Results: Higher SES students reported greater participation in optional summer science experiences and higher post-summer science attitudes and sensemaking skills. Fascination for science was more important for participation in home-related and nature-related experiences for higher SES participants, whereas science competency beliefs were more important for lower SES participants. For STEM camp experiences, higher SES participants with higher competency beliefs and lower SES participants with lower scientific sensemaking skills were more likely to participate. After controlling for self-selection biases that may influence participation in these experiences, we found that home-related and nature-related experiences had a positive impact on students' attitudes toward science. Conclusions: Our findings suggest two pathways for increasing participation in optional summer science experiences for higher SES and lower SES students. Specifically, it may be helpful to support interest in science for higher SES students and competency beliefs for lower SES students. Greater participation in home-related and nature-related summer science experiences can also increase science attitudes during middle and high school.

Gender differences in students' physics identity in introductory physics courses can influence students' interest in science, technology, engineering, and mathematics and their career decisions. Exploring the components that influence these identities is critical to developing a better understanding of the underrepresentation of women in physics courses and physics-related majors. We used a revised version of the physics identity framework developed by Hazari et al. [J. Res. Sci. Teach. 47, 978 (2010)] to investigate whether the relation between gender and physics identity was mediated by motivational factors, such as competency belief, interest, and perceived recognition by others. We surveyed approximately 500 students in introductory level calculus-based physics courses in which 30% of the students are women. Analysis revealed that the relation between gender and physics identity was mediated by students' self-reported motivation at the end of the semester. The model showed that perceived recognition by others played a major role in students' endorsement of physics identity with female students less likely to endorse statements that others perceived them as a "physics person."

Students' intentions to persevere and their career choices in science, technology, engineering, and math fields can be impacted by their physics identities. Women are severely underrepresented at all levels in physics and engineering. Physics in particular has stereotypes about being a discipline for brilliant men. Therefore, it is particularly difficult for women who do not fit the description of a stereotypical physicist to develop a physics identity. Thus, understanding the factors underlying physics identity in introductory physics classrooms is important for creating an equitable and inclusive physics learning environment and has the potential to at least partly explain the current under-representation of women in physics-related majors and careers. In this study, we examined physics identity and several other motivational constructs of male and female students by administering a survey in introductory calculus-based physics courses at a large research university. We found gender differences in how students identify as a physics person and how their perceived recognition from others, such as their teaching assistants or instructors, peers, or family members relates to their physics identities. We tested separate models by gender that examined how different motivational constructs relate to students' physics identities. We found that the perception of being recognized by influential others such as the course instructor or teaching assistants was differentially related to female and male students' physics self-efficacy and sense of belonging in the physics classroom. These findings call for improving the physics learning environments to make them equitable so that all students have a high sense of belonging and self-efficacy and opportunity to develop a strong physics identity.

Science learning is most often examined within formal education contexts, even though students spend more of their lives outside of the classroom. Students may interact with scientific phenomena during these out-of-school experiences, providing additional opportunities for learning to take place. Prior studies have found that optional science experiences have positive effects on science knowledge and attitudes. However, these studies do not always account for initial differences between students who are able to participate in many optional experiences and those who cannot. Moreover, many studies focus on high-quality science programs, which may not be representative of the average out-of-school experience. Using a longitudinal dataset of 3,700 6th and 8th grade students in urban and suburban schools from two regions in the United States, the current study investigates the effects of typically occurring optional science experiences during the school year on the development of science knowledge and attitudes. Using propensity score matching, we matched participating and nonparticipating students for characteristics that drive self-selection into these experiences, then analyzed separately for effects of school-related versus home-related science learning experiences. Stable patterns across analytic models reveal growth in science attitudes with both school- and home-related science experiences, but a greater relative decline in science knowledge with school-related science experiences. Thus, typically occurring optional science experiences can influence students' attitudes and knowledge, but the effects can vary by the type of experience that students attend.

We examined the influence of accountability on the consistency of peer ratings and quality of peer feedback by comparing three conditions: only rating accountability, only feedback accountability, or both rating and feedback accountability. From a large undergraduate course, 287 students' peer ratings and peer feedback were coded for rating consistency, comment helpfulness, amount of feedback, and feedback features. Because only 30% of the students accurately perceived their assigned condition, data were analyzed according to the perceived condition. Students who believed their reviewing grade would be influenced by the helpfulness of their feedback not only provided more feedback, but also more criticism, solutions, and localized comments. These students also provided more consistent ratings than those who thought their reviewing grade would be influenced by the consistency of their ratings. These findings indicate that constructing helpful comments could have a broad influence on peer assessment and consistent ratings are grounded in commenting.

The number of women studying STEM careers and pursuing graduate degrees has not changed in the last decade (National Student Clearinghouse Research Center, 2015; Science & Engineering Degree Attainment: 2004-2014). Most prior research to explain this problem has focused on the topics of identity, access, pedagogy, and choice (Brotman & Moore, 2008; Journal of Research in Science Teaching, 45, 971-1002). Additional research is needed on how internal and external factors interact with one another to demotivate girls and young women from pursuing science careers. Here, we show how girls' competency beliefs are an essential foundation for science content learning during middle school and how these effects of competency beliefs are mediated by in and out-of-school factors. We recruited over 2,900 6th and 8th grade students from two different regions in the United States. At two different time points, students completed surveys asking about their stance toward science such as competency beliefs in science, willingness to engage in argumentation, and choice preferences toward optional science experiences. We also collected a reasoning ability measure, and pre- and post-tests on science content knowledge. Moreover, students also reported on their cognitive behavioral engagement during a sampled science class on two separate occasions. Multiple regression and mediation analyses show that as boys grow older, their willingness to engage in argumentation and to participate in science experiences suppresses the role of competency beliefs on their learning science content. By contrast, as girls grew older, they showed an increasing need to have high competency beliefs to achieve strong content learning gains. Our results demonstrate that despite girls' willingness to participate in scientific argumentation and to take part in science experiences, they probably do not receive enough support in their environment to access the benefits of these experiences, and hence they have a stronger need to have high competency beliefs in order to achieve significant growth in science learning.

Learners encounter science in a wide variety of contexts beyond the science classroom which collectively could be quite influential on student attitudes and abilities. But relatively little is known about the relative influence of different forms of informal science experiences, especially for the kinds of experiences that students typically access. We conduct factor and regression analyses on data collected from a large number of diverse public-school attending 6th and 8th graders drawn from two regions in the USA. Students completed a science reasoning measure and surveys of attitudes, previously completed informal science learning experiences, and demographic factors. Factor analyses identify four dimensions of informal science learning participation (in home, semiformal, nature, and museums). Regression analyses find a relative specificity of effects, with particular outcomes associated with a subset of the forms of informal science participation, highlighting the importance of controlling for correlated factors. There were also a few differences by grade level, with different experiences influencing the development of competency beliefs in science in early vs. late middle schoolers.

Background: Robotics competitions are increasingly popular and potentially provide an on-ramp to computer science, which is currently highly gender imbalanced. However, within competitive robotics teams, student participation in programming is not universal. This study gathered surveys from over 500 elementary, middle, and high school robotics competition participants to examine (1) whether programming involvement in these competitions is associated with motivation to pursue additional programming experiences and (2) whether opportunities to learn programming varied by gender, age, and competition type. Results: Results showed a significant association of students' programming involvement with their motivation to learn more programming. Interestingly, in the youngest groups/entry-level competitions, girls were heavily involved in programming. Unfortunately, in older/more advanced competitions, girls were generally less involved in programming, even after controlling for prior programming experience. These gendered effects were substantially explained by programming interest. Conclusions: While robotics competition experiences may motivate students to learn more programming, gender gaps in programming involvement persist in these learning environments and appear to widen as students grow older and enter more advanced competitions. Therefore, addressing gender imbalances in programming will likely require greater attention to particular curricular and pedagogical characteristics of robotics competitions that support girls' interest and involvement in programming.

Prior research on peer assessment often overlooks how much students learn from providing feedback to peers. By practicing revision skills, students might strengthen their ability to detect, diagnose, and solve writing problems. However, both reviewer ability and the quality of the peers' texts affect the amount of practice available to learners. Therefore, the goal of the current study is to provide a first step towards a theoretical understanding about why students learn from peer assessment, and more specifically from providing feedback to peers. Students from a large Introduction to Psychological Science course were assigned four peers' papers to review. The reviewing ability of each student was determined, and to whom the students provided feedback was manipulated. The features and focus of the comments from a sample of 186 participants were coded, and the amount of each type was analyzed. Overall, reviewer ability and text quality did not affect the amount of feedback provided. Instead, the content of the feedback was affected by reviewer ability. Low reviewers provided more praise than high reviewers, whereas high reviewers provided more criticism than low reviewers. This criticism from high reviewers described more problems and offered more solutions, and it focused more often on high prose and substance. In the only significant reviewer ability × text quality interaction, high reviewers described more problems in the low-quality texts than in the high-quality texts, whereas low reviewers did not make this distinction. These results suggest that high reviewers and low reviewers may utilize different commenting styles, which could significantly impact the benefits of peer assessment.

We investigate students' negative perceptions about an online peer assessment system for undergraduate writing across the disciplines. Specifically, we consider the nature of students' resistance to peer assessment; what factors influence that resistance; and how students' perceptions impact their revision work. We do this work by first examining findings from an end-of-course survey administered to 250 students in ten courses across six universities using an online peer assessment system called SWoRD for their writing assignments. Those findings indicate that students have the most positive perceptions of SWoRD in those courses where an instructor graded their work in addition to peers (as opposed to peer-only grading). We then move to an in-depth examination of perceptions and revision work among 84 students using SWoRD and no instructor grading for assessment of writing in one university class. Findings from that study indicate that students sometimes regard peer assessment as unfair and often believe that peers are unqualified to review and assess students' work. Furthermore, students' perceptions about the fairness of peer assessment drop significantly following students' experience in doing peer assessment. Students' fairness perceptions--and drops in those perceptions--are most significantly associated with their perceptions about the extent to which peers' feedback is useful and positive. However, students' perceptions appear to be unrelated to the extent of their revision work. This research fills a considerable gap in the literature regarding the origin of students' negative perceptions about peer assessment, as well as how perceptions influence performance.

Across several decades, educational researchers have investigated the contribution of the learning environment to the attainment of educational goals, such as improving academic achievement and motivation to learn. The term learning environment not only includes physical activities in the classroom (e.g. experiments kits, computers), but also includes teaching methods, the type of learning in which pupils are engaged, and assessment methods. In this research, we refined an approach to measuring the impact of a variety of classroom features on many different learning outcomes through the lens of students' perception. A new instrument, the Design-Based Learning Environment Questionnaire (DBLEQ), was field-tested in an eighth-grade USA science classroom setting. This study examined pre-post changes for two different curricula, one emphasising design-based learning (n = 464) and another emphasising scripted inquiry (n = 248). The value of the instrument and ways of analysing its data are illustrated through the range of differences that were found between conditions over time.

Although peer reviewing of writing is a way to create more writing opportunities in college and university settings, the validity and reliability of peer-generated grades are a major concern. This study investigated the validity and reliability of peer-generated writing grades of 708 students across sixteen different courses from four universities in a particular scaffolded reviewing context: Students were given guidance on peer assessment, used carefully constructed rubrics, and were provided clear incentives to take the assessment task seriously. Distinguishing between instructor and student perspectives of reliability and validity, the analyses suggest that the aggregate ratings of at least four peers on a piece of writing are both highly reliable and as valid as instructor ratings while (paradoxically) producing very low estimates of reliability and validity from the student perspective. The results suggest that instructor concerns about peer evaluation reliability and validity should not be a barrier to implementing peer evaluations, at least with appropriate scaffolds. Future research needs to investigate how to address student concerns about reliability and validity and to identify scaffolds that may ensure high levels of reliability and validity.

Across disciplines and educational levels, peer feedback has emerged as a broadly useful pedagogical strategy. However, the value of peer feedback depends upon students being willing to provide each other substantial feedback. We develop a novel application of game theory to study whether students' experiences with interaction inequality in peer feedback predict lower participation in future peer feedback assignments. Two kinds of inequality are explored: inequality in amount and inequality in the helpfulness of received vs. providing feedback. We examine data from students (N = 732) enrolled in the three different courses (varying in discipline and level) using the same online peer review platform. We use advanced multiple regression models to test four theoretically-derived hypotheses regarding the overall effects of experienced inequality, three hypotheses involving contextual moderators of that effect. The results confirmed the hypothesized relationship of experiencing inequality in the number and quality of feedback provided vs. received with the subsequent depth of participation in peer reviewing. The results also generally confirmed the three predicted moderators of the relationship: stronger observed relationships with inequality experiences for weaker students, in later assignments, and in more advanced coursework. This study is unique in applying and extending game theory to computerized peer review and in its approach for studying the relationship of specific prior experiences with future peer review participation. The findings provide new practical insights into what is essential to manage in peer review processes. • Novel application of Game Theory to study peer reviewing behaviors. • Experienced inequality is associated with lower peer feedback participation. • Inequality moderated by task performance, course level, and amount of experience. [ABSTRACT FROM AUTHOR]

Although peer reviewing of writing is a way to create more writing opportunities in college and university settings, the validity and reliability of peer-generated grades are a major concern. This study investigated the validity and reliability of peer-generated writing grades of 708 students across 16 different courses from 4 universities in a particular scaffolded reviewing context: Students were given guidance on peer assessment, used carefully constructed rubrics, and were provided clear incentives to take the assessment task seriously. Distinguishing between instructor and student perspectives of reliability and validity, the analyses suggest that the aggregate ratings of at least 4 peers on a piece of writing are both highly reliable and as valid as instructor ratings while (paradoxically) producing very low estimates of reliability and validity from the student perspective. The results suggest that instructor concerns about peer evaluation reliability and validity should not be a barrier to implementing peer evaluations, at least with appropriate scaffolds. Future research needs to investigate how to address student concerns about reliability and validity and to identify scaffolds that may ensure high levels of reliability and validity. [ABSTRACT FROM AUTHOR]

• Investigation of effects of feedback features and perceptions on implementation. • Understanding and agreement with feedback predicted implementation. • Presence of solutions predicted understanding feedback. • Mitigating praise predicted agreement of the problem. • Explanation and hedges predicted implementation directly. Although the effects of peer feedback have been studied from a number of perspectives, much remains to be learned about what leads students to act (or not) on their peers' feedback in revisions. The present study examined the relationship between peer feedback features, student perceptions as potential mediators (understanding versus agreement with the feedback), and the likelihood of students' implementation of the feedback. Peer feedback, back-evaluation comments, and revisions from 185 US high school students were analyzed. Investigated feedback features included four cognitive features (identification, explanation, solution, suggestion), and two affective features (mitigating praise, hedges). Logistic regression analyses revealed that: (1) both understanding and agreement with feedback predicted implementation; (2) presence of solutions predicted understanding of feedback; (3) mitigating praise predicted agreement of the problem; and (4) explanation and hedges predicted implementation separately from perception effects. Theoretical and practical implications of the findings are discussed. [ABSTRACT FROM AUTHOR]