Search

Online peer feedback has proven to be practically useful for instructors and to be useful for learning, especially for the feedback provider. Because students can vary widely in skill level, some research has explored matching reviewer and author by performance level. However, past research on the impacts of reviewer matching has found little effect but used a simple binary high-low approach, which may mask the relative benefits of performance matching. In the current study, we leveraged a large dataset involving three large biology courses implementing multiple assignments with online peer feedback. This large dataset enabled dividing students into four levels of relative task performance to tease apart the relative contributions of providing and receiving feedback within the 16 different author-reviewer performance pairings. The results reveal that changes in task performance over assignments attributable to reviewing experiences vary by these finer-grained prior performance distinctions. In particular, providing to students at the same performance level appears to be beneficial, and receiving feedback from students at the same level is helpful except for very low-performing students. A simulation was used to examine the combined effects of receiving and providing under different algorithms for assigning reviewers to documents. The simulations suggest a matching algorithm will produce overall better outcomes than a random assignment algorithm for students at each of the four performance levels.

Background: Skill integration is vital in students' mastery development and is especially prominent in developing code tracing skills which are foundational to programming, an increasingly important area in the current STEM education. However, instructional design to support skill integration in learning technologies has been limited. Objectives: The current work presents the development and empirical evaluation of instructional design targeting students' difficulties in code tracing particularly in integrating component skills in the Trace Table Tutor (T3), an intelligent tutoring system. Methods: Beyond the instructional features of active learning, step-level support, and individualized problem selection of intelligent tutoring systems (ITS), the instructional design of T3 (e.g., hints, problem types, problem selection) was optimized to target skill integration based on a domain model where integrative skills were represented as combinations of component skills. We conducted an experimental study in a university-level introductory Python programming course and obtained three findings. Results and Conclusions: First, the instructional features of the ITS technology support effective learning of code tracing, as evidenced by significant learning gains (medium-to-large effect sizes). Second, performance data supports the existence of integrative skills beyond component skills. Third, an instructional design focused on integrative skills yields learning benefits beyond a design without such focus, such as improving performance efficiency (medium-to-large effect sizes). Major Takeaways: Our work demonstrates the value of designing for skill integration in learning technologies and the effectiveness of the ITS technology for computing education, as well as provides general implications for designing learning technologies to foster robust learning.

Research has shown that engaging students in peer feedback can help students revise documents and improve their writing skills. But the mechanistic pathways by which skills develop have remained untested: Does receiving and providing feedback lead to learning because it produces more extensive revision behavior or is such immediate implementation of feedback unnecessary? These pathways were tested through analyses of the relationships between feedback provided and received, feedback implemented and overall revisions, and improved writing quality in a new article. Overall, the number of revisions predicted growth in writing ability, and both amount of received and provided feedback were associated with being more likely to make revisions. However, providing feedback was also directly related to growth in writing ability.

Approximately 2 million students take Advanced Placement (AP) examinations annually. However, departmental policies that allow students to replace introductory courses with AP credit greatly vary within and across universities, even across relatively similar universities. This study examines the impact of AP credit policies on second-course success in introductory Biology, Chemistry, and Physics course sequences at six large public research universities (N = 48,230 students). Applying logistic regression analyses, we found that general performance indicators and measures of systemic inequities during high school were predictive of students earning skip-eligible scores. Interestingly, we descriptively discovered wide variation across institutions and disciplines in the percentage of students who chose to skip when meeting their local policies. However, logistic regression analysis did not find general trends of inequalities for skipping courses for students with skip-eligible scores. We then applied inverse-probability weights with regression adjustment to examine the effects of course skipping. We found that students who skipped their first course with AP credit actually performed similarly well or better in subsequent courses than did students who did not skip, even in contexts where lower AP scores were accepted. Therefore, our study suggests administrators to increase coherence in AP credit policies across departments and disciplines. In particular, we encourage modifying AP credit policies and academic advising to encourage students to skip when they have eligible AP scores. Benefits may include reductions in unnecessary coursework that is burdensome for both students and universities, potentially freeing up resources to better support students who were not privileged to enroll in AP courses during their high school education.

Background: Many studies have shown that ambitious, "student centered" approaches to STEM instruction benefit K-12 student learning. However, relatively little research has systematically investigated the learning processes that support teachers to skillfully enact these challenging pedagogies. In this study, we used a mixed-methods, case-comparison design to examine one kind of teacher learning routine, Mental Simulations for Teacher Reflection (MSTR), for advancing robust teacher learning in the context of one mathematics-focused instructional coaching intervention. Specifically, this study draws from a large, state-wide representative dataset to select carefully matched, contrasting cases to analyze the quality of coach-teacher conversations for teachers who showed very similar baseline instructional quality but then large differences in levels of improvement. We began by qualitatively coding detailed transcripts from selected coach-teacher pairs as they reflected on lesson artifacts (i.e., lesson plans, student work, and coach observations) using MSTR as an analytical lens. Next, quantitative analyses were conducted to determine the extent to which mental simulations characterized significant differences in the conversations of high- vs. low-instructional growth pairs. Lastly, additional qualitative analyses explored finer-grain distinctions in the quality of mental simulation talk in high- vs. low-growth pairs. Results: Quantitative analyses showed high-growth pairs were significantly more likely to engage in mental simulation talk compared to their low-growth counterparts. Moreover, the high-growth pairs were much more likely to initiate (i.e., raise an instructional ambiguity or problem for discussion) as well as complete (i.e., generate and weigh alternative instructional strategies) a MSTR routine. Qualitative analyses further revealed that engaging teachers' in-depth pedagogical reasoning to connect specific teaching moves to conceptual learning goals in mental simulations was a key distinction of the high-growth coaches. Conclusions: These findings indicate MSTR captured meaningful variation in coaching quality in this context. Notably, all coaches discussed the same instructional topics with teachers (i.e., teaching--learning goals and dimensions) and engaged in the same training that did not explicitly include MSTR, suggesting the possibility that MSTR captured a more implicit process of effective coaches. This study thus offers insight into the 'black box' of teacher learning and how it can be supported in similar professional learning contexts.

Although peer review has been widely used for formative assessment in writing instruction, there remain concerns about whether assessors are at a sufficient writing performance level that would allow them to identify major problems in the reviewed work and provide helpful feedback to improve draft quality. Little empirical research has examined how assessor writing performance specifically influences problem identification accuracy and helpfulness of feedback, nor has it acknowledged different grain sizes of assessor performance. Assessor writing performance at different grain sizes (i.e., performance at the levels of genre, dimension of a genre, and specific problem topic) was assessed alongside problem identification accuracy and feedback helpfulness in 234 high school students who participated in an anonymous multipeer review in a secondary writing course in the United States. A correlation analysis showed that assessor performance levels on specific problem topics were meaningfully separable, thereby allowing for consideration of the effects of assessor performance at genre, dimension, and topic levels. Multiple regression results indicated that assessor writing performance was unrelated to problem identification accuracy at any grain size. Therefore, scaffolds in the reviewing process appear sufficient to support problem identification accuracy. However, assessor writing performance, particularly on specific dimensions and specific topics, consistently predicted helpfulness of feedback, even though lower performing assessors rarely produce incorrect advice. Theoretical and practical implications of the findings are discussed.

Peer feedback activities are characterized by peers performing both roles of feedback provider and receiver, and peers are found to benefit from both roles. However, in current feedback literacy studies, the emphasis has been overwhelmingly given to students' understandings, capacities and dispositions in receiving feedback, and existing student feedback literacy scales measure only the receiving side of feedback literacy. Students learn from providing and what they provide fundamentally shapes what others receive. This study conceptualizes peer feedback literacy as students' capacities and attitudes in providing and receiving peer feedback, and developed and validated a peer feedback literacy scale on the basis of responses from 474 Chinese university students. Exploratory and confirmatory factor analyses were conducted, and four factors emerged: feedback-related knowledge and abilities, cooperative learning ability, appreciation of peer feedback, and willingness to participate. The resulting scale showed good psychometric properties and revealed some surprising associations with student major, year and amount of prior experience.

Students benefit from receiving and providing peer feedback, but the degree of participation limits the benefit. Further, students sometimes resist participation, providing few or only short comments. Prior researchers have examined the role of general attitudes toward peer feedback in limiting participation. However, little research has examined how peer feedback experiences predict the subsequent amount of feedback that students provide to peers. Data on peer feedback experiences and behaviors across multiple assignments were taken from students across two psychology courses (N = 360), two biology courses (N = 483), and one astronomy course (N = 170). The zero-inflated negative binomial (ZINB) regression analyses reveal that receiving fewer critical peer comments in the prior assignment, recognition for higher quality feedback in the prior assignment, and stronger performance on the current assignment predicted higher participation in peer feedback, but norm-setting did not appear to have a role. Implications for practitioners are discussed.

Background: For peer assessment, reliability (i.e., consistency in ratings across peers) and validity (i.e., consistency of peer ratings with instructors or experts) are frequently examined in the research literature to address a central concern of instructors and students. Although the average levels are generally promising, both reliability and validity can vary substantially from context to context. Meta-analyses have identified a few moderators that are related to peer assessment reliability/validity, but they have lacked statistical power to systematically investigate many moderators or disentangle correlated moderators. Objectives: The current study fills this gap by addressing what variables influence peer assessment reliability/validity using a large-scale, cross-context dataset from a shared online peer assessment platform. Methods: Using multi-level structural equation models, we examined three categories of variables: (1) variables related to the context of peer assessment; (2) variables related to the peer assessment task itself; and (3) variables related to rating rubrics of peer assessment. Results and Conclusions: We found that the extent to which assessment documents varied in quality on the given rubric played a central role in mediating the effect from different predictors to peer assessment reliability/validity. Other variables that are significantly associated with reliability and validity included: Education Level, Language, Discipline, Average Ability of Peer Raters, Draft Number, Assignment Number, Class Size, Average Number of Raters, and Length of Rubric Description. The results provide information to guide practitioners on how to improve reliability and validity of peer assessments.

Gender disparities in retention in pathways to science continue to vary widely by course. Undergraduates intending to study prehealth and premedicine often represent a majority of students enrolled in introductory science courses, contribute to a large number of eventual science degree earners, and are a population that typically includes a high number of women. However, gender differences in attrition, grades, and attitudes persist in the introductory science courses required by undergraduate preheath and premedical programs, particularly within the physical sciences (i.e., Chemistry and Physics). We use structural equation modeling to study 416 undergraduate students across multiple sections of an Algebra-based Physics course, a common course on the prehealth and premedical track where large gender differences in grades, retention and competency beliefs have been documented. Our analysis focuses on identifying potential academic and attitudinal sources for gender differences in students' beliefs about their Physics abilities at the end of the course, and retention to the second physics course, which is often influenced by these competency beliefs. Results suggest that while men's ability beliefs in Physics are relatively stable and largely derived from early performance indicators, this is a smaller source of ability beliefs for women. Instead, women's ability beliefs are mediated during the course through their sense of belonging in Physics, and the extent to which they believe that Physics ability is fixed or malleable. These findings can inform the design of interventions in Physics courses that specifically target the development of ability beliefs for women intending medical careers.

Peer feedback has become a common practice in MOOCs for its capacity to scale formative assessment and feedback on higher-order abilities. Though many practices for improving peer assessment have been examined, there is a lack of knowledge of how instructional design and platform features affect the quality of peer assessment and the relative frequency of different types of peer feedback comments. This study aimed to improve understanding of the relationship between quality of feedback and peer-feedback' pedagogical design. Peer feedback instructional design and peer feedback comment data were examined from two MOOCS in a similar domain of personal relevance but with substantially different designs. Country of origin of the feedback provider was also examined to control for cultural/linguistic effects. Differences between the two courses were observed in both the pedagogical designs and in the focus of peer comments, suggesting that peer feedback design is an important guide for the focus of peer feedback comments. Furthermore, the results support the idea that instructional design features, mainly the guide' structure and focus, determine the type of comments that participants will produce and hence receive.

Large lecture classes in higher education continue to be a context in which large performance differences between underrepresented minorities and their White and Asian peers are observed. In the current study, we sought to develop a package of interventions that may reduce this gap in a multi-section Micro Economics course. The focus of this intervention was to re-design instruction in students' recitation sections, while also iteratively training teaching assistants on their instruction during recitation sections. Participants (N = 2,679) who were enrolled in Micro Economics worked together in groups where teaching assistants facilitated their learning in their recitation section. Results indicated that while all students demonstrated higher Micro Economics grades after the course transformation than their peers in prior semesters, this was particularly the case for underrepresented minorities, essentially eliminating the performance gap observed in prior semesters. Findings highlight the importance of instructional training for teaching assistants and employing teaching practices that can promote engagement and potentially promote inclusion during the learning process.

This study investigated the revision effectiveness of peer feedback in comparison with teacher feedback, before and after a peer feedback training intervention that was designed to be implementable in large teaching load contexts and with EFL students. Fifty-six EFL students across two different class sections received teacher or peer feedback before and after training across five different writing assignments, with feedback conditions manipulated within and between class sections. Results show that EFL peer reviewers can provide more meaning-focused feedback than do teachers and that the impact of each piece of feedback on revision quality improves after training.

The success of peer feedback approaches to instruction depends upon students contributing in-depth feedback to their peers. Prior researchers have examined the role of general attitudes towards peer feedback, but how experiences, especially the performance information during peer feedback, influence the subsequent amount of feedback that students provide to peers has received little attention. This study investigated what experience factors from one assignment predicted growth or declines in the amount of peer feedback provided on the next assignment in a course with many peer feedback assignments. Data on peer feedback experiences and behaviors across multiple assignments were taken from students across two programming courses (N = 149). Negative binomial regression analyses reveal three experiences in the prior assignment predicted growth in length of comments provided to peers: receiving more comments, doing well on the task, and receiving recognition for good reviewing (when not doing well). Implications for practice are presented.

Background: Motivational factors are one active area of research that aims to increase the inclusion of women in physics. One of these factors that has only recently gained traction in physics is intelligence mindset (i.e., the belief that intelligence is either innate and unchangeable or can be developed). We studied 781 students in calculus-based Physics 1 to investigate if their mindset views were separable into more nuanced dimensions, if they varied by gender/sex and over time, and if they predicted course grade. Confirmatory factor analysis was used to divide mindset survey questions along two dimensions: myself versus others and growth versus ability aspects of mindset. Paired and unpaired t-tests were used to compare mindset factors over time and between genders, respectively. Multiple regression analysis was used to find which mindset factors were the best predictors of course grade. Results: This study shows that intelligence mindset can be divided into four factors: My Ability, My Growth, Others' Ability, and Others' Growth. Further, it reveals that gender differences are more pronounced in the "My" categories than the "Others'" categories. At the start of the course, there are no gender differences in any mindset component, except for My Ability. However, gender differences develop in each component from the start to the end of the course, and in the My Ability category, the gender differences increase over time. Finally, we find that My Ability is the only mindset factor that predicts course grade. Conclusion: These results allow for a more nuanced view of intelligence mindset than has been suggested in previous interview and survey-based work. By looking at the differences in mindset factors over time, we see that learning environments affect women's and men's intelligence mindsets differently. The largest gender difference is in My Ability, the factor that best predicts course grade. This finding has implications for developing future mindset interventions and opens new opportunities to eliminate classroom inequities.

Physics is a science, technology, engineering, and mathematics discipline in which women are severely underrepresented. Prior work has identified motivation-based explanations for low participation and retention rates of women in physics. Among various motivational factors, intelligence mindsets (i.e., having fixed or growth mindsets) have been rarely examined in the context of physics. Because physics is commonly associated with requiring brilliance to be successful, many students are likely to hold fixed mindset views for physics, which can be especially detrimental for students from underrepresented groups. We examined physics mindset views of 755 engineering and physical-science majors enrolled in calculus-based Physics 1 to understand separable aspects of such views, how these views varied by gender or sex (i.e., for female versus male students), and whether they predicted physics course grade. Multidimensional scaling analyses revealed four different mindset views that are relatively independently held beliefs. Multiple regression analyses showed that physics course grade is most closely associated with whether students deny or accept a fixed mindset view about themselves in particular. One particular view also had the largest gender difference. Therefore, understanding why students hold different mindset views and designing appropriate interventions for physics courses are important areas to consider in efforts seeking to improve outcomes in physics for diverse student groups.

Teacher learning is a huge challenge in instructional change, but relatively little work has carefully examined the mechanisms by which teachers learn, in contrast to the extensive work on programs that help teachers learn and the high-leverage instructional practices that are strong predictors of student learning. Specifically, relatively little is known about how teachers learn to effectively implement these new instructional practices. Using a mixed-methods, case-comparison design, this study examines specific instructional coaching practices that support 4th-8th grade mathematics teachers in learning to implement ambitious instructional practices. The study leverages a large, state-wide representative dataset in order to purposively select carefully-matched contrasting cases for qualitative analysis from a starting sample of hundreds of teachers, which enabled selecting teachers that began in a very similar place but then progressed at different rates. In-depth qualitative coding was systematically conducted on detailed transcripts of coach-teacher conversations from these carefully selected cases. Finally, these codes were analyzed quantitatively to determine whether the content and form of these conversations predicted improvement in teachers' instructional practices. Results showed that coach-teacher pairs who discuss when and why certain practices should be implemented, and provide more opportunities for teacher input, see larger gains in ambitious instruction in later lessons. Implications for a coaching model based in the cognitive sciences are discussed.

Prior research suggests that students endorsing a science identity are more likely to participate in optional science experiences and choose STEM careers. Science identity is a topical identity, which refers to an identity related to a topic rather than a social or cultural group. However, studies of topical identities typically examine them in isolation. The current study identified typically occurring combinations of topical identities as identity complexes to determine whether science identity would tend to occur within STEM-only complexes or together within larger topical identity complexes. Over 1200 urban public-school students in 6th, 7th, and 9th grades from two different regions in the USA completed surveys asking about their topical identities, choice preferences, and optional science experiences. Latent class analyses revealed that students often endorse science identities in the context of other unrelated identities like athletic and artistic identities. Further, the frequency (overall and relative to each other) of the two high-science identity complexes varied substantially by gender, ethnicity, and grade. These patterns were not simple reflects of the commonly observed overall rates of science identity by demographics. Further, students in topical complexes with high science identity still had high participation in optional science experiences despite having many topical identities that could compete for time.

Implementing peer feedback in revisions is a complex process involving first planning to fix problems and then actual implementing feedback through revisions. Both phases are influenced by features of the peer feedback itself, but potentially in different ways, and yet prior research has not examined their separate role in planning or the mediating role of planning in the relationship of feedback features and implementation. We build on a process model to investigate whether feedback features had differing relationships to plans to ignore or act on feedback versus actual implementation of feedback in the revision, and whether planning mediated the relationship of feedback features and actual implementation. Source data consisted of peer feedback comments received, revision plans made, and revisions implemented by 125 US high school students given a shared writing assignment. Comments were coded for feedback features and implementation in the revision. Multiple regression analyses revealed that having a comment containing a specific solution or a general suggestion predicted revision plans whereas having a comment containing an explanation predicted actual implementation. Planning mediated the relationship to actual implementation for the two feedback features predicting plans, suggestion and solution. Implications for practice are discussed.

The Coalition for Science After School highlights the dual nature of outcomes for science learning during out-of- school time (OST): Learning experiences should not only be positive in the moment, but also position youth for future success. Several frameworks speak to the first set of immediate outcomes--what youth learn, think, and feel as the result of informal learning experiences. Much less research has been conducted on longer-term outcomes--how OST experiences affect engagement over time, prepare youth for future learning, or even influence career trajectories. In general, the field urgently needs research and practice frameworks that speak directly to the ways OST programming produces longer-term learning, engagement, and career outcomes. Responding to this need, the authors have been developing a new framework and set of assessments built on the idea of "science learning activation". Building on recent advances in science education, sociocultural studies, and cognitive and social psychology, the authors define "science learning activation" as the dispositions, practices, and knowledge that enable learners to be successful in science learning and that are, in turn, influenced by success. This paper draws on in-depth interviews with and observations of adults and youth to explore this new concept. The authors' work so far supports a positive feedback model: the four dimensions of activation (fascination, valuing science, competency beliefs, and scientific sensemaking) all have positive effects on one or more of the aspects of success--choice, engagement, perceived success, and learning--which in turn predict increases in the dimensions of activation. Thus, science learning activation appears to provide developmental momentum that can support persistent success in science learning.

Peer review is regularly found to be a powerful and efficient technique for assessment and feedback, but many students are inexperienced and sometimes struggle to provide meaningful feedback. It is considered best practice to provide students with some training on how to be a good reviewer, but few classes can afford to devote much time to such training, and the assumption is that review quality will improve with experience. This study directly examines what kinds of experiences during peer feedback activities improve reviewing quality. In particular, organized by theories of norm-setting and practice-based learning, it examines the relationship of the amount of provided and received feedback on one assignment to improvements in the quality of feedback on the next assignment. Data on peer feedback experiences and behaviors across multiple assignments were taken from students across two introductory level undergraduate courses (N = 360). Multi-regression analyses reveal that the number and length of feedback provided predicted growth in helpfulness rates, and both improvements in domain performance and the reviewer's preference for length explains the effects on review helpfulness. Further, compared with high-performing students, low-performing students show more remarkable growth in helpfulness from providing feedback.

This study examines and compares how developers designed two primary science curricula to support teacher adaptation and enable use of innovative materials at scale. The two cases--"Literacy Science" (a science and literacy curriculum for grades 2-5) and "Science as Inquiry" (a curriculum focused on matter for grades 3-5)--were selected because the curricula shared many key features, yet the designers undertook the challenge of designing for adaptation in substantially different ways. Data sources for analysis included interviews with design team members, the curriculum materials, and a range of project documentation. A comparative case study approach was chosen to enable an examination of key contrasting features within the context of each curriculum. Both curricula provide teachers with supports to enact an inquiry-based curriculum in ways that honor science epistemologies. However, one designer team designed explicitly for adaptation by providing "worked examples" that described a range of possible classroom and learner contingencies, along with alternative solutions. By contrast, the other design 9team sought to "build teachers' pedagogical capacity" by providing access to content and explanations from the cognitive and natural sciences. The paper examines how these design stances informed materials developed to support teachers' content knowledge, as well as students' scientific inquiry and classroom discourse. These approaches represent different points on a continuum of design for adaptation, each with its own consequences for enactment and the design of written materials. The cases provide models for designers seeking to support teachers at scale.

As the population of emergent bilingual students in the United States continues to grow, it has become increasingly important to ensure that content area instruction is linguistically and culturally inclusive and accessible. Longitudinal survey and state mathematics assessment data were used to examine how the use of students' home language in the English-medium mathematics classroom related to growth in student achievement and attitudes towards mathematics learning. Surveys from 1,274 students attending four middle schools in a Spanish-dominant community reported on teacher and student use of Spanish as well as attitudes towards learning strategies and mathematics. Structural equation modelling revealed that teachers' use of Spanish predicted students' use of Spanish, which was associated with growth in comfort participating in the mathematics classroom. Increased comfort, moreover, was related to growth in mathematics self-efficacy and, in turn, increased interest in mathematics as well as growth on the state mathematics assessment. [ABSTRACT FROM AUTHOR]

The presentation of economics in introductory courses has been highlighted as potentially exacerbating the underrepresentation of women in economics. The authors study the impact of a gender-neutral change in content and instruction in introductory economics courses intended to increase student engagement. By implementing meaningful applied problems and structured group work, their intervention focuses on the students' perceptions of "what" economics is and "how" economics is used. Using institutional data from 8,727 students over nine semesters, they find that the intervention improved women's grades relative to men's in both Introductory Microeconomics and Macroeconomics and eliminated underperformance by women in Introductory Macroeconomics relative to men at baseline. These effects are evidence that course content and delivery impact the experiences and outcomes of female students in economics education. [ABSTRACT FROM AUTHOR]

Expanding on recent advances in science education, cognitive and social psychology, and sociocultural studies, the paper explores a construct called "science learning activation" and a theoretical framework that describes the characteristics, function, and impact of this construct. Authors define "science learning activation" as a set of dispositions, skills, and knowledge that commonly enable success in proximal science learning experiences and are in turn influenced by these successes. This study investigated the relationship between four dimensions of "science learning activation" (fascination, values, competency beliefs, and scientific sensemaking) and three indicators of success (choice, emotional and cognitive/behavioral engagement, and learning) in temporally proximal science learning experiences. Science learning activation, preferences to choose optional science experiences, engagement ratings, and learning outcomes were collected over multiple time points from diverse group of 681 fifth and sixth grade students from two different regions of the United States. Regression analyses, and hierarchical linear models controlling for demographic characteristics, revealed that: choice preferences were predicted by fascination, values, and sensemaking; engagement levels were predicted by competency believes, fascination, and values; and learning outcomes were predicted by scientific sensemaking. Further, successes themselves predicted further growth in activation: growth in fascination, values, and competency belief themselves were predicted by choice preferences and engagement levels; and growth in sensemaking was predicted by content learning. Thus, "science learning activation" provides a theory (and corresponding set of measurement tools) for proximal outcomes of early science learning interventions that can produce positive long-term outcomes through a reoccurring reinforcement process wherein the effects of an early intervention can lead toward additional positive effects from subsequent interventions. Conversely, poor experiences can lead to negative attitudes that hinder the next learning experience and eventually away from seeking future science learning opportunities. These findings have implications for theory, practice, and research.

Digital badges have long been assumed to possess motivational qualities that could encourage learners to engage with learning content. However, prior studies have found the effects of badges to be complex, differing by learner, type of badge, and potentially other factors. Qualitative reports suggest that individuals' perceptions of digital badges may play a role in moderating badges' effects: badges are only motivating when they are perceived as relevant and desirable. In the current study, we examine longitudinal episodes of data from 2,410 middle and high school users of a badged online programming curriculum to test whether there is evidence that learners' perceptions of badges' relevance predict changes in engagement over time; and whether that relationship is equitable with respect to age, sex, and ethnicity. We also investigate whether a reciprocal relationship may exist in which engagement predicts relative increases in learners' perceptions of badges as relevant to them. Learners' positive perceptions of badges' relevance predicted rank-order increases in engagement over time. Further, this relationship did not vary by age, sex, or minoritized racial/ethnic status. In addition, higher engagement also predicted upward shifts in perceived badge relevance. These results suggest that learners' subjective evaluations of digital badges are closely related to changes in their engagement with program activities, that badged engagement neither widens nor closes educational equity gaps, and that learners' regard for badges and engagement in program activities may be mutually reinforcing.

This research is focused on how to support students' acquisition of program construction skills through worked examples. Although examples have been consistently proven to be valuable for student's learning, the learning technology for computer science education lacks program construction examples with interactive elements that could engage students. The goal of this work is to investigate the value of the "engaging" features in programming examples. We introduce PCEX, an online tool developed to present program construction examples in an engaging fashion. We also present the results of a controlled study with a between-subject design that was conducted in a large introductory Python programming class to compare PCEX with non-interactive worked examples focused on program construction. The results of our study show the positive impact of interactive program construction examples on student's engagement, problem-solving performance, and learning.