Search

This complete research paper will present a model for offering students flexibility in assignment deadlines and the practical ramifications of enabling that flexibility on student performance in the course. Background - It is generally acknowledged that students benefit from reviewing feedback and revising their work. It's also widely recognized that this kind of iteration is logistically difficult to implement and rarely occurs in engineering courses. Purpose - The purpose of this paper is to describe the structure of a first-year Introduction to Graphical Communications course that provided deadline flexibility and the effects that flexibility had on students iterating on their work. The course was taught at a medium-sized, private, Business+STEM-only institution in the southeast United States. Design/Method - The course was designed using the ILEARN flipped-classroom framework. The ILEARN framework divides course content into six components. (I)ntroduction lists the learning objectives of that module. (L)essons are theory-focused passive content with comprehension quizzes at the end. (E)mulates are worked examples with a think-aloud protocol where students are required to submit the emulated problem solution. (A)ctivities are akin to traditional homework assignments; new problems that can be solved using the tools and techniques demonstrated in the Lesson and Emulate tasks. (R)eflections are meta-cognitive reflection surveys. (N)ext Steps are an application of the content toward their final project. Outof-class instruction is contained across the Lesson and Emulate tasks. In-class time is now spent in one-on-one or small group consultation to answer questions and help students move forward through the content. The flipped nature of the course enabled a more flexible deadline policy and removed the requirement for content lockstep. All assignments were given a recommended due date throughout the week. A hard deadline for each module was set 2 weeks after the Next Steps task's recommended due date. There was no grade penalty for submitting in this two-week window, but multiple nudges were in place to encourage students to submit their assignments by the recommended due date. Most notably, this 2-week buffer often enabled graders to provide feedback on work submitted on time and for students to revise and resubmit their work. Results - On 50 unique tasks assigned to 90 students, there were 4352 total assignment submissions. Students averaged 1.05 submissions per task. The maximum number of resubmissions was 4 attempts. 56 of the students resubmitted at least one task throughout the semester. [ABSTRACT FROM AUTHOR]

This paper is an update of a Work-in-Progress presented at the ASEE 2021 virtual conference [1] and includes new data from after the 2021 paper was accepted for publication. An undergraduate research experience was developed in response to an Office of Naval Research program seeking to develop "innovative solutions that directly maintain, or cultivate a diverse, world-class STEM workforce in order to maintain the U.S. Navy and Marine Corps' technological superiority." During the fall 2020 semester, nine students were recruited to participate in a UAS cybersecurityfocused undergraduate research experience. Three faculty members each identified a small topic area for undergraduate students to pursue. The three areas are: 1. Small UAS (sUAS) Vulnerability and Threat Assessment and Mitigation 2. Effects of Cyber Attacks on Communication in UAS Swarms with Distributed Swarm Control 3. Enhancing Security of Cloud-Connected UAS Services. Students were placed onto teams based on their prior course experiences and the project requirements. Common resources were provided for all students to train them in conducting research. Teams were then tasked with developing a more comprehensive research plan for their specific project and carrying out that plan throughout the 2020-2021 academic year. A no-cost extension was granted to enable six students who had not graduated to continue into the Fall 2021 semester with the goals of completing their work and publishing the results in an academic outlet. Three students continued into the Spring 2022 semester finishing their publication work. Year 1 of the program was assessed by having students take a pre-, mid-, and post-survey and conducting a brief interview about their experiences. Students took a pre-survey at the start of the project and a mid-project survey shortly before the winter break. They completed the post survey and interview during the final week of the Spring semester in Year 1. Five students who remained with the project in Year 2 also completed a "final year" survey at the end of the Fall term in year 2. The surveys were a combination of project specific skills questions as well as relevant questions from the Undergraduate Research Student Self-Assessment (URSSA), an NSF funded assessment tool to measure student gains in research skills. This paper will describe the broader project, the individual student team projects and the outcomes of those projects, and results from the surveys and interviews. The purpose of this paper is to provide an example approach for future undergraduate research programs looking for practical approaches to implementing undergraduate research programs, particularly those in the cybersecurity area. [ABSTRACT FROM AUTHOR]

This Research paper discusses the opportunities that utilizing a computer program can present in analyzing large amounts of qualitative data collected through a survey tool. When working with longitudinal qualitative data, there are many challenges that researchers face. The coding scheme may evolve over time requiring re-coding of early data. There may be long periods of time between data analysis. Typically, multiple researchers will participate in the coding, but this may introduce bias or inconsistencies. Ideally the same researchers would be analyzing the data, but often there is some turnover in the team, particularly when students assist with the coding. Computer programs can enable automated or semi-automated coding helping to reduce errors and inconsistencies in the coded data. In this study, a modeling survey was developed to assess student awareness of model types and administered in four first-year engineering courses across the three universities over the span of three years. The data collected from this survey consists of over 4,000 students' open-ended responses to three questions about types of models in science, technology, engineering, and mathematics (STEM) fields. A coding scheme was developed to identify and categorize model types in student responses. Over two years, two undergraduate researchers analyzed a total of 1,829 students' survey responses after ensuring intercoder reliability was greater than 80% for each model category. However, with much data remaining to be coded, the research team developed a MATLAB program to automatically implement the coding scheme and identify the types of models students discussed in their responses. MATLAB coded results were compared to human-coded results (n = 1,829) to assess reliability; results matched between 81%-99% for the different model categories. Furthermore, the reliability of the MATLAB coded results are within the range of the interrater reliability measured between the 2 undergraduate researchers (86-100% for the five model categories). With good reliability of the program, all 4,358 survey responses were coded; results showing the number and types of models identified by students are presented in the paper. [ABSTRACT FROM AUTHOR]

This complete research paper describes the impact of a modeling intervention on first-year engineering students' modeling skills in an introductory computer programming course. Five sections of the first-year engineering introductory programming course at a private, STEM+Business institution were revised to center around modeling concepts. These five sections made up the experimental group for this study. The comparison group consisted of four sections of the course that were not revised. Students in all these sections were given two different versions of a modeling problem two times in the semester to test their progress in gaining modeling skills. Each version required two submissions - a written solution and a coded solution. The assessment of these four submissions based on the three established dimensions of modeling were quantitatively analyzed in this study. The three dimensions within mathematical modeling that were the focus of this study were mathematical model complexity, modifiability, and reusability. Mathematical model complexity is being able to address the complexity of the problem. Modifiability addresses the generalizability of the model solution. Reusability is showing an understanding of the problem and the user. Statistical analysis showed that students in the experimental group had more gains in their demonstrated modeling abilities across all three dimensions than the students in the comparison group. This study demonstrated that intentional and explicit instructional strategies targeting model development resulted in greater gains in students' demonstrated modeling skills and both their written and coded solutions to a complex modeling problem. [ABSTRACT FROM AUTHOR]

Background Peer review is a cornerstone of academic research dissemination. It is a fundamental prerequisite for "good" research, even though the process of selecting reviewers is largely shrouded in mystery. Under ideal conditions, reviewers are experts on a paper's topic, but the process of identifying that expertise varies widely from publication venue to venue. Purpose (Hypothesis) The purpose of this paper is to (1) identify some of the methods currently used by conference and journal editors, and (2) present an alternative methodology used by the author to identify and assign reviewers for a division of ASEE during the summer of 2017. Design/Method The author, as program chair of a large division, developed software that data-mined the prior conference papers of any potential reviewer to identify the words frequently used in their prior work. Using this data, a Reviewer Profile was developed that identified areas of expertise. Using a similar process, papers being reviewed were profiled to the types of expertise necessary - a Paper Need Profile. These profiles were then used to assign peer reviewers for the ASEE 2017 conference. Results This paper will present details of the Reviewer Profile algorithm, the Paper Need Profile algorithm, the matching algorithm, and an analysis of the effectiveness of the overall assignment methodology. Conclusion The conclusion will explore how this work may translate into classroom peer review as well as a discussion of how this methodology could be applied across ASEE and in other professional publication venues. [ABSTRACT FROM AUTHOR]

Background Peer review can be a beneficial pedagogical tool for providing students both feedback and varied perspectives on their work. Despite being a valuable tool, the best mechanism for assigning reviewers to reviewees is still often blind random assignment. While better mechanisms must exist, they necessarily rely on having some prior knowledge about the work being reviewed. Purpose (Hypothesis) The purpose of this paper is to present the findings from an effort to classify student team performance on Model-Eliciting Activities (MEAs) using a trained reviewer's gut instinct about the quality of the work. Design/Method MEAs are realistic, open-ended, client-driven engineering problems where teams of students produce a written document describing the steps of how to solve the problem. Using an archival data set, nearly 450 MEA solutions were evaluated by two trained student researchers in approximately two minutes per solution. Their evaluations are compared against other, more detailed, analyses of the solutions to identify if their evaluations are sufficiently accurate enough to use as baseline data for making peer review matching decisions with a comparatively miniscule investment in time. Results Results indicate that both researchers performed less accurately than computer-based classification but were largely consistent with the more detailed evaluations conducted by teaching assistants. Conclusion The conclusion this research made was that gut reaction based classification was not wholly sufficient to address the needs for informed peer review matching. The results may be useful as an additional data source for computer-based classification to reduce the amount of training or to increase accuracy. [ABSTRACT FROM AUTHOR]

As an educational tool, peer review can be a valuable way to provide students feedback without a significant increase in instructor workload. Despite all that is currently known about our students, the best mechanism for assigning reviewers to reviewees in a peer review of artifacts is still considered to be blind, random assignment. The underlying conjecture of this research project is that "there has to be a better way". This paper represents a follow-up to earlier work by the author1. That study presented the results of an attempt to develop a classification schema using a large archival database of student work. This paper takes the resulting algorithms produced from that archival dataset and applies them to new student work, identifying how well the archival-based classification works on a new data set. The implications of that application on future algorithm design will be discussed as well as the next steps for the research. [ABSTRACT FROM AUTHOR]

Peer review can be a beneficial pedagogical tool for providing students both feedback and varied perspectives. Despite being a valuable tool, the best mechanism for assigning reviewers to reviewees is still often blind random assignment. This research represents the first step in a larger effort to find an improved method for matching reviewers to reviewees. By automating the classification of student work, reviewer quality and reviewee need can potentially be assessed. With that assessment, the best reviewers can be assigned to the neediest teams, while the most self-sufficient teams can be assigned reviewers who may need to see higher quality work. The purpose of this paper is to present the preliminary findings from an effort to classify student team performance on Model-Eliciting Activities (MEAs) using natural language processing tools. MEAs are realistic, open-ended, client-driven engineering problems where teams of students produce a written document describing the steps of how to solve the problem. Archival data containing expert evaluations to MEAs were used to test different natural language processing tools in an attempt to identify which tools could most accurately assign scores similar to an expert. The research did not re-implement the selected algorithms, but rather used off-theshelf libraries to explore the value of their application to this context. Using a split-sample training-testing set, the "Bagged Decision Tree" and "Random Forest" algorithms were used to classify sample solutions against 11 MEA rubric dimensions. Performance on each rubric item averaged between 60% and 85% accurate, depending on the item. The implementation of these algorithms also revealed words and phrases commonly used in higher quality samples. This paper will focus on how the data was obtained and prepared, how the different algorithms were utilized, how the algorithms performed in the classification tests, what the results indicate about our implementation of MEAs and how the results will be informing the next stages of the research project. [ABSTRACT FROM AUTHOR]

The PEER Collaborative National Network is a national peer mentoring network for early career tenure-track or mid-career tenured faculty who conduct and are primarily evaluated based on their research related to engineering education. This paper discusses the development, logistics, and outcomes of two PEER workshops built around a community of practice framework. Data from internal and external evaluations are presented to provide insights into aspects that worked well and aspects that need further development. Additionally, by reflecting on the workshops, participants crafted vignettes describing the impact the PEER workshops had on their personal and professional lives. The paper concludes with a discussion on the future of PEER (and potential spin-off groups from the PEER cohorts), and the changes that will be made in future workshops. Recommendations are provided for other organizers interested in developing successful "near peer" groups to address specific community needs. [ABSTRACT FROM AUTHOR]