Search

The landscape of undergraduate biology education has been shaped by decades of reform efforts calling for instruction to integrate core concepts and scientific skills as a means of helping students become proficient in the discipline. Assessments can be used to make inferences about how these reform efforts have translated into changes in department curriculum and course practices. Such changes can be measured using student scores on researcher-developed programmatic and concept assessments. Scores on these assessments are often assumed to be accurate representations of student biology content knowledge, but my work indicates that the validity of these interpretations may be threatened when students complete the assessments in low-stakes contexts that are more likely to elicit low test-taking effort. Score validity is also threatened in high-stakes out-of-class contexts in which students may be incentivized to leverage external resources to increase their score. My findings suggest that departments and instructors using programmatic and concept assessments to evaluate the progress of their curriculum and courses in meeting the goals of reform effort should carefully interpret scores in light of the conditions in which students completed the assessment. The impacts of reform efforts may also be detected in the types of skills and content that are assessed on course exams. I studied the skills and content of lower-division undergraduate biology exams in the context of a three-dimensional framework consisting of scientific practices, interdisciplinary crosscutting concepts, and disciplinary core ideas. I found that very few exam items were three-dimensional, primarily due to the low number of items assessing scientific practices. Although there were few three-dimensional items, those items were more likely to use a constructed-response format and assess higher-order cognitive skills compared to items not aligned with all three dimensions. To achieve the goals of reform efforts in undergraduate biology education, my research indicates instructors may need time, resources, and training for writing and grading three-dimensional assessments. Altogether, this dissertation sheds critical insight into the process and content of evaluating student learning, thereby refining our understanding of the impact of education reforms. Advisor: Brian A. Couch

Classroom tests are a common and default form of assessments in concept-heavy, fundamental engineering courses. Tests have benefits to learning, such as the testing effect that helps with the retrieval of knowledge, but there are also disadvantages, like discouraging deep learning approaches and decreasing motivation to learn, that warrant examining and questioning why tests are common, which engineering education literature lacks. Furthermore, the advancement of assessment research has led to alternative assessments that can diversify types of assessments and promote intentionality in test usage in these courses, supporting the need for scholarship on understanding test usage. My research began to address this by studying fundamental engineering course instructors' test beliefs and behaviors because engineering instructors have shown to have autonomy in making course decisions and barriers to adopting scholarship-based assessment practices among these engineering instructors persist. This dissertation study, grounded in the Situated Expectancy Value Theory (SEVT), explored, uncovered, and articulated seven fundamental engineering course instructors' test beliefs and behaviors from mechanical engineering and engineering science departments in a public, land-grant, Research 1 institution. Leveraging the case study research methodology from a pragmatic perspective, my multi-case study, with each participant being defined as a case, answered an overarching research question and five sub-research questions that yielded findings on five test aspects: test usage, design, administration, cheating, and fairness. Eight collected data sources in the form of qualitative interviews, course, department, and institution documents became the database to answer the questions. Analyses of these data involved coding and content analysis, and subsequent thematic analysis. The outcome of these analyses shaped the individual case profiles for cross-case analysis to understand belief and behavior patterns at a higher level. My research has found three groups of test usage beliefs. These are enthusiastic test users, default test users, and skeptical test users. All participants featured tests heavily in their courses and justified with learning outcomes and some non-course-content factors like large class sizes for grading conveniences. However, those in default and skeptical test user groups also acknowledged some non-course-content factors, like inertia and peer pressure, that influenced their test usage beliefs and behaviors. All participants acknowledged some disadvantages with tests, but those who are skeptical with test usage presented stronger beliefs about test disadvantages, arguing for the need to move away from tests when necessary. Some participants also presented conflicting beliefs and behaviors regarding their test usage. My study has also found all participants using problem-solving questions, emphasizing the need to curb cheating especially during the Covid-19 pandemic, preferring in-person test administration, and defining test fairness with reasonable completion time and adequate content coverage. These findings contribute to addressing identified research gaps in the literature and have implications for future research on tests with assessment philosophies, classroom practices on diversifying assessments and intentional test usage, and future research on possible assessment roles in addressing systemic inequity in engineering.

Background: Writing multiple choice questions may be a valuable tool for medical education. We asked medical students to generate multiple choice questions and studied its effect on their exams. We hypothesized that students generating questions would improve their learning. Methods: We randomized students in their second and third years at the School of Medicine to write four multiple choice questions on two different sections of General Pathology (Immunopathology and Electrolyte and acid-base status; second year) and Pathophysiology (Blood and Respiratory system; third year). We analyzed whether students writing questions on a section had better results in the exam test in that section than the rest of the students. Results: Seventy-five (38.2%) students wrote questions for General Pathology and 109 (47.6%) for Pathophysiology. Students that wrote questions obtained significantly better results in the exam than those who did not. In General Pathology, students who wrote questions about Immunopathology obtained better results in that section than those who wrote questions about the other section (5.13 versus 3.86 over 10; P = 0.03). In Pathophysiology, the differences between both groups were not significant, but students who wrote good questions about Respiratory system obtained better results in that section than those who wrote good questions about Blood (6.07 versus 4.28 over 10; P = 0.015). Male students wrote good questions in Pathophysiology more frequently than female students (28.1% versus 10.4%; P = 0.02). Conclusions: The writing of multiple choice questions by medical students may improve their learning. A gender effect may also influence this intervention. Future investigations should refine its potential role in teaching.

Background: Writing multiple choice questions may be a valuable tool for medical education. We asked medical students to generate multiple choice questions and studied its effect on their exams. We hypothesized that students generating questions would improve their learning. Methods: We randomized students in their second and third years at the School of Medicine to write four multiple choice questions on two different sections of General Pathology (Immunopathology and Electrolyte and acid-base status; second year) and Pathophysiology (Blood and Respiratory system; third year). We analyzed whether students writing questions on a section had better results in the exam test in that section than the rest of the students. Results: Seventy-five (38.2%) students wrote questions for General Pathology and 109 (47.6%) for Pathophysiology. Students that wrote questions obtained significantly better results in the exam than those who did not. In General Pathology, students who wrote questions about Immunopathology obtained better results in that section than those who wrote questions about the other section (5.13 versus 3.86 over 10; P= 0.03). In Pathophysiology, the differences between both groups were not significant, but students who wrote good questions about Respiratory system obtained better results in that section than those who wrote good questions about Blood (6.07 versus 4.28 over 10; P= 0.015). Male students wrote good questions in Pathophysiology more frequently than female students (28.1% versus 10.4%; P= 0.02). Conclusions: The writing of multiple choice questions by medical students may improve their learning. A gender effect may also influence this intervention. Future investigations should refine its potential role in teaching.

Evidence suggests that college students may increase caffeine consumption to cope with the stress and demands of college. The relationships between perceived stress, caffeine consumption, and GPA were examined in students at a large public university. Students were surveyed to determine perceived stress (Cohen’s Perceived Stress Questionnaire), beliefs about caffeine, caffeine consumption, workload in and outside of the classroom, and GPA. Surveys were administered at the beginning of the semester and again at midterm. Based on Cohen’s 40-point scale, average stress scores increased from 15.95 ± 6.34 at the beginning of the semester to 18.89 ± 6.94 at midterms. Additionally, 88% of students reported having consumed caffeine in the past week at the beginning of the semester; by midterms, this was 90%. Caffeine consumption increased from an average of 167.90 ± 159.08 mg/day to 197.59 ± 167.16 mg/day. Overall, there were not significant correlations between average perceived stress scores and average daily caffeine consumption, or by consumers, gender, or class. As well, there was not a significant association between average daily caffeine consumption and cumulative GPA. However, significant positive associations were found for the consumer group and within the senior consumer group.

Estuaries are complex ecosystems that are composed of a number of sensitive and inter-dependent environments. An abundance of nutrients combine with the dynamic conditions to create some of the most productive environments on the planet. Almost 85% of the commercially harvested fish in the United States depend on estuaries and the surrounding coastal waters at some stage in their life history. Many estuarine drainage basins contain large quantities of agricultural acreage. Over 29 million pounds of active ingredient of pesticides are applied in coastal drainage basins in the United States each year. Studies have shown that the overall condition of the nation's estuaries is fair, while benthic condition is poor. Seventy five percent of estuarine sediments are contaminated with pesticides. Concentrations of pesticides in 30% of the estuaries exceed the levels that are known to result in ecological effects at least 10% of the time. There have been few comprehensive pesticide studies in estuaries to date. The Albemarle-Pamlico (A-P) Drainage Basin of North Carolina forms the second largest estuarine system in the United States and supports heavy agricultural production with high pesticide use. I evaluated measurement and modeling strategies to assess exposure in the A-P drainage basin. Atrazine and metolachlor were the most frequently detected pesticides in water samples that were collected in the A-P drainage basin in 2000 and 2001. Concentrations of these compounds exceeded both human health and aquatic life criteria in 2000. No toxicity thresholds were exceeded in samples collected in 2001. Due to the expense associated with field sampling and pesticide analysis alternative methods for estimating pesticide exposure have been developed, including fate models. The Exposure Analysis Modeling System (EXAMS) was modified to model the fate of pesticides, specifically atrazine and metolachlor, in a small tidal estuary of the larger A-P estuarine system. Based on the estimate that 10% of the total amount of atrazine and metolachlor applied in the Bath Creek drainge basin would enter the estuary the EXAMS steady state model predicts concentrations of both atrazine and metolachlor that fall between the mean and maximum values that were measured in Bath Creek in 2002. Concentrations of pesticides in the A-P drainage basin, both measured and modeled, are significantly less than acute toxicity thresholds for even the most sensitive aquatic species. Due to the short lived nature of these compounds it is unlikely that organisms in the region would experience adverse health effects due to exposure to the existing concentrations.

The goal of this research was to develop a prototype expert system (EXAMS) for monitoring workplace air by Fourier transform infrared (FT-IR) spectrometry. FT-IR can provide real-time analysis of complex gas-phase mixtures without the need for time-averaging sample pre-concentration or pre-separation of individual components. The expert system enables a non-spectroscopist, or a spectroscopist with limited occupational health training, to utilize this powerful analytical technique for assessment of health hazards. The expert system operates in MS-Windows$\sp\circler$ 3.1 and includes an occupational hygiene and toxicological evaluation of the air monitoring results. EXAMS currently operates in occupational environments of known atmospheric contaminants, but with the ability to identify the presence of unanticipated compounds. EXAMS is a hybrid blackboard system utilizing three different knowledge-based systems that communicate with each other to accomplish the system goal. The mid-infrared spectrum of pure compounds is analyzed for analytical regions using a hybrid algorithm/heuristic expert system written in Borland C. The selection of optimal analytical regions for sensitivity and selectivity within a sample mixture is based on the compounds present in the environment. The hypertext user interface and database routines were written with Borland Object Vision$\sp\circler$ 2.0. Additional heuristic or rule-based programming for quality assurance and decision support is provided with Information Builder's Level5 Object$\sp\circler$ inference engine. The expert system was evaluated for individual compounds against previously published information using a metric for measuring the health protection value of proposed analytical regions. EXAMS performed as well as or better than the expert spectroscopists in determining analytical regions for these samples. Quantitative evaluation of EXAMS using five known composition mixtures of varied complexity against eight expert spectroscopists produced results in which EXAMS was statistically superior (P $

Thesis studying instances of cheating in junior high school. According to the introduction, the purpose of the study is: 1. To determine the extent to which pupils in the junior high school grades will cheat when allowed to score their own examination papers. 2. To determine the relation between the professed attitude toward cheating on examinations and actual cheating behavior. 3. To determine some factors related to cheating (p. 1).