Search

Abstract Instructional reform in STEM aims for the widespread adoption of evidence based instructional practices (EBIPS), practices that implement active learning. Research recognizes that faculty social networks regarding discussion or advice about teaching may matter to such efforts. But teaching is not the only priority for university faculty – meeting research expectations is at least as important and, often, more consequential for tenure and promotion decisions. We see value in understanding how research networks, based on discussion and advice about research matters, relate to teaching networks to see if and how such networks could advance instructional reform efforts. Our research examines data from three departments (biology, chemistry, and geosciences) at three universities that had recently received funding to enhance adoption of EBIPs in STEM fields. We evaluate exponential random graph models of the teaching network and find that (a) the existence of a research tie from one faculty member $$i$$ i to another $$j$$ j enhances the prospects of a teaching tie from $$i$$ i to $$j$$ j , but (b) even though faculty highly placed in the teaching network are more likely to be extensive EBIP users, faculty highly placed in the research network are not, dimming prospects for leveraging research networks to advance STEM instructional reforms.

Assessing student knowledge based on their writing using traditional qualitative methods is time-consuming. To improve speed and consistency of text analysis, we present our mixed methods development of a machine learning predictive model to analyze student writing. Our approach involves two stages: first an exploratory sequential design, and second an iterative complex design. We first trained our predictive model using qualitative coding of categories (ideas) in student writing. We next revised our model based on feedback from instructor-users. The model itself highlighted categories in need of revision. The contribution to mixed methods research lies in our innovative use of the machine learning tool as a rapid, consistent additional coder, and a resource that can predict codes for new student writing.

Programs seeking to transform undergraduate science, technology, engineering, and mathematics courses often strive for participating faculty to share their knowledge of innovative teaching practices with other faculty in their home departments. Here, we provide interview, survey, and social network analyses revealing that faculty who use innovative teaching practices preferentially talk to each other, suggesting that greater steps are needed for information about innovative practices to reach faculty more broadly.

Background Calls for science education reform have been made for decades in the USA. The recent call to produce one million new science, technology, engineering, and math (STEM) graduates over 10 years highlights the need to employ evidence-based instructional practices (EBIPs) in undergraduate STEM classes to create engaging and effective learning environments. EBIPs are teaching strategies that have been empirically demonstrated to positively impact student learning, attitudes, and achievement in STEM disciplines. However, the mechanisms and processes by which faculty learn about and choose to implement EBIPs remain unclear. To explore this problem area, we used social network analysis to examine how an instructor’s knowledge and use of EBIPs may be influenced by their peers within a STEM department. We investigated teaching discussion networks in biology and chemistry departments at three public universities. Results We report that tie strength and tie diversity vary between departments, but that mean indegree is not correlated with organizational rank or tenure status. We also describe that teaching discussion ties can often be characterized as strong ties based on two measures of tie strength. Further, we compare peer influence models and find consistent evidence that peer influence in these departments follows a network disturbances model. Conclusions Our findings with respect to tie strength and tie diversity indicate that the social network structures in these departments vary in how conducive they might be to change. The correlation in teaching practice between discussion partner and peer influence models suggest that change agents should consider local social network characteristics when developing change strategies. In particular, change agents can expect that faculty may serve as opinion leaders regardless of their academic rank and that faculty can increase their use of EBIPs even if those they speak to about teaching use EBIPs comparatively less.

The relationship between structure and function is a core concept in physiology education. Written formative assessments can provide insight into student learning of the structure and function relationship, which can then inform pedagogy. However, question order may influence student explanations. We explored how the order of questions from different cognitive levels affects student explanations. A junior level General Physiology class was randomly split in half. One-half of the students answered, “Define the principle: form reflects function,” followed by “Give an example of the principle: form reflects function” (format DX), whereas the other half answered, “Give an example of the principle: form reflects function,” followed by “Define the principle: form reflects function” (format XD). Human grading and computerized lexical analysis were used to evaluate student responses. Two percent of students in the format DX group related structure and function in their definition, whereas 48% of students related structure and function in their examples. In the format XD group, 17% related structure and function in their definition, and 26% related structure and function in their example of the principle. Overall, students performed better on the last question in the sequence, which may be evidence for conceptual priming. Computerized lexical analysis revealed that students draw on only a few levels of organization and may be used by instructors to quickly assess the levels of organization students use in their responses. Written assessment coupled with lexical analysis has the potential to reveal student understanding of core concepts in anatomy and physiology education.

Higher education institutions prepare future faculty members for multiple roles, including teaching. However, teaching professional development programs for graduate students vary widely. We present evaluation data from a high engagement program for STEM doctoral students. We analyzed the impact on three cohorts of participants over three academic years and identified the components most influential upon their teaching professional development. Participants found the year-long teaching assessment project and the disciplinary and reflective focus instrumental for improving their knowledge of teaching and learning. We recommend these components for the design of other such high-engagement programs.

College science instructors need continuous professional development (PD) to meet the call to evidence-based practice. New PD efforts need to focus on the nuanced blend of factors that influence instructors’ teaching practices. We used persona methodology to describe the diversity among instructors who were participating in a long-term PD initiative. Persona methodology originates from ethnography. It takes data from product users and compiles those data in the form of fictional characters. Personas facilitate user-centered design. We identified four personas among our participants: Emma the Expert views herself as the subject-matter expert in the classroom and values her hard-earned excellence in lecturing. Ray the Relater relates to students and focuses on their points of view about innovative pedagogies. Carmen the Coach coaches her students by setting goals for them and helping them develop skill in scientific practices. Beth the Burdened owns the responsibility for her students’ learning and feels overwhelmed that students still struggle despite her use of evidence-based practice. Each persona needs unique PD. We suggest ways that PD facilitators can use our personas as a reflection tool to determine how to approach the learners in their PD. We also suggest further avenues of research on learner-centered PD.

Online discussions are widely viewed as a valuable tool for encouraging student engagement and promoting interaction with course material outside of the traditional classroom. Strategies for conducting online discussions vary and are not confined to traditional, university-sponsored learning management systems (LMS). Social media platforms such as Facebook, which provide a variety of social benefits to students, might also represent a viable mechanism for educational information exchange and learning. Our study tested this proposition by comparing the effects of a Facebook-based and LMS-based online discussion forum on students' participation, achievement of learning goals, and overall course performance. Our findings suggest that different forums can affect classroom dynamics and student learning in different ways. While Facebook may be better at fostering student participation and encouraging peer-to-peer dialogue, the university-sponsored LMS may be a more effective tool for encouraging student...

Helping faculty develop high-quality instruction that positively affects student learning can be complicated by time limitations, a lack of resources, and inexperience using student data to make iterative improvements. We describe a community of 16 faculty from five institutions who overcame these challenges and collaboratively designed, taught, iteratively revised, and published an instructional unit about the potential effect of mutations on DNA replication, transcription, and translation. The unit was taught to more than 2000 students in 18 courses, and student performance improved from preassessment to postassessment in every classroom. This increase occurred even though faculty varied in their instructional practices when they were teaching identical materials. We present information on how this faculty group was organized and facilitated, how members used student data to positively affect learning, and how they increased their use of active-learning instructional practices in the classroom as a result of participation. We also interviewed faculty to learn more about the most useful components of the process. We suggest that this professional development model can be used for geographically separated faculty who are interested in working together on a known conceptual difficulty to improve student learning and explore active-learning instructional practices.

The understanding of core concepts and processes of science in solving problems is important to successful learning in biology. We have designed and developed a Web-based, self-directed tutorial program, SOLVEIT, that provides various scaffolds e.g., prompts, expert models, visual guidance to help college students enhance their skills and abilities in solving problems in science. An initial version of SOLVEIT was used in this study. This paper details the features of SOLVEIT that are contextualized within the biological domains of evolution and ecology. A qualitative case study was conducted to evaluate the usability of the program. Selected students were recruited from an introductory biology course at a large public university in the south-eastern United States. Data for this study were collected through the SOLVEIT database and semi-structured interviews. The findings of this study demonstrate the potential of the program for improving students' problem solving in biology. Suggestions for the use of SOLVEIT and its further improvement and development are discussed, along with suggestions for future research. This study also provides more general guidance for researchers and practitioners who are interested in the design, development and evaluation of Web-based tutorial programs in science education.

Findings from a mixed-methods investigation of undergraduate biology problem solving are reported. Students used a variety of problem-solving procedures that are domain general and domain specific. This study provides a model for research on alternative problem types and can be applied immediately in the biology classroom., This study uses the theoretical framework of domain-specific problem solving to explore the procedures students use to solve multiple-choice problems about biology concepts. We designed several multiple-choice problems and administered them on four exams. We trained students to produce written descriptions of how they solved the problem, and this allowed us to systematically investigate their problem-solving procedures. We identified a range of procedures and organized them as domain general, domain specific, or hybrid. We also identified domain-general and domain-specific errors made by students during problem solving. We found that students use domain-general and hybrid procedures more frequently when solving lower-order problems than higher-order problems, while they use domain-specific procedures more frequently when solving higher-order problems. Additionally, the more domain-specific procedures students used, the higher the likelihood that they would answer the problem correctly, up to five procedures. However, if students used just one domain-general procedure, they were as likely to answer the problem correctly as if they had used two to five domain-general procedures. Our findings provide a categorization scheme and framework for additional research on biology problem solving and suggest several important implications for researchers and instructors.

Students’ writing can provide better insight into their thinking than can multiple-choice questions. However, resource constraints often prevent faculty from using writing assessments in large undergraduate science courses. We investigated the use of computer software to analyze student writing and to uncover student ideas about chemistry in an introductory biology course. Students were asked to predict acid–base behavior of biological functional groups and to explain their answers. Student explanations were rated by two independent raters. Responses were also analyzed using SPSS Text Analysis for Surveys and a custom library of science-related terms and lexical categories relevant to the assessment item. These analyses revealed conceptual connections made by students, student difficulties explaining these topics, and the heterogeneity of student ideas. We validated the lexical analysis by correlating student interviews with the lexical analysis. We used discriminant analysis to create classification functions that identified seven key lexical categories that predict expert scoring (interrater reliability with experts = 0.899). This study suggests that computerized lexical analysis may be useful for automatically categorizing large numbers of student open-ended responses. Lexical analysis provides instructors unique insights into student thinking and a whole-class perspective that are difficult to obtain from multiple-choice questions or reading individual responses.

Plant species with separate genders often exhibit gender differences in traits related to reproductive allocation. In gynodioecious species, females often produce more seeds than do hermaphrodites, leading to a higher reproductive cost. The mechanisms that allow females to meet the high costs of reproduction are currently under debate. In this study, we test the hypothesis that there are genetically based gender differences in physiological traits that enable females to finance these costs through higher photosynthetic carbon gain in the gynodioecious perennial Geranium maculatum. Females and hermaphrodites were compared in a greenhouse study that minimized environmental and selfing rate differences between the genders. We found that females produced smaller flowers but more of them and more fruits than did hermaphrodites. However, genders did not differ in their seed number, seed mass, fruit set, and reproductive allocation. In addition, genders did not differ in photosynthetic rate (A), leaf N, and wate...

One challenge in science education assessment is that students often focus on question surface features rather than the underlying scientific principles. The authors investigated how student responses to photosynthesis constructed-response questions vary based on two surface features of a question and found no significant difference in the content of responses., One challenge in science education assessment is that students often focus on surface features of questions rather than the underlying scientific principles. We investigated how student written responses to constructed-response questions about photosynthesis vary based on two surface features of the question: the species of plant and the order of two question prompts. We asked four versions of the question with different combinations of the two plant species and order of prompts in an introductory cell biology course. We found that there was not a significant difference in the content of student responses to versions of the question stem with different species or order of prompts, using both computerized lexical analysis and expert scoring. We conducted 20 face-to-face interviews with students to further probe the effects of question wording on student responses. During the interviews, we found that students thought that the plant species was neither relevant nor confusing when answering the question. Students identified the prompts as both relevant and confusing. However, this confusion was not specific to a single version.

Woodlots are forest islands embedded within an urban matrix, and often represent the only natural areas remaining in suburban areas. Woodlots represent critical conservation areas for native plants, and are important habitat for wildlife in urban areas. Invasion by non-indigenous (NIS) plants can alter ecological structure and function, and may be especially severe in remnant forests where NIS propagule pressure is high. Woody shrubs in the Family Berberidaceae have been well documented as invaders of the forest–urban matrix in North America. Mahonia bealei (Berberidaceae) is a clonal shrub native to China, and is a popular ornamental in the Southeastern United States. Mahoni bealei is listed as ‘‘present’’ on some local and state floras, but almost nothing is known regarding its invasion potential in the United States. We sampled 15 woodlots in Clemson, South Carolina, to assess the invasion of M. bealei and other woody non-indigenous species (NIS). M. bealei invaded 87% of the woodlots surveyed and species richness of NIS on these woodlots varied from 5 to 14. Stepwise-multiple regression indicated that less canopy cover and older M. bealei predicted greater abundance of M. bealei, and that not all subdivisions were equally invaded (P < 0.0001; r 2 ¼ 0.88). The impact of M. bealei on native flora and fauna may be considerable, and it is likely to continue to spread in the Southeastern United States. M. bealei should be recognized as an aggressive invader in the Southeastern United States, with the potential for negative impacts on native flora and fauna.

This qualitative study uses expectancy-value theory to explore the motivation for college biology instructors to participate and persist in teaching professional development for 2.5 years., We conducted a study of 19 biology instructors participating in small, local groups at six research-intensive universities connected to the Automated Analysis of Constructed Response (AACR) project (www.msu.edu/∼aacr). Our aim was to uncover participants’ motivation to persist in a long-term teaching professional development effort, a topic that is understudied in discipline-based educational research. We interviewed each participant twice over a 2-year period and conducted qualitative analyses on the data, using expectancy-value theory as a framework for considering motivation. Our analyses revealed that motivation among instructors was high due to their enjoyment of the AACR groups. The high level of motivation is further explained by the fact that AACR groups facilitated instructor involvement with the larger AACR project. We also found that group dynamics encouraged persistence; instructors thought they might never talk with colleagues about teaching in the absence of AACR groups; and groups were perceived to have a low-enough time requirement to warrant sustained involvement. We conclude that instructors have persisted in AACR groups because the groups provided great value with limited cost. The characterization of instructor experiences described here can contribute to a better understanding of faculty needs in teaching professional development.

We have been exploring a variety of computerized techniques for analyzing student writing in introductory biology. We achieve computer-to-expert inter-rater reliability (IRR) on par with expert-to-expert IRR (> .8). In Fall, 2012, we piloted the use of automated text analysis to facilitate the use of written formative assessment for Just-in-Time Teaching (JiTT) in a large-enrollment introductory biology course at a large public Midwestern university. A total of 12,677 student responses to 15 online homework questions were collected in three 300+ student course sections with four instructors. We used automated analysis to create feedback for instructors before the next class period (less than one working day), so that instructions could use this feedback to inform their instruction. Instructors used many of the questions pre- and post-instruction and the reports we provided to them allowed them to see how their students' answers changed as a result of their instruction. Focus groups with the instructors revealed that they already knew some of the topics that challenged students, as revealed in previous semesters with multiple-choice examinations. However, the instructors pointed out that the written assessments were particularly important for gaining insight as to why students have struggled continuously with these ideas.

Computerized lexical analysis paired with human scoring was used to explore student ideas about the effect of a stop codon mutation on replication, transcription, and translation. It was found that student ideas about one process can affect their understanding of subsequent and previous processes, leading to mixed conceptual models of the central dogma., Previous work has shown that students have persistent difficulties in understanding how central dogma processes can be affected by a stop codon mutation. To explore these difficulties, we modified two multiple-choice questions from the Genetics Concept Assessment into three open-ended questions that asked students to write about how a stop codon mutation potentially impacts replication, transcription, and translation. We then used computer-assisted lexical analysis combined with human scoring to categorize student responses. The lexical analysis models showed high agreement with human scoring, demonstrating that this approach can be successfully used to analyze large numbers of student written responses. The results of this analysis show that students’ ideas about one process in the central dogma can affect their thinking about subsequent and previous processes, leading to mixed models of conceptual understanding.

Thermodynamics can be challenging to students, thus improving thermodynamics instruction and assessment is an important area of science and engineering education research. Constructed response assessments can reveal the complexity of students' ideas about thermodynamics. We investigate the use of lexical analysis software for examining students' constructed responses using a group of three questions related to reaction thermodynamics. These questions were administered to students in a large enrollment undergraduate introductory science course and examined learning at two levels of Bloom's Taxonomy: comprehension and application. Our results show that students are able to identify correct statements about thermodynamics in a multiple choice context but fail to construct correct explanations using thermodynamic concepts. Lexical analysis revealed that students who provided correct explanations incorporated more correct concepts and/or made more connections among these concepts than did students with incorrect explanations. Lexical analysis provided insight into student understanding by revealing heterogeneous ideas that were masked in multiple choice versions of the assessment.

Despite the growing prominence of Facebook in the lives of college students, few studies have investigated the potential of these innovative web-based communication tools for engaging students in academic discussions. This study used a pre-test, post-test design in two introductory-level courses at a large public university to compare students’ (n = 107) perceptions of, attitudes toward, and perceived learning associated with two different online discussion tools: the Facebook group forum and a university-sponsored online tool. Although pre-course surveys indicated that few students enjoyed online discussions, postcourse analysis revealed significant changes in students’ opinions regarding the value and functionality of web-based discussion forums, with Facebook as their clear preference. Students who participated in Facebook discussions enjoyed the site’s familiarity, navigability, and aesthetically appealing interface. Facebook users also reported that they were able to become better acquainted with classmates, felt like valued participants in the course, and learned more course material. This study suggests that, if used appropriately, Facebook may help to increase college student engagement in certain learning contexts by cultivating classroom community and stimulating intellectual discourse.