Search

In this work, the authors document an expansion of the Public Speaking Competency Rubric (PSCR). First developed in 2012 by Schreiber, et al., the original rubric has only one item related to non-verbal communication. The authors of this work expanded the rubric to include 10 items related to the non-verbal aspects of public speaking and had it critiqued by 10 outside experts. The rubric was tested on recorded speeches given by college students. The expanded rubric, dubbed the PSCR+, has been found to be both valid and reliable. Minimal training is needed to apply the rubric in a classroom setting, and it has the benefit of being useful for both formative and summative assessment. Finally, this rubric is complete enough that it can be used to provide students with detailed feedback regarding their speaking skills without the addition of further notes or comments.

Items that are chained, or blocked, together appear on many of the conceptual assessments utilized for physics education research. However, when items are chained together there is the potential to introduce local dependence between those items, which would violate the assumption of item independence required by classical test theory, unidimensional item response theory, and other measurement theories. Local dependence can be divided into two categories: (i) underlying local dependence, which can be adequately modeled with multidimensional measurement theories, and (ii) surface local dependence (SLD), which cannot be modeled using multidimensional measurement theories. The act of chaining items is thought to be one of the many potential sources of SLD between items. Using previous local dependence research results, this study proposes two methods for detecting the presence of local dependence and SLD between items on an assessment. These methods were applied to the Force Concept Inventory (FCI) and the Force and Motion Conceptual Evaluation (FMCE). It was found that the assumption of item independence was violated for both assessments, implying that unidimensional measurement theories may not adequately model either the FCI or FMCE. Further, both detection methods identified the potential for a minimal amount of SLD present for FCI and a significant amount of SLD present for the FMCE. This implies that even multidimensional measurement theories may not be capable of adequately modeling the FMCE when scoring items individually. This result supports the claim made by Thornton et al. that the items on the FMCE should be scored in groups; however, the currently proposed grading scheme was found to be inadequate.

As targeted, single-conception curriculum research becomes more prevalent in physics education research (PER), the need for a more sophisticated statistical understanding of the conceptual surveys used becomes apparent. Previously, the factor structure of the Force Concept Inventory (FCI) was examined using exploratory factor analysis (EFA) and exploratory multitrait item response theory (EMIRT). The resulting exploratory driven factor model and two expert-proposed models were tested using confirmatory factor analysis (CFA) and were found to be adequate representations of the FCI’s factor structure for postinstruction student responses. This study compliments the literature by using confirmatory multitrait item response theory (CMIRT) on matched preinstruction to postinstruction student responses (N=19745). It was found that the proposed models did not fit the data within a CMIRT framework. To determine why these models did not fit the data, an EMIRT was performed to find suggested modifications to the proposed models. This resulted in a bi-factor structure for the FCI. After calculating the student ability scores for each trait in the modified factor models, only two traits in each of the models were found to exhibit student gains independent of the general, “Newtonian-ness” trait. These traits measure students’ understanding of Newton’s third law and their ability to identify forces acting on objects.

Traditionally, multiple choice assessments are graded in a dichotomous manner, where selecting the correct option for a question awards 1 point and the selection of an incorrect option awards 0 points. As a result of this grading scheme, all incorrect response options are treated as being equally incorrect regardless of potential differences in their relative correctness, intentional or otherwise. We propose a partial credit grading model for the Force Concept Inventory (FCI) that allots points to the incorrect responses. This was done using slope parameters from the two-parameter logistics nominal response model (2PLNRM), a model from polytomous item response theory (PIRT). The resulting scores from the partial credit model represent student growth towards a proper Newtonian mindset, as measured by the FCI. Observations indicate that this model accounts for student progression through prominent misconceptions (i.e., impetus) as their world views become more Newtonian. As expected, we find that student total scores increase as a result of the model, but the average overall gains on the assessment are essentially unchanged. The data used in this analysis were maintained and organized by PhysPort and included about 20 000 responses from first semester introductory physics courses at multiple universities. Ultimately, this partial credit model allows instructors to more accurately gauge the growth of their students over the course of instruction. Additionally, as a result of these partial credit scores, we are able to identify potentially malfunctioning questions on the FCI that may be sources of error in measuring student abilities.

While numerous studies have analyzed the conceptions probed by the Force Concept Inventory (FCI), assessments dedicated to electricity and magnetism lack similar analyses. This paper investigated the conceptions explored by the Brief Electricity and Magnetism Assessment (BEMA) and the Conceptual Survey of Electricity and Magnetism (CSEM) using exploratory factor analysis techniques. Exploratory factor analysis (EFA) was performed on both assessments using 5368 and 4941 postinstruction student responses for the BEMA and CSEM, respectively. A 6-factor EFA generated model was found for the CSEM, and was fit against another sample of 4964 student responses using confirmatory factor analysis to supply evidence for the possible generalizability of the model. The 5-factor EFA generated model for the BEMA could not be fit against another sample when trying to check for generalizability. The EFA generated factor models for the BEMA and CSEM were then compared and found to be similar in conceptual content, with the exception of one or two factors. Thus, from a factor analysis perspective, the BEMA and CSEM were found to be similar in conceptual content as revealed by student responses. With a better understanding of these electricity and magnetism assessments, future research into this domain of physics will then be able to make stronger conclusions based on students’ results within these assessments.

For proper assessment selection understanding the statistical similarities amongst assessments that measure the same, or very similar, topics is imperative. This study seeks to extend the comparative analysis between the brief electricity and magnetism assessment (BEMA) and the conceptual survey of electricity and magnetism (CSEM) presented by Pollock. This is accomplished by using large samples (N_{BEMA}=5368 and N_{CSEM}=9905) within classical test theory (CTT) and item response theory (IRT) frameworks. For the IRT comparison, after consideration of the conceptual content addressed in each assessment, it was assumed that each of these assessments are measuring the same student latent ability (θ), specifically a student’s ability to do introductory electricity and magnetism. Via a CTT and IRT analysis it was found that both assessments are essentially equal in overall difficulty. Classical item analysis applied to 7 questions used by both assessments revealed that each assessment functions slightly differently internally. The test information curves found from IRT show that the CSEM has superior information compared to the BEMA in estimating student latent abilities for the entire range of typical latent abilities achieved by students on each assessment, θ≈-2 to θ≈3. Information in this case is interpreted as how well a student’s latent ability was estimated by an assessment as a function of latent ability. When the circuits questions are removed from the BEMA the majority of the information is lost in the θ≈0 to θ≈2 range. This means the circuits questions on the BEMA are information heavy for higher ability scores. So, special considerations should be made as to which assessment a study uses depending on the specific questions a researcher is attempting to answer.

Supporting efforts to grow the scientific workforce means articulating and comparing the content of science field stereotypes. To do this, data were collected from the general public [undergraduates (n=121) and Amazon Mechanical Turk workers (n=223)] as well as from people within science [attendees of an undergraduate conference for women in physics (n=34)]. Participants were randomly assigned to consider either biologists or physicists and then produce both spontaneous judgments and rate various person traits (e.g., ratings related to looks and personality and hobbies) and field characteristics (e.g., ratings related to the working conditions, norms, and expectations for the field). Analyses show stereotypes of the scientist and the science field were statistically significantly negative overall, with stereotypes about physicists and the field of physics more negative than biology. Compared to biologists, physicists were perceived as statistically significantly more competent, but statistically significantly more unattractive, tech oriented, awkward, and loners. Furthermore, compared to biology, a job in physics was viewed as having fewer opportunities for working with and helping others, but more opportunities for agency, a greater requirement for innate brilliance and effort to succeed, and as more difficult. That said, physicists were more envied than biologists. Data were triangulated with open-ended responses illustrating that across samples, people are more likely to reproduce science stereotypes for physicists. Implications for stereotype research and broadening participation of the science workforce are discussed, with a focus on the utility of role models and classroom interventions that negate stereotypes such as writing activities and encouraging students to approach physics with a growth mindset. Instructors are encouraged to consider what stereotypes students have about the field of physics and physicists. At the department level, instructors are encouraged to consider hosting a Conference for Undergraduate Women in Physics sponsored in part by the American Physical Society.

This article discusses our investigation regarding nature of science (NOS) implementations and epistemological beliefs within an undergraduate introductory astronomy course. The five year study consists of two years of baseline data in which no explicit use of NOS material was implemented, then three years of subsequent data in which specific NOS material was integrated into the classroom. Our original study covered two years of baseline data and one year of treatment data. Two additional years of treatment course data have revealed intriguing new insights into our students’ epistemic belief structure. To monitor the evolution of belief structures across each semester we used student pre-post data on the Epistemological Beliefs About the Physical Sciences (EBAPS) assessment. The collected data were also partitioned and analyzed according to the following variables: college (Letters of Science, Business, Education, etc.), degree (BA or BS), status (freshman, sophomore, etc.), and gender (male or female). We find that the treatment course no longer undergoes significant overall epistemic deterioration after a semester of instruction. We also acquire a more detailed analysis of these findings utilizing the aforementioned variables. Most notably, we see that this intervention had a pronounced positive impact on males and on students within the college of Education, Arts & Architecture, and those with no concentration. Lastly, whether or not students believe their ability to learn science is innate or malleable did not seem to change, remaining a rigid construct with student epistemologies.