In deciding on a student’s grade in a class, an instructor generally needs to combine many individual grading judgments into one overall judgment. Two relatively common numerical scales used to specify individual grades are the 4-point scale (where each whole number 0–4 corresponds to a letter grade) and the percent scale (where letter grades A through D are uniformly distributed in the top 40% of the scale). This paper uses grading data from a single series of courses offered over a period of 10 years to show that the grade distributions emerging from these two grade scales differed in many ways from each other. Evidence suggests that the differences are due more to the grade scale than to either the students or the instructors. One major difference is that the fraction of students given grades less than $\mathrm{C}-$ was over 5 times larger when instructors used the percent scale. The fact that each instructor who used both grade scales gave more than 4 times as many of these low grades under percent scale grading suggests that the effect is due to the grade scale rather than the instructor. When the percent scale was first introduced in these courses in 2006, one of the authors of this paper, who is also one of the instructors in this dataset, had confidently predicted that any changes in course grading would be negligible. They were not negligible, even for this instructor.

Though several conceptual inventories have been developed for thermal physics, none target upper-division material and all focus specifically on thermodynamics without including statistical mechanics content. In this paper, we outline the development process of an upper-division thermal physics assessment that captures both thermodynamics and statistical mechanics content. To ensure that the assessment can be broadly usable, as a first step in this process, we administered a survey to physics faculty to determine the scope and content variability of thermal physics courses across institutions. We received over 70 responses from 63 unique institutions, approximately half of which are minority-serving institutions and women’s colleges. Our findings support the claim that there is significant variation in the content covered at different institutions, but also some general agreement on a number of core content areas. We identified 10 key topics which were listed by the majority (95%) of survey respondents to focus on for the assessment development process. We then wrote assessment objectives that encompass core content goals within these 10 topics, which guided the writing of free-response assessment items that were piloted with students. Using student responses to the free-response items, we developed multiple-response versions, which includes both multiple-choice and coupled, multiple-response items. In this paper, we present details of the faculty survey, including methods of developing and distributing the survey to solicit a broad range of perspectives. Additionally, we present results of the survey, including core content covered by faculty in upper-division thermal physics courses, and discuss how these results were used to guide development of assessment objectives and assessment items. We include the full development process of one assessment item as an example.

This article shares analysis regarding quantum mechanics students’ metarepresentational competence (MRC) that is expressed as they engaged in solving an expectation value problem, which involves linear algebra concepts. The particular characteristic of MRC that is the focus of this analysis is students’ critiquing and comparing the adequacy of representations, specifically matrix notation and Dirac notation, and judging their suitability for various tasks. With data of students’ work during semistructured individual interviews, components of students’ MRC were analyzed and categorized according to value-based preferences, problem-based preferences, and purpose and utility awareness. Detail is provided on two students who serve as paradigmatic examples of students’ power and flexibility within different notation systems, and detail of a third student is given as a point of contrast. In addition to adapting MRC as a helpful construct for characterizing student understanding at the intersection of undergraduate mathematics and physics, we aim to demonstrate how students’ rich understanding of linear algebra and quantum mechanics includes and is aided by their understanding and flexible use of different notational systems. For example, the problem-based preference aspects of MRC highlight that any particular problem-solving approach is itself intrinsically tied to a notational system. We suggest that any instruction with the goal of helping students develop a deep understanding of quantum mechanics and linear algebra should provide opportunities for students to use and improve their MRC.

Educators and education researchers in postsecondary physics have rarely centered (i.e., intentionally directed attention to) the experiences of students with disabilities, leading to an instructional environment that is not designed to support students with disabilities. In this study, we interviewed five students who identified with the diagnosis of attention-deficit/hyperactivity disorder (ADHD) and were enrolled in introductory physics courses at a public four-year institution. We framed our investigation with a social relational perspective of disability, which posits that an individual’s impairments (referred to as diagnosis characteristics in this paper) interact with social structures to result in disabling barriers (i.e., characteristics of social structures which prevent equal access for individuals with disabilities). We analyzed interview transcripts with interpretative phenomenological analysis (IPA). We found that the participating students discussed diagnosis characteristics including difficulties with focus, being prone to distractions, difficulties with keeping mental track of tasks and structures, and thinking often about abstract concepts. Diagnosis characteristics identified as challenges could result in negative self-perceptions, possibly as a result of internalized ableism. However, students also expressed that understanding their diagnosis led to benefits such as making more informed choices about their study strategies (e.g., using a planner or chunking their studying time). In alignment with our social relational perspective of disability, we found that course design could support or hinder participants’ ability to use their preferred planning or studying strategies. We also found that students experienced increased barriers in their physics courses compared to other courses, specifically due to the increased time needed to process information and a lack of guidance for how to effectively study content for conceptual understanding. SCALE-UP courses introduced supports due to increased student autonomy but could also introduce barriers due to increased distractions. We present recommendations that instructors can implement to increase course supports. Researchers need to continue to center the experiences of students with disabilities in STEM courses so that researchers and practitioners can identify disciplinarily specific strategies to support student engagement and learning.

In this work we present the application of design based research (DBR) methodology to conduct a systematic iterative study of the design and implementation of a teaching-learning sequence (TLS) on emf (electromotive force). This work is the final part of a broader study that started with the analysis of students’ difficulties with emf in the contexts of transitory current, direct current, and electromagnetic induction. We complete our research by investigating to what extent students see emf as a cross-cutting concept. We establish the design parameters based on an epistemological analysis and the empirical data, and describe how we devised and implemented the first version of the TLS that takes 4.5 h of classroom time. We analyze the data obtained in the first implementation and describe the redesign process that resulted in a second version of the TLS that takes 1.5 h of classroom time. We conclude that both versions of the TLS effect a similar boost in student attainment of learning objectives compared to the control group.

Professional problem-solving practice in physics and engineering relies on mathematical sense making—reasoning that leverages coherence between formal mathematics and conceptual understanding. A key question for physics education is how well current instructional approaches develop students’ mathematical sense making. We introduce an assessment paradigm that operationalizes a typically unmeasured dimension of mathematical sense making: use of calculations on qualitative problems and use of conceptual arguments on quantitative problems. Three assessment items embodying this calculation-concept crossover assessment paradigm illustrate how mathematical sense making can positively benefit students’ problem solving, leading to more efficient, insightful, and accurate solutions. These three assessment items were used to evaluate the efficacy of an instructional approach focused on developing students’ mathematical sense making skills. In a quasi-experimental study, three parallel lecture sections of first-semester, introductory physics were compared: two mathematical sense making sections, one with an experienced instructor and one with a novice instructor, and a traditionally taught section, as a control group. Compared to the control group, mathematical sense making groups used calculation-concept crossover approaches more often and gave more correct answers on the crossover assessment items, but they did not give more correct answers to associated standard problems. In addition, although students’ postcourse epistemological views on problem-solving coherence were associated with their crossover use, they did not fully account for the differences in crossover approach use between the mathematical sense making and control groups. These results demonstrate a new assessment paradigm for detecting a typically unmeasured dimension of mathematical sense making and provide evidence that a targeted instructional approach can enhance engagement with mathematical sense making in introductory physics.

With limited time available in the classroom, $e$-learning tools can supplement in-class learning by providing opportunities for students to study and learn outside of class. Such tools can be especially helpful for students who lack adequate prior preparation. However, one critical issue is ensuring that students, especially those in need of additional help, engage with the tools as intended. Here we first discuss an empirical investigation in which students in a large algebra-based physics course were given opportunities to work through research-validated tutorials outside of class as self-study tools. Students were provided these optional tutorials after traditional instruction in relevant topics and were then given quizzes that included problems that were identical to the tutorial problems with regard to the physics principles involved but had different contexts. We find that students who worked through the tutorials as self-study tools struggled to transfer their learning to solve problems that used the same physics principles. On the other hand, students who worked on the tutorials in supervised, one-on-one situations performed significantly better than them. These empirical findings suggest that many introductory physics students may not engage effectively with self-paced learning tools unless they are provided additional incentives and support, e.g., to aid with self-regulation. Inspired by the empirical findings, we propose a holistic theoretical framework to help create learning environments in which students with diverse backgrounds are provided support to engage effectively with self-study tools.

We present the development and validation of a new assessment tool, the Moon Phases Concept Inventory for Middle School (MPCI-MS), a concept inventory about the phases of the moon targeting students aged 10 to 14 years old. Items in the questionnaire are based on a careful examination of the concept domain of phases of the moon, ideas and concepts necessary to understand the mechanism of lunar phases, as chosen by a panel of seven professional astronomers. Questions and multiple-choice answers were tested for readability with 5th grade students, tested for reading level, and submitted to a second panel of professional astronomers to check for face and construct validity of the items. The MPCI-MS was tested with $N=296$ students from grade 5 in elementary school to secondary 2 ($M_{\mathrm{age}}=10.2$ to 14.1). One item about global perspective on lunar phases had to be removed because of poor psychometric properties. The revised MPCI-MS has a post-test Cronbach alpha score of 0.786 and good overall psychometric properties: the mean difficulty index for the MPCI-MS pretest is 0.47, and 0.61 for the post-test; mean point-biserial correlation (post-test) is 0.376. Test-retest without instruction at one-week interval showed high test-retest reliability [$M_{\mathrm{pre}}=13.696$, $M_{\mathrm{post}}=14.523$; $t(45)=1.315$, $p=0.192$]. We conclude that the MPCI-MS is a reliable and valid instrument that can discriminate between novices and experts, and can be used to assess 10 to 14 year-old students’ learning gains on the topic of lunar phases. The final version of MPCI-MS is a 19-item instrument, including two new questions about eclipses, that takes between 15 and 25 min for students to complete.

Is physics education research based on a representative sample of students? To answer this question we skimmed physics education research papers from three journals for the years 1970–2015 looking for the number of research subjects, the course the subjects were enrolled in, and the institution where the research was conducted. We combined this data with demographics data about these institutions to compile a profile of physics education research subjects, and compared the demographics of this population to those of all students taking physics in the United States. Our results suggest that physics education research subjects, as a whole, are better prepared mathematically and are from a narrow and unrepresentative subset of our intended target physics student populations. For this reason, findings from research may not be as generalizable to all student populations as we have previously assumed.

Much work in physics education research (PER) characterizes faculty teaching practice in terms of whether faculty use specific named PER-based teaching methods, either with fidelity or with adaptation; we call this research paradigm the “teaching-method-centered paradigm.” However, most faculty do not frame their teaching in terms of which particular named methods they use, but rather in terms of their own ideas and values, suggesting that the teaching-method-centered paradigm misses key features of faculty teaching. These key features include the productive ideas that faculty have about student learning and faculty agency around teaching. We present three case studies of faculty talking about their teaching, and analyze them in terms of two theoretical frameworks: a framework of teaching principles (How Learning Works) and a framework of faculty agency (self-determination theory). We show that these frameworks well characterize key features of faculty teaching practices and agency, and can be combined in a new paradigm for modeling faculty teaching which we call an “asset-based agentic paradigm.” We therefore encourage physics education researchers to move beyond the teaching-method-centered paradigm and think about faculty teaching using an asset-based agentic paradigm.

We present the findings of a pilot plan of active learning implemented in introductory physics in a Chilean public university. The model is research based as it considered a literature review for adequate selection and design of activities consistent with the levels of students’ reasoning skills. The level of scientific reasoning is positively correlated to student success. By contrast to a control group of students following traditional lectures, we observed a significant reduction in failure rate for students that do not yet possess formal scientific reasoning. This profile of students being the majority, we conclude that implementing active learning is particularly suited to the first year of higher education in the context of a developing country. It fits the particularities of student profile and typical classroom size, leading to learning improvement and reduction of academic risk, as well as being financially sound.

The Colorado Learning Attitudes about Science Survey (CLASS) is an instrument which is widely used in physics education to characterize students’ attitudes toward physics and learning physics and compare them with those of experts. While CLASS has been extensively validated for use in the context of higher education institutions in the United States, there has been less information about its use with European students. We have studied the structural, content, and substantive aspects of validity of CLASS by first doing a confirmatory factor analysis of $N=642$ sets of student answers from the University of Helsinki, Finland. The students represented a culturally and demographically different subset of university physics students than in previous studies. The confirmatory factor analysis used a 3-factor, 15-item factor structure as a starting point and the resulting factor structure was similar to the original. Just minor modifications were needed for fit parameters to be in the acceptable range. We explored the differences by student interviews and consultation of experts. With the exception of one item, they supported the new 14-item, 3-factor structure. The results show that the interpretations made from CLASS results are mostly transferable, and CLASS remains a useful instrument for a wide variety of populations.

Concept inventories (CIs) are commonly used in pre-post instruction to study student conceptual change. For consistency in assessment interpretation, a CI’s assessment construct is desired to maintain invariance across different test times. In this study, the longitudinal measurement invariance (LMI) analysis under the confirmatory factor analysis framework was used to examine the stability of the factor structure between pretest and post-test of two commonly used CIs, i.e., the Force Concept Inventory (FCI) and Conceptual Survey of Electricity and Magnetism (CSEM). A number of existing and modified models were examined in this paper. The results confirmed that all factor models of the FCI fitted well with both pre- and post-test data. For CSEM, acceptable fits were obtained with a reduced version of the CI. When reliability analysis was performed for the factors of these models, most modified models were found to be more reliable than the existing models. The modified models were further tested in LMI analysis, in which a sequence of models with increasingly restrictive parameter constraints was examined. For the FCI, LMI analysis demonstrated the existence of partial strict invariance, i.e., common factor structures, factor loadings, and item thresholds, and equally observed residual variances for all the items except items 2 and 29. For the CSEM, after excluding 10 items, a reduced version was found to hold the strict invariance criteria. These findings reveal that changes in scores of the whole FCI and the reduced CSEM can be attributed to changes in the latent constructs measured by the CIs, which confirms these two CIs as reliable instruments to study students’ conceptual change over time in introductory physics courses.

At universities where introductory physics labs are taught by graduate student teaching assistants (TAs), there is a need for specialized professional development for those TAs. This paper presents a specific instantiation of a model for lab TA professional development that uses a combination of cognitive apprenticeship and expectancy-value theories as its framework. We describe how our model was implemented in the lab TA professional development program, which included reflections, role playing, and other pedagogical activities offered through weekly meetings. Our evaluation included an analysis of TA writing and interactions with students alongside informal observations and interviews. We discuss the importance of accounting for TAs’ interest and self-efficacy development in teaching the labs, as well as the challenge of motivating TAs who have very low initial levels of interest in supporting student learning. We find that many TAs in our lab TA professional development program demonstrated an improvement in TA performance in supporting student learning. Given that the professional development activities require only a modest investment of time, these positive results suggest that the model of lab TA professional development may be usefully adopted and adapted at other institutions where introductory labs are led by graduate student TAs.

Assessments are usually thought of as ways for instructors to get information from students. In this work, we flip this perspective and explore how assessments communicate information to students. Specifically, we consider how assessments may provide information about what faculty and/or researchers think it means to know and do physics, i.e., their epistemologies. Using data from students completing assessment questions during one-on-one think aloud interviews, we explore how assessment features did (or did not) impact student engagement with the assessment problems. We analyze video recordings and transcripts to infer the epistemological framings and resources students use while completing introductory-level physics problems. Students’ framings tended to be fairly stable, but when shifts occurred, they were triggered by a shift in epistemological resource, which can be activated by assessment feature. This work extends existing work on epistemological framing into the realm of assessment and allows us to consider the effects of assessments on our students’ understanding of physics teaching and learning.

We present the development and validation of a construct map addressing introductory quantum mechanics topics at the high school level, as a subset of a larger learning progression on quantum mechanics. Topics include energy quantization, photon absorption and emission, the Heisenberg uncertainty principle, atom stability, orbitals, wave function, and electronic properties of materials. To validate the hypothesized construct map, we designed a multiple-choice questionnaire and a 14-h teaching-learning sequence (TLS) designed in strict relation with the levels of the construct map. Twenty-three classes of Italian students (ages 17–18, $N=408$) were involved in the study: about half ($N=200$) were exposed to the TLS, while $N=208$ students received a typical textbook instruction on the same topics targeted by the construct map. Data were analyzed using a 1D Rasch model. Results show that the proposed construct map consistently describes the increasing abilities of students when exposed to instruction. In particular, when exposed to the TLS activities, students more likely move towards the upper levels of the construct map. Findings have implications for instruction designed to support students’ learning in quantum mechanics at the high school level.

Students’ ability to assess their own knowledge is an important skill in science education. However, students often overestimate their actual performances. In such cases, overconfidence bias arises. Previous studies in physics education have shown that overconfidence bias concerns mainly content areas, such as Newtonian mechanics, where misconceptions are strongly held by students. However, how the received instruction and the levels of understanding of a given topic influence overconfidence bias is yet to be proved. In this paper, we address this issue choosing as content area introductory quantum mechanics (QM). Overall, 408 high school students were involved in the study and randomly assigned to two experimental groups. One group received a textbook-based instruction about introductory QM, whereas the other one received instruction on the same topics through an innovative guided inquiry teaching-learning sequence (TLS), which included also potential pedagogical countermeasures for overconfidence bias. Students of both experimental groups completed a multiple-choice questionnaire and indicated for each item the degree of their confidence in the given answer using a 5-point Likert scale. The overconfidence bias was quantitatively defined and evaluated at person level using a 1D Rasch model. Progress in knowledge about the targeted topics was modeled according to a construct map validated in a previous paper. Results show that, for the whole sample, the overconfidence bias decreased as students progressed along the levels of the construct map. However, findings indicate that students of the TLS group achieved a significantly higher accuracy and a better confidence calibration, while the textbook group exhibited a lower performance and a significantly greater overconfidence bias. Implications for research into overconfidence bias in physics education are briefly discussed.

We have used an unsupervised machine learning method called latent Dirichlet allocation (LDA) to thematically analyze all papers published in the Physics Education Research Conference Proceedings between 2001 and 2018. By looking at co-occurrences of words across the data corpus, this technique has allowed us to identify ten distinct themes or “topics” that have seen varying levels of prevalence in physics education research (PER) over time and to rate the distribution of these topics within each paper. Our analysis suggests that although all identified topics have seen sustained interest over time, PER has also seen several waves of increased interest in certain topics, beginning with initial interest in qualitative, theory-building studies of student understanding, which gave way to a focus on problem solving in the late 2000s. Since 2010 the field has seen a shift toward more sociocultural views of teaching and learning with a particular focus on communities of practice, student identities, and institutional change. Based on these results, we suggest that unsupervised text analysis techniques like LDA may hold promise for providing quantitative, independent, and replicable analyses of educational research literature.

Physics education research (PER) has long used concept inventories to investigate student learning over time and to compare performance across various student subpopulations. PER has traditionally used normalized gain to explore these questions but has begun to use established methods from other fields, including Cohen’s $d$, multiple linear regression, and linear mixed effects models. The choice of analysis method for examining student learning gains in PER is a current subject of debate. We synthesize this debate here by focusing on the research questions and interpretations that can be drawn using various statistical tools. We focus particularly on how results can be interpreted through an equity lens and what questions can and cannot be answered using different methods. Our results demonstrate the importance of clearly defining a research question and using appropriate tools to answer that question.

The cyclic format of the undergraduate physics curriculum depends on students’ ability to recall and utilize material covered in prior courses in order to reliably build on that knowledge in later courses. However, there is evidence to suggest that people often do not retain all, or even most, of what they learned previously. How much information is retained appears to be dependent on the individuals’ approach to learning as well as the style of instruction. In particular, there is evidence to suggest that active engagement techniques in the classroom can improve students’ retention of the material over time. Here, we report the findings of a longitudinal investigation of students’ retention of conceptual understanding as measured by the Force and Motion Conceptual Evaluation (FMCE) following a first-semester, calculus-based introductory physics course, which features significant active engagement in both lecture and recitation. By administering the FMCE at the end of a first-semester physics course and again at the beginning of the subsequent second-semester physics course, we examine students’ knowledge retention over time periods ranging from 1 to 15 months. We find that the shift in students’ FMCE scores between these two courses is positive but corresponds to a small effect size, indicating that students retained effectively all of their conceptual learning (as measured by the FMCE). This finding largely persists even as the length of the gap between the two courses increases. We also find that, when breaking out students’ performance on individual questions, the majority of students maintain their score on individual questions. Averaged over all questions, roughly a fifth of the students switched their answers from right to wrong or wrong to right on any given item.