This study investigates the cognitive processes of novice students in science learning, with a specific focus on how inhibitory control is employed to overcome a common student misconception about the buoyant force in liquid, which leads to the belief that “the greater the depth an object is in a liquid, the greater the buoyant force it experiences.” Utilizing a negative priming paradigm integrated with eye-tracking technology, the research examines the role of inhibitory control in facilitating conceptual change among novice learners. The findings reveal that successful problem- solving in novices is contingent not only on the acquisition of scientific knowledge but also on the effective activation of inhibitory control over misconceptions. A repeated measures analysis of variance (ANOVA) demonstrated significant effects of student expertise levels and experimental conditions on reaction times, with novices exhibiting significantly longer reaction times compared to experts, indicative of lower inhibitory control ability. However, following targeted intervention, novices’ average reaction times progressively decreased, approaching those of experts, suggesting an improvement in inhibitory control. Additionally, eye movement data analysis revealed distinct differences in information processing strategies between novice and expert students. Novices frequently engaged in reviewing and comparing information across multiple areas during problem solving, whereas experts exhibited more efficient information processing, rapidly identifying and suppressing misconceptions. The eye-tracking data provided further insights into the cognitive processes and behavioral patterns of novices across different intervention phases, highlighting the dynamic interplay between inhibitory control and conceptual learning. These findings underscore the critical role of inhibitory control in overcoming misconceptions and offer implications for the design of targeted instructional interventions.

Large language models (LLMs) are now widely accessible, reaching learners across all educational levels. This development has raised concerns that their use may circumvent essential learning processes and compromise the integrity of established assessment formats. In physics education, where problem solving plays a central role in both instruction and assessment, it is therefore essential to understand the physics-specific problem-solving capabilities of LLMs. Such understanding is key to informing responsible and pedagogically sound approaches to integrating LLMs into instruction and assessment. This study therefore compares the problem-solving performance of a general-purpose LLM ($GPT\text{−}4o$, using varying prompting techniques) and a reasoning-optimized model ($o1$-preview) with that of participants in the German Physics Olympiad, based on a set of well-defined Olympiad problems. In addition to evaluating the correctness of the generated solutions, the study analyzes the characteristic strengths and limitations of LLM-generated solutions. The results of this study indicate that both tested LLMs ($GPT\text{−}4o$ and $o1$-preview) demonstrate advanced problem-solving capabilities on Olympiad-type physics problems, on average outperforming the human participants. Prompting techniques had little effect on $GPT\text{−}4o$’s performance, and $o1$-preview almost consistently outperformed both $GPT\text{−}4o$ and the human benchmark. The main implications of these findings are twofold: LLMs pose a challenge for summative assessment in unsupervised settings, as they can solve advanced physics problems at a level that exceeds top-performing students, making it difficult to ensure the authenticity of student work. At the same time, their problem-solving capabilities offer potential for formative assessment, where LLMs can support students in evaluating their own solutions to problems.

One expected outcome of physics instruction is that students develop quantitative reasoning skills, including strategies for evaluating solutions to problems. Examples of well-known “canonical” evaluation strategies include special case analysis, unit analysis, and checking for reasonable numbers. We report on responses from three tasks in different physics contexts prompting students in an introductory calculus-based physics sequence to evaluate expressions for various quantities: the velocity of a block at the bottom of an incline with friction, the final velocities of two masses involved in an elastic collision, and the electric field due to three point charges. Responses from written ($\mathit{N}=580$) and interview ($\mathit{N}=18$) data were analyzed using modified grounded theory and phenomenology. We also employed the analytical framework of epistemic frames. Students’ evaluation strategies were classified into three broad categories: consulting external sources, checking through computation, and comparing to the physical world. Some of the evaluation strategies observed in our data, including canonical as well as noncanonical strategies, have been reported in prior research on evaluation, albeit sometimes with different names and with varying levels of generalizability. We note four major, general observations prompted by our results. First, most students did not evaluate solutions to physics problems using an approach that an expert would consider an evaluation strategy. Second, many students used evaluation strategies that emphasized computation. Third, many students used evaluation strategies that are not canonical but are nonetheless useful. Fourth, the relative prevalence of different strategies was highly dependent on the task context. We conclude with remarks including implications for classroom instruction.

In 2003, Bailey and Slater [A review of astronomy education research, Astron. Educ. Rev. 2, 20 (2003)] published a review of astronomy education research (AER) to survey the field, identify areas of robust coverage, and uncover gaps in the research to help guide future researchers. While our review is not intended as a direct comparison to their original work, we aim to update the field’s progress since that time. After initially considering more than 450 works, primarily peer-reviewed publications, we focused on research conducted in undergraduate settings or related to non-K-12-teaching adults, such as planetarium professionals, participants in massive open online courses, and citizen scientists. We identified seven broad categories of research, continuing several categories from the original study, such as student understanding and efficacy of instructional strategies, while also introducing new topics, such as authentic research experiences. These findings demonstrate the evolution of AER over the past two decades. The largest body of research remains focused on the efficacy and implementation of instructional strategies. Progress has also been made in exploring technology use (e.g., simulations) in introductory astronomy classrooms and integrating active learning into fully online courses. Notably, course-based undergraduate research experiences have emerged as a new focus in AER. Moving forward, future research should better incorporate educational theory (e.g., mechanisms of learning) and explore the unique educational opportunities offered within astronomy.

Despite the recognition that reflection is an essential part of problem solving, it is often not emphasized in upper-division physics courses. In this paper, we discuss homework corrections (HWCs) as a pedagogical tool to motivate reflection on homework assignments. We focus on gaining a qualitative understanding of how students may engage with the process of homework corrections, considering potential impacts on content understanding, metacognitive skills, and affect. To do so, we present three comparative case studies to elucidate different aspects of student engagement with HWCs; we also present descriptive statistics of participation in HWCs. We find that HWCs can be a useful pedagogical tool, but more scaffolding may be necessary to support students in identifying and learning from their mistakes, even in upper-division courses.

This exploratory field study investigates the integration of innovative forms of recitation tasks in a first-year introductory mechanics course, focusing on smartphone-based experimental tasks alongside programming and standard recitation tasks. Smartphones, combined with external sensor modules, serve as a gateway enabling students to conduct various low-cost and authentic physics experiments with first-hand data collection outside traditional lab settings. These tasks aim to enhance students’ agency in independent physics experimentation and enrich homework assignments by dissolving boundaries between lectures, recitation sessions, and traditional labs, and thereby linking theoretical and experimental aspects of undergraduate physics education. To explore this potential, we implemented and evaluated a sample set of nine smartphone-based experimental tasks and, for comparison, three programming tasks as weekly exercises in a first-year physics course at RWTH Aachen University. We investigated students’ perceptions of learning with these new tasks through 12 short surveys involving up to 188 participants, focusing on factors such as goal clarity, difficulty, or feasibility at home. In two additional surveys with 108 and 78 participants, students assessed affective responses to the smartphone-based experimental tasks relative to the programming and standard recitation tasks. Our findings indicate that the smartphone-based experimental tasks were generally well suited to the students and tended to outperform the programming tasks in terms of perceptions of learning with the tasks and affective responses. Overall, students responded positively to the new experimental tasks, with perceptions comparable to, or only partly below, those of long-established standard recitation tasks. These results suggest that smartphone-based experimental tasks can be successfully integrated into teaching and contribute to refining traditional recitation tasks. Students’ differentiated perceptions of the three task types investigated offer valuable insights into how students perceived technology-enhanced recitation tasks in terms of feasibility, engagement, and instructional value. This provides a meaningful basis for instructors and researchers aiming to design more effective and student-centered learning environments in undergraduate physics education.

In this quantitative study, we explored learners’ intrinsic motivation towards three different experimental physics educational activities offered by our department to upper-grade high school students. The inspected activities were lecture demonstrations, a science show, and hands-on practical work. The respondents were Czech high school students ($N=9647$) who participated in these activities. From each of these activities, a highly comparable subset of 1128 respondents was selected for the comparative analysis. To assess their intrinsic motivation and its covariates, we used three dimensions of the Intrinsic Motivation Inventory: interest/enjoyment, effort/importance, and value/usefulness. The data were evaluated using a unidimensional IRT Graded Response model. Our study revealed that the science show was more appealing to high school students than traditional lecture demonstrations and, surprisingly, even more appealing than practical lab work. This increased interest was more pronounced among students who identified themselves as less diligent in physics. Paradoxically, the respondents perceived the science show as less useful than the other two activities. Delving deeper, gender differences in perceived intrinsic motivation were generally minor, with one exception: in a comparison of the interest generated by the science show and practical work, girls favored the science show more than boys did. These findings underscore the need for a nuanced understanding of pedagogical strategies and offer insights into different approaches to promoting student engagement and interest in physics. Our research could assist educators in distinguishing the activities that might be more beneficial in terms of generating interest among particular groups of students.

Accuracy on assessments is commonly studied in education research but response time (RT) is relatively less investigated even though decades of research in cognitive sciences indicate that time can be an important dimension for understanding student learning. To better understand RT and the potentially important relations between accuracy and RT in physics education, we conducted an exploratory investigation by collecting and analyzing both accuracy and RT data on physics-relevant math skills on low-stakes pre and posttests as well as course exam scores in algebra-based and calculus-based introductory physics courses over two semesters for a total of $N=1936$ participants. Overall, we found a high level of variation in response times revealing weak but consistent patterns of associations between RT and accuracy on skills and exam scores. First, we found a nonlinear relationship between RT and accuracy on the pretest and on the post-test, which may indicate a variety of strategies and engagement among students on these participation-credit-only tests. Second, the results indicate that while RT alone does not predict course grade, when controlling for accuracy on pre or posttest math skills, students with lower RT on these skills are more likely to get better grades. Therefore, both pre or posttest accuracy and speed predicted course grades, though accuracy explained a substantial amount of variance ($\sim 35\%$) while pretest RT explained a much smaller amount of variance ($\sim 1\%$). Third, controlling for both pretest accuracy and pretest RT, we found that students who sped up from pre to posttest were likely to get higher exam scores; however, students who slowed down were on average likely to have a higher post-test score. Fourth, since systemic inequities in STEM education have been documented via measured mean differences between some demographic groups for exam scores and accuracy on math skills, we compared RTs by sex, race, first-generation status, and citizenship to potentially gain more insight into these inequities. We found no consistent or conclusive evidence of demographic differences, though in multiple comparisons, Black, Hispanic, Native American, and Pacific Islander students had larger RTs on average, and in one comparison they were slightly faster. We found that RT was not a mediator of demographic differences in physics grades, though, as expected, accuracy on math skills was a mediator. We briefly discuss how our results relate to various cognitive models such as cognitive ability, speed-accuracy trade-offs, fluency and cognitive load, dual-process theories, and student psychological factors like self-efficacy, anxiety, and motivation. We argue that, based on which (if any) of the above mechanisms are at play, valuing speed in physics may have benefits, such as improving fluency to reduce cognitive load and drawbacks, such as unintentionally using speed as a proxy for achievement or inducing excessive stress that may interfere with performance and student well-being.

Following the global COVID-19 pandemic, digital learning environments have gained in importance in education all around the world. A new digital learning format called a “Digital Learning Module” (DLM) was developed within the Physics Education Research (PER) group at CERN, consisting of a series of prerecorded explanatory and experiment videos, interactive elements, an interactive simulation experiment, and expert interviews. So far, one DLM has been created on the topic of positron emission tomography (PET). Aiming to establish a set of effective design principles that can be employed in the design of future DLMs or similar digital learning environments, the goal of this research is to evaluate the already existing PET DLM. In doing so, this research brings together different theoretical and empirical studies, in order to identify design principles fitting the format of the DLM. It also reviews key physics literature on PET to identify its disciplinary-relevant aspects. First, an analysis of the PET DLM was performed in terms of the identified design principles and disciplinary-relevant aspects. Then, nine semistructured interviews were conducted with high-school students from Greece and Austria as they worked through the PET DLM. A qualitative content analysis of the interview transcripts has determined the set of effective design principles that will constitute the foundation of future DLMs developed by the CERN PER group and can serve as a starting point and reference for researchers developing similar digital learning environments elsewhere.

Inequity is persistent in science, technology, engineering, and mathematics classrooms despite significant attention to upending it. In order to better understand how and why inequity persists, an analysis is presented of 27 self-identified white male physics faculty and graduate students responding to an interview prompt about the social exclusion of a Black woman by peers in a physics classroom. Three main research questions are addressed: (i) How familiar are these men with the alienation Black women frequently experience?, (ii) To what do they attribute this alienation?, and (iii) What do they think an instructor should do in a classroom where this alienation is occurring? Interviewees were found to have little understanding of the bias and discrimination Black women face, were hesitant to name the underlying cause as racism or sexism, and generally had limited insight into how to address racial and gender bias in their own classrooms. These findings indicate majority group instructors are ill equipped to recognize or disrupt oppressive structures in their own classrooms and may inadvertently exacerbate hostile interactions.

Dual-process theories of reasoning suggest that humans reason using two processes often referred to as process 1 (heuristic) and process 2 (analytic). When presented with a situation requiring any sort of reasoning or decision making, process 1 automatically engages and generates an initial mental model to address the situation. Process 2 may or may not be engaged to assess the initial model as a plausible solution. In a study by Kryjevskaia et al., a “screening” question regarding a pulse on a spring aimed to identify students with relevant content knowledge who nevertheless seemed to rely on process 1 when answering a subsequent “target” question. The study was offered as evidence that dual-process theories can explain some discrepancies in student responses to related questions. The study described here assesses the same pair of questions for their ability to distinguish between incorrect answers that stem from inadequate conceptual understanding and those that stem from reasoning approaches. We use Frederick’s cognitive reflection test as part of this analysis. Our results largely support a dual-process-theories perspective of student reasoning.

[This paper is part of the Focused Collection in Artificial Intelligence Tools in Physics Teaching and Physics Education Research.] This study examines the feasibility and potential advantages of using large language models, in particular GPT-4o, to perform partial credit grading of large numbers of student written responses to introductory level physics problems. Students were instructed to write down verbal explanations of their reasoning process when solving one conceptual and two numerical calculation problems on two exams. The explanations were then graded according to a three-item rubric with each item graded as binary (1 or 0). We first demonstrate that machine grading using GPT-4o with no examples or reference answers can reliably agree with human graders in 70%–80% of all cases, which is equal to or higher than the level at which two human graders agree with each other. Two methods are essential for achieving this level of accuracy: (i) Adding explanation language to each rubric item that targets the errors of initial machine grading. (ii) Running the grading process 5 times and taking the most frequent outcome. Next, we show that the variation in outcomes across five machine grading attempts can serve as a grading confidence index. The index allows a human expert to identify $\sim 40\%$ of all potentially incorrect gradings by reviewing just 10%–15% of all responses with the highest variation. Finally, we show that it is straightforward to use GPT-4o to write a clear and detailed explanation of the partial credit grading outcome. Those explanations can be used as feedback for students, which will allow students to understand their grades and raise different opinions when necessary. Almost all feedback messages generated were rated three or above on a five-point scale by two instructors who had taught the course multiple times. The entire grading and feedback generating process costs roughly $5 per 100 student answers, which shows immense promise for automating labor-intensive grading process through a combination of machine grading with human input and supervision.

Video instruction has been widely studied and is generally considered as effective as live instruction for teaching scientific concepts and procedures. However, there is limited research on the effectiveness of prerecorded videos for teaching more challenging content. This study conducted a controlled experiment on over 300 high school students to compare the effectiveness of live instruction versus recorded video in teaching physics problem solving, a representative example of challenging content. Our results indicate that for simpler problems, there is no significant difference in student learning outcomes between live and video instruction, corroborating previous research. However, as problem complexity increases, live instruction significantly outperforms video instruction, potentially highlighting the importance of real-time adaptability, engagement, and sustained attention in face-to-face settings. We found a negative correlation between the effectiveness of video instruction and the difficulty level of the content. This study is the first to empirically investigate the use of video instruction for complex problem solving, making a significant contribution to the current body of literature. Additionally, the findings offer practical insights for school administrators or department heads, suggesting that while video instruction can effectively substitute for live teaching in simpler scenarios, live instruction remains crucial for more complex material, impacting decisions related to human resource allocation and instructional planning.

We have previously described the reformed introductory physics course, collaborative learning through active sense-making in physics (CLASP), for bioscience students at a large public research one university (Original University) and presented evidence that the course was more successful and more equitable than the course it replaced by several measures. Now we compare the original success of CLASP with an implementation at a second institution. We find that the original results hold at another institution despite some changes to the original curriculum and a somewhat different student population. We find that students who take CLASP are (1) less likely to drop, (2) less likely to fail, and (3) do as well in later coursework when compared to students who took the courses that CLASP replaced, even if that coursework is not similarly reformed. We find the above items to be independently true for historically marginalized students and remarkably, also find that (4) marginalized students who take CLASP are more likely to graduate from a science, technology, engineering, and mathematics field. We use a course deficit model perspective to examine these results and discuss some of the factors that may have contributed to this success. We argue that higher education has the tools needed to significantly increase equity and improve student success and retention.

In physics education research, instructors and researchers often use research-based assessments (RBAs) to assess students’ skills and knowledge. In this paper, we support the development of a mechanics cognitive diagnostic to test and implement effective and equitable pedagogies for physics instruction. Adaptive assessments using cognitive diagnostic models provide significant advantages over fixed-length RBAs commonly used in physics education research. As part of a broader project to develop a cognitive diagnostic assessment for introductory mechanics within an evidence-centered design framework, we identified and tested the student models of four skills that cross content areas in introductory physics: apply vectors, conceptual relationships, algebra, and visualizations. We developed the student models in three steps. First, we based the model on learning objectives from instructors. Second, we coded the items on RBAs using the student models. Finally, we then tested and refined this coding using a common cognitive diagnostic model, the deterministic inputs, noisy “and” gate model. The data included 19 889 students who completed either the Force Concept Inventory, Force and Motion Conceptual Evaluation, or Energy and Momentum Conceptual Survey on the LASSO platform. The results indicated a good to adequate fit for the student models with high accuracies for classifying students with many of the skills. The items from these three RBAs do not cover all of the skills in enough detail, however, they will form a useful initial item bank for the development of the mechanics cognitive diagnostic.

Studying the factors that influence the quality of physics Ph.D. students’ doctoral experiences, especially those that motivate them to stay or leave their programs, is critical for providing them with more holistic and equitable support. Prior literature on doctoral attrition has found that students with clear research interests who establish an advisor-advisee relationship early in their graduate careers are most likely to persist. However, these trends have not been investigated in the context of physics, and the underlying reasons for why these characteristics are associated with leaving remain unstudied. Using semistructured interviews with 40 first and second year physics Ph.D. students, we construct a model describing the characteristic pathways that physics Ph.D. students take while evaluating the interest congruence of prospective research groups. We show how access to undergraduate research and other formative experiences helped some students narrow their interests and look for research groups before arriving at graduate school. In turn, these students reported fewer difficulties finding a group than students whose search for an advisor took place during the first year of their Ph.D. Finally, we identify two characteristic types of students at a higher risk of leaving their programs: students who enter graduate school with broad interests and struggle to find a group and students who join a research group early based on research interest alone and subsequently encounter issues with a negative mentoring relationship. This work serves as a major step toward creating a comprehensive model of how Ph.D. students find a research group and opens the door for future work to investigate how factors such as group culture and working environment impact the search process.

Despite knowing physics and astronomy doctoral programs are laden with identity-based inequities, they continue to push minoritized students to the margins. This qualitative social network analysis of 100 women and/or lesbian, gay, bisexual, transgender, queer, and more ($\mathrm{LGBT}+$) physics and astronomy Ph.D.’s explores how minoritized physics and astronomy students utilize social networks to navigate departmental exclusion. Our findings indicate that many of the participants’ identities were often unacknowledged or negatively addressed within their graduate education, with only four participants reporting a positive or favorable experience during this period of their career. Direct support from peers, faculty, and identity-based affinity groups was necessary for participants to navigate their educations. This study demonstrated that generic best practices often cannot fully support the diverse range of persons who come to physics and that identity-neutral values in physics further isolate students by insinuating that their own minoritized experiences are not valid.

As a result of the pandemic, many physics courses moved online. Alongside, the popularity of Internet-based problem-solving sites and forums rose. With the emergence of large language models, another shift occurred. One year into the public availability of these models, how has online help-seeking behavior among introductory physics students changed, and what is the effect of different patterns of online resource usage? In a mixed-method approach, we investigate student choices and their impact on assessment components of an online introductory physics course for scientists and engineers. We find that students still mostly rely on traditional Internet resources and that their usage strongly influences the outcome of low-stake unsupervised quizzes. We empirically found distinct clusters of help-seeking and resource-usage patterns among the students; the impact of students’ cluster membership on the supervised assessment components of the course, however, is nonsignificant.

Joining a research group is one of the most important events on a graduate student’s path to becoming an independent physics researcher and earning a Ph.D. However, graduate students’ perspectives on the experience of finding a research group are not well documented in the literature. Understanding these perspectives is crucial for evaluating whether departments are providing students with adequate support while they search for a research group, and how difficulties during this process contribute to attrition. Semistructured interviews with $N=20$ first and second year physics Ph.D. students reveal that incoming graduate students see joining a research group as a significant decision, and recognize that it may impact whether they will be able to complete the program. We found that students who struggled to find a group felt isolated and worried about falling behind their peers, whereas students who were able to immerse themselves in a positive group environment reported increased sense of belonging in their programs. The process of finding a research group often held differential importance for students identifying as women and nonbinary, who at times reported having to deprioritize their preferred research topic in order to be part of a more inclusive working environment. Although incoming graduate students characterized joining a research group as a significant decision, they often felt unprepared to make it. Moreover, they perceived an overall lack of guidance and structure from their departments, and characterized coursework as a barrier to searching for a group. Our findings suggest that providing students with better support during their group search process could help improve retention, particularly for traditionally underrepresented students, and improve students’ overall satisfaction in their graduate programs.

Physics education researchers are interested in using the tools of machine learning and natural language processing to make quantitative claims from natural language and text data, such as open-ended responses to survey questions. The aspiration is that this form of machine coding may be more efficient and consistent than human coding, allowing much larger and broader datasets to be analyzed than is practical with human coders. Existing work that uses these tools, however, does not investigate norms that allow for trustworthy quantitative claims without full reliance on cross-checking with human coding, which defeats the purpose of using these automated tools. Here we propose a four-part method for making such claims with supervised natural language processing: evaluating a trained model, calculating statistical uncertainty, calculating systematic uncertainty from the trained algorithm, and calculating systematic uncertainty from novel data sources. We provide evidence for this method using data from two distinct short response survey questions with two distinct coding schemes. We also provide a real-world example of using these practices to machine code a dataset unseen by human coders. We offer recommendations to guide physics education researchers who may use machine-coding methods in the future.