- Physical Review Journals
All JournalsPhysics Magazine

Physical Review Physics Education Research

Reuse & Permissions

It is not necessary to obtain permission to reuse this article or its components as it is available under the terms of the Creative Commons Attribution 4.0 International license. This license permits unrestricted use, distribution, and reproduction in any medium, provided attribution to the author(s) and the published article's title, journal citation, and DOI are maintained. Please note that some figures may have been included with permission from other third parties. It is your responsibility to obtain the proper permission from the rights holder directly for these figures.

Export citation

Export citation

Choose format for download:

Download Citation
  • Open Access

Uncovering student conceptual structure by a multimethod evaluation of the energy and momentum conceptual survey

Xian Wu1,*, Yaoguang Li2,3, and N. Sanjay Rebello4,5

  • *Contact author: xian.wu@uconn.edu
PDF

Phys. Rev. Phys. Educ. Res. 21, 020140 – Published 5 November, 2025

DOI: https://doi.org/10.1103/kvph-l899

Abstract

The Energy and Momentum Conceptual Survey (EMCS) is an instrument designed to assess students’ understanding of key physics concepts in introductory mechanics. In this study, we applied a combination of classical test theory (CTT), item response theory (IRT), and exploratory factor analysis (EFA) to a dataset of over 10 000 students collected across 5 semesters at a large U.S. university. CTT and IRT analyses identified several problematic items with weak discrimination, low reliability, or evidence of guessing. We then explored the factor structure of the full and reduced versions of the EMCS, comparing models with differing numbers of factors. While a five-factor model offered the best statistical fit, a four-factor model excluding three problematic items (Q16, Q22, and Q23) provided improved interpretability and conceptual coherence. In both models, energy-related items were separated into distinct clusters, suggesting multiple modes of student reasoning. Finally, we evaluated a theoretically grounded two-factor model reflecting the EMCS’s design intent. This structure, representing energy and momentum as distinct yet stable factors, proved robust across model variations. The study demonstrated the best practice of exploratory factor analysis. Our findings demonstrate best practices for EFA while also highlighting the importance of instructional context in shaping learning gains. The low course-level gains observed in this study suggest the need to examine teaching practices along a continuum from traditional to active. Future work linking EMCS outcomes with detailed instructional-context data across multiple institutions will help clarify how different enactments of active teaching influence student conceptual understanding in physics.

View figure in article

Physics Subject Headings (PhySH)

Corrections

12 December, 2025

Correction: The last author’s name was presented incorrectly and has been fixed.

Article Text

References (77)

  1. S. B. McKagan, L. E. Strubbe, L. J. Barbato, B. A. Mason, A. M. Madsen, and E. C. Sayre, PhysPort use and growth: Supporting physics teaching with research-based resources since 2011, Phys. Teach. 58, 465 (2020).
  2. W. K. Adams and C. E. Wieman, Development and validation of instruments to measure learning of expert-like thinking, Int. J. Sci. Educ. 33, 1289 (2011).
  3. T. M. Haladyna and M. C. Rodriguez, Developing and Validating Test Items (Routledge, New York, 2013).
  4. L. P. McGinness and C. M. Savage, Developing an action concept inventory, Phys. Rev. Phys. Educ. Res. 12, 010133 (2016).
  5. J. Li and C. Singh, Developing and validating a conceptual survey to assess introductory physics students’ understanding of magnetism, Eur. J. Phys. 38, 025702 (2017).
  6. L. Ding and R. Beichner, Approaches to data analysis of multiple-choice questions, Phys. Rev. ST Phys. Educ. Res. 5, 020103 (2009).
  7. I. T. Koponen and M. Nousiainen, Concept networks of students’ knowledge of relationships between physics concepts: Finding key concepts and their epistemic support, Appl. Netw. Sci. 3, 14 (2018).
  8. J. Stewart, C. Zabriskie, S. DeVore, and G. Stewart, Multidimensional item response theory and the Force Concept Inventory, Phys. Rev. Phys. Educ. Res. 14, 010137 (2018).
  9. C. Zabriskie and J. Stewart, Multidimensional item response theory and the conceptual survey of electricity and magnetism, Phys. Rev. Phys. Educ. Res. 15, 020107 (2019).
  10. J. Hansen and J. Stewart, Multidimensional item response theory and the brief electricity and magnetism assessment, Phys. Rev. Phys. Educ. Res. 17, 020139 (2021).
  11. X. Wu, M. W. Guthrie, E. M. Scanlon, and Y. Li, Analyzing the dimensionality of the energy and momentum conceptual survey using item response theory, presented at PER Conf. 2003, Sacramento, CA, 10.1119/perc.2023.pr.Wu.
  12. B. Van Dusen and J. Nissen, Modernizing use of regression models in physics education research: A review of hierarchical linear modeling, Phys. Rev. Phys. Educ. Res. 15, 020108 (2019).
  13. J. Von Korff, B. Archibeque, K. A. Gomez, T. Heckendorf, S. B. McKagan, E. C. Sayre, E. W. Schenk, C. Shepherd, and L. Sorell, Secondary analysis of teaching methods in introductory physics: A 50 k-student study, Am. J. Phys. 84, 969 (2016).
  14. T. Martín-Blas, L. Seidel, and A. Serrano-Fernández, Enhancing Force Concept Inventory diagnostics to identify dominant misconceptions in first-year engineering physics, Eur. J. Eng. Educ. 35, 597 (2010).
  15. G. Poutot and B. Blandin, Exploration of students’ misconceptions in mechanics using the FCI, Am. J. Educ. Res. 3, 116 (2015).
  16. J. Wells, R. Henderson, J. Stewart, G. Stewart, J. Yang, and A. Traxler, Exploring the structure of misconceptions in the Force Concept Inventory with modified module analysis, Phys. Rev. Phys. Educ. Res. 15, 020122 (2019).
  17. K. L. Malone, Correlations among knowledge structures, Force Concept Inventory, and problem-solving behaviors, Phys. Rev. ST Phys. Educ. Res. 4, 020107 (2008).
  18. J. Wang and L. Bao, Analyzing Force Concept Inventory with item response theory, Am. J. Phys. 78, 1064 (2010).
  19. M. Planinic, L. Ivanjek, and A. Susac, Rasch model based analysis of the Force Concept Inventory, Phys. Rev. ST Phys. Educ. Res. 6, 010103 (2010).
  20. G. A. Morris, N. Harshman, L. Branum-Martin, E. Mazur, T. Mzoughi, and S. D. Baker, An item response curves analysis of the Force Concept Inventory, Am. J. Phys. 80, 825 (2012).
  21. T. F. Scott, D. Schumayer, and A. R. Gray, Exploratory factor analysis of a Force Concept Inventory data set, Phys. Rev. ST Phys. Educ. Res. 8, 020105 (2012).
  22. P. Eaton and S. D. Willoughby, Confirmatory factor analysis applied to the Force Concept Inventory, Phys. Rev. Phys. Educ. Res. 14, 010124 (2018).
  23. J. Docktor, K. Heller, C. Henderson, M. Sabella, and L. Hsu, Gender differences in both Force Concept Inventory and introductory physics performance, AIP Conf. Proc. 1064, 15 (2008).
  24. A. Traxler, R. Henderson, J. Stewart, G. Stewart, A. Papak, and R. Lindell, Gender fairness within the Force Concept Inventory, Phys. Rev. Phys. Educ. Res. 14, 010103 (2018).
  25. J. B. Buncher, J. M. Nissen, B. Van Dusen, and R. M. Talbot, Is the Force Concept Inventory biased across the intersections of gender and race?, Phys. Rev. Phys. Educ. Res. 21, 010137 (2025).
  26. P. Nieminen, A. Savinainen, and J. Viiri, Force Concept Inventory-based multiple-choice test for investigating students’ representational consistency, Phys. Rev. ST Phys. Educ. Res. 6, 020109 (2010).
  27. P. Nieminen, A. Savinainen, and J. Viiri, Relations between representational consistency, conceptual understanding of the force concept, and scientific reasoning, Phys. Rev. ST Phys. Educ. Res. 8, 010123 (2012).
  28. R. K. Thornton and D. R. Sokoloff, Assessing student learning of Newton’s laws: The Force and Motion Conceptual Evaluation and the evaluation of active learning laboratory and lecture curricula, Am. J. Phys. 66, 338 (1998).
  29. T. I. Smith and M. C. Wittmann, Applying a resources framework to analysis of the force and motion conceptual evaluation, Phys. Rev. ST Phys. Educ. Res. 4, 020101 (2008).
  30. R. K. Thornton, D. Kuhl, K. Cummings, and J. Marx, Comparing the force and motion conceptual evaluation and the Force Concept Inventory, Phys. Rev. ST Phys. Educ. Res. 5, 010105 (2009).
  31. J. Wells, R. Henderson, A. Traxler, P. Miller, and J. Stewart, Exploring the structure of misconceptions in the Force and Motion Conceptual Evaluation with modified module analysis, Phys. Rev. Phys. Educ. Res. 16, 010121 (2020).
  32. C. Singh and D. Rosengrant, Multiple-choice test of energy and momentum concepts, Am. J. Phys. 71, 607 (2003).
  33. L. C. McDermott and P. S. Shaffer, Tutorials in Introductory Physics, 1st ed. (Prentice Hall, Upper Saddle River, NJ, 2002).
  34. R. W. Chabay and B. A. Sherwood, Matter & Interactions, 4th ed. (Wiley, Hoboken, NJ, 2015).
  35. https://www.physport.org/.
  36. R. A. Lawson and L. C. McDermott, Student understanding of the work-energy and impulse-momentum theorems, Am. J. Phys. 55, 811 (1987).
  37. B. A. Lindsey, P. R. L. Heron, and P. S. Shaffer, Student ability to apply the concepts of work and energy to extended systems, Am. J. Phys. 77, 999 (2009).
  38. N. Jorion, B. D. Gane, K. James, L. Schroeder, L. V. DiBello, and J. W. Pellegrino, An analytic framework for evaluating the validity of concept inventory claims, J. Eng. Educ. 104, 454 (2015).
  39. L. Plonsky, Advancing Quantitative Methods in Second Language Research, 1st ed. (Taylor & Francis Group, Oxford, 2015).
  40. M. C. Howard, A review of exploratory factor analysis decisions and overview of current practices: What we are doing and how can we improve?, Int. J. Hum. Comput. Interact. 32, 51 (2016).
  41. M. W. Watkins, Exploratory factor analysis: A guide to best practice, J. Black Psychol. 44, 219 (2018).
  42. M. C. Howard, A systematic literature review of exploratory factor analyses in management, J. Bus. Res. 164, 113969 (2023).
  43. M. J. Brundage, A. Maries, and C. Singh, Using the Energy and Momentum Conceptual Survey to investigate progression in student understanding from introductory to advanced levels, Phys. Rev. Phys. Educ. Res. 19, 020132 (2023).
  44. R. F. DeVellis, Classical test theory, Med. Care 44, S50 (2006).
  45. P. Eaton, K. Johnson, B. Frank, and S. Willoughby, Classical test theory and item response theory comparison of the brief electricity and magnetism assessment and the conceptual survey of electricity and magnetism, Phys. Rev. Phys. Educ. Res. 15, 010102 (2019).
  46. E. E. Cureton, The upper and lower twenty-seven per cent rule, Psychometrika 22, 293 (1957).
  47. R. L. Doran, Implications for measurement and evaluation from the trends of science education, Sci. Educ. 60, 199 (1976).
  48. T. Kline, Psychological Testing: A Practical Approach to Design and Evaluation (SAGE Publications, Inc., Thousand Oaks, CA, 2005).
  49. M. Tavakol and R. Dennick, Making sense of Cronbach’s alpha, Int. J. Med. Educ. 2, 53 (2011).
  50. Handbook of Modern Item Response Theory, edited by W. J. Van Der Linden and R. K. Hambleton (Springer, New York, NY, 1997).
  51. J. P. Stevens, Applied Multivariate Statistics for the Social Sciences, 5th ed. (Taylor and Francis, London, 2009).
  52. M. S. Bartlett, Tests of significance in factor analysis, Br. J. Stat. Psychol. 3, 77 (1950).
  53. H. F. Kaiser, A second generation little jiffy, Psychometrika 35, 401 (1970).
  54. C. D. Dziuban and E. C. Shirkey, When is a correlation matrix appropriate for factor analysis? Some decision rules., Psychol. Bull. 81, 358 (1974).
  55. R. K. Henson and J. K. Roberts, Use of exploratory factor analysis in published research: Common errors and some comment on improved practice, Educ. Psychol. Meas. 66, 393 (2006).
  56. A. B. Costello and J. Osborne, Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis, Pract. Assess. Res. Eval. 10, 7 (2005)..
  57. J. M. Conway and A. I. Huffcutt, A review and evaluation of exploratory factor analysis practices in organizational research, Organ. Res. Meth. 6, 147 (2003).
  58. R Core Team, r: A Language and Environment for Statistical Computing (R Foundation for Statistical Computing, Vienna, Austria, 2024).
  59. H. Wickham, R. François, L. Henry, K. Müller, D. Vaughan, and Posit Software, PBC, dplyr: A grammar of data manipulation (2023), https://dplyr.tidyverse.org/.
  60. H. Wickham, L. Henry, and Posit Software, PBC, purrr: Functional programming tools (2025), https://purrr.tidyverse.org/.
  61. W. Revelle, psych: Procedures for psychological, psychometric, and personality research (2025), https://cran.r-project.org/package=psych.
  62. P. Chalmers et al., mirt: A multidimensional item response theory package for the R environment, J. Stat. Softw., 48, 1 (2012).
  63. S. Epskamp, G. Costantini, J. Haslbeck, A. Isvoranu, A. O. J. Cramer, L. J. Waldorp, V. D. Schmittmann, and D. Borsboom, qgraph: Graph plotting methods, psychometric data visualization and graphical model estimation (2023), https://cran.r-project.org/package=qgraph.
  64. R. R. Hake, Interactive-engagement versus traditional methods: A six-thousand-student survey of mechanics test data for introductory physics courses, Am. J. Phys. 66, 64 (1998).
  65. J. D. Marx and K. Cummings, Normalized change, Am. J. Phys. 75, 87 (2007).
  66. L. Bao, Theoretical comparisons of average normalized gain calculations, Am. J. Phys. 74, 917 (2006).
  67. J. M. Nissen, R. M. Talbot, A. Nasim Thompson, and B. Van Dusen, Comparison of normalized gain and Cohen’s d for analyzing gains on concept inventories, Phys. Rev. Phys. Educ. Res. 14, 010115 (2018).
  68. H. F. Kaiser, An index of factorial simplicity, Psychometrika 39, 31 (1974).
  69. R. Driver and L. Warrington, Students’ use of the principle of energy conservation in problem situations, Phys. Educ. 20, 171 (1985).
  70. J. Tuminaro and E. F. Redish, Elements of a cognitive model of physics problem solving: Epistemic games, Phys. Rev. ST Phys. Educ. Res. 3, 020101 (2007).
  71. D. Tong, J. Liu, Y. Sun, Q. Liu, X. Zhang, S. Pan, and L. Bao, Assessment of student knowledge integration in learning work and mechanical energy, Phys. Rev. Phys. Educ. Res. 19, 010127 (2023).
  72. R. Driver and L. Warrington, Students’ use of the principle of energy conservation in problem situations, Phys. Educ. 20, 171 (1985).
  73. C. Henderson, M. Dancy, and M. Niewiadomska-Bugaj, Use of research-based instructional strategies in introductory physics: Where do faculty leave the innovation-decision process?, Phys. Rev. ST Phys. Educ. Res. 8, 020104 (2012).
  74. M. Dancy, C. Henderson, N. Apkarian, E. Johnson, M. Stains, J. R. Raker, and A. Lau, Physics instructors’ knowledge and use of active learning has increased over the last decade but most still lecture too much, Phys. Rev. Phys. Educ. Res. 20, 010119 (2024).
  75. M. Sundstrom, J. Gambrell, C. Green, A. L. Traxle, and E. Brewe, Relative benefits of different active learning methods to conceptual physics learning, arXiv:2505.04577.
  76. I. Bukola, M. Sundstrom, J. Gambrell, O. Ross, A. L. Traxler, and E. Brewe, Multi-institutional assessment of peer instruction implementation and impacts using the framework for interactive learning in lectures, arXiv:2508.08422.
  77. X. Wu, Master Data File (2025), 10.17605/OSF.IO/KV9PA.

Outline

Information

Phys. Rev. Phys. Educ. Res. 21, 020140– Published 5 November, 2025
  • Received 9 June 2025
  • Accepted 10 October 2025
  • Corrected 12 December 2025
Crossmark - Check for updates button
Creative Commons Logo

Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article’s title, journal citation, and DOI.

Published by the American Physical Society

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation