- Physical Review Journals
All JournalsPhysics Magazine

Physical Review Physics Education Research

Reuse & Permissions

It is not necessary to obtain permission to reuse this article or its components as it is available under the terms of the Creative Commons Attribution 4.0 International license. This license permits unrestricted use, distribution, and reproduction in any medium, provided attribution to the author(s) and the published article's title, journal citation, and DOI are maintained. Please note that some figures may have been included with permission from other third parties. It is your responsibility to obtain the proper permission from the rights holder directly for these figures.

Export citation

Export citation

Choose format for download:

Download Citation
  • Open Access

Validity of Force Concept Inventory evaluated by students’ explanations and confirmation using modified item response curve

Yoshihiko Shoji*

Shuji Munejiri

Eiko Kaga

  • Laboratory of Advanced Science and Technology for Industry, University of Hyogo, NweSUBARU/SPring-8, Kamigori-cho, Ako-gun, Hyogo, Japan 678-1205

  • Graduate School of Advanced Science and Engineering, Hiroshima University, 1-7-1 Kagamiyama, Higashi-Hiroshima City Hiroshima, Japan 739-8521

  • Graduate School of Integrated Arts and Sciences, Hiroshima University, 1-7-1 Kagamiyama, Higashi-Hiroshima City Hiroshima, Japan 739-8521

  • *shoji@lasti.u-hyogo.ac.jp
PDF

Phys. Rev. Phys. Educ. Res. 17, 020120 – Published 20 September, 2021

DOI: https://doi.org/10.1103/PhysRevPhysEducRes.17.020120

Abstract

In this study, we evaluate the validity of question 29 (Q.29) in the Force Concept Inventory (FCI), which requires an understanding of atmospheric pressure. At least in some Japanese physics classes, the rate of the correct answer to Q.29 is reduced between the pretest and post-test. To examine this deterioration of a student’s response, we analyzed 429 explanations written by 358 respondents about the force described in the question. Many learners, who were not aware of atmospheric force, chose the correct answer. On the other hand, many of the wrong respondents had learned about air pressure but did not understand it. As a result, postinstruction learners tended to have a worse rate of correct response. This made Q.29 less sensitive to a Japanese student’s ability. This characteristic feature of Q.29 is visually confirmed by a modified item response curve, which represents correlation between the response to one item and the total score of the other items.

View figure in article

Physics Subject Headings (PhySH)

Article Text

References (45)

  1. D. Hestenes, M. Wells, and G. Swackhamer, Force Concept Inventory, Phys. Teach. 30, 141 (1992).
  2. The latest version of the FCI revised in 1995 and its Japanese version are available at PhysPort, http://modeling.asu.edu/R&E/Research.html (Retrieved 11/11/2017).
  3. R. R. Hake, Interactive-engagement versus traditional methods: A six-thousand-student survey of mechanics test data for introductory physics courses, Am. J. Phys. 66, 64 (1998).
  4. J. Cohen, Statistical Power Analysis for the Behavioral Sciences (Academic Press, New York, 1977).
  5. K. F. Wilson and D. J. Low, On second thoughts…: Changes of mind as an indication of competing knowledge structures, Am. J. Phys. 83, 802 (2015).
  6. D. J. Low and K. F. Wilson, The role of competing knowledge structures in undermining learning: Newton’s second and third laws, Am. J. Phys. 85, 54 (2017).
  7. Y. Hasegawa, S. Kishizawa, T. Yamazaki, J. Yasuda, H. Minoda, M. Goda, T. Murata, and R. Lang, Analysis of mechanics conceptual understanding through “Item Response Patterns”, in 2017 Annual meeting of the Physical Society of Japan, 19pC11-5, written in Japanese, https://www.jstage.jst.go.jp/article/jpsgaiyo/72.1/0/72.1_3378/_pdf/-char/ja.
  8. E. F. Redish, Teaching Physics with the Physics Suite (John Wiley & Sons, Hoboken, NJ, 2003), Chap. 2.
  9. A. Perez-Lemonche, J. Stewart, B. Drury, R. Henderson, A. Shvonski, and D. E. Pritchard, Mining students pre-instruction beliefs for improved learning, in Proceedings of the Sixth ACM Conference on Learning, Chicago, USA, June 2019, Article No. 15 (Association for Computing Machinery, 2019), 10.1145/3330430.3333637.
  10. D. Hestenes and I. Halloun, Interpreting the force concept inventory: A response to March 1995 critique by Huffman and Heller, Phys. Teach. 33, 502 (1995).
  11. L. Bao, Theoretical comparisons of average normalized gain calculations, Am. J. Phys. 74, 917 (2006).
  12. G. A. Morris, L. Branum-Martin, N. Harshman, S. D. Baker, E. Mazur, S. Dutta, Taha Mzoughi, and V. McCaule, Testing the test: Item response curves and test quality, Am. J. Phys. 74, 449 (2006).
  13. G. A. Morris, N. Harshman, L. Branum-Martin, E. Mazur, T. Mzoughi, and S. D. Baker, An item response curves analysis of the Force Concept Inventory, Am. J. Phys. 80, 825 (2012).
  14. R. K. Thornton and D. R. Sokoloff, Assessing student learning of Newton’s laws: The Force and Motion Conceptual Evaluation and the Evaluation of Active Learning Laboratory and Lecture Curricula, Am. J. Phys. 66, 338 (1998).
  15. M. Ishimoto, G. Davenport, and M. C. Wittmann, Use of item response curves of the Force and Motion Conceptual Evaluation to compare Japanese and American students’ views on force and motion, Phys. Rev. Phys. Educ. Res. 13, 020135 (2017).
  16. A. E. Lawson, The development and validation of a classroom test of formal reasoning, J. Res. Sci. Teach. 15, 11 (1978).
  17. Y. Xiao, J. Han, K. Koenig, J. Xiong, and L. Bao, Multilevel Rasch modeling of two-tier multiple choice test: A case study using Lawson’s classroom test of scientific reasoning, Phys. Rev. Phys. Educ. Res. 14, 020104 (2018).
  18. L. Bao, Y. Xiao, K. Koenig, and J. Han, Validity evaluation of the Lawson classroom test of scientific reasoning, Phys. Rev. Phys. Educ. Res. 14, 020106 (2018).
  19. P. Eaton, K. Vavruska, and S. Willoughby, Exploring the preinstruction and postinstruction non-Newtonian world views as measured by the Force Concept Inventory, Phys. Rev. Phys. Educ. Res. 15, 010123 (2019).
  20. J. Stewart, B. Drury, J. Wells, A. Adair, R. Henderson, Y. Ma, AA. Perez-Lemonche, and D. Pritchard, Examining the relation of correct knowledge and misconceptions using the nominal response model, Phys. Rev. Phys. Educ. Res. 17, 010122 (2021).
  21. I. A. Halloun and D. Hestenes, Common sense concepts about motion, Am. J. Phys. 53, 1056 (1985).
  22. T. F. Scott, D. Schumayer, and A. R. Gray, Exploratory factor analysis of a Force Concept Inventory data set, Phys. Rev. ST Phys. Educ. Res. 8, 020105 (2012).
  23. M. R. Semak, R. D. Dietz, R. H. Pearson, and C. W. Willis, Examining evolving performance on the Force Concept Inventory using factor analysis, Phys. Rev. Phys. Educ. Res. 13, 010103 (2017).
  24. T. F. Scott and D. Schumayer, Conceptual coherence of non-Newtonian worldviews in Force Concept Inventory data, Phys. Rev. Phys. Educ. Res. 13, 010126 (2017).
  25. P. Eaton and S. D. Willoughby, Confirmatory factor analysis applied to the Force Concept Inventory, Phys. Rev. Phys. Educ. Res. 14, 010124 (2018).
  26. J. Stewart, C. Zabriskie, S. DeVore, and G. Stewart, Multidimensional item response theory and the Force Concept Inventory, Phys. Rev. Phys. Educ. Res. 14, 010137 (2018).
  27. J. Wang and L. Bao, Analyzing Force Concept Inventory with Item Response Theory, Am. J. Phys. 78, 1064 (2010).
  28. M. Planinic, L. Ivanjek, and A. Susac, Rasch model based analysis of the Force Concept Inventory, Phys. Rev. ST Phys. Educ. Res. 6, 010103 (2010).
  29. T. F. Scott and D. Schumayer, Students’ proficiency scores within multitrait item response theory, Phys. Rev. ST Phys. Educ. Res. 11, 020134 (2015).
  30. E. Brewe, J. Bruun, and I. G. Bearden, Using module analysis for multiple choice responses: A new method applied to Force Concept Inventory data, Phys. Rev. Phys. Educ. Res. 12, 020131 (2016).
  31. A. Traxler, R. Henderson, J. Stewart, G. Stewart, A. Papak, and R. Lindell, Gender Fairness within the Force Concept Inventory, Phys. Rev. Phys. Educ. Res. 14, 010103 (2018).
  32. P. Eaton, K. Johnson, and S. Willoughby, Generating a growth-oriented partial credit grading model for the Force Concept Inventory, Phys. Rev. Phys. Educ. Res. 15, 020151 (2019).
  33. P. Eaton and S. Willoughby, Identifying a preinstruction to postinstruction factor model for the Force Concept Inventory within a multitrait item response theory framework, Phys. Rev. Phys. Educ. Res. 16, 010106 (2020).
  34. J.-i. Yasuda, N. Mae, M. M. Hull, and M.-a. Taniguchi, Optimizing the length of computerized adaptive testing for the Force Concept Inventory, Phys. Rev. Phys. Educ. Res. 17, 010115 (2021).
  35. J. Wells, R. Henderson, J. Stewart, G. Stewart, J. Yang, and A. Traxler, Exploring the structure of misconceptions in the Force Concept Inventory with modified module analysis, Phys. Rev. Phys. Educ. Res. 15, 020122 (2019).
  36. J. Yang, J. Wells, R. Henderson, E. Christman, G. Stewart, and J. Stewart, Extending modified module analysis to include correct responses: Analysis of the Force Concept Inventory, Phys. Rev. Phys. Educ. Res. 16, 010124 (2020).
  37. Y. Xiao, G. Xu, J. Han, H. Xiao, J. Xiong, and Lei Bao, Assessing the longitudinal measurement invariance of the Force Concept Inventory and the Conceptual Survey of Electricity and Magnetism, Phys. Rev. Phys. Educ. Res. 16, 020103 (2020).
  38. J. Stewart, M. Miller, C. Audo, and G. Stewart, Using cluster analysis to identify patterns in students’ responses to contextually different conceptual problems, Phys. Rev. ST Phys. Educ. Res. 8, 020112 (2012).
  39. L. McCullough and D. Meltzer, Differences in male/female response patterns on alternative-format versions of FCI items, in Proceedings of the 2001 Physics Education Research Conference, Rochester, NY(AIP, New York, 2001), pp. 103–106.
  40. C. Henderson, Common concerns about the Force Concept Inventory, Phys. Teach. 40, 542 (2002).
  41. N. Lasry, S. Rosenfield, H. Dedic, A. Dahan, and O. Reshef, The puzzling reliability of the Force Concept Inventory, Am. J. Phys. 79, 909 (2011).
  42. Y. Shoji, Cognitive acceleration program in undergraduate school, in Proceedings of the 2019 8th International Congress on Advanced Applied Informatics (IIAI-AAI), Toyama, Japan, 7–11 July 2019, available in IEEE Xplore (IEEE, 2019), https://ieeexplore.ieee.org/document/8992713.
  43. C. S. Wallace and J. M. Bailey, Do concept inventories actually measure anything?, Astron. Educ. Rev. 9, 010116 (2010).
  44. L. J. Cronbach, Coefficient alpha and the internal structure of tests, Psychometrica 16, 297 (1951).
  45. G. F. Kuder and M. W. Richardson, The theory of the estimation of test reliability, Psychometrika 2, 151 (1937).

Outline

Information

Phys. Rev. Phys. Educ. Res. 17, 020120– Published 20 September, 2021
  • Received 18 February 2021
  • Accepted 23 August 2021
Crossmark - Check for updates button
Creative Commons Logo

Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article’s title, journal citation, and DOI.

Published by the American Physical Society

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation