- Physical Review Journals
All JournalsPhysics Magazine

Physical Review Physics Education Research

Reuse & Permissions

It is not necessary to obtain permission to reuse this article or its components as it is available under the terms of the Creative Commons Attribution 4.0 International license. This license permits unrestricted use, distribution, and reproduction in any medium, provided attribution to the author(s) and the published article's title, journal citation, and DOI are maintained. Please note that some figures may have been included with permission from other third parties. It is your responsibility to obtain the proper permission from the rights holder directly for these figures.

Export citation

Export citation

Choose format for download:

Download Citation
  • Open Access

Visualizing depth of student conceptual understanding using subquestions and alluvial diagrams

Jun-ichiro Yasuda*

Michael M. Hull

Naohiro Mae

  • Institute of Arts and Sciences, Yamagata University, Yamagata, Yamagata 990-8560, Japan

  • Department of Physics, University of Alaska Fairbanks, 1930 Yukon Dr, Fairbanks, Alaska 99775, USA

  • Research Center for Nuclear Physics, Osaka University, Ibaraki, Osaka 567-0047, Japan

  • *Corresponding author: phys.cat.collaboration@gmail.com
PDF

Phys. Rev. Phys. Educ. Res. 19, 020121 – Published 31 August, 2023

DOI: https://doi.org/10.1103/PhysRevPhysEducRes.19.020121

Abstract

We aim to graphically analyze the depth of conceptual understanding behind the Force Concept Inventory (FCI) responses of students, focusing on three questions (questions 1, 15, and 28). In our study, we created and implemented subquestions to clarify and quantify the students’ reasoning steps in reaching their responses to the original FCI questions. We used alluvial diagrams to visualize the responses of students on the subquestions and original FCI questions, representing the transition rates across the question options as flows. The flows represent common patterns of consistency and inconsistency in the students’ conceptual understandings across the original FCI question and the subquestions. By combining subquestions and alluvial diagrams as we did, we are able to efficiently analyze the depth of understanding for a given situation and to visualize the existence of a specific misconception. For example, we found that even among those who correctly answered FCI question 15 about Newton’s third law, many students incorrectly answered a corresponding subquestion asking about whether Newton’s third law holds at the end of the interaction between the two objects. This misconception turned out to be fairly robust, as these same students had a similar answering pattern for question 28. Alluvial diagrams can be used for a broad range of applications, including analysis of multiple-tier tests, testlets (e.g., FCI questions 8 through 11), tests for representational consistency (e.g., a representational variant of the FCI), and so on.

View figure in article

Physics Subject Headings (PhySH)

Article Text

References (50)

  1. D. Hestenes, M. Wells, and G. Swackhamer, Force Concept Inventory, Phys. Teach. 30, 141 (1992).
  2. D. Hestenes and I. Halloun, Interpreting the Force Concept Inventory: A response to March 1995 critique by Huffman and Heller, Phys. Teach. 33, 502 (1995).
  3. N. S. Rebello and D. A. Zollman, The effect of distracters on student performance on the Force Concept Inventory, Am. J. Phys. 72, 116 (2004).
  4. S. DeVore, J. Stewart, and G. Stewart, Examining the effects of testwiseness in conceptual physics evaluations, Phys. Rev. Phys. Educ. Res. 12, 020138 (2016).
  5. D. K. Gurel, A. Eryilmaz, and L. C. McDermott, A review and comparison of diagnostic instruments to identify students’ misconceptions in science, Eurasia J. Math. Sci. Technol. Educ. 11, 989 (2015).
  6. I. Testa, S. Galano, S. Leccia, and E. Puddu, Development and validation of a learning progression for change of seasons, solar and lunar eclipses, and moon phases, Phys. Rev. ST Phys. Educ. Res. 11, 020102 (2015).
  7. P. Barniol and G. Zavala, Mechanical waves conceptual survey: Its modification and conversion to a standard multiple-choice test, Phys. Rev. Phys. Educ. Res. 12, 010107 (2016).
  8. P. Klein, A. Muller, and J. Kuhn, Assessment of representational competence in kinematics, Phys. Rev. Phys. Educ. Res. 13, 010132 (2017).
  9. Y. Xiao, J. Han, K. Koenig, J. Xiong, and L. Bao, Multilevel Rasch modeling of two-tier multiple choice test: A case study using Lawson’s classroom test of scientific reasoning, Phys. Rev. Phys. Educ. Res. 14, 020104 (2018).
  10. L. Bao, Y. Xiao, K. Koenig, and J. Han, Validity evaluation of the Lawson classroom test of scientific reasoning, Phys. Rev. Phys. Educ. Res. 14, 020106 (2018).
  11. U. ScottidiUccio, A. Colantonio, S. Galano, I. Marzoli, F. Trani, and I. Testa, Design and validation of a two-tier questionnaire on basic aspects in quantum mechanics, Phys. Rev. Phys. Educ. Res. 15, 010137 (2019).
  12. J.-P. Burde and T. Wilhelm, Teaching electric circuits with a focus on potential differences, Phys. Rev. Phys. Educ. Res. 16, 020153 (2020).
  13. H.-S. Wang, S. Chen, and M.-H. Yen, Effects of metacognitive scaffolding on students’ performance and confidence judgments in simulation-based inquiry, Phys. Rev. Phys. Educ. Res. 17, 020108 (2021).
  14. L. Ivanjek, L. Morris, T. Schubatzky, M. Hopf, J.-P. Burde, C. Haagen-Schutzenhofer, L. Dopatka, V. Spatz, and T. Wilhelm, Development of a two-tier instrument on simple electric circuits, Phys. Rev. Phys. Educ. Res. 17, 020123 (2021).
  15. H. Şahin Kızılcık, M. Aygün, E. Şahin, N. Önder Çelikkanlı, O. Türk, T. Taşkın, and B. Güneş, Possible misconceptions about solid friction, Phys. Rev. Phys. Educ. Res. 17, 023107 (2021).
  16. G. Resbiantoro, R. Setiani, and Dwikoranto, A review of misconception in physics: The diagnosis, causes, and remediation: Research article, J. Turk. Sci. Educ. 19, 403 (2022), https://www.tused.org/index.php/tused/article/view/924.
  17. M. M. Hull, A. Jansky, and M. Hopf, Does confidence in a wrong answer imply a misconception?, Phys. Rev. Phys. Educ. Res. 18, 020108 (2022).
  18. S. Mi, J. Ye, L. Yan, and H. Bi, Development and validation of a conceptual survey instrument to evaluate senior high school students’ understanding of electrostatics, Phys. Rev. Phys. Educ. Res. 19, 010114 (2023).
  19. J. ichiro Yasuda and M. aki Taniguchi, Validating two questions in the Force Concept Inventory with subquestions, Phys. Rev. ST Phys. Educ. Res. 9, 010113 (2013).
  20. J. ichiro Yasuda, N. Mae, M. M. Hull, and M. aki Taniguchi, Analyzing false positives of four questions in the Force Concept Inventory, Phys. Rev. Phys. Educ. Res. 14, 010112 (2018).
  21. M. C. Wittmann and K. E. Black, Visualizing changes in pretest and post-test student responses with consistency plots, Phys. Rev. ST Phys. Educ. Res. 10, 010114 (2014).
  22. R. Tapping, G. P. Lepage, and N. G. Holmes, Visualizing patterns in CSEM responses to assess student conceptual understanding, presented at PER Conf. 2018, Washington, DC, 10.1119/PERC.2018.PR.TAPPING.
  23. M. Rosvall and C. T. Bergstrom, Mapping change in large networks, PLoS One 5, e8694 (2010).
  24. J. C. Brunson, ggalluvial: Layered grammar for alluvial plots, J. Open Source Softw. 5, 2017 (2020).
  25. A. L. Traxler, T. Suda, E. Brewe, and K. Commeford, Network positions in active learning environments in physics, Phys. Rev. Phys. Educ. Res. 16, 020129 (2020).
  26. A. Werth, J. R. Hoehn, K. Oliver, M. F. Fox, and H. Lewandowski, Instructor perspectives on the emergency transition to remote instruction of physics labs, Phys. Rev. Phys. Educ. Res. 18, 020129 (2022).
  27. M. B. Atkinson and S. L. Bretz, Measuring changes in undergraduate chemistry students’ reasoning with reaction coordinate diagrams: A longitudinal, multi-institution study, J. Chem. Educ. 98, 1064 (2021).
  28. J. H. Doherty, J. A. Cerchiara, and M. P. Wenderoth, Undergraduate students’ neurophysiological reasoning: What we learn from the attractive distractors students select, Adv. Physiol. Educ. 47, 222 (2023).
  29. J. ichiro Yasuda, H. Uematsu, and H. Nitta, Validating a Japanese version of Force Concept Inventory (in Japanese), J. Phys. Educ. Soc. Japan 59, 90 (2011).
  30. Revised Table I [Table I. Newtonian concepts in the revised Force Concept Inventory (form 081695r)], https://www.modelinginstruction.org/effective/evaluation-instruments/.
  31. Revised Table II [Table II for the Force Concept Inventory (revised form 081695r], https://www.modelinginstruction.org/effective/evaluation-instruments/.
  32. M. M. Hull, J. ichiro Yasuda, and N. Mae (to be published).
  33. M. J. Kolen and R. L. Brennan, Test Equating, Scaling, and Linking (Springer, New York, 2014), p. 2.
  34. S. L. Wise, Rapid-guessing behavior: Its identification, interpretation, and implications, Educ. Meas. 36, 52 (2017).
  35. E. F. Redish, Teaching Physics with the Physics Suite (John Wiley & Sons, New York, 2003), p. 26.
  36. L. Bao, K. Hogg, and D. Zollman, Model analysis of fine structures of student models: An example with Newton’s third law, Am. J. Phys. 70, 766 (2002).
  37. T. I. Smith and M. C. Wittmann, Comparing three methods for teaching Newton’s third law, Phys. Rev. ST Phys. Educ. Res. 3, 020105 (2007).
  38. C. Chen, L. Bao, J. C. Fritchman, and H. Ma, Causal reasoning in understanding Newton’s third law, Phys. Rev. Phys. Educ. Res. 17, 010128 (2021).
  39. L. Bao and J. C. Fritchman, Knowledge integration in student learning of Newton’s third law: Addressing the action-reaction language and the implied causality, Phys. Rev. Phys. Educ. Res. 17, 020116 (2021).
  40. K. F. Wilson and D. J. Low, on second thoughts…: Changes of mind as an indication of competing knowledge structures, Am. J. Phys. 83, 802 (2015).
  41. D. J. Low and K. F. Wilson, The role of competing knowledge structures in undermining learning: Newton’s second and third laws, Am. J. Phys. 85, 54 (2017).
  42. H. Dedic, S. Rosenfield, and N. Lasry, Are all wrong FCI answers equivalent?, AIP Conf. Proc. 1289, 125 (2010).
  43. D. E. Brown and J. Clement, Misconceptions concerning Newton’s law of action and reaction: The underestimated importance of the third law, in Proceedings of the Second International Seminar: A Misconceptions and Educational Strategies in Science and Mechanics, edited by J. D. Novak (Cornell University, Ithaca, 1987), Vol. 3, p. 39, https://www.google.co.jp/books/edition/Proceedings_of_the_Second_International/rWJQAAAAYAAJ.
  44. E. F. Redish, J. M. Saul, and R. N. Steinberg, On the effectiveness of active-engagement microcomputer-based laboratories, Am. J. Phys. 65, 45 (1998).
  45. D. Hammer, Misconceptions or p-prims: How may alternative perspectives of cognitive structure influence instructional perceptions and intentions, J. Learn. Sci. 5, 97 (1996).
  46. P. Nieminen, A. Savinainen, and J. Viiri, Force Concept Inventory-based multiple-choice test for investigating students’ representational consistency, Phys. Rev. ST Phys. Educ. Res. 6, 020109 (2010).
  47. S. Bogart, Sankeymatic, http://sankeymatic.com.
  48. J. ichiro Yasuda, Sample code for alluvial diagrams, https://github.com/Jun-ichiro/sample-alluvial.
  49. J. ichiro Yasuda, N. Mae, M. M. Hull, and M. aki Taniguchi, Optimizing the length of computerized adaptive testing for the Force Concept Inventory, Phys. Rev. Phys. Educ. Res. 17, 010115 (2021).
  50. J. ichiro Yasuda, M. M. Hull, and N. Mae, Improving test security and efficiency of computerized adaptive testing for the Force Concept Inventory, Phys. Rev. Phys. Educ. Res. 18, 010112 (2022).

Outline

Information

Phys. Rev. Phys. Educ. Res. 19, 020121– Published 31 August, 2023
  • Received 20 April 2023
  • Accepted 20 July 2023
Crossmark - Check for updates button
Creative Commons Logo

Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article’s title, journal citation, and DOI.

Published by the American Physical Society

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation