- Physical Review Journals
All JournalsPhysics Magazine

Physical Review Physics Education Research

Reuse & Permissions

It is not necessary to obtain permission to reuse this article or its components as it is available under the terms of the Creative Commons Attribution 4.0 International license. This license permits unrestricted use, distribution, and reproduction in any medium, provided attribution to the author(s) and the published article's title, journal citation, and DOI are maintained. Please note that some figures may have been included with permission from other third parties. It is your responsibility to obtain the proper permission from the rights holder directly for these figures.

Export citation

Export citation

Choose format for download:

Download Citation
  • Open Access

How to evaluate students’ decisions in a data comparison problem: Correct decision for the wrong reasons?

Karel Kok*, Sophia Chroszczinsky, and Burkhard Priemer

  • Physics Education, Department of Physics, Humboldt-Universität zu Berlin, Newtonstraße 15, Berlin 12489, Germany

  • *karel.kok@physik.hu-berlin.de
PDF

Phys. Rev. Phys. Educ. Res. 20, 010129 – Published 26 April, 2024

DOI: https://doi.org/10.1103/PhysRevPhysEducRes.20.010129

Abstract

Data comparison problems are used in teaching and science education research that focuses on students’ ability to compare datasets and their conceptual understanding of measurement uncertainties. However, the evaluation of students’ decisions in these problems can pose a problem: e.g., students making a correct decision for the wrong reasons. Three previous studies, that share the same context and data comparison problem but where participants had increasing conceptual knowledge of measurement uncertainties, are revisited. The comparison shows a troublesome result: increasing conceptual knowledge does not lead to better decision making in the data comparison problem. In this research, we have looked into this apparent discrepancy by comparing and reanalyzing the data from these three studies. We have analyzed students’ justifications by coding them based on the compared quantity and the deciding criterion, giving a highly detailed insight into what they do when comparing the datasets. The results show clear differences in the quality of the justifications across the studies and by combining the results with the decisions, we could successfully identify four cases of correct and incorrect decisions for right or wrong reasons. This analysis showed a high prevalence of correct decisions for wrong reasons in two of the studies, resolving the discrepancy in the initial comparison of these studies. The implication of our analysis is that simply asking students to make a decision in data comparison problems is not a suitable probe to gauge their ability to compare datasets or their conceptual understanding of measurement uncertainties and a probe like this should always be complemented by an analysis of the justification.

View figure in article

Physics Subject Headings (PhySH)

Article Text

References (37)

  1. J. Kozminski, H. Lewandowski, N. Beverly, S. Lindaas, D. L. Deardorff, A. Reagan, R. Dietz, R. Tagg, M. Eblen-Zayas, J. Williams, R. Hobbs, and B. M. Zwickl, AAPT Recommendations for the Undergraduate Physics Laboratory Curriculum, Tech. Rep. (American Association of Physics Teachers, College Park, MD, 2014).
  2. B. Pollard, R. Hobbs, R. Henderson, M. D. Caballero, and H. J. Lewandowski, Introductory physics lab instructors’ perspectives on measurement uncertainty, Phys. Rev. Phys. Educ. Res. 17, 010133 (2021).
  3. D. Hu and B. M. Zwickl, Examining students’ views about validity of experiments: From introductory to Ph.D. students, Phys. Rev. Phys. Educ. Res. 14, 010121 (2018).
  4. M. M. Hull, A. Jansky, and M. Hopf, Probability-related naïve ideas across physics topics, Stud. Sci. Educ. 57, 45 (2020).
  5. A. Buffler, S. Allie, and F. Lubben, The development of first year physics students’ ideas about measurement in terms of point and set paradigms, Int. J. Sci. Educ. 23, 1137 (2001).
  6. K. Kok, Certain about Uncertainty—What students need to know about measurement uncertainties to compare data sets, doctoral thesis, Humboldt-Universität, Berlin, 2022, https://doi.org/10.18452/24782.
  7. S. Allie, A. Buffler, B. Campbell, and F. Lubben, First-year physics students’ perceptions of the quality of experimental measurements, Int. J. Sci. Educ. 20, 447 (1998).
  8. M.-G. Séré, R. Journeaux, and C. Larcher, Learning the statistical analysis of measurement errors, Int. J. Sci. Educ. 15, 427 (1993).
  9. A. M. Masnick and B. J. Morris, Investigating the development of data evaluation: The role of data characteristics, Child Dev. 79, 1032 (2008).
  10. S. Allie, A. Buffler, F. Lubben, and B. Campbell, Point and set paradigms in students’ handling of experimental measurements, in Research in Science Education—Past, Present, and Future, edited by H. Behrendt, H. Dahncke, R. Duit, W. Gräber, M. Komorek, A. Kross, and P. Reiska (Kluwer Academic Publishers, Dordrecht, 2002), pp. 331–336, https://doi.org/10.1007/0-306-47639-8_47.
  11. S. M. Coelho and M.-G. Séré, Pupils’ reasoning and practice during hands-on activities in the measurement phase, Res. Sci. Technol. Educ. 16, 79 (1998).
  12. P. Warwick, R. S. Linfield, and P. Stephenson, A comparison of primary school pupils’ ability to express procedural understanding in science through speech and writing, Int. J. Sci. Educ. 21, 823 (1999).
  13. F. Lubben and R. Millar, Children’s ideas about the reliability of experimental data, Int. J. Sci. Educ. 18, 955 (1996).
  14. S. Pillay, A. Buffler, F. Lubben, and S. Allie, Effectiveness of a GUM-compliant course for teaching measurement in the introductory physics laboratory, Eur. J. Phys. 29, 647 (2008).
  15. T. S. Volkwyn, S. Allie, A. Buffler, and F. Lubben, Impact of a conventional introductory laboratory course on the understanding of measurement, Phys. Rev. ST Phys. Educ. Res. 4, 010108 (2008).
  16. R. L. Kung, Teaching the concepts of measurement: An example of a concept-based laboratory course, Am. J Phys. 73, 771 (2005).
  17. B. Pollard, A. Werth, R. Hobbs, and H. J. Lewandowski, Impact of a course transformation on students’ reasoning about measurement uncertainty, Phys. Rev. Phys. Educ. Res. 16, 020160 (2020).
  18. J. Day and D. Bonn, Development of the concise data processing assessment, Phys. Rev. ST Phys. Educ. Res. 7, 010114 (2011).
  19. C. Walsh, K. N. Quinn, C. Wieman, and N. G. Holmes, Quantifying critical thinking: Development and validation of the physics lab inventory of critical thinking, Phys. Rev. Phys. Educ. Res. 15, 010135 (2019).
  20. M. Vignal, G. Geschwind, B. Pollard, R. Henderson, M. D. Caballero, and H. J. Lewandowski, Survey of physics reasoning on uncertainty concepts in experiments: An assessment of measurement uncertainty for introductory physics labs, Phys. Rev. Phys. Educ. Res. 19, 020139 (2023).
  21. J. Schulz, Entwicklung eines testinstrumentes zur erfassung von kompetenzen im umgang mit messunsicherheiten, doctoral thesis, Humboldt-Universität, Berlin, 2022, http://dx.doi.org/10.18452/23957.
  22. K. Kok, B. Priemer, W. Musold, and A. Masnick, Students’ conclusions from measurement data: The more decimal places, the better?, Phys. Rev. Phys. Educ. Res. 15, 010103 (2019).
  23. T. Wan, Investigating student reasoning about measurement uncertainty and ability to draw conclusions from measurement data in inquiry-based university physics labs, Int. J. Sci. Educ. 45, 223 (2023).
  24. P. van Kampen and O. Gkioka, Undergraduate students’ reasoning about the quality of experimental measurements of covarying secondary data, Eur. J. Phys. 42, 045704 (2021).
  25. E. Cauzinille-Marmeche, M. Meheut, M.-G. Séré, and A. Weil-Barais, The influence of a priori ideas on the experimental approach, Sci. Educ. 69, 201 (1985).
  26. V. Munier, H. Merle, and D. Brehelin, Teaching Scientific Measurement and Uncertainty in Elementary School, Int. J. Sci. Educ. 35, 2752 (2013).
  27. T. Ludwig, Argumentieren beim experimentieren in der physik—die bedeutung personaler und situationaler faktoren, doctoral thesis, Humboldt-Universität, Berlin, 2017, https://doi.org/10.18452/18408.
  28. R. L. Kung and C. Linder, University students’ ideas about data processing and data comparison in a physics laboratory course, Nordic Stud. Sci. Educ. 2, 40 (2006).
  29. B. Priemer and J. Hellwig, Learning about measurement uncertainties in secondary education: A model of the subject matter, Int. J. Sci. Math. Educ. 16, 45 (2018).
  30. G. Geschwind, M. Vignal, and H. J. Lewandowski, Representational differences in how students compare measurements, presented at PER Conf. 2023, Sacramento, CA, 10.1119/perc.2023.pr.Geschwind.
  31. E. Kardaş, Der einfluss von datenkompetenz auf das argumentieren beim experimentieren: Entwicklung und evaluation von lernapps zur förderung von datenkompetenz, doctoral thesis, PH Karlsruhe, Karslruhe, 2023.
  32. M.-G. Séré, M. Fernandez-Gonzalez, J. A. Gallegos, F. Gonzalez-Garcia, E. De Manuel, F. J. Perales, and J. Leach, Images of science linked to labwork: A survey of secondary school and university students, Res. Sci. Educ. 31, 499 (2001).
  33. K. Kok and B. Priemer, Assessment tool to understand how students justify their decisions in data comparison problems, Phys. Rev. Phys. Educ. Res. 19, 020141 (2023).
  34. S. Chroszczinsky, doctoral thesis, Humboldt-Universität, Berlin (to be published).
  35. E. Kardaş and T. Ludwig, Den umgang mit daten und messunsicherheiten lernen—digitale apps für ein wichtiges thema, Plus Lucis, 24 (2021), https://www.pluslucis.org/ZeitschriftenArchiv/2021-4_PL.pdf.
  36. J. R. Landis and G. G. Koch, The measurement of observer agreement for categorical data, Biometrics 33, 159 (1977).
  37. W. K. Adams and C. E. Wieman, Development and validation of instruments to measure learning of expertlike thinking, Int. J. Sci. Educ. 33, 1289 (2011).

Outline

Information

Phys. Rev. Phys. Educ. Res. 20, 010129– Published 26 April, 2024
  • Received 10 November 2023
  • Accepted 25 March 2024
Crossmark - Check for updates button
Creative Commons Logo

Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article’s title, journal citation, and DOI.

Published by the American Physical Society

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation