- Physical Review Journals
All JournalsPhysics Magazine

Physical Review Physics Education Research

Reuse & Permissions

It is not necessary to obtain permission to reuse this article or its components as it is available under the terms of the Creative Commons Attribution 4.0 International license. This license permits unrestricted use, distribution, and reproduction in any medium, provided attribution to the author(s) and the published article's title, journal citation, and DOI are maintained. Please note that some figures may have been included with permission from other third parties. It is your responsibility to obtain the proper permission from the rights holder directly for these figures.

Export citation

Export citation

Choose format for download:

Download Citation
  • Open Access

Assessment tool to understand how students justify their decisions in data comparison problems

Karel Kok* and Burkhard Priemer

  • Physics Education, Department of Physics, Humboldt-Universität zu Berlin, Newtonstraße 15, Berlin 12489, Germany

  • *karel.kok@physik.hu-berlin.de
PDF

Phys. Rev. Phys. Educ. Res. 19, 020141 – Published 9 October, 2023

DOI: https://doi.org/10.1103/PhysRevPhysEducRes.19.020141

Abstract

Students at all levels of education experience difficulties with the concepts of measurement uncertainties. One task that includes concepts of measurement uncertainties is a data comparison problem where students decide whether two datasets are in agreement or not—an authentic scientific practice. To aid students with these concepts and tasks, sensitive instruments are needed that probe students’ understanding and track their progress. We present a tool in the form of a coding manual that analyzes students’ written justifications for such a data comparison problem. With this coding manual, justifications are coded based on the quantity that is compared and the deciding criterion in this comparison. Our coding manual was used in the evaluation of a digital learning environment (DLE). In this evaluation, 154 participants (aged 14–17 years) wrote a justification for a data comparison problem before and after going through the DLE. These participants were randomly assigned to one of three groups where each group was taught an increased amount of concepts regarding measurement uncertainties. By analyzing the participants’ justifications with our coding manual, we could probe their ability to compare datasets on a fine-grained level. The sensitivity of our tool was illustrated by the increase in higher quality codes with the increased conceptual teaching between the three groups. We argue that our tool for coding justifications can be applied to any data comparison problem giving detailed information about what data analysis methods students use and how they reach their conclusion and thus contributes to the research on students’ understanding of measurement uncertainties.

View figure in article

Physics Subject Headings (PhySH)

Article Text

References (59)

  1. D. Boyd and K. Crawford, Critical questions for big data, Inf. Commun. Soc. 15, 662 (2012).
  2. S. Duggan and R. Gott, What sort of science education do we really need? Int. J. Sci. Educ. 24, 661 (2002).
  3. M. K. Kjelvik and E. H. Schultheis, Getting messy with authentic data: Exploring the potential of using data from scientific research to support student data literacy, CBE—Life Sci. Educ. 18, es2 (2019).
  4. J. Day and D. Bonn, Development of the concise data processing assessment, Phys. Rev. ST Phys. Educ. Res. 7, 010114 (2011).
  5. R. Millar, F. Lubben, R. Got, and S. Duggan, Investigating in the school science laboratory: Conceptual and procedural knowledge and their influence on performance, Res. Pap. Educ. 9, 207 (1994).
  6. N. G. Holmes, C. E. Wieman, and D. A. Bonn, Teaching critical thinking, Proc. Natl. Acad. Sci. U.S.A. 112, 11199 (2015).
  7. Z. Kanari and R. Millar, Reasoning from data: How students collect and interpret data in science investigations, J. Res. Sci. Teach. 41, 748 (2004).
  8. C. A. Chinn and B. A. Malhotra, Epistemologically authentic inquiry in schools: A theoretical framework for evaluating inquiry tasks, Sci. Educ. 86, 175 (2002).
  9. Beyond 2000: Science Education for the Future, edited by R. Millar and J. Osborne (King’s College London, London, 1998).
  10. S. V. Sharma, High school students interpreting tables and graphs: Implications for research, Int. J. Sci. Math. Educ. 4, 241 (2006).
  11. K. Kok, B. Priemer, W. Musold, and A. Masnick, Students’ conclusions from measurement data: The more decimal places, the better? Phys. Rev. Phys. Educ. Res. 15, 010103 (2019).
  12. A. M. Masnick and B. J. Morris, Investigating the development of data evaluation: The role of data characteristics, Child Dev. 79, 1032 (2008).
  13. S. Allie, A. Buffler, B. Campbell, and F. Lubben, First-year physics students’ perceptions of the quality of experimental measurements, Int. J. Sci. Educ. 20, 447 (1998).
  14. J. Schulz, Entwicklung eines Testinstrumentes zur Erfassung von Kompetenzen im Umgang mit Messunsicherheiten, Doctoral thesis, Humboldt-Universität, Berlin, 2022, http://dx.doi.org/10.18452/23957.
  15. A. Buffler, S. Allie, and F. Lubben, The development of first year physics students’ ideas about measurement in terms of point and set paradigms, Int. J. Sci. Educ. 23, 1137 (2001).
  16. M. M. Hull, A. Jansky, and M. Hopf, Probability-related naïve ideas across physics topics, Stud. Sci. Educ. 57, 45 (2020).
  17. B. Priemer and J. Hellwig, Learning about measurement uncertainties in secondary education: A model of the subject matter, Int. J. Sci. Math. Educ. 16, 45 (2018).
  18. M. J. Goedhart and A. H. Verdonk, The development of statistical concepts in a design-oriented laboratory course in scientific measuring, J. Chem. Educ. 68, 1005 (1991).
  19. S. Heinicke, Aus Fehlern Wird Man Klug: Eine Genetisch-Didaktische Rekonstruktion des Messfehlers (Logos Verlag Berlin GmbH, Berlin, 2012).
  20. K. Kampourakis and K. McCain, Uncertainty: How It Makes Science Advance (Oxford University Press, New York, NY, 2019).
  21. L. Kirkup, A guide to GUM, Eur. J. Phys. 23, 483 (2002).
  22. S. Pillay, A. Buffler, F. Lubben, and S. Allie, Effectiveness of a GUM-compliant course for teaching measurement in the introductory physics laboratory, Eur. J. Phys. 29, 647 (2008).
  23. M. Rollnick, B. Dlamini, S. Lotz, and F. Lubben, Views of South African chemistry students in University Bridging Programs on the reliability of experimental data, Res. Sci. Educ. 31, 553 (2001).
  24. F. Lubben and R. Millar, Children’s ideas about the reliability of experimental data, Int. J. Sci. Educ. 18, 955 (1996).
  25. P. Warwick, R. S. Linfield, and P. Stephenson, A comparison of primary school pupils’ ability to express procedural understanding in science through speech and writing, Int. J. Sci. Educ. 21, 823 (1999).
  26. M.-G. Séré, R. Journeaux, and C. Larcher, Learning the statistical analysis of measurement errors, Int. J. Sci. Educ. 15, 427 (1993).
  27. S. Allie, A. Buffler, F. Lubben, and B. Campbell, Point and set paradigms in students’ handling of experimental measurements, in Research in Science Education—Past, Present, and Future, edited by H. Behrendt, H. Dahncke, R. Duit, W. Gräber, M. Komorek, A. Kross, and P. Reiska (Kluwer Academic Publishers, Dordrecht, 2002), pp. 331–336.
  28. J. Leach, R. Millar, J. Ryder, M.-G. Séré, D. Hammelev, H. Niedderer, and V. Tselfes, Survey 2: Students’ images of science as they relate to labwork learning. Working Paper 4, Labwork in Science Education Project, Centre for Studies in Science and Mathematics Education, Leeds, Technical Report No. Project PL 95-2005, 1998.
  29. P. van Kampen and O. Gkioka, Undergraduate students’ reasoning about the quality of experimental measurements of covarying secondary data, Eur. J. Phys. 42, 045704 (2021).
  30. M. J. Ford, The game, the pieces, and the players: Generative resources from two instructional portrayals of experimentation, J. Learn. Sci. 14, 449 (2005).
  31. N. Majiet and S. Allie, Student understanding of measurement and uncertainty: Probing the mean, presented at PER Conf. 2018, Washington, DC, http://doi.org/10.1119/perc.2018.pr.Majiet.
  32. J. Mokros and S. J. Russell, Children’s concepts of average and representativeness, J. Res. Math. Educ. 26, 20 (1995).
  33. C. Konold and A. Pollatsek, Data analysis as the search for signals in noisy processes, J. Res. Math. Educ. 33, 259 (2002).
  34. S. M. Coelho and M.-G. Séré, Pupils’ reasoning and practice during hands-on activities in the measurement phase, Res. Sci. Technol. Educ. 16, 79 (1998).
  35. R. Fairbrother and M. Hackling, Is this the right answer? Int. J. Sci. Educ. 19, 887 (1997).
  36. F. Lubben, B. Campbell, A. Buffler, and S. Allie, Point and set reasoning in practical science measurement by entering university freshmen, Sci. Educ. 85, 311 (2001).
  37. E. M. Smith, M. M. Stein, and N. G. Holmes, How expectations of confirmation influence students’ experimentation decisions in introductory labs, Phys. Rev. Phys. Educ. Res. 16, 010113 (2020).
  38. E. Cauzinille-Marmeche, M. Meheut, M.-G. Séré, and A. Weil-Barais, The influence of a priori ideas on the experimental approach, Sci. Educ. 69, 201 (1985).
  39. T. Wan, Investigating student reasoning about measurement uncertainty and ability to draw conclusions from measurement data in inquiry-based university physics labs, Int. J. Sci. Educ. 45, 223 (2023).
  40. D. Hu and B. M. Zwickl, Examining students’ views about validity of experiments: From introductory to Ph.D. students, Phys. Rev. Phys. Educ. Res. 14, 010121 (2018).
  41. B. Pollard, R. Hobbs, R. Henderson, M. D. Caballero, and H. J. Lewandowski, Introductory physics lab instructors’ perspectives on measurement uncertainty, Phys. Rev. Phys. Educ. Res. 17, 010133 (2021).
  42. T. S. Volkwyn, S. Allie, A. Buffler, and F. Lubben, Impact of a conventional introductory laboratory course on the understanding of measurement, Phys. Rev. ST Phys. Educ. Res. 4, 010108 (2008).
  43. R. L. Kung, Teaching the concepts of measurement: An example of a concept-based laboratory course, Am. J. Phys. 73, 771 (2005).
  44. V. Munier, H. Merle, and D. Brehelin, Teaching scientific measurement and uncertainty in elementary school, Int. J. Sci. Educ. 35, 2752 (2013).
  45. R. L. Kung and C. Linder, University students’ ideas about data processing and data comparison in a physics laboratory course, Nordic Stud. Sci. Educ. 2, 40 (2006).
  46. B. Pollard, A. Werth, R. Hobbs, and H. J. Lewandowski, Impact of a course transformation on students’ reasoning about measurement uncertainty, Phys. Rev. Phys. Educ. Res. 16, 020160 (2020).
  47. K. Kok and B. Priemer, Comparing different uncertainty measures to quantify measurement uncertainties in high school science experiments, Int. J. Phys. Chem. Educ. 14, 1 (2022).
  48. C. A. Chinn and W. F. Brewer, The role of anomalous data in knowledge acquisition: A theoretical framework and implications for science instruction, Rev. Educ. Res. 63, 1 (1993).
  49. C. A. Chinn and W. F. Brewer, An empirical test of a taxonomy of responses to anomalous data in science, J. Res. Sci. Teach. 35, 623 (1998).
  50. M.-G. Séré, M. Fernandez-Gonzalez, J. A. Gallegos, F. Gonzalez-Garcia, E. De Manuel, F. J. Perales, and J. Leach, Images of science linked to labwork: A survey of secondary school and university students, Res. Sci. Educ. 31, 499 (2001).
  51. A. Estepa, C. Batanero, and F. Sánchez, Students’ intuitive strategies in judging association when comparing two samples, Hiroshima J. Math. Educ. 7, 17 (1999), https://eric.ed.gov/?id=EJ586616.
  52. T. Ludwig, B. Priemer, and D. Lewalter, Assessing secondary school students’ justifications for supporting or rejecting a scientific hypothesis in the physics lab, Res. Sci. Educ. 51, 819 (2021).
  53. K. Kok, Certain about uncertainty—what students need to know about measurement uncertainties to compare data sets, Doctoral thesis, Humboldt-Universität, Berlin, 2022, https://doi.org/10.18452/24782.
  54. K. Kok, Digital learning environment, https://lernen.physik.hu-berlin.de/measurementuncertainties (2020).
  55. K. Kok, Digitale Lerneinheit, https://lernen.physik.hu-berlin.de/Messunsicherheiten (2020).
  56. A. Buffler, S. Allie, and F. Lubben, Teaching measurement and uncertainty the GUM way, Phys. Teach. 46, 539 (2008).
  57. P. Mayring, Qualitative Content Analysis: Theoretical Foundation, Basic Procedures and Software Solution (Social Science Open Access Repository, Klagenfurt, 2014).
  58. J. R. Landis and G. G. Koch, The measurement of observer agreement for categorical data, Biometrics 33, 159 (1977).
  59. S. Chroszczinsky, B. Priemer, and A. Masnick, The influence of information added to data, in Bridging the Gap: Empowering and Educating Today’s Learners in Statistics. Proceedings of the Eleventh International Conference on Teaching Statistics (International Association for Statistical Education, 2022), https://doi.org/10.52041/iase.icots11.T8E3.

Outline

Information

Phys. Rev. Phys. Educ. Res. 19, 020141– Published 9 October, 2023
  • Received 19 April 2023
  • Accepted 5 September 2023
Crossmark - Check for updates button
Creative Commons Logo

Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article’s title, journal citation, and DOI.

Published by the American Physical Society

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation