- Physical Review Journals
All JournalsPhysics Magazine

Physical Review Physics Education Research

Reuse & Permissions

It is not necessary to obtain permission to reuse this article or its components as it is available under the terms of the Creative Commons Attribution 4.0 International license. This license permits unrestricted use, distribution, and reproduction in any medium, provided attribution to the author(s) and the published article's title, journal citation, and DOI are maintained. Please note that some figures may have been included with permission from other third parties. It is your responsibility to obtain the proper permission from the rights holder directly for these figures.

Export citation

Export citation

Choose format for download:

Download Citation
  • Editors' Suggestion
  • Open Access

Cheat sites and artificial intelligence usage in online introductory physics courses: What is the extent and what effect does it have on assessments?

Gerd Kortemeyer*

Wolfgang Bauer†

  • Rectorate and AI Center, ETH Zurich, 8092 Zurich, Switzerland

  • *kgerd@ethz.ch Also at Michigan State University, East Lansing, MI 48823, USA.
  • †bauerw@msu.edu
PDF

Phys. Rev. Phys. Educ. Res. 20, 010145 – Published 23 May, 2024

DOI: https://doi.org/10.1103/PhysRevPhysEducRes.20.010145

Abstract

As a result of the pandemic, many physics courses moved online. Alongside, the popularity of Internet-based problem-solving sites and forums rose. With the emergence of large language models, another shift occurred. One year into the public availability of these models, how has online help-seeking behavior among introductory physics students changed, and what is the effect of different patterns of online resource usage? In a mixed-method approach, we investigate student choices and their impact on assessment components of an online introductory physics course for scientists and engineers. We find that students still mostly rely on traditional Internet resources and that their usage strongly influences the outcome of low-stake unsupervised quizzes. We empirically found distinct clusters of help-seeking and resource-usage patterns among the students; the impact of students’ cluster membership on the supervised assessment components of the course, however, is nonsignificant.

View figure in article

Physics Subject Headings (PhySH)

Article Text

References (52)

  1. E. Kim and S.-J. Pak, Students do not overcome conceptual difficulties after solving 1000 traditional problems, Am. J. Phys. 70, 759 (2002).
  2. R. Feynman, R. Leighton, and M. Sands, The Feynman Lectures on Physics (Addison Wesley, Reading, MA, 1965).
  3. G. Kortemeyer and W. Bauer, Multimedia collaborative content creation (mc3): The MSU lecture online system, J. Eng. Educ. 88, 421 (1999).
  4. W. Bauer and G. D. Westfall, University Physics with Modern Physics (ebook) (McGraw-Hill Higher Education, New York, NY, 2023).
  5. G. Kortemeyer, Over two decades of blended and online physics courses at Michigan State University, eleed 10 (2014), https://www.eleed.de/archive/10/4011.
  6. T. L. Russell, The “No Significant Difference” Phenomenon as Reported in 248 Research Reports, Summaries, and Papers (North Carolina State University, 1997).
  7. J. Cavanaugh and S. J. Jacquemin, A large sample comparison of grade based student learning outcomes in online vs. face-to-face courses, Online Learn. 19, 1 (2015).
  8. E. Bergeler and M. F. Read, Comparing learning outcomes and satisfaction of an online algebra-based physics course with a face-to-face course, J. Sci. Educ. Technol. 30, 97 (2021).
  9. G. Kortemeyer, W. Bauer, and W. Fisher, Hybrid teaching: A tale of two populations, Phys. Rev. Phys. Educ. Res. 18, 020130 (2022).
  10. G. Kortemeyer, C. Kortemeyer, and W. Bauer, Taking introductory physics in studio, lecture, or online format: What difference does it make in subsequent courses, and for whom?, Phys. Rev. Phys. Educ. Res. 19, 020148 (2023).
  11. H. Ibrahim, F. Liu, R. Asim, B. Battu, S. Benabderrahmane, B. Alhafni, W. Adnan, T. Alhanai, B. AlShebli, R. Baghdadi et al., Perception, performance, and detectability of conversational artificial intelligence across 32 university courses, Sci. Rep. 13, 12187 (2023).
  12. F. R. A. Ahmed, T. E. Ahmed, R. A. Saeed, H. Alhumyani, S. Abdel-Khalek, and H. Abu-Zinadah, Analysis and challenges of robust e-exams performance under COVID-19, Res. Phys. 23, 103987 (2021).
  13. M. Farrokhnia, S. K. Banihashem, O. Noroozi, and A. Wals, A swot analysis of ChatGPT: Implications for educational practice and research, Innov. Educ. Teach. Int. 61, 460 (2023).
  14. J. A. Oravec, Artificial intelligence implications for academic cheating: Expanding the dimensions of responsible human-ai collaboration with ChatGPT, J. Interact. Learn. Res. 34, 213 (2023), https://www.learntechlib.org/primary/j/JILR/v/34/n/2/.
  15. C. Ruggieri, Students’ use and perception of textbooks and online resources in introductory physics, Phys. Rev. Phys. Educ. Res. 16, 020123 (2020).
  16. Chegg, Chegg, https://www.chegg.com [accessed December 2023].
  17. T. Lancaster and C. Cotarlan, Contract cheating by stem students through a file sharing website: A COVID-19 pandemic perspective, Int. J. Educ. Integr. 17, 3 (2021).
  18. G. Kortemeyer, E. Kashy, W. Benenson, and W. Bauer, Experiences using the open-source learning content management and assessment system LON-CAPA in introductory physics courses, Am. J. Phys. 76, 438 (2008).
  19. H. Busch, One method for inhibiting the copying of online homework, Phys. Teach. 55, 422 (2017).
  20. OpenAI, ChatGPT, https://chat.openai.com/ [accessed December 2023].
  21. Google, Google Bard, https://bard.google.com/ [accessed December 2023].
  22. P. Prakash, Chegg’s shares tumbled nearly 50% after the edtech company said its customers are using ChatGPT instead of paying for its study tools, Fortune (2023), https://fortune.com/2023/05/02/chegg-shares-tumble-students-fleeing-chatgpt-a-i/.
  23. D. Paresh, Chegg embraced ai. ChatGPT ate its lunch anyway, Wired (2023), https://www.wired.com/story/chegg-embraced-ai-chatgpt-ate-its-lunch-anyway/.
  24. G. Kortemeyer, Could an artificial-intelligence agent pass an introductory physics course?, Phys. Rev. Phys. Educ. Res. 19, 010132 (2023).
  25. W. Yeadon and T. Hardy, The impact of AI in physics education: A comprehensive review from GCSE to university levels, Phys. Educ. 59, 025010 (2024).
  26. G. Polverini and B. Gregorcic, Performance of ChatGPT on the test of understanding graphs in kinematics, Phys. Rev. Phys. Educ. Res. 20, 010109 (2024).
  27. T. Kumar and M. A. Kats, ChatGPT-4 with code interpreter can be used to solve introductory college-level vector calculus and electromagnetism problems, Am. J. Phys. 91, 955 (2023).
  28. M. N. Dahlkemper, S. Z. Lahme, and P. Klein, How do physics students evaluate artificial intelligence responses on comprehension questions? A study on the perceived scientific accuracy and linguistic quality of ChatGPT, Phys. Rev. Phys. Educ. Res. 19, 010142 (2023).
  29. J. T. Laverty, W. Bauer, G. Kortemeyer, and G. Westfall, Want to reduce guessing and cheating while making students happier? Give more exams!, Phys. Teach. 50, 540 (2012).
  30. R Core Team, R: A Language and Environment for Statistical Computing (R Foundation for Statistical Computing, Vienna, Austria 2021).
  31. S. Epskamp, A. O. J. Cramer, L. J. Waldorp, V. D. Schmittmann, and D. Borsboom, qgraph: Network visualizations of relationships in psychometric data, J. Stat. Softw. 48, 1 (2012).
  32. J. T. Willse, CTT: An R package for classical test theory functions (2014), https://CRAN.R-project.org/package=CTT.
  33. D. J. Palazzo, Y.-J. Lee, R. Warnakulasooriya, and D. E. Pritchard, Patterns, correlates, and reduction of homework copying, Phys. Rev. ST Phys. Educ. Res. 6, 010104 (2010).
  34. E. Gönülateş and G. Kortemeyer, Modeling unproductive behavior in online homework in terms of latent student traits: An approach based on item response theory, J. Sci. Educ. Technol. 26, 139 (2017).
  35. W. M. Rand, Objective criteria for the evaluation of clustering methods, J. Am. Stat. Assoc. 66, 846 (1971).
  36. T. M. Fruchterman and E. M. Reingold, Graph drawing by force-directed placement, Software 21, 1129 (1991).
  37. Y. Cheng and H. Liu, A short note on the maximal point-biserial correlation under non-normality, Br. J. Math. Stat. Psychol. 69, 344 (2016).
  38. G. Kortemeyer, Extending item response theory to online homework, Phys. Rev. ST Phys. Educ. Res. 10, 010118 (2014).
  39. K. Wang, J. Pan, W. Shi, Z. Lu, M. Zhan, and H. Li, Measuring multimodal mathematical reasoning with math-vision dataset, arXiv:2402.14804.
  40. D. A. Kashy, G. Albertelli, W. Bauer, E. Kashy, and M. Thoennessen, Influence of nonmoderated and moderated discussion sites on student success, J. Asynchronous Learn. Netw. 7, 31 (2003).
  41. M. Castro, S. Manoharan, U. Speidel, X. Ye, and J. Zu, Observations of cheating behaviours in online examinations and tools for mitigation, in 2023 IEEE Frontiers in Education Conference (FIE) (IEEE, New York, 2023), pp. 1–7.
  42. M. Alguacil, N. Herranz-Zarzoso, J. C. Pernías, and G. Sabater-Grande, Academic dishonesty and monitoring in online exams: A randomized field experiment, J. Comput Higher Educ. (2023), 10.1007/s12528-023-09378-x.
  43. B. Sorenson and K. Hanson, Statistical comparison between in-person and online general chemistry exam outcomes: A COVID-induced case study, J. Chem. Educ. 100, 3454 (2023).
  44. Respondus, Respondus Browser, https://web.respondus.com/he/lockdownbrowser/ [accessed December 2023].
  45. SEB Alliance, ETH Zurich, Save Exam Browser, https://www.safeexambrowser.org/ [accessed December 2023].
  46. Google, Gemini, https://deepmind.google/technologies/gemini/ [accessed December 2023].
  47. D. Dalalah and O. M. Dalalah, The false positives and false negatives of generative AI detection tools in education and academic research: The case of ChatGPT, Int. J. Manage. Educ. 21, 100822 (2023).
  48. A. M. Elkhatat, K. Elsaid, and S. Almeer, Evaluating the efficacy of ai content detection tools in differentiating between human and AI-generated text, Int. J. Educ. Integrity 19, 17 (2023).
  49. A. Shoufan, Exploring students’perceptions of CHATGPT: Thematic analysis and follow-up survey, IEEE Access (2023).
  50. K. E. Gray, W. K. Adams, C. E. Wieman, and K. K. Perkins, Students know what physicists believe, but they don’t agree: A study using the class survey, Phys. Rev. ST Phys. Educ. Res. 4, 020106 (2008).
  51. H. Lin, Learning physics vs. passing courses, Phys. Teach. 20, 151 (1982).
  52. G. Kortemeyer, N. Dittmann-Domenichini, C. Schlienger, E. Spilling, A. Yaroshchuk, and G. Dissertori, Attending lectures in person, hybrid or online—how do students choose, and what about the outcome?, Int. J. Educ. Technol. Higher Educ. 20, 19 (2023).

Outline

Information

Phys. Rev. Phys. Educ. Res. 20, 010145– Published 23 May, 2024
  • Received 27 December 2023
  • Accepted 7 May 2024
Crossmark - Check for updates button
Creative Commons Logo

Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article’s title, journal citation, and DOI.

Published by the American Physical Society

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation