- Open Access
Comment on “Rubric-based holistic review represents a change from traditional graduate admissions approaches in physics”
Phys. Rev. Phys. Educ. Res. 19, 028001 – Published 6 December, 2023
DOI: https://doi.org/10.1103/PhysRevPhysEducRes.19.028001
Abstract
A recent paper evaluating a new rubric-based graduate admissions approach using generic methods tentatively suggested that its decisions differed noticeably from the previous approach in an unspecified way. Using prior knowledge that the often-stated specific goal was to open a path to increased diversity by reducing barriers to the admission of applicants with low undergraduate grade point averages and graduate record exam (GRE) scores allows simple statistical tests of changes in the distributions of the metrics. The simple tests confirm with good statistical confidence that the barrier-reduction changes were achieved. Nevertheless, the paper’s argument that the de-emphasized tests and grades are not predictive of graduate outcomes is not supported by the prior literature. On technique, although a method used in some of the analyses for dropping data points before running the machine algorithm is likely to bias those results, it helps to clarify why models of the rubric-based system were only weakly predictive.
Physics Subject Headings (PhySH)
Article Text
Original Article
Rubric-based holistic review represents a change from traditional graduate admissions approaches in physics
References (13)
- N. T. Young, N. Verboncoeur, D. C. Lam, and M. D. Caballero, Rubric-based holistic review represents a change from traditional graduate admissions approaches in physics, Phys. Rev. Phys. Educ. Res. 19, 010134 (2023).
- C. Miller and K. Stassun, A test that fails, Nature (London) 510, 303 (2014).
- C. W. Miller, B. M. Zwickl, J. R. Posselt, R. T. Silvestrini, and T. Hodapp, Typical physics Ph.D. admissions criteria limit access to underrepresented groups but fail to predict doctoral completion, Sci. Adv. 5, eaat7550 (2019).
- M. Verostek, C. W. Miller, and B. Zwickl, Analyzing admissions metrics as predictors of graduate GPA and whether graduate GPA mediates Ph.D. completion, Phys. Rev. Phys. Educ. Res. 17, 020115 (2021).
- N. T. Young and M. D. Caballero, Physics graduate record exam does not help applicants “stand out”, Phys. Rev. Phys. Educ. Res. 17, 010144 (2021).
- A. Gelman, Rubric-based holistic word salad represents a change from traditional approaches to publishing papers in physics, Statistical Modeling, Causal Inference, and Social Science (2023), https://statmodeling.stat.columbia.edu/2023/10/10/rubric-based-holistic-word-salad-represents-a-change-from-traditional-approaches-to-publish-papers-in-physics/.
- F. J. Massey, The Kolmogorov-Smirnov test for goodness of fit, J. Am. Stat. Assoc. 46, 68 (1951).
- T. Wilkerson, The relationship between admission credentials and the success of students admitted to a physics doctoral program, Ph.D. thesis, University of Central Florida, 2007.
- Sealy L, Saunders C, Blume J, and C. R., The GRE over the entire range of scores lacks predictive ability for PhD outcomes in the biomedical sciences. PLoS One 14, e0201634 (2019).
- B. E. Huitema and C. R. Stein, Validity of the GRE without restriction of range, Psychol. Rep. 72, 123 (1993).
- M. B. Weissman, Do GRE scores help predict getting a physics Ph.D.? A comment on a paper by Miller et al., Sci. Adv. 6, eaax3787 (2020).
- M. B. Weissman, Invalid methods and false answers: Physics education research and the use of GREs, Econ J. Watch 19, 4 (2022), https://econjwatch.org/articles/invalid-methods-and-false-answers-physics-education-research-and-the-use-of-gres.
- Yijun Zhao, Q. Xu, M. Chen, and G. Weiss, Predicting student performance in a master’s program in data science using admissions data, Proceedings of the 13th International Conference on Educational Data Mining (EDM 2020) (2020), p. 325, https://eric.ed.gov/?id=ED607995.
