- Physical Review Journals
All JournalsPhysics Magazine

Physical Review Physics Education Research

Reuse & Permissions

It is not necessary to obtain permission to reuse this article or its components as it is available under the terms of the Creative Commons Attribution 4.0 International license. This license permits unrestricted use, distribution, and reproduction in any medium, provided attribution to the author(s) and the published article's title, journal citation, and DOI are maintained. Please note that some figures may have been included with permission from other third parties. It is your responsibility to obtain the proper permission from the rights holder directly for these figures.

Export citation

Export citation

Choose format for download:

Download Citation
  • Open Access

Statistically equivalent models with different causal structures: An example from physics identity

Yangqiuting Li1 and Chandralekha Singh2

  • 1Department of Physics, Oregon State University, Corvallis, Oregon 97331, USA
  • 2Department of Physics and Astronomy, University of Pittsburgh, Pittsburgh, Pennsylvania 15260, USA

PDF

Phys. Rev. Phys. Educ. Res. 20, 010101 – Published 16 January, 2024

DOI: https://doi.org/10.1103/PhysRevPhysEducRes.20.010101

Abstract

Structural equation modeling (SEM) is a statistical method widely used in educational research to investigate relationships between variables. SEM models are typically constructed based on theoretical foundations and assessed through fit indices. However, a well-fitting SEM model alone is not sufficient to verify the causal inferences underlying the proposed model, as there are statistically equivalent models with distinct causal structures that equally well fit the data. Therefore, it is crucial for researchers using SEM to consider statistically equivalent models and to clarify why the proposed model is more accurate than the equivalent ones. However, many SEM studies did not explicitly address this important step, and no prior study in physics education research has delved into potential methods for distinguishing statistically equivalent models with differing causal structures. In this study, we use a physics identity model as an example to discuss the importance of considering statistically equivalent models and how other data can help to distinguish them. Previous research has identified three dimensions of physics identity: perceived recognition, self-efficacy, and interest. However, the relationships between these dimensions have not been thoroughly understood. In this paper, we specify a model with perceived recognition predicting self-efficacy and interest, which is inspired by individual interviews with students in physics courses to make physics learning environments equitable and inclusive. We test our model with fit indices and discuss its statistically equivalent models with different causal inferences among perceived recognition, self-efficacy, and interest. We then discuss potential experiments that could further empirically test the causal inferences underlying the models, aiding the refinement to a more accurate causal model for guiding educational improvements.

View figure in article

Physics Subject Headings (PhySH)

Article Text

References (111)

  1. R. B. Kline, Principles and Practice of Structural Equation Modeling (Guilford Publications, New York, 2015).
  2. Y. Li and C. Singh, Effect of gender, self-efficacy, and interest on perception of the learning environment and outcomes in calculus-based introductory physics courses, Phys. Rev. Phys. Educ. Res. 17, 010143 (2021).
  3. A. Godwin, G. Potvin, Z. Hazari, and R. Lock, Identity, critical agency, and engineering: An affective model for predicting engineering as a career choice, J. Engin. Educ. 105, 312 (2016).
  4. V. Adlakha and E. Kuo, Critical issues in statistical causal inference for observational physics education research, Phys. Rev. Phys. Educ. Res. 19, 020160 (2023).
  5. G. Shmueli, To explain or to predict? Stat. Sci. 25, 289 (2010).
  6. M. Kuhn and K. Johnson, Applied Predictive Modeling (Springer, New York, 2013), Vol. 26.
  7. J. Pearl and D. Mackenzie, The Book of Why: The New Science of Cause and Effect (Basic Books, New York, 2018).
  8. R. C. MacCallum and J. T. Austin, Applications of structural equation modeling in psychological research, Ann. Rev. Psychol. 51, 201 (2000).
  9. D. Kaplan, Structural Equation Modeling: Foundations and Extensions (Sage Publications, Thuosand Oaks, 2008), Vol. 10.
  10. S. L. Hershberger and G. Marcoulides, The problem of equivalent structural models, in Structural Equation Modeling: A Second Course, edited by G. R. Hancock and R. O. Mueller (IAP Information Age Publishing, Charlotte, NC, 2006), p. 13.
  11. D. B. Rubin, Comment: The design and analysis of gold standard randomized experiments, J. Am. Stat. Assoc. 103, 1350 (2008).
  12. S. A. Mulaik, Linear Causal Modeling with Structural Equations (CRC Press, Boca Raton, 2009).
  13. D. Eden, E. F. Stone-Romero, and H. R. Rothstein, Synthesizing results of multiple randomized experiments to establish causality in mediation testing, Hum. Resour. Manag. Rev. 25, 342 (2015).
  14. J. G. Bullock, D. P. Green, and S. E. Ha, Yes, but what’s the mechanism? (don’t expect an easy answer), J. Pers. Soc. Psychol. 98, 550 (2010).
  15. S. J. Spencer, M. P. Zanna, and G. T. Fong, Establishing a causal chain: Why experiments are often more effective than mediational analyses in examining psychological processes, J. Pers. Soc. Psychol. 89, 845 (2005).
  16. Y. Kifer, D. Heller, W. Q. E. Perunovic, and A. D. Galinsky, The good life of the powerful: The experience of power and authenticity enhances subjective well-being, Psychol. Sci. 24, 280 (2013).
  17. Z. Hazari, G. Sonnert, P. M. Sadler, and M.-C. Shanahan, Connecting high school physics experiences, outcome expectations, physics identity, and physics career choice: A gender study, J. Res. Sci. Teach. 47, 978 (2010).
  18. Y. Li and C. Singh, Inclusive learning environments can improve student learning and motivational beliefs, Phys. Rev. Phys. Educ. Res. 18, 020147 (2022).
  19. Z. Hazari and C. Cass, Towards meaningful physics recognition: What does this recognition actually look like?, Phys. Teach. 56, 442 (2018).
  20. Z. Y. Kalender, E. Marshman, C. D. Schunn, T. J. Nokes-Malach, and C. Singh, Why female science, technology, engineering, and mathematics majors do not identify with physics: They do not think others see them that way, Phys. Rev. Phys. Educ. Res. 15, 020148 (2019).
  21. Y. Li and C. Singh, Do female and male students’ physics motivational beliefs change in a two-semester introductory physics course sequence?, Phys. Rev. Phys. Educ. Res. 18, 010142 (2022).
  22. Y. Li, K. Whitcomb, and C. Singh, How perception of being recognized or not recognized by instructors as a “physics person” impacts male and female students’ self-efficacy and performance, Phys. Teach. 58, 484 (2020).
  23. S. Cwik and C. Singh, Not feeling recognized as a physics person by instructors and teaching assistants is correlated with female students’ lower grades, Phys. Rev. Phys. Educ. Res. 18, 010138 (2022).
  24. A. Bandura, Self-efficacy, in Encyclopedia of Psychology, edited by R. J. Corsini (Wiley, New York, 1994), 2nd ed., Vol. 3, p. 368.
  25. P. Vincent-Ruz and C. D. Schunn, The increasingly important role of science competency beliefs for science learning in girls, J. Res. Sci. Teach. 54, 790 (2017).
  26. B. J. Zimmerman, Self-efficacy: An essential motive to learn, Contemp. Educ. Psychol. 25, 82 (2000).
  27. D. H. Schunk and F. Pajares, The development of academic self-efficacy, in Development of Achievement Motivation: A Volume in the Educational Psychology Series, edited by A. Wigfield and J. S. Eccles (Academic Press, San Diego, 2002), p. 15.
  28. H. M. Watt, The role of motivation in gendered educational and occupational trajectories related to maths, Educ. Res. Eval. 12, 305 (2006).
  29. S. Hidi, Interest: A unique motivational variable, Educ. Res. Rev. 1, 69 (2006).
  30. J. L. Smith, C. Sansone, and P. H. White, The stereotyped task engagement process: The role of interest and achievement motivation, J. Educ. Psychol. 99, 99 (2007).
  31. P. Häussler and L. Hoffmann, An intervention study to enhance girls’ interest, self-concept, and achievement in physics classes, J. Res. Sci. Teach. 39, 870 (2002).
  32. S. Hidi and K. A. Renninger, The four-phase model of interest development, Educ. Psychol. 41, 111 (2006).
  33. R. M. Lock, Z. Hazari, and G. Potvin, Impact of out-of-class science and engineering activities on physics identity and career intentions, Phys. Rev. Phys. Educ. Res. 15, 020137 (2019).
  34. Z. Hazari, D. Chari, G. Potvin, and E. Brewe, The context dependence of physics identity: Examining the role of performance/competence, recognition, interest, and sense of belonging for lower and upper female physics undergraduates, J. Res. Sci. Teach. 57, 1583 (2020).
  35. S. A. Crockett, A five-step guide to conducting SEM analysis in counseling research, Couns. Outcome Res. Eval. 3, 30 (2012).
  36. Y. Li and C. Singh, Impact of perceived recognition by physics instructors on women’s self-efficacy and interest, Phys. Rev. Phys. Educ. Res. 19, 020125 (2023).
  37. D. Doucette and C. Singh, Why are there so few women in physics? Reflections on the experiences of two women, Phys. Teach. 58, 297 (2020).
  38. L. M. Santana and C. Singh, Negative impacts of an unwelcoming physics environment on undergraduate women, presented at PER Conf. 2021, virtual conference, 10.1119/perc.2021.pr.Santana.
  39. D. Doucette, R. Clark, and C. Singh, Hermione and the Secretary: How gendered task division in introductory physics labs can disrupt equitable learning, Eur. J. Phys. 41, 035702 (2020).
  40. L. Santana and C. Singh, Investigating experiences of a black woman in physics and astronomy, presented at PER Conf. 2022, 10.1119/perc.2022.pr.Santana.
  41. J. C. Chan and S.-f. Lam, Effects of different evaluative feedback on students’ self-efficacy in learning, Instruc. Sci. 38, 37 (2010).
  42. R. Ruegg, The effect of peer and teacher feedback on changes in EFL students’ writing self-efficacy, Lang. Learn. J. 46, 87 (2018).
  43. B. W. Tuckman and T. L. Sexton, The effect of teacher encouragement on student self-efficacy and motivation for self-regulated performance, J. Soc. Behav. Pers. 6, 137 (1991).
  44. J. Henderlong and M. R. Lepper, The effects of praise on children’s intrinsic motivation: A review and synthesis, Psychol. Bull. 128, 774 (2002).
  45. M. E. Shanab, D. Peterson, S. Dargahi, and P. Deroian, The effects of positive and negative verbal feedback on the intrinsic motivation of male and female subjects, J. Soc. Psychol. 115, 195 (1981).
  46. E. L. Deci, R. Koestner, and R. M. Ryan, A meta-analytic review of experiments examining the effects of extrinsic rewards on intrinsic motivation, Psychol. Bull. 125, 627 (1999).
  47. D. A. Donnay and F. H. Borgen, The incremental validity of vocational self-efficacy: An examination of interest, self-efficacy, and occupation, J. Counsel. Psychol. 46, 432 (1999).
  48. R. W. Lent, S. D. Brown, and G. Hackett, Toward a unifying social cognitive theory of career and academic interest, choice, and performance, J. Vocat. Behav. 45, 79 (1994).
  49. N. E. Betz, Self-efficacy theory as a basis for career assessment, J. Career Assess. 8, 205 (2000).
  50. P. J. Silvia, Interest and interests: The psychology of constructive capriciousness, Rev. General Psychol. 5, 270 (2001).
  51. P. J. Silvia, Self-efficacy and interest: Experimental studies of optimal incompetence, J. Vocat. Behav. 62, 237 (2003).
  52. D. E. Berlyne, Conflict, Arousal, and Curiosity (McGraw-Hill Book Company, 1960), https://doi.org/10.1037/11164-000.
  53. S. Cwik and C. Singh, How perception of learning environment predicts male and female students’ grades and motivational outcomes in algebra-based introductory physics courses, Phys. Rev. Phys. Educ. Res. 17, 020143 (2021).
  54. E. Marshman, Z. Kalender, Yasemin, T. Nokes-Malach, C. Schunn, and C. Singh, Female students with A’s have similar physics self-efficacy as male students with C’s in introductory courses: A cause for alarm?, Phys. Rev. Phys. Educ. Res. 14, 020123 (2018).
  55. G. C. Marchand and G. Taasoobshirazi, Stereotype threat and women’s performance in physics, Int. J. Sci. Educ. 35, 3050 (2013).
  56. A. Maries, N. I. Karim, and C. Singh, Is agreeing with a gender stereotype correlated with the performance of female students in introductory physics?, Phys. Rev. Phys. Educ. Res. 14, 020119 (2018).
  57. A. Maries, N. I. Karim, and C. Singh, Does stereotype threat affect female students’ performance in introductory physics?, AIP Conf. Proc. 2109, 120001 (2019).
  58. A. A. Eaton, J. F. Saunders, R. K. Jacobson, and K. West, How gender and race stereotypes impact the advancement of scholars in STEM: Professors’ biased evaluations of physics and biology post-doctoral candidates, Sex Roles 82, 127 (2020).
  59. Y. Li and C. Singh, Sense of belonging is an important predictor of introductory physics students’ academic performance, Phys. Rev. Phys. Educ. Res. 19, 020137 (2023).
  60. Learning Activation Lab, Activation lab tools: Measures, and data collection instruments (2017), http://www.activationlab.org/tools/. (Accessed 4 February 2019).
  61. J. Schell and B. Lukoff, Peer instruction self-efficacy instrument [Developed at Harvard University] (unpublished) (2010).
  62. S. M. Glynn, P. Brickman, N. Armstrong, and G. Taasoobshirazi, Science motivation questionnaire II: Validation with science majors and nonscience majors, J. Res. Sci. Teach. 48, 1159 (2011).
  63. E. Marshman, Z. Y. Kalender, C. Schunn, T. Nokes-Malach, and C. Singh, A longitudinal analysis of students’ motivational characteristics in introductory physics courses: Gender differences, Can. J. Phys. 96, 391 (2018).
  64. T. Nokes-Malach, E. Marshman, Z. Y. Kalender, C. Schunn, and C. Singh, Investigation of male and female students’ motivational characteristics throughout an introductory physics course sequence, presented at AIP Conf. 2017, 10.1119/perc.2017.pr.064.
  65. Z. Y. Kalender, E. Marshman, C. D. Schunn, T. J. Nokes-Malach, and C. Singh, Large gender differences in physics self-efficacy at equal performance levels: A warning sign?, presented at AIP Conf. 2018, 10.1119/perc.2018.pr.Kalender.
  66. B. Thompson, Exploratory and Confirmatory Factor Analysis (American Psychological Association, Washington, DC, 2004).
  67. D. Hooper, J. Coughlan, and M. Mullen, Structural equation modeling: Guidelines for determining model fit, Electronic J. Bus. Res. Methods 6, 53 (2008).
  68. L. J. Cronbach, Coefficient alpha and the internal structure of tests, Psychometrika 16, 297 (1951).
  69. K. Pearson and F. Galton, VII. Note on regression and inheritance in the case of two parents, Proc. R. Soc. London 58, 240 (1895).
  70. Z. Hazari, G. Potvin, R. M. Lock, F. Lung, G. Sonnert, and P. M. Sadler, Factors that affect the physical science career interest of female students: Testing five common hypotheses, Phys. Rev. ST Phys. Educ. Res. 9, 020115 (2013).
  71. Z. Hazari, R. H. Tai, and P. M. Sadler, Gender differences in introductory university physics performance: The influence of high school physics preparation and affective factors, Sci. Educ. 91, 847 (2007).
  72. R. Likert, A technique for the measurement of attitudes, Arch. Sci. Psychol. 22, 55 (1932), https://legacy.voteview.com/pdf/Likert_1932.pdf.
  73. S. E. Embretson and S. P. Reise, Item Response Theory for Psychologists (Lawrence Erlbaum Associates Publishers, Mahwah, NJ, US, 2000).
  74. R. P. Chalmers, mirt: A multidimensional item response theory package for the R environment, J. Stat. Softw. 48, 1 (2012).
  75. F. Samejima, Estimation of latent ability using a response pattern of graded scores, in Psychometrika Monograph (Psychometric Society, Richmond, VA, 1969), p. 17.
  76. M. G. Bulmer, Principles of Statistics (Dover Publications, New York, 1979).
  77. J. F. Hair, W. C. Black, B. J. Babin, and R. E. Anderson, Multivariate Data Analysis: International Version (Pearson, New Jersey, 2010).
  78. B. M. Byrne, Structural equation modeling with AMOS: Basic concepts, applications, and programming (Routledge/Taylor & Francis Group, New York, NY, 2010), 2nd ed.
  79. J. Pallant, SPSS Survival Manual: A Step by Step Guide to Data Analysis Using IBM SPSS (Routledge, London, 2020).
  80. R Core Team, R: A Language and Environment for Statistical Computing (2013) (R Foundation, 2013), https://www.r-project.org/ (accessed 6 February 2019).
  81. A. J. Tomarken and N. G. Waller, Structural equation modeling: Strengths, limitations, and misconceptions, Annu. Rev. Clin. Psychol. 1, 31 (2005).
  82. S. J. Finney and C. DiStefano, Non-normal and categorical data in structural equation modeling, in Structural Equation Modeling: A Second Course (Information Age, Greenwich, CT, 2006), p. 269.
  83. D. Mindrila, Maximum likelihood (ML) and diagonally weighted least squares (DWLS) estimation procedures: A comparison of estimation bias with ordinal and multivariate non-normal data, Int. J. Digital Soc. 1, 60 (2010).
  84. M. Rhemtulla, P. É. Brosseau-Liard, and V. Savalei, When can categorical variables be treated as continuous? A comparison of robust continuous and categorical SEM estimation methods under suboptimal conditions, Psychol. Methods 17, 354 (2012).
  85. K. S. Betts, G. M. Williams, J. M. Najman, and R. Alati, The role of sleep disturbance in the relationship between post-traumatic stress disorder and suicidal ideation, J. Anxiety Disord. 27, 735 (2013).
  86. C. A. Kronauge, The Effects of Mixing Metrics and Distributions Simultaneously in Structural Equation Modeling: A Simulation Study (University of Northern Colorado, Greeley, CO, 2012).
  87. H. Akoglu, User’s guide to correlation coefficients, Turkish J. Emergency Medicine 18, 91 (2018).
  88. G. D. Garson, Structural Equation Modeling (Statistical Associates Publishers, Asheboro, NC, 2014).
  89. J. Cohen, Statistical Power Analysis for the Behavioral Sciences (L. Erlbaum Associates, Hillsdale, NJ, 1988).
  90. J. Cohen, Statistical Power Analysis for the Behavioral Sciences (Routledge, London, 2013).
  91. J. B. Grace and K. A. Bollen, Interpreting the results from multiple regression and structural equation models, Bull. Ecol. Soc. Am. 86, 283 (2005).
  92. M. Seitchik, Confidence and gender: Few differences, but gender stereotypes impact perceptions of confidence, Psychologist-Manager J. 23, 194 (2020).
  93. A. Master, A. N. Meltzoff, and S. Cheryan, Gender stereotypes about interests start early and cause gender disparities in computer science and engineering, Proc. Natl. Acad. Sci. U.S.A. 118, e2100030118 (2021).
  94. J. Wade, How you can change gender stereotypes about physicists, Nat. Rev. Phys. 4, 690 (2022).
  95. C. Singh, Inclusive mentoring: The mindset of an effective mentor, arXiv:2112.01227.
  96. C. M. Mueller and C. S. Dweck, Praise for intelligence can undermine children’s motivation and performance, J. Pers. Soc. Psychol. 75, 33 (1998).
  97. J. Wang and Z. Hazari, Promoting high school students’ physics identity through explicit and implicit recognition, Phys. Rev. Phys. Educ. Res. 14, 020111 (2018).
  98. A. Bandura, Self-efficacy: The Exercise of Control (W. H. Freeman, New York, 1997).
  99. W. Speering and L. Rennie, Students’ perceptions about science: The impact of transition from primary to secondary school, Res. Sci. Educ. 26, 283 (1996).
  100. Z. Hazari, C. Cass, and C. Beattie, Obscuring power structures in the physics classroom: Linking teacher positioning, student engagement, and physics identity development, J. Res. Sci. Teach. 52, 735 (2015).
  101. R. Elmesky, “I am science and the world is mine”: Embodied practices as resources for empowerment, School Sci. Math. 105, 335 (2005).
  102. D. A. Luzzo, P. Hasper, K. A. Albert, M. A. Bibby, and E. A. Martinelli Jr, Effects of self-efficacy-enhancing interventions on the math/science self-efficacy and career interests, goals, and actions of career undecided college students, J. Counsel. Psychol. 46, 233 (1999).
  103. R. A. Bartsch, K. A. Case, and H. Meerman, Increasing academic self-efficacy in statistics with a live vicarious experience presentation, Teach. Psychol. 39, 133 (2012).
  104. D. Cordova and M. Lepper, Intrinsic motivation and the process of learning: Beneficial effects of contextualization, personalization, and choice, J. Educ. Psychol. 88, 715 (1996).
  105. S. Hidi and W. Baird, Strategies for increasing text-based interest and students’ recall of expository texts, Read. Res. Q. 23, 465 (1988).
  106. S. Hidi, J. Weiss, D. Berndorff, and J. Nolan, The role of gender, instruction and a cooperative learning technique in science education across formal and informal settings, in Interest and Learning: Proceedings of the Seeon Conference on Interest and Gender, IPN Kiel, Germany (The Institute for Science Education (IPN), Kiel, Germany, 1998), p. 215.
  107. M. R. Lepper and D. I. Cordova, A desire to be taught: Instructional consequences of intrinsic motivation, Motiv. Emot. 16, 187 (1992).
  108. M. Mitchell, Situational interest: Its multifaceted structure in the secondary school mathematics classroom, J. Educ. Psychol. 85, 424 (1993).
  109. J. A. Sloboda and J. Davidson, The young performing musician, in Musical Beginnings: Origins and Development of Musical Competence (Oxford University Press, New York, 1996), p. 171.
  110. P. W. Lei and Q. Wu, Introduction to structural equation modeling: Issues and practical considerations, Educ. Meas. 26, 33 (2007).
  111. R. P. Bagozzi and Y. Yi, On the evaluation of structural equation models, J. Acad. Marketing Sci. 16, 74 (1988).

Outline

Information

Phys. Rev. Phys. Educ. Res. 20, 010101– Published 16 January, 2024
  • Received 24 March 2023
  • Accepted 18 December 2023
Crossmark - Check for updates button
Creative Commons Logo

Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article’s title, journal citation, and DOI.

Published by the American Physical Society

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation