- Open Access
Values affirmation replication at the University of Illinois
Phys. Rev. Phys. Educ. Res. 17, 020121 – Published 20 September, 2021
DOI: https://doi.org/10.1103/PhysRevPhysEducRes.17.020121
Abstract
Values affirmation exercises have been implemented in many contexts to combat stereotype threat in students from marginalized populations; the exercises are intended to fortify students by prompting them to self-affirm their values in short writing activities. Within the physics education research community, the style of intervention was underlined by a positive result from the University of Colorado Boulder; researchers were able to use the intervention to minimize the achievement gap between men and women in an introductory physics course. These results inspired a replication experiment in two physics courses at the University of Illinois in Urbana-Champaign, and this article provides some history and context of these interventions, describes our specific implementation, and reveals that we were unable to reproduce the positive results despite thorough attention to the details of the replication. Our findings suggest that the values affirmation exercises are not understood at a level where they should be considered a positive intervention to help marginalized populations.
Physics Subject Headings (PhySH)
Article Text
References (114)
- J. M. Nissen and J. T. Shemwell, Gender, experience, and self-efficacy in introductory physics, Phys. Rev. Phys. Educ. Res. 12, 020105 (2016).
- M. Ong, C. Wright, L. L. Espinosa, and G. Orfield, Inside the double bind: A synthesis of empirical research on undergraduate and graduate women of color in science, technology, engineering,and mathematics, Harv. Educ. Rev. 81, 172 (2011).
- V. Sawtelle, E. Brewe, and L. H. Kramer, Exploring the relationship between self-efficacy and retention in introductory physics, J. Res. Sci. Teach. 49, 1096 (2012).
- A. Traxler and E. Brewe, Equity investigation of attitudinal shifts in introductory physics, Phys. Rev. ST Phys. Educ. Res. 11, 020132 (2015).
- E. M. Marshman, Z. Yasemin Kalender, T. Nokes-Malach, C. Schunn, and C. Singh, Female students with A’s have similar physics self-efficacy as male students with C’s in introductory courses: A cause for alarm?, Phys. Rev. Phys. Educ. Res. 14, 020123 (2018).
- K. L. Lewis, J. G. Stout, S. J. Pollock, N. D. Finkelstein, and T. A. Ito, Fitting in or opting out: A review of key social-psychological factors influencing a sense of belonging for women in physics, Phys. Rev. Phys. Educ. Res. 12, 020110 (2016).
- S. L. Eddy and S. E. Brownell, Beneath the numbers: A review of gender disparities in undergraduate education across science, technology, engineering, and math disciplines, Phys. Rev. Phys. Educ. Res. 12, 020106 (2016).
- A. L. Traxler, X. C. Cid, J. Blue, and R. Barthelemy, Enriching gender in physics education research: A binary past and a complex future, Phys. Rev. Phys. Educ. Res. 12, 020114 (2016).
- M. Bruun, S. Willoughby, and J. L. Smith, Identifying the stereotypical who, what, and why of physics and biology, Phys. Rev. Phys. Educ. Res. 14, 020125 (2018).
- A. Maries, N. I. Karim, and C. Singh, Is agreeing with a gender stereotype correlated with the performance of female students in introductory physics?, Phys. Rev. Phys. Educ. Res. 14, 020119 (2018).
- Z. Hazari, P. M. Sadler, and G. Sonnert, The science identity of college students: Exploring the intersection of gender, race, and ethnicity, J. Coll. Sci. Teach. 42, 82 (2013), https://www.jstor.org/stable/43631586.
- L. E. Kost-Smith, S. J. Pollock, and N. D. Finkelstein, Gender disparities in second-semester college physics: The incremental effects of a “smog of bias”, Phys. Rev. ST Phys. Educ. Res. 6, 020112 (2010).
- V. Seyranian, A. Madva, N. Duong, N. Abramzon, Y. Tibbetts, and J. M. Harackiewicz, The longitudinal effects of STEM identity and gender on flourishing and achievement in college physics, Int. J. STEM Educ. 5, 40 (2018).
- K. Rosa and F. Moore Mensah, Educational pathways of Black women physicists: Stories of experiencing and overcoming obstacles in life, Phys. Rev. Phys. Educ. Res. 12, 020113 (2016).
- X. R. Quichocho, J. Conn, E. M. Schipull, and E. W. Close, Who does physics? Understanding the composition of physicists through the lens of women of color and , Proceedings of the Physics Education Research Conference 2019, Provo, UT, 10.1119/perc.2019.pr.Quichocho.
- L. Bian, A. Cimpian, and S.-J. Leslie, Gender stereotypes about intellectual ability emerge early and influence children’s interests, Science 355, 389 (2017).
- A. M. Kelly, Physics teachers’ perspectives on factors that affect urban physics participation and accessibility, Phys. Rev. ST Phys. Educ. Res. 9, 010122 (2013).
- N. Reid and E. A. Skryabina, Gender and physics, Int. J. Sci. Educ. 25, 509 (2003).
- L. E. Kost, S. J. Pollock, and N. D. Finkelstein, Characterizing the gender gap in introductory physics, Phys. Rev. ST Phys. Educ. Res. 5, 010101 (2009).
- I. Rodriguez, G. Potvin, and L. H. Kramer, How gender and reformed introductory physics impacts student success in advanced physics courses and continuation in the physics major, Phys. Rev. Phys. Educ. Res. 12, 020118 (2016).
- P. W. Irving and E. C. Sayre, Becoming a physicist: The roles of research, mindsets, and milestones in upper-division student perceptions, Phys. Rev. ST Phys. Educ. Res. 11, 020120 (2015).
- V. Sawtelle, E. Brewe, R. Michelle Goertzen, and L. H. Kramer, Identifying events that impact self-efficacy in physics learning, Phys. Rev. ST Phys. Educ. Res. 8, 020111 (2012).
- E. W. Close, J. Conn, and H. G. Close, Becoming physics people: Development of integrated physics identity through the Learning Assistant experience, Phys. Rev. Phys. Educ. Res. 12, 010109 (2016).
- R. M. Lock, Z. Hazari, and G. Potvin, Physics career intentions: The effect of physics identity, math identity, and gender, AIP Conf. Proc. 1513, 262 (2013).
- Z. Hazari, G. Sonnert, P. M. Sadler, and M.-C. Shanahan, Connecting high school physics experiences, outcome expectations, physics identity, and physics career choice: A gender study, J. Res. Sci. Teach. 47, 978 (2010).
- G. L. Cohen, J. Garcia, N. Apfel, and A. Master, Reducing the racial achievement gap: A social-psychological intervention, Science 313, 1307 (2006).
- A. Miyake, L. E. Kost-Smith, N. D. Finkelstein, S. J. Pollock, G. L. Cohen, and T. A. Ito, Reducing the gender achievement gap in college science: A classroom study of values affirmation, Science 330, 1234 (2010).
- S. Andersson and A. Johansson, Gender gap or program gap? Students’ negotiations of study practice in a course in electromagnetism, Phys. Rev. Phys. Educ. Res. 12, 020112 (2016).
- R. Gutiérrez, A “gap-gazing” fetish in mathematics education? Problematizing research on the achievement gap, J. Res. Math. Educ. 39, 357 (2008), https://www.jstor.org/stable/40539302.
- A. Danielsson, Gender in physics education research: A review and a look forward, in Never Mind the gap! Gendering Science in Transgressive Encounters (2010), http://urn.kb.se/resolve?urn=urn:nbn:se:uu:diva-254006.
- G. Ladson-Billings, From the achievement gap to the education debt: Understanding achievement in U.S. schools, Educ. Res. 35, 3 (2006).
- D. Bauder, AP says it will capitalize Black but not white, 2020, https://www.ap.org/ap-in-the-news/2020/ap-says-it-will-capitalize-black-but-not-white.
- K. A. Appiah, The case for capitalizing the B in Black, The Atlantic 18 (2020), https://www.theatlantic.com/ideas/archive/2020/06/time-to-capitalize-blackand-white/613159/.
- M. Laws, Why we capitalize ‘Black’ (and not ‘white’), Columbia Journalism Rev. 16 (2020), https://www.cjr.org/analysis/capital-b-black-styleguide.php.
- E. Zorn, Column: Should ‘white’ be capitalized? It feels wrong, but it’s the way to go, Chicago Tribune (2020), https://www.chicagotribune.com/columns/eric-zorn/ct-column-capitalize-white-black-language-race-zorn-20200709-e42fag6ivbazdblizpopsp4p2a-story.html.
- N. Irvin Painter, Opinion: Why ‘White’ should be capitalized, too, The Washington Post (July 22, 2020), https://www.washingtonpost.com/opinions/2020/07/22/why-white-should-be-capitalized/.
- G. L. Cochran, A. Gupta, S. Hyater-Adams, A. V. Knaub, and B. Zamarripa Roman, Emerging reflections from the people of color (POC) at PERC discussion space, 2019, arXiv:1907.01655.
- K. Crenshaw, Demarginalizing the Intersection of Race and Sex: A Black Feminist Critique of Antidiscrimination Doctrine, Feminist Theory and Antiracist Politics, University of Chicago Legal Forum Vol. 1989, p. 8, https://chicagounbound.uchicago.edu/uclf/vol1989/iss1/8.
- S. Kanim and X. C. Cid, Demographics of physics education research, Phys. Rev. Phys. Educ. Res. 16, 020106 (2020).
- S. Theule Lubienski, On “gap gazing” in mathematics education: The need for gaps analyses, J. Res. Math. Educ. 39, 350 (2008), https://www.jstor.org/stable/40539301.
- C. M. Steele, S. J. Spencer, and J. Aronson, Contending with group image: The psychology of stereotype and social identity threat, Adv. Exp. Soc. Psychol. 34, 379 (2002).
- G. M. Walton and S. J. Spencer, Latent ability: Grades and test scores systematically underestimate the intellectual ability of negatively stereotyped students’, Psychol. Sci. 20, 1132 (2009).
- G. L. Cohen and D. K. Sherman, The psychology of change: Self-affirmation and social psychological intervention, Annu. Rev. Psychol. 65, 333 (2014).
- A. Madsen, S. B. McKagan, and E. C. Sayre, Gender gap on concept inventories in physics: What is consistent, what is inconsistent, and what factors influence the gap?, Phys. Rev. ST Phys. Educ. Res. 9, 020121 (2013).
- D. S. Yeager and G. M. Walton, Social-psychological interventions in education: They’re not magic, Rev. Educ. Res. 81, 267 (2011).
- N. Shnabel, V. Purdie-Vaughns, J. E. Cook, J. Garcia, and G. L. Cohen, Demystifying values-affirmation interventions: Writing about social belonging is a key to buffering against identity threat, Pers. Soc. Psychol. Bull. 39, 663 (2013).
- S. Stoessner and C. Good, Stereotype threat: An overview, 2011, https://diversity.arizona.edu/sites/default/files/stereotype_threat_overview.pdf.
- C. M. Steele and J. Aronson, Stereotype threat and the intellectual test performance of African Americans, J. Personality Soc. Psychol. 69, 797 (1995).
- S. J. Spencer, C. M. Steele, and D. M. Quinn, Stereotype threat and women’s math performance, J. Exp. Soc. Psychol. 35, 4 (1999).
- M. Walsh, C. Hickey, and J. Duffy, Influence of item content and stereotype situation on gender differences in mathematical problem solving, Sex Roles 41, 219 (1999).
- C. Good, J. Aronson, and J. Ann Harder, Problems in the pipeline: Stereotype threat and women’s achievement in high-level math courses, J. Appl. Dev. Psychol. 29, 17 (2008).
- M. Inzlicht and T. Ben-Zeev, A threatening intellectual environment: Why females are susceptible to experiencing problem-solving deficits in the presence of males, Psychol. Sci. 11, 365 (2000).
- N. Ambady, M. Shih, A. Kim, and T. L. Pittinsky, Stereotype susceptibility in children: Effects of identity activation on quantitative performance, Psychol. Sci. 12, 385 (2001).
- J.-C. Croizet and T. Claire, Extending the concept of stereotype threat to social class: The intellectual underperformance of students from low socioeconomic backgrounds, Pers. Soc. Psychol. Bull. 24, 588 (1998).
- L. A. Harrison, C. M. Stevens, A. N. Monty, and C. A. Coakley, The consequences of stereotype threat on the academic performance of White and non-White lower income college students, Social Psychol. Educ. 9, 341 (2006).
- D. K. Sherman, K. A. Hartson, K. R. Binning, V. Purdie-Vaughns, J. Garcia, S. Taborsky-Barba, S. Tomassetti, A. David Nussbaum, and G. L. Cohen, Deflecting the trajectory and changing the narrative: How self-affirmation affects academic performance and motivation under identity threat, J. Personality Soc. Psychol. 104, 591 (2013).
- P. M. Gonzales, H. Blanton, and K. J. Williams, The effects of stereotype threat and double-minority status on the test performance of Latino women, Personality Soc. Psychol. Bull. 28, 659 (2002).
- T. Schmader and M. Johns, Converging evidence that stereotype threat reduces working memory capacity, J. Personality Soc. Psychol. 85, 440 (2003).
- B. Levy, Improving memory in old age through implicit self-stereotyping, J. Personality Soc. Psychol. 71, 1092 (1996).
- J. K. Bosson, E. L. Haymovitz, and E. C. Pinel, When saying and doing diverge: The effects of stereotype threat on self-reported versus non-verbal anxiety, J. Exp. Soc. Psychol. 40, 247 (2004).
- J. Stone, M. Sjomeling, C. I. Lynch, and J. M. Darley, Stereotype threat effects on black and white athletic performance, J. Personality Soc. Psychol. 77, 1213 (1999).
- L. J. Kray, A. D. Galinsky, and L. Thompson, Reversing the gender gap in negotiations: An exploration of stereotype regeneration, Organ. Behav. Human Decis. Processes 87, 386 (2002).
- N. C. J. Yeung and C. von Hippel, Stereotype threat increases the likelihood that female drivers in a simulator run over jaywalkers, Accid. Anal. Prev. 40, 667 (2008).
- C. M. Steele, S. J. Spencer, and J. Aronson, Contending with group image: The psychology of stereotype and social identity threat, in Advances in Experimental Social Psychology (Academic Press, New York, 2002), Vol. 34, pp. 379–440.
- C. M. Steele, A threat in the air: How stereotypes shape intellectual identity and performance, Am. Psychol. 52, 613 (1997).
- A. C. Krendl, J. A. Richeson, W. M. Kelley, and T. F. Heatherton, The negative consequences of threat: A functional magnetic resonance imaging investigation of the neural mechanisms underlying women’s underperformance in math, Psychol. Sci. 19, 168 (2008).
- J. C. Croizet, G. Despres, M. E. Gauzins, P. Huguet, J. P. Leyens, and A. Meot, Stereotype threat undermines intellectual performance by triggering a disruptive mental load, Pers. Soc. Psychol. Bull. 30, 721 (2004).
- J. Keller and D. Dauenheimer, Stereotype threat in the classroom: Dejection mediates the disrupting threat effect on women’s math performance, Pers. Soc. Psychol. Bull. 29, 371 (2003).
- C. Stangor, C. Carr, and L. Kiang, Activating stereotypes undermines task performance expectations, J. Personality Soc. Psychol. 75, 1191 (1998).
- J. Thomas Kellow and B. D. Jones, The effects of stereotypes on the achievement gap: Reexamining the academic performance of African American high school students’, J. Black Psychol. 34, 94 (2008).
- C. Graham, R. W. Baker, and S. Wapner, Prior interracial experience and Black student transition into predominantly White colleges, J. Personality Soc. Psychol. 47, 1146 (1984).
- J. Aronson, C. B. Fried, and C. Good, Reducing the effects of stereotype threat on African American college students by shaping theories of intelligence, J. Exp. Soc. Psychol. 38, 113 (2002).
- C. S. Dweck, C.-y. Chiu, and Y.-y. Hong, Implicit theories and their role in judgments and reactions: A word from two perspectives, Psychol. Inquiry 6, 267 (1995).
- C. S. Dweck, Mindset: The New Psychology of Success (Random House Digital, Inc., New York, 2008), ISBN: 9780345472328, 0345472322.
- C. M. Steele, The psychology of self-affirmation: Sustaining the integrity of the self, Adv. Exp. Soc. Psychol. 21, 261 (1988).
- C. M. Steele, Name-calling and compliance, J. Personality Soc. Psychol. 31, 361 (1975).
- A. McQueen and W. M. P. Klein, Experimental manipulations of self-affirmation: A systematic review, Self & Identity 5, 289 (2006).
- D. K. Sherman and G. L. Cohen, The psychology of self-defense: Self-affirmation theory, Adv. Exp. Soc. Psychol. 38, 183 (2006).
- G. L. Cohen, J. Garcia, V. Purdie-Vaughns, N. Apfel, and P. Brzustoski, Recursive processes in self-affirmation: Intervening to close the minority achievement gap, Science 324, 400 (2009).
- D. K. Sherman, G. L. Cohen, L. D. Nelson, A. David Nussbaum, D. P. Bunyan, and J. Garcia, Affirmed yet unaware: Exploring the role of awareness in the process of self-affirmation, J. Personality Soc. Psychol. 97, 745 (2009).
- N. K. Bowen, K. M. Wegmann, and K. C. Webber, Enhancing a brief writing intervention to combat stereotype threat among middle-school students, J. Educ. Psychol. 105, 427 (2013).
- K. Woolf, I. Chris McManus, D. Gill, and J. Dacre, The effect of a brief social intervention on the examination results of UK medical students: a cluster randomised controlled trial, BMC Med. Educ. 9, 35 (2009).
- J. M. Harackiewicz, E. A. Canning, Y. Tibbetts, C. J. Giffen, S. S. Blair, D. I. Rouse, and J. S. Hyde, Closing the social class achievement gap for first-generation students in undergraduate biology, J. Educ. Psychol. 106, 375 (2014).
- E. D. B. Riggle, K. A. Gonzalez, S. S. Rostosky, and W. W. Black, Cultivating positive LGBTQA identities: An intervention study with college students, J. LGBT Issues Counseling 8, 264 (2014).
- G. M. Walton, C. Logel, J. M. Peach, S. J. Spencer, and M. P. Zanna, Two brief interventions to mitigate a “Chilly Climate” transform women’s experience, relationships, and achievement in engineering, J. Educ. Psychol. 107, 468 (2015).
- D. N. Bradley, E. P. Crawford, and S. E. Dahill-Brown, Defining and assessing FoI in a large-scale randomized trial: Core components of values affirmation, Studies Educ. Eval. 49, 51 (2016).
- G. Borman, Examination of a Self-Affirmation Intervention in St. Paul Public Schools, The Senior Urban Education Research Fellowship Series (2012), Vol. IX, https://www.cgcs.org/cms/lib/DC00001581/Centricity/Domain/85/St_Paul_Self-Aff irmation.pdf.
- T. S. Dee, Social identity and achievement gaps: Evidence from an affirmation intervention, J. Res. Educ. Effectiveness 8, 149 (2015).
- G. D. Borman, J. Grigg, and P. Hanselman, An effort to close achievement gaps at scale through self-affirmation, Educ. Eval. Policy Anal. 38, 21 (2016).
- G. D. Borman, J. Grigg, C. S. Rozek, P. Hanselman, and N. A. Dewey, Self-affirmation effects are produced by school context, student engagement with the intervention, and time: Lessons from a district-wide implementation, Psychol. Sci. 29, 1773 (2018).
- H. Jordt, S. L. Eddy, R. Brazil, I. Lau, C. Mann, S. E. Brownell, K. King, and S. Freeman, Values affirmation intervention reduces achievement gap between underrepresented minority and white students in introductory biology classes, CBE Life Sci. Educ. 16 (2017).
- S. Lauer, J. Momsen, E. Offerdahl, M. Kryjevskaia, W. Christensen, and L. Montplaisir, Stereotyped: Investigating gender in introductory science courses, CBE Life Sci. Educ. 12, 30 (2013).
- A. Franco, N. Malhotra, and G. Simonovits, Publication bias in the social sciences: Unlocking the file drawer, Science 345, 1502 (2014).
- J. Mervis, Research transparency: Why null results rarely see the light of day, Science 345, 992 (2014).
- D. Fanelli, Negative results are disappearing from most disciplines and countries, Scientometrics 90, 891 (2012).
- D. Chavalarias, J. D. Wallach, A. H. T. Li, and J. P. A. Ioannidis, Evolution of reporting P values in the biomedical literature, 1990-2015: Evolution of reporting P values in the biomedical literature, 1990–2015, J. Am. Med. Assoc. 315, 1141 (2016).
- R. Rosenthal, The file drawer problem and tolerance for null results, Psychol. Bull. 86, 638 (1979).
- L. D. Conlin, E. Kuo, and N. R. Hallinen, Nothing’s plenty: The significance of null results in physics education research, arXiv:1810.10071.
- T. D. Sterling, W. L. Rosenbaum, and J. J. Weinkam, Publication Decisions Revisited: The effect of the outcome of statistical tests on the decision to publish and vice versa, Am. Statistician 49, 108 (1995).
- Z. Wu, T. F. Spreckelsen, and G. L. Cohen, A meta-analysis of the effect of values affirmation on academic achievement, J. Soc. Issues 1 (2021).
- L. E. Kost-Smith, S. J. Pollock, N. D. Finkelstein, G. L. Cohen, T. A. Ito, and A. Miyake, Replicating a self-affirmation intervention to address gender differences: Successes and challenges, AIP Conf. Proc. 1413, 231 (2012).
- M. Rifkin, Addressing underrepresentation: Physics teaching for all, Phys. Teach. 54, 72 (2016).
- Z. Hazari, G. Potvin, R. M. Lock, F. Lung, G. Sonnert, and P. M. Sadler, Factors that affect the physical science career interest of female students: Testing five common hypotheses, Phys. Rev. ST Phys. Educ. Res. 9, 020115 (2013).
- E. A. Eschenbach, M. Virnoche, E. M. Cashman, S. M. Lord, and M. M. Camacho, Proven practices that can reduce stereotype threat in engineering education: A literature review, in Proceedings of the 2014 IEEE Frontiers in Education Conference (FIE) (2014), pp. 1–9, 10.1109/FIE.2014.7044011.
- L. Aguilar, G. Walton, and C. Wieman, Psychological insights for improved physics teaching, Phys. Today 67, No. 5, 43 (2014).
- J. L. Docktor and J. P. Mestre, Synthesis of discipline-based education research in physics, Phys. Rev. ST Phys. Educ. Res. 10, 020119 (2014).
- B. W. Fitzgerald, Using superheroes such as Hawkeye, Wonder Woman and the Invisible Woman in the physics classroom, Phys. Educ. 53, 035032 (2018).
- N. Abramzon, P. Benson, E. Bertschinger, S. Blessing, G. L. Cochran, A. Cox, B. Cunningham, J. Galbraith-Frew, J. Johnson, L. Kerby, E. Lalanne, C. O’Donnell, S. Petty, S. Sampath, S. Seestrom, C. Singh, C. Spencer, K. SparksWoodle, and S. Yennello, Women in physics in the United States: Recruitment and retention, AIP Conf. Proc. 1697, 060045 (2015).
- T. Scott, A. Gray, and P. Yates, A controlled comparison of teaching methods in first-year university physics, J. R. Soc. New Zealand 43, 88 (2013).
- J. M. Valla and W. M. Williams, Increasing achievement and higher-education representation of under-represented groups in science, technology, engineering, and mathematics fields: A review of current L-12 intervention programs, J. Women Minorities Sci. Eng. 18, 21 (2012).
- R. Henderson, G. Stewart, J. Stewart, L. Michaluk, and A. Traxler, Exploring the gender gap in the Conceptual Survey of Electricity and Magnetism, Phys. Rev. Phys. Educ. Res. 13, 020114 (2017).
- K. Kreutzer and A. Boudreaux, Preliminary investigation of instructor effects on gender gap in introductory physics, Phys. Rev. ST Phys. Educ. Res. 8, 010120 (2012).
- W. K. Adams, K. K. Perkins, N. S. Podolefsky, M. Dubson, N. D. Finkelstein, and C. E. Wieman, New instrument for measuring student beliefs about physics and learning physics: The Colorado Learning Attitudes about Science Survey, Phys. Rev. ST Phys. Educ. Res. 2, 010101 (2006).
- B. Efron, Bootstrap methods: Another look at the jackknife, Ann. Stat. 7, 1 (1979).