Abstract

Women continue to be underrepresented and less visible in the fields of science, technology, engineering, mathematics, and medicine (STEMM). 500 Women Scientists created and launched in January 2018 a global (>140 countries to date), online, open-access directory of women in STEMM fields. This directory—recently renamed gage—now also includes gender diverse persons (i.e., additional underrepresented genders) in STEMM fields. The purpose of the directory is to make these scientists’ expertise easier to locate and access for conference organizers, journalists, policy makers, educators, and others. Here, we undertake an assessment of the directory using surveys, Google Analytics, and focus groups to understand its efficacy and direction to date and identify future improvements we pledge to undertake. Through this assessment—conducted externally and in accordance with privacy protocols by Concolor Research—we identified who and how people are using our directory, why people signed up to be a resource, and areas for improvement. Through such assessment, we can learn how to enhance the directory’s efficacy and our broader efforts to boost the visibility of underrepresented people in STEMM.

Introduction

In many fields of STEMM (science, technology, engineering, math, medicine), white men dominate, including in the media and at key scientific events (Palermo et al. 2008; Shor et al. 2019; Roper 2019; Begeny et al. 2020; Makarem and Wang 2020). Decisions to predominately (or only) feature white men—as opposed to scientists with intersectional identities that are marginalized in society (Crenshaw 1989; Carbado et al. 2013; Collins 2015)—can perpetuate the belief that science is only for certain types of people (Loverock and Hart 2018; James et al. 2019; Makarova et al. 2019), thwarting the United Nations’ Sustainable Development Goals (SDG), including goal 5 of “achiev[ing] political, economic, and social equality for all women” (sdgs.un.org/goals/goal5). While this SDG is specific to women, other underrepresented gender identities are also struggling to thrive in STEMM (Restar and Operario 2019).
In response to the lack of diversity in public representations of experts in STEMM, different stakeholders have put in place a variety of strategies such as adding profiles of women to Wikipedia (Wade and Zaringhalam 2018), developing curricula featuring a diversity of scientist identities (Palermo et al. 2008; Roper 2019; Shor et al. 2019; Begeny et al. 2020; Makarem and Wang 2020), conducting bias literacy workshops (National Academies of Sciences 2020), featuring women scientists in television shows (National Academies of Sciences 2020), identifying barriers that women in science face (Handelsman et al. 2005), establishing discipline-specific databases (e.g., Diversify Chemistry, DiversifyEEB), and organizing grassroots social media campaigns (e.g., #BlackInSTEM, #DisabledInSTEM, #QueerInSTEM). As more specific interventions are developed and taken up, evaluation of these programs will be critical for identifying their strengths, areas for improvements, and opportunities to maximize outcomes and impact.
In this Note, we describe lessons learned from an evaluation of one grassroots, online tool for redressing gender imbalances in the selection of experts in STEMM—the Request a Woman in STEMM (Request) directory, recently renamed gage after Matilda Gage and the “Matilda Effect”, which refers to the phenomenon of ignoring, denying credit, or erasing women’s contributions in science (Rossiter 1993). We will refer to the directory as gage throughout this piece, though the assessment presented below was conducted prior to the name change and specific inclusion of gender diverse persons.

Gage directory

In January 2018, 500 Women Scientists—a grassroots, nonprofit organization with the mission to serve society by making science more open, inclusive, and accessible—launched Request a Woman in STEMM. The goal of this directory is to help diversify the face of science by developing a central location where conference organizers, educators, policymakers, and members of the media can find women and gender diverse persons in STEMM to contact for interviews, features, panels, and other forms of engagement (gage.500womenscientists.org/). While not the only directory of its kind (see 500womenscientists.org/related-resources), gage seeks to be comprehensive in its coverage of disciplines and expansive in its geographic representation. In other words, the mission behind gage is to be the largest global directory (now representing >140 countries) of women and gender diverse STEMM persons and experts. While we always strived to be inclusive of gender diverse persons in the Request directory, a big motivator for the name change to gage was to remove the word “woman” thus allowing for diverse identities to feel welcome in our directory. In the directory’s first year, more than 8000 people signed up (McCullagh et al. 2019) and by the end of 2020, the directory grew to more than 14 000 registered experts.

Methods

As the platform underwent this rapid expansion, several changes to the platform’s structure became necessary and were made. This included migrating the directory from a free tool to an improved user interface and user experience web platform designed by Critigen (Seattle, WA) to meet data protection standards and improve the user experience. During this time, the gage team began examining other impacts of the platform, in part through an assessment conducted by Concolor Research (concolorresearch.com/) and Google Analytics tracking. Results of this assessment are the focus of the manuscript and were used to design a major update to the then Request directory that was relaunched as gage in January 2021.
Assessment is an important step in this directory’s development because it provides a structured process for measuring the effects of an implemented program. Our assessment plan included a mixed-methods approach (Miller and Fredericks 2006), which combined online analytics, surveys, focus groups, and interviews to identify who was using the platform, why they were using it, and how the platform could be improved. Google Analytics for the directory were analyzed from the start of tracking on 12 January 2020 to when this piece was written on 31 December 2020.
Assessment focused on two groups of gage users: (i) “directory participants” who signed up as experts and (ii) “external users” of the directory. Qualitative and quantitative data about their experience with the directory were gathered through:
1.
Compiling Google Analytics to track the number of users and time spent on the website.
2.
Surveying (January–March 2020, survey questions available as Supplementary Material 1) directory participants to solicit general feedback, inquiring about who was contacted through the directory and why, and tracking outcomes of those contacts. Questions covered three topics: (i) contacts made through the directory, (ii) types of use and motivation for using the directory, and (iii) experiences with the directory. We received 704 responses out of approximately 12 000 people in the database, giving a response rate of approximately 6%.
3.
Conducting four focus groups of directory participants (total participants = 16, focus group questions available as Supplementary Material 2) who responded to the abovementioned survey and represented a range of disciplines, career stages, positions, and geographic locations. Focus group discussions covered reasons the directory participants joined gage, details of contacts that participants generated by being in and using the directory, recommendations for increasing impact of gage, and identification of major media outlets and conferences in their fields.
4.
Conducting interviews with nine external users (interview questions available as Supplementary Material 3). The interviews included questions about how they generally search for and contact scientists, their experience with the database both specific to the interaction with the scientist and in general in their work, and if they consider diverse expertise (gender, race/ethnicity, geographical region) when searching for scientists. To increase access for the public and invite more external users, no login is required to access data in gage. External users included students, nonprofit organizations, freelance journalists, and educators—they were sourced from contacts with the focus group participants.

Results

Who uses gage?

In 2020, gage was accessed by 7682 new users who spent an average of 5:59 minutes exploring the site—compared with Google Analytics’ benchmarking reported average of 5628 new users and 1:44 minutes on “science pages” during the same time period. “Science pages” are a defined benchmark for comparison built within Google Analytics with a similar number of average daily sessions (in this case 0–99). Our survey of directory participants showed that 16% (115 respondents) reported being contacted by an external user since joining the directory. Of those 115 scientists, 70% were contacted once, 21% were contacted twice, 7% were contacted three times, and 2% were contacted four or more times. Directory participants were most frequently contacted by members of the media, followed by preschool through secondary schools, and then organizers of conferences, meetings, or panels (Table 1).
Table 1.
Table 1. Roles of users contacting directory participants.
RoleNumber of contacts (% of total contacts)
Member of the media58 (43)
Educator from a preschool through secondary school20 (15)
Organizer of a conference, meeting, or panel11 (8)
Member of 500 Women Scientists leadership10 (7)
Affiliate of a university or institute of higher education9 (6.5)
Person from an organization other than educational or media8 (6)
Another woman scientist7 (5)
Associate of a government organization/institute2 (1.5)
Other (e.g., members of the public, organizer of a mentorship program)11 (8)

Why do STEMM experts join a public directory?

In the survey, directory participants reported two main motivations for joining: to be a resource to others and to amplify women’s voices in science. They described the directory’s most valuable attributes as amplifying women’s voices in science and society, providing a resource for finding other women, having a way to become an easily accessible resource to others, and increasing their own visibility.

How do directory participants and external users use the directory?

Approximately 33% of survey respondents reported using the directory to search for other women scientists and 67% reported not having searched the directory. Users’ most frequent use of gage was to find colleagues in their own field and geographic area (Table 2), highlighting two important features (discipline and location) to consider strengthening in the platform’s search functions.
Table 2.
Table 2. Participant uses of the directory.
Type of useNumber of uses (% of total number of uses reported)
To find potential colleagues in my own field138 (30)
To find women scientists in my geographic area irrespective of specialty125 (28)
To find women scientists in my specialty around the world113 (25)
To find a relevant expert in another field60 (13)
Other (e.g., to find research collaborators, head hunting for a professional position, to find speakers for a conference, to find reviewers for a peer-reviewed journal)18 (4)
We also assessed, through interviews, how external users used this resource. All external users interviewed were interested in promoting the voices of women scientists. Most external users had specific needs, such as looking for a speaker from a particular scientific field (e.g., biochemistry) or multiple speakers representing a range of disciplines. Many also wanted access to STEMM experts in a specific geographical area, and valued the “one-stop” single database representing many areas of scientific expertise. Most external users indicated that they typically find STEMM experts through word of mouth or their own extended networks (e.g., contacting faculty from relevant departments) and using Google searches.
Gage can be searched using numerous criteria, including scientific expertise, personal identity, geographical location, and by type of engagement sought (e.g., career and professional development, delivering a lecture, joining a panel, outreach, science activism, speaking to a journalist, etc.). The search bar on gage suggests entering keywords such as country, city, or discipline to find registered database users, but all keywords are searchable individually or as a list. Determining the available search categories has been an iterative process and additional options have been included in the recent relaunch of the platform. When registering to become a member of the database, directory participants are required to provide their name, email address, title, and geographical location. Adding other pieces of information—such as keywords that are professionally or personally descriptive—is optional. The directory’s search engine leverages profile content to identify registered users.

How can user feedback inform future directory development?

A key part of the assessment process is using identified areas of improvement to design evidence-based changes that help a program, product, or tool better reach its intended goals or purpose. When asked how the directory had met the expectations of directory participants, 47% of directory participants who responded to our survey reported feeling neutral, 29% said the platform was below their expectations, and 24% said the platform met their expectations. Respondents’ comments (Table 3)—including both negative and positive experiences—can be grouped into nine major categories: contact related, visibility of the directory, profiles, reminders, search, organization, uses, functionality, and representation. People who felt the directory had not met their expectations noted that they wanted more engagement, for example, feeling more connected to other scientists. Some respondents were also disappointed by the lack of contact initiated through the platform so far.
Table 3.
Table 3. Comments received describing expectations and improvements.
Type of CommentDescriptionFrequency (% of total comments)Example
Contact related
Described expectations with respect to contacts via the platform
73 (22)
“I was hopeful I would be contacted by people interested in my work through the platform, but I haven’t been. I’m curious what percentage of people registered have been contacted and what I can do to spread the word.”
Visibility of the platform
Commented on the need for greater visibility
67 (20)
“The challenge of most sites like this—we need people beyond the women scientists to know about its existence. For journalists, teachers, or conference organizers to use it, they need to know about it. Not sure how to fix [this], it is the core issue.”
Profiles
Gave feedback on their experience with the profile feature
39 (12)
“My language other than English is not mentioned (American Sign Language); some of my bibliography is older—and yet still relevant—than the internet and thus doesn’t have URLs.”
Reminders
Desire for more engagement and activity from the platform
33 (10)
“As it grows and more people join, it will become more valuable. Also need reminders 1/quarter or something to check the platform, otherwise I tend to forget to check because we are all so busy.”
Search
Described issues or positive aspects of the search performance
33 (10)
“I am able to find other women scientists very easily, it makes initiating collaborations easier.”
Organization of the platform
Described positive and negative experiences of the organization of the platform
27 (8)
“This system is easy to use and reliable—even the teachers seem happy to use it and support expanding its use in classrooms.”
Uses
A broad category to capture what people have used or plan to use the platform for
23 (7)
“I was impressed that a regional organizer knew to go to this site to find potential speakers/participants.”
Functionality
Comments on issues or positive aspects of platform functionality
21 (6)
“Filters should be checked. Switzerland, for example, is not in the filter Location when I look for colleagues in my field. However, going by hand on to Switzerland, I found one.”
Representation
Identified need for additional representation (e.g., age, discipline) in the platform
12 (5)
“I was surprised how few female scientists in my region are registered.”
Gaps were further explored in focus groups with directory participants who identified several areas for improvement. Participants suggested strategic steps to increase awareness and use of the directory, such as featuring stories about gage interactions/outcomes with external users, sharing member profiles to highlight expertise, and targeting outreach to specific potential external users such as educational groups, the media, conference organizers, and journal editorial boards. Additional feedback showcased a need for professional development resources (e.g., when to ask for compensation, how to talk to the media, etc.) and for clearer expectations from users of the directory during their communications about engagement (e.g., receiving questions in advance of an interview, information about travel expectations, acknowledgement of use of the directory in publications, etc.). In addition, the focus group participants recommended increasing engagement with directory members, including through social media, newsletters, and stories of directory interactions to boost the likelihood of repeated usage of the directory.
While these recommendations were specifically made with the gage directory in mind, there are some guiding questions that stem out of gage’s assessment that may be more broadly applicable to similar types of public resources with the goal of improving interactions with and within an increasingly diverse scientific workforce. There are some guiding questions that stem out of gage’s assessment that may be more broadly applicable for designers to ask themselves when creating similar types of public resources. For example, which methods might be more effective in increasing awareness and use of a platform? What resources might different stakeholders need to maximize their satisfaction and use of a platform? Who is left out of the conversation and how might we bring them in? How do we better support links between a platform’s different user types?

Conclusion

Moving forward for gage

The gage directory is one of many initiatives borne out of a need to increase gender diversity and visibility in STEMM and challenge assumptions of what a scientist looks like. For this directory—and other resources and programs aimed at addressing inequity in STEMM—to reach its full potential, it is critical to take stock of what works and what needs improving. Assessment through “Monitoring, Evaluation, Learning” (MEL) is one way we can ensure that the gage directory is fulfilling its intended goals. For example, our assessment indicated that gage connected directory participants with members of the media, educators, students, conference organizers, and other scientists. But to our knowledge, policy makers are not using the directory yet, and this is an area we would like to strengthen.
Gage, like other technologies or knowledge infrastructures, is “value-laden, privileging some interests and disenfranchising others”—which means that the (re)design and use of gage has important implications for shaping future labor and access conditions around knowledge sharing (Hammond et al. 2020). A consequence we would like to avoid with this directory is creating additional onus or pressure on women and gender diverse persons to fulfill service roles, knowing that their involvement in, for example, membership on committees and advising of students, surpasses that of men (Park 1996; Guarino and Borden 2017; Hanasono et al. 2019). A key part of gage’s mission is to make it simple for opportunities to present themselves to participants who are seeking them, but not to overburden them.
Moving forward, we will be implementing many of the suggested changes to our directory including increasing our engagement with directory members through offerings specifically for gage members (such as workshops and trainings), newsletters, and social media campaigns; highlighting connections made between external users and directory members; developing tools for improving capacity in science communication for directory members; and improving directory search capabilities. While visual impairments were considered in the design of the new platform during the 2020 transition and the gage rebrand, we are continuing to improve the directory’s accessibility. In addition, we are seeking additional ways to be more inclusive of gender diverse persons (Restar and Operario 2019) and expanding outreach and recruitment of underrepresented disciplines, other historically marginalized social identities, and neglected geographic areas. As we implement these changes, we plan to continue monitoring, evaluating, and learning to determine other areas for improvement so that the gage platform can more effectively meet the needs of users and help transform the face of STEMM.

Acknowledgements

We would like to acknowledge the funders of the gage directory including Simons’ Science Sandbox and Lyda Hill Philanthropies. We thank all the members of the gage directory to date and are grateful to those who participated in our evaluation of the directory.

References

Begeny CT, Ryan MK, Moss-Racusin CA, and Ravetz G. 2020. In some professions, women have become well represented, yet gender bias persists—perpetuated by those who think it is not happening. Science Advances, 6(26): eaba7814.
Carbado DW, Crenshaw KW, Mays VM, and Tomlinson B. 2013. Intersectionality. Du Bois Review: Social Science Research on Race, 10(2): 303–312.
Collins, PH. 2015. Intersectionality’s definitional dilemmas. Annual Review of Sociology, 41(1): 1–20.
Crenshaw K. 1989. Demarginalizing the intersection of race and sex: a black feminist critique of antidiscrimination doctrine, feminist theory and antiracist politics. University of Chicago Legal Forum, 1: 8.
Guarino CM, and Borden VMH. 2017. Faculty service loads and gender: are women taking care of the academic family? Research in Higher Education, 58(6): 672–694.
Hammond JW, Moss PA, Huynh MQ, and Lagoze C. 2020. Research synthesis infrastructures: shaping knowledge in education. Review of Research in Education, 44(1): 1–35.
Hanasono LK, Broido EM, Yacobucci MM, Root KV, Peña S, and O’Neil DA. 2019. Secret service: revealing gender biases in the visibility and value of faculty service. Journal of Diversity in Higher Education, 12(1): 85–98.
Handelsman J, Cantor N, Carnes M, Denton D, Fine E, Grosz B, et al. 2005. More women in science. Science 309(5738): 1190–1191.
James A, Chisnall R, and Plank MJ. 2019. Gender and societies: a grassroots approach to women in science. Royal Society Open Science, 6(9): 190633.
Loverock B, and Hart MM. 2018. What a scientist looks like: portraying gender in the scientific media. FACETS, 3(1): 754–763.
Makarem Y, and Wang J. 2020. Career experiences of women in science, technology, engineering, and mathematics fields: a systematic literature review. Human Resource Development Quarterly, 31(1): 91–111.
Makarova E, Aeschlimann B, and Herzog W. 2019. The gender gap in stem fields: the impact of the gender stereotype of math and science on secondary students’ career aspirations. Frontiers in Education, 4.
McCullagh EA, Nowak K, Pogoriler A, Metcalf JL, Zaringhalam M, and Zelikova TJ. 2019. Request a woman scientist: a database for diversifying the public face of science. PLoS Biology, 17(4): e3000212.
Miller SI, and Fredericks M. 2006. Mixed-methods and evaluation research: trends and issues. Qualitative Health Research 16(4): 567–579.
National Academies of Sciences, Engineering. 2020. Promising practices for addressing the underrepresentation of women in science, engineering, and medicine: opening doors. The National Academies Press.
Palermo S, Giuffra E, Arzenton V, and Bucchi M. 2008. Gender and science. EMBO Reports 9(6): 494–495.
Park SM. 1996. Research, teaching, and service: why shouldn’t women’s work count? The Journal of Higher Education, 67(1): 46–84.
Restar AJ, and Operario D. 2019. The missing trans women of science, medicine, and global health. The Lancet 393(10171): 506–508.
Roper RL. 2019. Does gender bias still affect women in science? Microbiology and Molecular Biology Reviews: MMBR, 83(3).
Rossiter MW. 1993. The Matthew Matilda Effect in Science. Social Studies of Science 23(2): 325–341.
Shor E, van de Rijt A, and Fotouhi B. 2019. A large-scale test of gender bias in the media. Sociological Science, 6: 526–550.
Wade J, and Zaringhalam M. 2018. Why we’re editing women scientists onto Wikipedia. Nature, August.

Supplementary materials

Supplementary Material 1 (PDF / 154 KB)
Supplementary Material 2 (PDF / 60.4 KB)
Supplementary Material 3 (PDF / 51.9 KB)

Information & Authors

Information

Published In

cover image FACETS
FACETS
Volume 6Number 1January 2021
Pages: 1601 - 1609
Editor: Imogen Coe

History

Received: 24 March 2021
Accepted: 23 June 2021
Version of record online: 16 September 2021

Data Availability Statement

All relevant data are within the paper and in the Supplementary Material.

Key Words

  1. gender equity
  2. STEMM
  3. online communities
  4. directory
  5. program assessment
  6. MEL

Sections

Subjects

Authors

Affiliations

Elizabeth A. McCullagh [email protected]
Department of Integrative Biology, Oklahoma State University, Stillwater, OK 74078, USA
Member, 500 Women Scientists
Francesca Bernardi
Member, 500 Women Scientists
Department of Mathematical Sciences, Worcester Polytechnic Institute, Worcester, MA 01609, USA
Monica Malta
Member, 500 Women Scientists
Institute for Mental Health Policy Research, Centre for Addiction and Mental Health, Toronto, ON M6J 1H4, Canada
Department of Psychiatry, Faculty of Medicine, University of Toronto, Toronto, ON M5S 1A1, Canada
Katarzyna Nowak
Faculty of Agricultural, Life, and Environmental Sciences, University of Alberta, Edmonton, Alberta T6G 2H1, Canada
Yukon University, 500 University Drive, Whitehorse, YT Y1A 5K4, Canada
The Safina Center, 80 North Country Road, Setauket, NY 11733 USA
Alison R. Marklein
Member, 500 Women Scientists
Department of Environmental Sciences, University of California – Riverside, Riverside, CA 92507, USA
Katie Van Horne
Concolor Research, Louisville, CO 80027, USA
Tiffany Lee Clark
Concolor Research, Louisville, CO 80027, USA
Susan J. Cheng
Member, 500 Women Scientists
Department of Ecology and Evolutionary Biology, University of Michigan, Ann Arbor, MI 48109, USA
Center for Research on Learning and Teaching, University of Michigan, Ann Arbor, MI 48109, USA
Maryam Zaringhalam
Member, 500 Women Scientists
Lauren L. Edwards
Member, 500 Women Scientists

Author Contributions

EAM, KVH, and TLC conceived and designed the study.
KVH and TLC performed the experiments/collected the data.
EAM, KVH, and TLC analyzed and interpreted the data.
LLE contributed resources.
All drafted or revised the manuscript.

Competing Interests

The authors have declared that no competing interests exist.

Metrics & Citations

Metrics

Other Metrics

Citations

Cite As

Export Citations

If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.

There are no citations for this item

View Options

View options

PDF

View PDF

Get Access

Media

Media

Other

Tables

Share Options

Share

Share the article link

Share on social media