Let’s do better: public representations of COVID-19 science

Since the start of 2020, tens of thousands of peer-reviewed academic articles and preprints on COVID-19 entered the public domain (Natureindex.com 2020). Submission rates to prestigious biomedical journals have increased substantially, with some journals receiving triple the usual number of submissions (Bauchner et al. 2020).

COVID science is being both done and disseminated at a furious pace. Currently, the median time from the submission of an article to acceptance is just six days (Palayew et al. 2020). That is an astonishing increase in pace of acceptance from the prepandemic speed of around 100 days. And some publications made it through peer review in just one day (Locher et al. 2020).

Wanting science to happen quickly during a pandemic is understandable (Gleick 2020). While it is inspiring to see the research community responding so vigorously to the pandemic crisis, all this activity has also added to a chaotic information environment by injecting bad data, conflicting results, and hyped headlines (Jaklevic 2020). One day a study, published in a renowned biomedical journal, is being hailed as definitive data that should (and does) guide our actions and policies (Sattui et al. 2020). The next day that same study is retracted (Joseph 2020) (or being asked to be retracted) (Mandavilli 2020).

With representations of science becoming increasingly polarized, twisted, and hyped, there is growing concern that the science is being represented to the public in a manner that may cause confusion, inappropriate expectations, and the erosion of public trust (Saitz and Schwitzer 2020).

Here we explore some of the key issues associated with the representations of science in the context of the COVID-19 pandemic. This is not meant to be a comprehensive analysis of how science is prioritized, done, incentivized, and evaluated (Council of Canadian Academies 2010)—though we will touch on those topics. Rather, we consider some of the sources and impact of problematic representations of COVID-19 science—including the potential to compromise public trust and public health initiatives. Many of these issues are not new. But, as we outline below, the COVID-19 pandemic has both placed a spotlight on the health research process and amplified the adverse ramifications of poor public communication.

While there are many examples of less-than-ideal representations of COVID-19 science, the hydroxychloroquine controversy stands as a good illustration of both the ways in which things can go wrong and the myriad harmful ramifications of those missteps (Sattui et al. 2020).

In mid-March, 2020, French researchers published a preprint suggesting that hydroxychloroquine had potential therapeutic benefits in the treatment of COVID-19 (Gautret et al. 2020). The study was small (n = 80), open-labelled, and almost immediately criticized as being too methodologically flawed to justify publication in a peer-reviewed journal (Voss 2020). Indeed, a later academic review of the study suggested it was “a non-informative manuscript with gross methodological shortcomings” (Rosendaal 2020). As a direct result of this study (which, at the time of this writing, has already and unfortunately been cited over 1700 times) and despite these scientific concerns, hydroxychloroquine started to receive a great deal of positive media attention and endorsements from prominent individuals like Elon Musk and Donald Trump. This drove up public interest in the drug. Indeed, one study found that Internet searches spiked as a result of these endorsements (M. Liu et al. 2020) and, more worrisome, so did off-label prescriptions by MDs (Vaduganathan et al. 2020).

As the hydroxychloroquine story unfolded—and the hype gained steam (Thompson 2020)—more and more researchers around the world started investigating the drug, despite the fact that there was (and remains) little evidence to suggest significant (or any) clinical utility (Boulware et al. 2020; Kupferschmidt 2020; Skipper et al. 2020; University of Oxford—News 2020). Then, in mid-June, a large study—published in the influential journal The Lancet—concluded that the drug had the potential to cause significant adverse events. The study prompted an immediate response from the international research community, including causing clinical trials to be temporarily stopped due to safety concerns. Noting anomalies in the data set used to support the conclusions of harm, many other scientists almost immediately criticized the study, and it was quickly retracted (Mahase 2020b).

The hydroxychloroquine story continues to evolve (Gonsalves 2020). More evidence—including both observational and clinical trials of varying methodological strengths—has been produced and suggests the drug is unlikely to be beneficial in the context of COVID-19 (Qaseem et al. 2020). Indeed, because of the lack of compelling evidence, key clinical trials have been discontinued, including trials sponsored by the National Institutes of Health (Kiley 2020) and the World Health Organization (WHO; WHO Newsroom 2020). In addition, concerns about significant side effects remain (Downes et al. 2020; U.S. Food & Drug Administration 2020).

There are, of course, other examples of both poor science and poor science communication in the context of COVID-19 related research (Day 2020; Schwitzer 2020). But the hydroxychloroquine controversy—which took flight due to public representations and celebrity endorsements of questionable preliminary research—highlights the breadth of adverse outcomes that can emerge when science is communicated poorly, including injecting inefficiencies into COVID-19 research efforts (e.g., making it more difficult to recruit participants into well-designed clinical trials) (Ledford 2020), spurring questionable public investment in further research (Herper and Riglin 2020), causing poor pharmaceutical allocation decisions (Mahase 2020a) and resource shortages (hydroxycholorquine shortages—potentially affecting people with chronic immunologic conditions—were reported in most Canadian provinces) (Mendel et al. 2021), encouraging unnecessary and potentially harmful prescriptions (Lovelace 2020), and creating unjustified and heightened public expectations (e.g., 23% of Canadians—and 30% of Quebecers—wrongly believe the drug is effective) (Everts and Greenberg 2020). The initial hype and subsequent retracted research results have fed conspiracy theories (Mikkelson 2020), been used to polarize public discourse, and perhaps most problematic, may have contributed to a decrease in public trust of science (Laurent 2020).

Despite the mounting evidence that it does not work, a belief in hydroxychloroquine endures for some—in part because it has become associated with a particular ideological position due to the connection with prominent politicians like US President Trump (Dearment 2020). Once a belief becomes part of an individual’s personal identity, it can be very difficult to change their mind (Kaplan et al. 2016).

Building and maintaining public trust—including in science and scientific institutions—is particularly critical during a pandemic (Balog-Way and McComas 2020; Udow-Phillips and Lantz 2020). For example, there is some evidence (albeit observational in nature) that correlates trust in relevant institutions with the likelihood of engaging in the needed preventative behaviours (Devine et al. 2020; Fukuyama 2020; Goldberg et al. 2020; Lep et al. 2020). An analysis from France, for instance, found that “high-trust regions decrease their mobility related to non-necessary activities significantly more than low-trust regions” (Bargain and Aminjonov 2020). This study is consistent with research from Australia that correlated trust in government and health authorities with greater adoption of physical distancing and hygiene-related behaviours (Seale et al. 2020). And research from Johns Hopkins University that looked specifically at distrust of science found that it was strongly correlated with a failure to adopt preventative strategies—like physical distancing—and, unsurprisingly, listening to public health experts’ advice on COVID-19 (Barry et al. 2020). A lack of trust in relevant institutions has also been found to be associated with an increased belief in COVID-19 myths and conspiracy theories (Pickles et al. 2020).

The Canadian public is following the developments surrounding the pandemic very closely. A June 2020 survey by Carleton University, for example, found that 82% of Canadians are monitoring COVID-19 news “every day” (53%) or “most days” (29%) (Greenberg and Everts 2020). As a result, the public are observing the messy but usually concealed process of building scientific understanding (which can include missteps) and the shifting nature of scientific consensus. And they are seeing the dramatic headlines, the retractions, and the disagreements among scientific experts. As a result, there is concern that the many less than ideal representations of science will erode public trust (Drage O’Reilly 2020), especially as the pandemic drags on and global sentiment shifts, as some research indicates, from fear towards frustration and anger (Lwin et al. 2020).

A recent study from the London School of Economics suggests that the COVID-19 crisis may have a negative impact on people’s perceptions of scientists, especially among those in the public with little or no scientific education (Aksoy et al. 2020a). Specifically, the researchers conclude—using past pandemics as a guide—that the COVID-19 crisis “will reduce confidence in individual scientists, worsen perceptions of their honesty, and weaken the belief that their activities benefit the public” (Aksoy et al. 2020a). Another study, a survey from France, found a 10% drop in trust in science—driven mostly by frustration surrounding the two polarizing topics: the hydroxychloroquine debacle and policy reversals surrounding the use of masks (Matthew 2020).

To date, the public discourse in Canada around COVID-19 has not been as polarized as we have seen in the United States and in some other countries (Merkley et al. 2020; Padilla and Hípola 2020). Trust in our health and science institutions remains relatively high (Statistics Canada 2020). If asked, most Canadians will still say they have confidence in academics, health care providers, and public health officials (Statistics Canada 2020). Still, we should not be complacent (Robinson 2020). There is some evidence—prepandemic—that trust in science is falling and that many Canadians view the scientific community as elitist (Ontario Science Centre 2017; Semeniuk 2018; Weber 2019). Other research has suggested the pandemic could result in a general and long-term erosion of trust in public institutions (Aksoy et al. 2020b). And there are complex trust issues to be considered in the context of particular populations—particularly those such as Indigenous peoples (Government of Canada 2019; Kolopenuk 2020)—that have been poorly served or even harmed by existing research institutions.

Trust can be lost quickly—and with dire consequences (Robinson 2020). Indeed, the public perception of science and issues of trust are likely to become even more salient in the context of vaccine uptake. Studies have found that a lack of trust in science is associated with decreased intention to get a COVID-19 vaccine (Palamenghi et al. 2020). Many in Canada already have concerns about any vaccine and hesitancy is on the rise. An August 2020 survey found that only 46% of Canadians agree that they “would get a vaccination as soon as one become available to me” and three in five Canadians worry about safety (Angus Reid Institute 2020). Rhetoric from the anti-vaccine community is clearly having an adverse impact on public perceptions, in part because it leverages (and stokes) concerns about the adequacy of relevant science (Crow and Stacey 2020). Given fear about the potential for political interference with the vaccine research process—particularly in the United States (LaFraniere et al. 2020)—the social and health issues associated with the erosion of trust in scientific institutions and science-informed policy decisions seem likely to intensify. Indeed, an August 2020 survey found that “78% of Americans worry the COVID-19 vaccine approval process is being driven more by politics than science” (Silverman 2020).

Obviously, an essential component in the creation of accurate, balanced, and trustworthy representations of science is to ensure that the science is done well and in a manner that is transparent, which includes ensuring public access to the relevant data (e.g., disposition, publicly accessible repositories). There is growing concern that the pressures associated with production and dissemination of COVID-19 science is leading to poor-quality science (Dinis-Oliveira 2020). One of the fastest ways to create confusion and lose public trust is to publish and publicize weak, careless, or worse, fraudulent research (Yarborough 2014). Unfortunately, this may be happening too often in this era of panicky, pandemic publishing (Retraction Watch 2020; Steinberg 2020)—which, as one commentator suggests, has created “a deluge of poor quality research” that is “sabotaging an effective evidence based response” (Glasziou et al. 2020). This includes high profile and frequent retractions of peer-reviewed journal articles (Yeo-Teh and Tang 2021), though it is still unclear how unusual—if at all—the retraction rate is (Abritis et al. 2021). (As of this writing, Retraction Watch, an entity that monitors this kind of activity, reports 36 retracted COVID-19 studies.)

While a detailed analysis of existing research institutions and incentive structures is beyond the scope of this paper, it seems axiomatic that maintaining integrity of the research process should be a priority. During a pandemic there is an understandable sense of urgency (Tingley 2020). But the desire for quick results should not be allowed to erode scientific standards (Pang and Elkhodiry 2020). As succinctly put by Alex John London and Jonathan Kimmelman, “Crises are no excuse for lowering scientific standards” (London and Kimmelman 2020). And this point was echoed by H. Clifford Lane and Anthony Fauci, “scientifically robust and ethically sound clinical research remains the quickest and most efficient pathway to effective treatment and prevention strategies for patients with COVID-19” (Lane and Fauci 2021). Unfortunately, there is some evidence that much of research that is currently being done—as measured by an analysis of registered clinical trials—is expected to produce only a “low level of evidence” as there are few high-quality randomized controlled trials and, as a result, “most studies likely will not yield meaningful scientific evidence” (Di Girolamo and Meursinge Reynders 2020; Pundi et al. 2020).

The push for speed can also create problems for how the relevant science is published and represented to the public and decision-makers. There is, for example, concern that the huge volume of paper submissions is straining the peer-review process—as highlighted by the paper that started the hydroxychloroquine controversy (Locher et al. 2020)—and that “weak, or even wrong, findings disseminate, amplify, and potentially enter into scientific and popular discourse” (Bell and Green 2020). While some of this work has been quickly retracted (Retraction Watch n.d.), once the work has been circulated by the popular press and on social media, it can be hard to undo the damage—including adversely impacting public trust (Wysong 2020).

As a result of these concerns, there have been recommendations about how to improve the publication and peer-review process (Bauchner et al. 2020), such as establishing new editorial standards to maintain quality during public health emergencies and requiring peer reviewers to be adequately trained (Bazdaric and Smart 2020; Palayew et al. 2020). Some journals—including The Lancet, the publication involved in one of the recent high-profile COVID-19 related retractions (Medical Xpress 2020; Rabin 2020)—have already suggested that the peer-review process will need to be adjusted to ensure greater scrutiny of the relevant methods and data (Caulfield 2020a).

Addressing the explosion of preprints—that is, the distribution of research prior to peer review—is also critical (Van Schalkwyk et al. 2020). Preprints can be a valuable tool for the dissemination of data and for generating constructive critiques from colleagues. (Most journals, including 80% of the highest impact journals, allow preprint dissemination prior to submission (Massey et al. 2020).) But preprints can also result in the circulation of unverified and poor research in a manner that can confuse public discourse (the hydroxychloroquine issue started with a preprint). Preprints are being produced and accessed at an incredibly high pace during the pandemic. And they are having an impact on public and policy discourse (Majumder and Mandl 2020). One study (a preprint about preprints, ironically) found that the pandemic has resulted in an increased academic, public, and news media engagement with preprints. For example, they found “COVID-19 preprints are accessed and distributed at least 15 times more than non-COVID-19 preprints” (Fraser et al. 2020). As a result, some scholars are striving to develop an informal and publicly accessible, rapid review of preprints that will help to inform both the public and policymakers (Eisen and Tibshirani 2020).

How scientists communicate their work to the media—on social media and directly to the public—also requires consideration. How work is shared on social media, for example, can shape both subsequent citations to the work but also public and policy discourse (Kousha and Thelwall 2020). There are growing pressures on the scientific community to present their work in overly enthusiastic terms. Indeed, there are forces and incentives throughout the knowledge creation process that can encourage hyped representations of science (Bubela 2006; Bubela et al. 2009; Caulfield and Condit 2012), from the submission of grants (Matthews 2016), to the write up of results (Vinkers et al. 2015), to the crafting of institutional press releases (Yavchitz et al. 2012), to the interactions with the popular press (Kamenova and Caulfield 2015). And as we have seen in other domains, this hype (Ball 2015) can have a profound impact on public understanding, science and health policy (Caulfield 2018), the marketing of associated products and therapies (Caulfield et al. 2016), and perhaps (Master and Resnik 2013), public trust (Resnick 2019).

It is essential for the scientific community to remain part of the public conversation, including challenging misrepresentations and spin used to further polarize public perceptions. But it is also essential for those in research community to portray their work in a measured and accurate manner (Leeming 2018), including reflecting on limitations of the work and how it fits in the broader body of evidence.

It is equally important for a wide range of communities to be meaningfully engaged in the scientific conversation, especially for research that informs public health interventions (Tworek et al. 2020). These interventions have both intended and unintended consequences, and the economic, social, and health burdens are unequally distributed. In the context of COVID-19 there is evidence that some communities are experiencing disproportionate disease burden and, at the same time, have increased levels of distrust toward, for example, the vaccine research process (Hoffman 2020). Research best practices in health and social science domains have increasingly integrated the voices of community partners and patients, from the inception of research questions, methodological design, research conduct, and the interpretation and dissemination or communication of results. This last point is imperative when scientific findings may be interpreted in a manner that leads to increased stigma or overt racism against individuals, communities, or populations. This philosophy of public engagement is enshrined in Canada’s Tri-Council Policy Statement: Ethical Conduct for Research Involving Humans (CIHR et al. 2018) and the national Strategy for Patient-Oriented Research (CIHR and Government of Canada 2019). The recognized need for engagement is based on the premise that public trust may be enhanced if those most impacted by the research are active partners. While beyond the scope of this paper, Indigenous health research goes one step further to be increasingly led and controlled by Indigenous communities (FNIGC 2021).

Public health authorities—regional, national, and international—are a vital source of scientific information during a pandemic. While controversy has surrounded some of the recommendations that have flowed from entities like the WHO, the US Centers for Disease Control and Prevention (CDC), and the Public Health Agency of Canada, clinicians, health care institutions, educators, the public and politicians turn to public health authorities for both updates on emerging evidence and recommendations on how best to proceed (Carleton Newsroom 2020; Goldberg et al. 2020). As such, it is critically important that communication of science is done in a manner that maintains public trust in both the science and the relevant institutions.

Public health authorities should, for example, be honest and clear about the state of the science used to inform recommendations (Leask 2020; Mello et al. 2020; Robinson 2020). This includes “being transparent and open about what is known and unknown about SARS-CoV-2 virus and COVID-19 disease” (Pak and Adegboye 2020). Unsupportable or oversimplified dogmatic pronouncements of benefit or harm—no matter how noble the justification—only help to feed a polarization process that, long-term, seems likely to do real damage to public trust and the perception of science and scientists. Attention must also be paid to the mode of communication—visual media are distinct from print sources, necessitating spokespeople, settings, and congruence in messaging for clarity and maintenance of public confidence (Luth et al. 2013).

As mentioned previously, the evolving recommendations about the use of masks (Zhang et al. 2020) in public has been pointed to as a possible engine of public distrust (Urback 2020). Commentators have claimed that this evolution in guidelines—or the “flip flop”, as those critical of the mask policies have labelled it (Toronto Sun 2020)—has facilitated a reduction in trust in public health authorities (Gerson 2020). In such situations, public health authorities should not shy away from being frank about the equivocal and changing nature of the evidence. As noted by public health experts, Rutter et al. (2020), during a pandemic “most data will be flawed or incomplete” and we need to “be honest and transparent about this”. Of course, this is how science almost always unfolds. This situation is not unique to the pandemic. As such, ensuring that the public understands the nature of scientific research and knowledge translation process is also critically important.

During a pandemic, public health decisions often need to be made using a less than ideal body of evidence (Greenhalgh 2020). And recommendations that are based on emerging science will (and should) evolve. Revising a position as new evidence and (or) social conditions change should not be viewed as a failure of the system (Dupré 2020). While it is understandable that public health officials may be tempted to provide strong and unequivocal messaging, it is important to be explicit about the ambiguities of the evidence. Indeed, there is some evidence that being transparent about uncertainties can actually heighten credibility (Ratcliff et al. 2018), trust (Fleerackers 2020), and public understanding (Jensen et al. 2011; Porter 2020). Public health authorities can provide a clear and actionable message that mobilizes our shared values in a manner that still accurately reflects the available science. Indeed, as noted by science communication expert Dominique Brossard, “at the end of the day, it’s better to say ‘the best practice is this, although we’re not 100% sure and we’ll let you know as soon as we know more’ ” (Drage O’Reilly 2020).

Some have suggested that it is important to prime the public with supportable rationales as to why additional preventative strategies may be required, including details about evidence and goals (Seale et al. 2020). And public health entities also need to use a wide range of communication platforms, especially social media, to ensure that science-informed messages play a dominant role in public discourse (Lovari 2020). This may include working with social media platforms to facilitate the “upranking” of “links to recommendations from recognised health authorities” (Limaye et al. 2020).

Perhaps most worrisome is the issue of political influence. Decisions by science-based health institutions must be done in an independent manner and devoid of politically motivated interference. To do otherwise can greatly compromise the ability for these institutions to have an impact on public health. Recent action by the US Food and Drug Administration (e.g., the messaging and questionable approval of convalescent plasma treatment) (Kupferschmidt and Cohen 2020; McGinley et al. 2020) and the CDC (e.g., the policy change on the testing of asymptomatic individuals) (Sheridan 2020; Troisi 2020) has highlighted how political interference can impact both public trust and the public representations of science (Wilson 2020).

How the media cover science, particularly during a pandemic (Gozzi et al. 2020; Q. Liu et al. 2020), is also important. It can have an impact on public perceptions and attitudes (Zheng et al. 2020), policy development, clinical practice, and research priorities. And news coverage can facilitate the spread of misinformation and the polarization of public discourse (Green et al. 2020).

Those working in the popular press—whether for TV, radio, print media, or online sources—should take care not to hype or misrepresent science, including the certainty of a result (Abbas and Lamb 2020; Strazewski 2020). True game-changing breakthroughs are vanishingly rare (for example, fewer than 10% of experimental drugs that are promising enough to be in a clinical trial will be approved for clinical use) (Lowe 2019). The reality is that scientific research is an iterative and, in general, slow process. The media, however, prefer definitive pronouncements of near future benefit.

There have been numerous journalist organizations that have emphasized the importance of accurate and measured reporting (First Draft 2020; Hanage and Lipsitch 2020; Mulcahey 2020; Science Media Centre of Canada 2020). Still, much of the reporting has been less than ideal and, as highlighted by the hydroxychloroquine situation, with significant ramifications. As noted in an analysis by science communication experts Saitz and Schwitzer (2020), the news media too often focus on (and hype) a single study and (or) overemphasize the potential meaning of the results without putting the research in the context of the existing available evidence.

Some have also argued that the content of what the media have covered, especially in the early days of the pandemic, was problematic. One study, for example, found that news TV coverage mostly emphasized death and death rates and said little about the science surrounding preventative behaviours (Basch et al. 2020).

While the news media can certainly improve their practices, it shouldn’t be forgotten that much of the misinformation and hype that appears in the press comes from researchers and research institutions (Caulfield and Condit 2012; Woolston 2014). There is a relationship between how research is represented in, for example, press releases—which often hype research results—and how the science is represented to the public. And, of course, much of the misinformation about the COVID-19 science is happening on social media. It is being created and circulated not by professional journalists but by the users of social media platforms. Indeed, social media has been identified as a primary driver of COVID-19 misinformation (and those who get their news from social media are more likely to believe misinformation) (Bridgman et al. 2020; Caulfield 2020b). Still, the popular press remains an important source of pandemic information and can have a significant impact on how the science is perceived and utilized.

Science has always been under various external pressures, including ideological mandates (Baran et al. 2019), military and national defence demands (Finkbeiner 2018), and the ever-present profit motive. And, of course, the incentive structures built into academia—rewarding publication quantity and “impact factors” over quality and social benefit (Plackett 2020)—also shape, for better or worse, the research enterprise. The policy decisions that are made by governments, funding agencies, and research institutions about how to support and fund research—including the commercialization push that has been embraced by almost every Canadian research funding entity (Caulfield and Ogbogu 2015)—help to influence how that research is done and represented to the public.

We recognize that a deep reflection on these complex, interrelated, and systemic influences on public representations of research is likely warranted. We also recognize that improving representations of science will not, on its own, necessarily lead to better policies and a more informed public. These are big and complex challenges. Our goal here, however, is narrower and is focused on several of the key actors involved in the communication of COVID-19 science (Fig. 1). How science has been communicated during this pandemic has had an impact on public perceptions, health and science policy, and the uptake of preventative strategies. But the communication problems that have unfolded during this public health crisis are not new. Indeed, in many ways they have served to highlight the adverse impact of many long-standing concerns about how science is being communicated including, inter alia, issues associated with interpretation of research results, the publication process, press releases, and media coverage. As such, we offer broad recommendations that we believe will have relevance beyond this pandemic.

We would like to thank the entire Royal Society of Canada Task Force on COVID-19 for their support and comments and Darren N. Wagner for his insight and excellent editorial help.