Original research•

Analysis of tweets discussing the risk of Mpox among children and young people in school (May–October 2022): a retrospective observational study

Abstract

Objective To determine the number of tweets discussing the risk of Mpox to children and young people in school and (1) determine accuracy, (2) for inaccurate tweets, determine if risk was minimised or exaggerated and (3) describe the characteristics of the accounts and tweets which contained accurate versus inaccurate information.

Design Retrospective observational study.

Setting Twitter advanced search in January 2023 of tweets spanning 18 May 2022–19 September 2022.

Participants Accounts labelled as: MD, DO, nurse, pharmacist, physical therapist, other healthcare provider, PhD, MPH, Ed. degree, JD, health/medicine/public policy reporter (including students or candidates) who tweeted about the risk of Mpox to children and young people in school.

Exposures Tweets containing the keywords ‘school’ and ‘mpox’, ‘pox’, or ‘monkeypox’ from May to October 2022.

Measures (1) The total and ratio of accurate versus inaccurate tweets, the latter further subdivided by exaggerating or minimising risk, and stratified by account author credential type. (2) The total likes, retweets and follower counts by accurate versus inaccurate tweets, by month and account credentials. (3) Twitter user exposure to inaccurate versus accurate tweets was estimated.

Results 262 tweets were identified. 215/262 (82%) were inaccurate and 215/215 (100%) of these exaggerated risks. 47/262 (18%) tweets were accurate. There were 163 (87%) unique authors of inaccurate tweets and 25 (13%) of accurate tweets. Among healthcare professionals, 86% (95/111) of tweets were inaccurate. Multiplying accuracy by followers and retweets, Twitter users were approximately 974× more likely to encounter inaccurate than accurate information.

Conclusion Credentialed Twitter users were 4.6 times more likely to tweet inaccurate than accurate messages. We also demonstrated how incorrect tweets can be quickly amplified by retweets and popular accounts. In the case of Mpox in children and young people, incorrect information always exaggerated risks.

What is already known on this topic

  • Twitter (now X) is used by health professionals to relay information about a wide range of topics; however, the accuracy of this information is unknown.

What this study adds

  • Tweets about Mpox in children and young people in school from credentialed sources were inaccurate 82% of the time overall and 86% of the time when coming from healthcare professionals. Inaccurate tweets had more likes, shares and were from accounts with higher mean follower counts than accurate tweets.

How this study might affect research, practice or policy

  • Our study raises concerns about relying on X (formerly Twitter) for accurate information about emerging diseases or health threats even from accounts of healthcare professionals. The extent to which our findings apply to other emerging health threats is beyond the scope of this paper, but it is advisable for the media and public alike to independently verify tweet accuracy.

Introduction

On 17 May 2022, the first case of Mpox was documented in the USA.1 Mpox is a DNA virus of the Orthopoxvirus genus which typically causes illness beginning with a febrile prodrome followed by an eruptive phase with a defining rash.2 By mid-June 2022, data indicated >95% of cases were in males and 90%–99% were in men who have sex with men (MSM). Cases largely remained confined to this population through July3–7 (figure 1) and peaked on 1 August at 6458 (online supplemental figure S1). Shortly after, U.S. Department of Health and Humans Services Secretary, Xavier Becerra announced that Mpox was a Public Health Emergency.9 Twitter was used by the lay public and health experts alike to provide information, draw awareness to and make predictions regarding the spread of the Mpox virus.

Figure 1
Request permissionrequest permission icon
Figure 1

Timeline of risk factors and transmission of Mpox overlayed with the number of inaccurate and accurate tweets per month. Number of tweets that were inaccurate versus accurate, broken down by month (left). Red bars are inaccurate (all exaggerated) tweets and blue bars are accurate tweets. Timeline (right) organised by publication date of studies meeting our search criteria, which describe patient characteristics/risk factors in confirmed Mpox cases and transmission. GBMSM, gay, bisexual, men who have sex with men; MSM, men who have sex with men.

X (formerly Twitter) is a social media platform with around 350 million monthly active users.10 In 2022, Twitter was found to be the top social media platform for journalists11 and the public to obtain news in the USA.11 It was also recently found to be the most popular social media outlet for disease outbreak surveillance information12 and health information, generally.13 However, the accuracy of health information is not rigorously monitored. The platform gained a reputation for spreading ‘misinformation’13 14 on numerous health topics,13 though a minority of studies of social media health information assess accuracy.13 Inaccurate public health information, especially in the context of new disease outbreaks, can be harmful if it creates undue fear or anxiety, promotes unproven treatments or mitigation, or undermines effective disease prevention strategies.

Information about Mpox on Twitter and other social media platforms has been criticised.15–17 A recent analysis of TikTok videos about Mpox found the information in general to be ‘poor’ and ‘incomplete’.15 An additional analysis of 599 YouTube videos about Mpox found that health professionals were responsible for 29 of the 122 videos categorized as ‘misleading’.16 Furthermore, an analysis in May 2022 found 52 of the top 100 tweets containing the keyword ‘monkeypox’ to be misinformation.17

Because Twitter was easily searchable and used to distribute health information, we sought to determine how often users, who might be perceived as domain experts on Mpox, public health or infectious disease, provided accurate versus inaccurate information on risks. We focused on comments regarding the school environment for children and young people because Mpox was shown early on to be spreading in MSM communities3–7 18–21 and not among children. Moreover, discussion of this pathogen was used in debates regarding fall school precautions or restrictions for children.

Methods

Overview

We identified tweets from accounts that could be viewed as having specific expertise in Mpox, science, medicine or public health policy that discussed risks of Mpox to children and young people in school. We described and calculated the ratio of inaccurate tweets with those which accurately described risks or areas of uncertainty without creating undue fear and ended up being objectively accurate or were objectively accurate at the time. We focused on children because their risk, particularly in the school setting, was objectively ‘minimal’,22–24 and thus more amenable to objective grading of tweets as inaccurate/accurate. We used Strengthening the Reporting of Observational Studies in Epidemiology guidelines for cross-sectional studies to report the findings.

Search strategy

We performed a Twitter advanced search in January of 2023 using the following search terms: school (pox OR monkeypox OR mpox) min_faves:1 lang:en until:2022-10-01 -filter:replies, which included tweets occurring prior to 1 October 2022. The search stop date corresponded with a decline in the number of Mpox cases (online supplemental figure S1).1 The search strategy was adapted to exclude tweets that were replies (comments) because replies receive less engagement and are less impactful. Additionally, we used the snowball method to retrieve additional tweets or replies that fit our inclusion criteria but did not appear in the search.

Inclusion criteria for tweets

One author (BK) analysed all tweets in the Twitter advanced search. First, the author and biography of each tweet was viewed (author’s Twitter profile) and excluded unless at least one of the following set of credentials was met: MD, DO, nurse, pharmacist, physical therapist, other healthcare provider, PhD, MPH, Ed. degree (doctor of education or masters of education), JD (lawyer), health/medicine reporter/journalist/columnist/expert, public policy reporter (students or candidates of these professions were also included). We categorised MDs, DOs, nurses, pharmacists, physical therapists, others who provide direct patient care, and students in these degree programmes as ‘health care’. PhDs, MPHs and Ed. degree as well as students in these degree programmes were placed in a second category. JDs were in their own category. While lawyers do not hold a degree specific to health, we included them because they have training in analysis of evidence and public policy, and many have domain expertise in health-related fields. All science/medicine journalists/reporters were placed in a fourth category called ‘health reporters’.

All tweets meeting the account credential inclusion criteria were reviewed for appropriateness for inclusion by a second author (TBH). All included tweets were subsequently analysed for content by the same two authors (BK, TBH). Tweets were placed into two categories: inaccurate or accurate (table 1). If the two authors differed in opinion, a third author (VP) analysed the tweet and made a final decision (table 1).

Table 1
|
Inclusion criteria for tweet content

Data extraction

In January 2023, we extracted the tweet date, tweet text, author credentials, number of favourites, number of retweets and number of followers of the tweet’s author. In April 2023, one author (BK) reviewed all tweets through the Twitter platform to determine if tweets had been deleted.

Data analysis

We calculated a ratio of inaccurate to accurate tweets and specified whether the inaccurate tweets exaggerated or minimised risks. Additionally, we separated all tweets into four bins based on the credentials of the author: healthcare; health/medicine reporter/journalist/columnist/expert; PhD, MPH, Ed. degree; JD.

We calculated the ratio of inaccurate to accurate tweets within each bin. We calculated the same ratio based on the month the tweet was published. We also compared the mean and median number of account followers between tweets which were inaccurate and accurate as well as mean and median number of likes and retweets for each tweet. Finally, we used the Wilcoxon Rank-sum test in R (V.4.2.2) to compare the distribution of the median follower count for tweets that were inaccurate to those providing accurate information. R was also used to generate a waterfall plot showing the number of tweets per user and to create a Kaplan-Meier plot of the cumulative likes and retweets over time. The Wilcoxon Rank-sum test in R was used to compare the cumulative likes and retweets for inaccurate and accurate tweets. Excel was used to tabulate descriptive statistics.

Patient and public involvement

Twitter users were exposed to information about the risk of Mpox to children and young people in school during the summer of 2022; however, the accuracy of this information was unknown. Members of the public were not involved in the creation of this article, nor were patients. The results will be published and available to everyone.

Research ethics approval

Ethical approval for use of the tweets was not needed because all tweets were publicly available on Twitter. Further, the author of the tweets consented to the tweets being public at the time they were tweeted. In accordance with 45 CFR §46.102(f), this study was not submitted for institutional review board approval because it involved publicly available data and did not involve individual patient data.

Results

We identified 262 tweets spanning 18 May 2022–19 September 2022, from 188 unique accounts. Among these, 187 were individuals and one was an institution. Among individuals, 48% (90/187) were categorised as healthcare, 7%, (13/187) health reporters, 39% (72/187) PhD, MPH, or Ed. degree, and 6% (12/187) JD. The median tweets per account was one (IQR one to one).

Two hundred fifteen (82%) tweets exaggerated the risk of Mpox to children and young people in the school setting, while 47 (18%) provided accurate information. The ratio of inaccurately to accurately stating risks was 4.6:1. Ten examples of inaccurate/exaggerated tweets are in table 2.

Table 2
|
Selected examples of inaccurate/exaggerated tweets

The ratio of inaccurate/exaggerated to accurate/appropriate tweets in May/June, July, August and September was 7:1, 6.6:1, 3.9:1 and 3:1, respectively (figure 1). August had the highest number of tweets from both categories: 116 tweets that were inaccurate and 30 that were accurate.

The percentage of tweets that were inaccurate by the credentials of the author were: among healthcare professionals 95/111 (86%), health reporters 18/41 (44%), PhDs, MPH, Ed. degree 93/97 (96%), JDs 9/13 (69%) (figure 2). Only health reporters were more likely to provide accurate information, though this is based predominantly on one outlier (figure 3, rightmost bar).

Figure 2
Request permissionrequest permission icon
Figure 2

The number of tweets categorised by the author’s occupation or degree and whether the tweet was inaccurate or accurate.

Figure 3
Request permissionrequest permission icon
Figure 3

Waterfall plot showing the number of tweets per author. Each bar represents a unique author. The red bars represent authors who had inaccurate tweets and the blue bars indicate authors who had accurate tweets.

There were 163 unique authors of the 215 tweets that overstated the risk (figure 3). There were 12 individuals with at least three tweets within this group. In contrast, we found 25 unique authors of the 47 tweets that provided accurate information (figure 3). There was 1 author in the health reporter group who had 14 accurate tweets.

The mean and median follower count of the users with inaccurate tweets was 37 229 and 6409 (Q1 2658–Q3 19 900), respectively (online supplemental table S1). For accounts that provided accurate information, the mean and median follower count was 31 334 and 17 700 (Q1 10 100–Q3 17 700) (online supplemental table S1). The distribution of the median follower count between users that provided accurate and inaccurate information was statistically significant (p=0.00014).

The cumulative number of likes was 201 811 and 7084 (28.5-fold difference) for tweets that were inaccurate and accurate, respectively (p<0.001) (figure 4a). However, two inaccurate tweets were outliers and had a significant number of likes: 126 000 and 41 600. The cumulative number of retweets was 50 710 and 1295 (39.2-fold difference) for tweets that were inaccurate and accurate, respectively (p<0.001) (figure 4b). The same two tweets were outliers with 33 502 and 6477 retweets. The cumulative number of account followers was 8 004 244 and 1 472 683 for inaccurate and accurate tweets, respectively.

Figure 4
Request permissionrequest permission icon
Figure 4

Kaplan-Meier plot showing the cumulative number of likes (A) and retweets (B) over time grouped by either being inaccurate or accurate. The figure legend denotes the two tweets that contributed substantially to the total sum of likes and retweets.

There were nine tweets that appeared to be deleted sometime between our initial search and April 2023 and all were in the inaccurate category.

Discussion

We found healthcare professionals and other credentialed Twitter profiles tended to provide inaccurate information, which exaggerated the risks of Mpox in school settings. Tweets exaggerating risks exceeded accurate ones by a ratio of 4.6:1. The category ‘PhDs, MPHs and Ed. degree’ had the highest tendency to overstate risks at 23 inaccurate tweets for every one accurate tweet. ‘Health reporters’ was the only category which was slightly more likely to tweet accurate than inaccurate information by a ratio of 1.3:1. However, this finding was driven primarily by one reporter who published 14 accurate tweets about Mpox. Healthcare providers also performed poorly with nearly six inaccurate for every one accurate tweet.

Inaccurate tweets were published by accounts that had, on average, 19% higher follower counts. Individual inaccurate tweets were liked and shared 28.5 times and 39.2 times more frequently than accurate tweets, respectively. Thus, the ratio of inaccurate to accurate tweets alone likely underestimates the real impact of inaccurate versus accurate tweets.

View counts were not available at the time of our study but multiplying the ratio of inaccurate to accurate by a share differential for inaccurate compared with accurate tweets by the follower number and retweet difference for each tweet (4.6 × 5.4 × 39.2) suggests the number of views may be 974 times higher for inaccurate information. Our study demonstrates the potential of Twitter (now X) to magnify inaccurate and, in this case, fear-based messaging. The 974-fold difference is likely an underestimate considering inaccurate tweets may have been deleted before our search. We observed 9/215 inaccurate and 0/47 accurate tweets deleted between the initial search and the second search (January 2023–April 2023). It is unclear how many inaccurate tweets were deleted prior to our initial search in January of 2023, several months after the Mpox cases started steadily declining.

Of concern, some of the accounts tweeting inaccurate information were esteemed experts in medicine, including a former surgeon general, and professors at major medical centres. In some cases, we observed that the inaccurate information—specifically comments creating undue fear of Mpox among children and/or in schools—was paired with longstanding advocacy for specific policy restrictions, such as masking in schools. It would be concerning if experts knowingly or inadvertently create fear in order to advance restrictions that they may support for other or pre-existing reasons (eg, COVID-19). Admittedly, we cannot know if experts knowingly or unknowingly provided inaccurate information.

Unlike real-life healthcare settings, accounts tweeting inaccurate information may not face repercussions for providing incorrect or even dangerous information. Exaggerated or fear-based messaging may even be incentivised though increased popularity and attention. Our study found incorrect tweets had higher cumulative likes and retweets and came from accounts with a higher number of followers on average. However, it is unclear whether the inaccurate information from these accounts was limited to Mpox or extended to other themes.

Strengths and limitations

Our study has three strengths and five limitations. We provide information on inaccurate tweets about an emerging health issue stratified by profession/degree, follower counts, likes and retweets. We found that inaccurate tweets were able to reach far more readers. Additionally, we analysed erroneous tweets pertaining to a vulnerable group (children/young people) which may be disproportionately susceptible to the negative impacts of inaccurate health information.

Regarding limitations, the accuracy criteria were based on a review of the current scientific literature but was not validated. However, the included tweets were reviewed for accuracy independently by two authors. We made all tweets available in the supplement and encourage independent reanalysis (online supplemental table S2, S3). Second, to our knowledge, the accuracy of the Twitter Advanced Search function has not been described in the scientific literature or made publicly available. The search function may have biased the search by removing or censoring certain tweets. Third, we investigated one infectious disease outbreak in 2022 which may not apply to other contexts or infectious diseases. Fourth, we could not state the impact that the incorrect information may have had on children, schools and society. Finally, the analysis was limited to Twitter which represents only a fraction of health information circulated online; thus, we encourage others to analyse other social media platforms.

Conclusion

Our study found that credentialed Twitter users were 4.6 times more likely to tweet inaccurate than accurate information about Mpox risks in children and young people in school. One hundred percent of the inaccurate information identified in our study exaggerated the risks of Mpox. Inaccurate/exaggerated tweets came from accounts with higher mean follower counts and had higher cumulative likes and retweets than tweets with accurate information.

This finding may have major and widespread societal ramifications including heightened anxiety and inaccurately informed public health decision-making. Those seeking and/or reporting health information from X (formerly Twitter) should be aware of our documented high rates of inaccuracy even from the accounts of credentialed health professionals.

Article metrics
Altmetricopen-url
Dimensionsopen-url