How Accurate Are Wikipedia Articles in Health, Nutrition, and Medicine? / Les articles de Wikipédia dans les domaines de la santé, de la nutrition et de la médecine sont-ils exacts ?

Abstract

Previous studies of Wikipedia have reported mixed results regarding the quality of information on health-related topics. We investigated the accuracy of Wikipedia entries in the areas of health, nutrition, medicine, and complementary and alternative medicine. We formulated 32 statements which are often stated but are probably incorrect ("common misconceptions"). Using Google we found 49 Wikipedia entries that provided information on these 32 statements. Most entries are accurate, but deficiencies are present in a minority. The information provided by Wikipedia has a high degree of accuracy for 23 (72%) of the 32 common misconceptions (19 had a score of 5, the highest mark possible, and 4 had a score of 4.5–4.7). Seven of them (22%) had a score of 4–4.1, indicating that the entry has a minor error or that significant information is missing. Two had a low score (3.5), which indicates a more serious problem. Of all the 49 Wikipedia entries that were evaluated, four had a score of only 3, indicating that the accuracy is seriously flawed or that no information is given. These findings together with those from other studies indicate that the information provided by Wikipedia is mostly of high quality but that significant errors and omissions are fairly common.

Résumé

Des études antérieures portant sur les articles Wikipédia ont fait mention de résultats inégaux concernant la qualité de l’information sur les sujets liés à la santé. Nous avons étudié l’exactitude des entrées de Wikipédia dans les domaines de la santé, de la nutrition, de la médecine et des médecines parallèles et complémentaires. Nous avons défini trente-deux énoncés qui sont souvent proposés, mais qui sont probablement incorrects (conceptions courantes mais fausses). Une recherche Google a trouvé quarante-trois entrées Wikipédia fournissant des informations sur ces trente-deux énoncés. La plupart des entrées fournissaient des informations exactes, mais nous avons constaté des lacunes dans une minorité d’entre elles. Pour les trente-deux conceptions courantes mais fausses, vingt-trois (72 %) des entrées Wikipédia atteignaient un degré élevé d’exactitude (dix-neuf avaient un score de 5 [note la plus élevée] et quatre atteignaient 4,5–4,7). Sept des entrées (22%) atteignaient un score de 4–4,1, ce qui indique que l’entrée comportait une petite erreur ou que des informations essentielles étaient manquantes. Deux entrées atteignaient un score faible (3,5). Sur les quarante-trois entrées de Wikipédia qui ont été évalués quatre atteignaient un score de seulement 3, ce qui indique que l’entrée était soit gravement inexacte soit lacunaire. Ces résultats ainsi que ceux d’autres études indiquent que les informations fournies par Wikipédia sont généralement de haute qualité, mais que des erreurs et des omissions importantes sont assez fréquentes.

Introduction

Wikipedia is an encyclopedia of knowledge that is accessed via the Internet. According to its website it offers over four million entries in English as well as millions more in other languages (Wikipedia 2013). These entries cover a very wide range of subjects and are easily accessible at no cost. Wikipedia is now the sixth most used website on the Internet (Alexa 2013). As a result, a Wikipedia entry routinely appears when an Internet search is carried out on almost any topic. This extends to information in most areas of health. A large survey reported that more than 70% of searches for health-related information produce a Wikipedia entry among the first 10 results (Laurent and Vickers 2009).

Evidence suggests that many health professionals use Wikipedia as a source of information. For example, one study reported that 28% of pharmacists in the United States use Wikipedia entries when searching for information on drugs (Brokowski and Sheehan 2009). There is little doubt that millions of members of the general public use Wikipedia as a source of information on diverse topics related to health. However, the accuracy of Wikipedia articles in the biomedical sciences is uncertain, and as a result Wikipedia is seldom cited in papers in peerreviewed journals.

Several studies have evaluated whether Wikipedia deserves this second-rate status. Some studies have indeed found problems with the overall quality, especially the accuracy and completeness, of Wikipedia entries. One study assessed entries on three health problems—otitis media, conjunctivitis, and multiple sclerosis (Pender et al. 2009). The entries omitted some key information, and there were factual errors. Another study looked at entries on the subject of statins (drugs that lower blood cholesterol; Kupferberg and Protus 2011). While none of the information was incorrect, much of it was incomplete in ways that could harm patients (a lack of information on drug interactions and on contraindications for the drugs). A much broader study on the quality of information on drugs determined that Wikipedia entries were inferior to those at Medscape Drug Reference (Clauson et al. 2008). The investigators concluded that Wikipedia is not authoritative in this area. Similarly, an assessment of the information in Wikipedia on 20 common drugs revealed that it was often incomplete or inaccurate (Lavsa et al. 2011). The investigators concluded that pharmacy students should not use Wikipedia.

In contrast to the above studies, others have rated Wikipedia more positively. One study reported that Wikipedia entries on the topic of osteosarcoma were generally of good quality but inferior to those on the website of the National Cancer Institute (Leithner et al. 2010). Another study found that for schizophrenia and depression the quality of information provided by Wikipedia compared favourably with that found in a psychiatric textbook (Reavley et al. 2012). A study evaluated content related to 10 types of cancer (Rajagopalan et al. 2011). In comparison with information provided by the National Cancer Institute’s Physician Data Query, that from Wikipedia was of similar accuracy and depth though it was less readable. One study observed that Wikipedia entries on a wide variety of health-related topics were based on reputable sources of information (Haigh 2011).

The study reported here was carried out to explore this subject further. Our specific aim was to determine the degree of accuracy of articles in Wikipedia in the areas of health, nutrition, medicine, and complementary and alternative medicine.

Methods

Formulation of Test Statements

We formulated 32 statements, each of which is often stated but is unlikely to be correct ("common misconceptions") (presented in the appendix). Most of the common misconceptions fell into one of these groups: (1) the statement was in older published literature that has been disproven by more recent findings; or (2) the statement is based on little more than speculation with only weak supporting evidence (often by persons selling supplements or promoting unproven ways to prevent or treat various health conditions). In most cases we have seen the common misconceptions stated numerous times. The table provides examples of where the common misconceptions have been stated. We also formulated a correct version of each statement that is based on strong supporting evidence. We included only statements where it would be feasible to categorize Wikipedia entries as being accurate or incorrect (or somewhere in between).

The rationale for this study design is as follows. If Wikipedia is an unreliable source of information, then we would predict that several of the above common misconceptions would be found in Wikipedia entries and presented as if they were proven facts. Conversely, if Wikipedia is a reliable source of information, then we would predict that only the accurate version of each statement would be found in Wikipedia entries.

The 32 statements came from the following areas:

• • herbal supplements (3 statements);

• • non-herbal supplements (2);

• • nutrition and health (11);

• • cancer (3);

• • cardiovascular disease (3);

• • complementary and alternative medicine (7); and

• • health (general aspects) (3).

Several statements overlapped between two or more areas.

Searches and Grading of Wikipedia Entries

We carried out Google searches for Wikipedia entries that covered information related to the 32 statements. This was done during November 2013. We searched for keywords or terms that might be used by a health-care professional, a scientist, or a layperson. For example, for the statement "Tea and coffee have a diuretic action which may therefore lead to dehydration," we searched for the terms "tea," "coffee," and "diuretic action." Similarly, for the statement "Red wine is more effective than other types of alcoholic beverage for preventing heart disease," we searched for the terms "red wine," "wine," "alcohol," and "heart disease." In several cases there was more than one entry with relevant information, in which case all entries were graded and an overall assessment made. However, in other cases Wikipedia had one main entry on a topic but also had other entries which contained only a brief mention of the topic in question and/or were cross-referenced to the entry that covered the topic in more detail. In such cases only the main entry was assessed. Each of the Wikipedia entries is a separate Web page.

Wikipedia entries were graded by level of accuracy as follows: 1 = major error, 2 = significant error, 3 = neutral (or no information given), 4 = mostly correct (but has a minor error or significant information is missing), or 5 = very accurate (including the inclusion of all important information).

For several of the common misconceptions the "accurate statement" contains a degree of uncertainty. For example, for the common misconception on calcium supplements, the accurate statement is "Evidence indicates that calcium supplements may cause an increased risk of cardiovascular disease." The supporting evidence for this is strongly suggestive but falls well short of being firmly established. Accordingly, a score of 5 would indicate that the Wikipedia entry accurately states our present knowledge, including the level of uncertainty.

Results

The findings are shown in detail in the appendix and are summarized in table 1. We evaluated the accuracy of 49 Wikipedia entries that provide information on the 32 common misconceptions. The distribution of the mean scores for the 32 sets of evaluations was as follows: nineteen had a score of 5 (highest mark), four scored 4.5–4.7, seven scored 4–4.1, two scored 3.5, and none scored below 3.5. As indicated in the appendix, 4 Wikipedia entries (out of 49) had a score of

Table 1.

Comparison of Wikipedia with Current Evidence: Summary

only 3 (indicating that the accuracy is neutral [i.e., neither correct nor incorrect] or that no information is given; common misconceptions 9, 10, 12, and 20). However, other Wikipedia entries on the same four common misconceptions scored higher marks.

Discussion

The findings reported here should be viewed cautiously. First, we assessed Wikipedia only for a limited number of areas related to health, nutrition, medicine, and complementary and alternative medicine. An assessment of the accuracy of Wikipedia entries on other topics within those areas might produce different findings. It must also be stressed that categorizing the accuracy of statements is not always clear-cut; some misclassification may therefore have occurred.

Our findings reveal that most Wikipedia entries have a high level of accuracy but that deficiencies are present in a significant minority. This is best shown by the distribution of scores. Of the 32 common misconceptions for which we evaluated the accuracy of Wikipedia entries, 19 (out of 32 mean scores) have a score of 5 (highest mark), and 4 scored 4.5–4.7. Thus, 23 out of 32 (72%) were judged to have a high degree of accuracy. Seven of the 32 mean scores (22%) have a lesser degree of accuracy (i.e., a score of 4–4.1, indicating that the entries are mostly correct but that minor errors are present or significant information is missing). More serious problems were seen with two sets of entries (mean scores of 3.5). Of the 49 Wikipedia entries that were evaluated, 4 had a score of only 3 (indicating that the accuracy is seriously flawed or that no information is given). To put this in context we should bear in mind that review and commentary papers in peer-reviewed journals are often met with strong disagreement.

Previous studies of Wikipedia entries have given mixed results regarding the quality of information on health-related topics. Some have been positive in their assessment (Leithner et al. 2010; Reavley et al. 2012; Rajagopalan et al. 2011; Haigh 2011), whereas others have pointed to significant errors or other deficiencies in the information (Pender et al. 2009; Kupferberg and Protus 2011; Clauson et al. 2008; Lavsa et al. 2011). These inconsistent findings suggest that Wikipedia is a "work in progress"; while it is mostly quite accurate it should not be relied on as an authoritative source of information. The accuracy of Wikipedia entries clearly requires further study across a range of subject areas. This research needs to be ongoing as Wikipedia entries are continually changing.

Conclusions

Looking at the information as a whole, including the study described here, a reasonable overall assessment is that much of the information provided by Wikipedia is of high quality but that significant errors and omissions are fairly common. Wikipedia should therefore not be relied on as a primary source of scholarly work. However, it can be a useful resource as a starting point for searches or as a supplement to other sources.