Abstract
BACKGROUND AND PURPOSE: More than 75 million Americans have less than adequate health literacy skills according to the National Center for Education Statistics. Readability scores are used as a measure of how well populations read and understand patient education materials. The purpose of this study was to assess the readability of Web sites dedicated to patient education for radiologic spine imaging and interventions.
MATERIALS AND METHODS: Eleven search terms relevant to radiologic spine imaging were searched on the public Internet, and the top 10 links for each term were collected and analyzed to determine readability scores by using 10 well-validated quantitative readability assessments from patient-centered education Web sites. The search terms included the following: x-ray spine, CT spine, MR imaging spine, lumbar puncture, kyphoplasty, vertebroplasty, discogram, myelogram, cervical spine, thoracic spine, and lumbar spine.
RESULTS: Collectively, the 110 articles were written at an 11.3 grade level (grade range, 7.1–16.9). None of the articles were written at the American Medical Association and National Institutes of Health recommended 3rd-to-7th grade reading levels. The vertebroplasty articles were written at a statistically significant (P < .05) more advanced level than the articles for x-ray spine, CT spine, and MR imaging spine.
CONCLUSIONS: Increasing use of the Internet to obtain health information has made it imperative that on-line patient education be written for easy comprehension by the average American. However, given the discordance between readability scores of the articles and the American Medical Association and National Institutes of Health recommended guidelines, it is likely that many patients do not fully benefit from these resources.
ABBREVIATIONS:
- AMA
- American Medical Association
- FRE
- Flesch Reading Ease
- GFI
- Gunning Fog Index
- NIH
- National Institutes of Health
As barriers to on-line access have decreased, the Internet has emerged as a primary resource for Americans desiring greater understanding of their health. According to a June 2015 report by the Pew Research Center,1 up to 84% of adults access the Internet, and within the past year, 72% of those users have searched for health information.2 Specifically, 55% wanted to learn more about a disease or medical problem; and 43%, about a medical treatment or procedure.2 Studies have confirmed that this on-line research impacts decision-making for many patients: the questions they ask, the types of treatment they pursue, and whether they visit a physician.2⇓⇓–5
Although more adults are accessing health care information on-line than ever before,2,4 it is uncertain how much of this information is fully comprehended due to poor health literacy. Health literacy, as defined by the US Department of Health and Human Services, is “the capacity to obtain, process, and understand basic health information and services needed to make appropriate health decisions.”6 In a 2003 assessment commissioned by the US Department of Education, only 12% of adults were found to have proficient health literacy. Proficiency was defined as having the skills necessary to locate, understand, and use information contained within documents commonly encountered in the medical system, such as medication dosing instructions, preventative care documentation, and insurance information. This definition indicates an ability to read, analyze, and synthesize complex content. More than 75 million Americans demonstrated either basic or below basic health literacy and would experience difficulty reading and comprehending health care–related text.7 The importance of health literacy cannot be understated because it has a direct influence on both health outcomes and health care expenditures. Studies have linked low health literacy to increased hospitalizations,8,9 higher mortality rates,8,10 and an annual cost to the US economy of up to $238 billion.11 In fact, the American Medical Association (AMA) has identified low health literacy as a strong independent predictor of health status.12
Readability, defined as the degree of ease with which a given text can be read and comprehended, is 1 correlative measure of health literacy.13 The reading level of the average American is between the 7th and 8th grade, while the average Medicaid enrollee reads at just a 5th grade level.12 Therefore, to maximize the number of individuals benefiting from patient education, the AMA and the National Institutes of Health (NIH) recommend that content be written at a level commensurate with the 3rd-to-7th grade levels.12,14 However, patient education materials across numerous specialties in medicine do not meet this recommendation. A 2013 readability study published in Journal of the American Medical Association analyzed material from 16 different medical specialties and determined that it was too complex for the average patient.15 Similar conclusions have been drawn regarding the surgical subspecialties.16
Readability analyses specific to spine-related patient education have also revealed a failure to meet reading level guidelines.17⇓⇓–20 However, research to date has only examined surgical procedures and material sourced from professional society Web sites. Three of the 4 studies were also limited by an analysis that incorporated just 1 readability assessment. The purpose of this study was to quantitatively determine the readability of patient education Web sites pertaining to radiologic diagnostic tests and interventions of the spine. We used 10 readability assessments that are well-vetted in the literature to avoid bias from any single test. This analysis does not include patient education materials related to imaging of the brain.
Materials and Methods
This study examined publicly available data; thus, institutional review board oversight was not required. In December 2015, Web sites dedicated to patient education relevant to spine imaging were sought on the public Internet by using the Google search engine. Eleven keywords were separately entered as search terms: x-ray spine, CT spine, MR imaging spine, lumbar puncture, kyphoplasty, vertebroplasty, discogram, myelogram, cervical spine, thoracic spine, and lumbar spine. The first 10 articles intended for patients for each term were included in the analysis. Web sites not specifically directed toward patients were excluded. The text of 110 articles was copied, pasted, and saved as individual Microsoft Word (Microsoft, Redmond, Washington) documents. Images, figures, tables, references, and other noneducational text were removed.
Each document was then analyzed, and a readability analysis was performed with Readability Studio Professional Edition (Oleander Software, Vandalia, Ohio). An individual readability score was calculated for each of the 10 following well-validated assessments (Table): the Coleman-Liau Index,21 Flesch Reading Ease (FRE),22 Flesch-Kincaid Grade Level,23 FORCAST,24 Fry Graph,25 Gunning Fox Index (GFI),26 New Dale-Chall,27 New Fog Count,23 Raygor Readability Estimate,28 and SMOG.29 The FRE reports scores on a 0–100 scale with lower numbers corresponding to more difficult-to-read text. The remaining 9 scales report the readability of the text as a grade level. For instance, a GFI score of 9.0 corresponds to a 9th grade reading level.
Formulas for the readability assessments
Statistical analysis was conducted by using OriginPro (OriginLab, Northamptom, Massachusetts) to compare readability scores among the 11 keywords. A 1-way ANOVA and a Tukey Honestly Significant Difference post hoc analysis were performed with P < .05.
Results
Collectively, the 110 articles had a mean FRE score of 51.9, classifying them as fairly difficult on the FRE scale, and an 11.3 mean grade level averaged across the other 9 assessments, scored on the basis of grade level (Fig 1). FRE scores ranged from 74 (fairly easy) to 14 (very difficult), and grade levels ranged from 7.1 to 16.9. None of the articles (0/110) met the recommendations of the AMA and NIH of being written within a 3rd-to-7th grade level. Approximately 35% (39/110) were written at a level that required a high school education or higher (score of ≥12). An additional 50 articles scored between a 9th and 12th grade levels (On-line Table).
The grade level taken as the mean of all readability scales examined in this study for the 10 top search results for each key term. The red box represents the AMA and NIH recommended 3rd-to-7th grade guidelines.
The articles consisted of many words characterized as complex, long, or unfamiliar. Words with at least 3 syllables were considered complex and composed 16.1% of the text of the articles, while words with at least 6 characters were considered long and composed 33.7%. More than 28% of words were classified as unfamiliar, as determined by an absence from the Dale-Chall list of simple words, which contains 3000 words known by most 4th grade children.27 In addition, unfamiliar words made up at least one-third of the text for 19 of the 110 (17.3%) articles. Sentences ranged from 23 to 127 words.
The 1-way ANOVA found a statistical difference among the 11 keywords (F(10,99) = 3.19, P = .001). Average grade levels for each searched term were as follows: x-ray spine, 9.4; CT spine,9.6; MR imaging spine, 10.2; discogram, 10.7; myelogram, 11.0; cervical spine, 11.2; thoracic spine, 11.8; lumbar spine,11.8; lumbar puncture, 12.0; kyphoplasty, 12.4; and vertebroplasty, 13.4. Tukey Honestly Significant Difference post hoc analysis indicated that the vertebroplasty articles were significantly more advanced than the articles for x-ray spine, CT spine, and MR imaging spine (P < .05).
Discussion
Due to the inherently complex nature of spine diagnoses and treatments, patients are apt to seek more information on the Internet. Up to 77% of individuals begin this process with a search engine such as Google.2 More than 90% do not look beyond the first page of results.30 Consequently, patients wishing to learn more about radiologic spine imaging and interventions would likely encounter 1 of the 110 articles in this study when searching for these 11 terms. With a mean readability score of 11.3, these articles would be too complex for the average American who reads at a 7th-to-8th grade level. In addition, the abundance of uncommon words and long sentences would make understanding difficult for those classified as having less than proficient health literacy, which indicates an inability to read and synthesize complex health care–related text. Therefore, 62% of the adult population identified by the US Department of Education as having either basic or below basic health literacy would not fully benefit from this information and may be led to uninformed decisions that negatively affect health outcomes.7
If on-line patient education resources were written at a 7th grade reading level or lower, more Americans would be able to read and understand the material more thoroughly. Consequently, patients would likely experience increased involvement in their care and improved communication with their physicians. When empowered with knowledge, patients have been shown to ask more questions, communicate concerns with greater confidence, and actively engage in the medical decision-making process.31⇓–33 Patients have also reported greater satisfaction, particularly with informed consent.34 In radiology, health literacy has been linked to differing rates of imaging use35 and patient knowledge of procedure details and radiation use.36 Complex examinations and interventions, including those of the spine, stand to benefit from the active patient engagement and enhanced patient-provider communication resulting from well-written education materials.
The results of this study are consistent with prior research investigating the readability of on-line patient education. Web sites for both medical and surgical subspecialties are routinely written at a level exceeding the 7th grade.37⇓⇓–40 Those dedicated to radiology, including radiologyinfo.org sponsored by the American College of Radiology and Radiological Society of North America, are written at a level too advanced for most patients.41 In addition, patient education materials from professional society Web sites, Wikipedia, WebMD, and hospital Web sites have all been written above the average comprehension level.42⇓⇓–45 This study, strengthened by the incorporation of text sourced from multiple Web site types and the use of 10 readability assessments, adds additional support to the conclusions drawn by prior spine imaging readability research. Collectively, these results highlight the need for further action to satisfy AMA and NIH readability recommendations. Authors and editors should use simpler words, construct shorter sentences, reduce abbreviations and acronyms, and eliminate medical jargon.14 Resources from the NIH,14 Centers for Disease Control and Prevention,46 and Center for Medicare and Medicaid Services are available to offer further guidance.47
This study is limited by the constraints of the readability assessments. Most important, the algorithms for certain quantitative parameters, such as the number of letters, syllables, words, and sentences used in the text, may lead to inaccurate scores for medical terminology. For instance, words with few syllables that are not necessarily familiar to the average person may lead to inappropriately low scores, while multisyllabic common words would be scored with a higher grade level. The FORCAST formula, which is based solely on the number of single-syllable words, is particularly susceptible to this bias. For example, “pia” would receive a lower rating than “operation,” despite being an uncommon term. The other assessments that use syllable counts, including the FRE, Flesch-Kincaid Grade Level, Fry Graph, GFI, and SMOG, may be affected to a somewhat lesser extent due to the use of additional variables. In this study, incorporation of 10 readability assessments reduces the bias of any single algorithm. An additional limitation is that none of the assessments evaluated the nontextual elements of readability, such as style, format, and organization13 or the use of supplemental material, such as images or diagrams. Further work is needed to determine the effect of these elements on the comprehension of patient education materials, specifically in radiology. Conducting readability and comprehension tests with target prospective patient populations may also be revealing.
Conclusions
With increasing use of the Internet for patient self-education, there is a growing need for the readability of material to fall within the limits of the average American's comprehension. However, an average reading level is often far exceeded in many disciplines of medicine. Spine imaging and radiologic interventions have not been an exception. It is imperative to broaden awareness of this discrepancy to mitigate the negative outcomes of poor health literacy. By adhering to the AMA and NIH guidelines, physicians, professional societies, and other authors can increase patient comprehension of on-line health care materials.
References
- Received October 7, 2016.
- Accepted after revision January 25, 2017.
- © 2017 by American Journal of Neuroradiology
Jump to section
Related Articles
Cited By...
- No citing articles found.