Content Analysis of Web-based Norovirus Education Materials

Content Analysis of Web-based Norovirus Education Materials Targeting Consumers Who Prepare and Handle Food: An Assessment of Alignment and Readability

Hillary Evans^a

Morgan Getty^a

Cortney M. Leone^a

Michael Finney^b

Angela Fraser^a*

^aDepartment of Food, Nutrition, and Packaging Sciences, Clemson University, Clemson, SC, USA

^bDepartment of Mathematical Sciences, Clemson University, Clemson, SC, USA

ABSTRACT - word count

Huma noroviruses sicken 19-21 million people in the U.S. each year, suggesting the need for education. The World Wide Web is a readily accessible source of information about how to prevent a norovirus infection but at present the accuracy and readability of materials targeting consumers is unknown. The aim of our study was to evaluate the alignment and readability of web-based norovirus education materials targeting consumers using CDC guidelines and Microsoft Word readability formulas. Our search yielded 60 artifacts. Most did not address duration of handwashing (83%), use of hand sanitizers (83%), type of drying devices (92%), or avoidance of bare-hand contact with ready-to-eat foods (97%). Less than half (n=29) recommended minimizing contact with sick persons. Two-thirds of the artifacts (n=40) also did not mention the recommended concentration of sodium hypochlorite (bleach) solution to be used to disinfect surfaces contaminated with Noroviruses. The mean Flesch Reading Ease score was 47.75 (score of >70 is easy to read), and the mean Flesch-Kincaid Grade Level was 10.36 so documents were written at a 10^th grade level. The alignment and readability of web-based educational materials about Noroviruses must be improved as knowledge is a prerequisite to application of behaviors that can prevent one from becoming infected with Noroviruses.

Key words: Huma noroviruses, content analysis, consumer education, food safety, readability

Content Analysis of Web-based Norovirus Education Materials Targeting Consumers who Prepare and Handle Food: An Assessment of Alignment and Readability

INTRODUCTION

Worldwide, noroviruses are the leading cause of acute gastroenteritis. In the U.S., an estimated 21 million individuals (1 in 14) are sickened each year by this group of viruses resulting in $777 million in healthcare costs (Hall et al., 2013). Most reported outbreaks are attributed to person-to-person contact (69%) followed by the consumption of contaminated food (23%) making noroviruses the most common cause of foodborne disease (58% of cases) in the U.S. ((Hall, Wikswo, Pringle, Gould, & Parashar, 2014; Scallan et al, 2011).

Get Help With Your Essay

If you need assistance with writing your essay, our professional essay writing service is here to help!
Find out more about our Essay Writing Service

Results from a nationally representative survey of 1051 U.S. adults revealed that consumers are not aware of noroviruses and have limited knowledge about how to prevent an infection. Forty-seven percent of respondents reported awareness of noroviruses, and 85% had heard of the terms ''cruise ship virus,'' ''the stomach bug,'' or ''the stomach flu,'' which are commonly used to describe norovirusess (Cates, Kosa, Brophy, Hall, & Fraser, 2015). Less than one-third (341) correctly answered 50% or more of the 22 true-and-false questions, suggesting consumers have limited knowledge on how to prevent a norovirus infection. These findings support the need to provide consumers with accurate and complete information about noroviruses as knowledge is a prerequisite to implementation of a behavior (DiClemente et al., 1989; Fisher & Fisher, 1992).

The World Wide Web (hereafter referred to as the Web) is one way to reach consumers with information about how to prevent a norovirus infection. Eighty-seven percent (87%) of U.S. adults use the internet, and of those, 72 percent use the Web to seek health information (Pew Research Center, 2014). While the internet has accelerated the speed and spread of information, it also has a downside - allowing the dissemination and diffusion of information without checks and balances, peer review, and/or fact checking. Three studies that evaluated health information available online reported a lack of quality information characterized by accuracy of content and/or appropriate design and usability (Berland et al., 2001; Scullard, Peacock, & Davies, 2010; Irwin et al., 2011). Moreover, a systematic review of medical information further supported the above-mentioned authors' conclusions that there is a lack of high-quality sites on the Web that offer complete and accurate information on various health topics (Eysenbach, Powell, Kuss, & Sa, 2002). One problem with inaccurate or misleading health information being so accessible on the Web is that the user of this information might implement recommendations that have no effect on preventing the targeted disease or condition or even worse one might implement recommendations that cause harm (Ilic, 2010). In addition to providing accurate and complete information, it is also important to determine if recommendations, or changes that might become recommendations in the near future, are relevant and practical under real-world conditions.

Another problem is the readability of the text. If the text is not easy to read its usability is greatly diminished. To illustrate how wide ranging this problem could be one needs to simply read the National Assessment of Adult Literacy (NAAL), released in 2006 by the U.S. Department of Education. The authors report that 30 million adults struggle with basic reading tasks, which represents nearly 10% of the U.S. population (Kutner, Greenburg, Jin, & Paulsen, 2006). Within the context of health information, Kessels (2003) pointed out that 40-80% of information provided to patients by health professionals is forgotten immediately not only because terminology is too difficult to understand but often too much information is presented at one time leading to poor recall. One conclusion is that when text is too difficult to read, the reader could become frustrated hence they stop reading, hence the purpose of the materials is not achieved - improving knowledge.

Using the results of a readability formula is also recommended by experts to guide the development of written work as it is recommended to write documents for U.S. adults at the 7^th grade level and documents about health, medicine, or safety at the 5^th grade level (DuBay, 2004). As an assessment tool, one of the several dozen readability formulas are useful and convenient but must be employed with caution. These formulas determine the grade level demand of specific written information based on an analysis of sentence structure and word length. But, readability does not necessarily guarantee comprehension, which is the degree to which one understands what they have read (U.S. Department of Health and Human Services, 2010). Assessing comprehension is a much more complex task, which is why readability is more commonly used as a general indicator of how easy a document is to read.

Therefore, we assert that if consumer education materials are aligned with evidence-based guidance documents and are easy to read, the reader is more likely to engage in behaviors known to prevent Noroviruses. Thus, the aim of our study was to evaluate the alignment and readability of Noroviruses consumer education materials available on the Web using evidence-based guidance documents and Microsoft Word readability formulas. Two research questions guided our study: 1) Are noroviruses materials targeting consumers who handle food aligned with the three prevention strategies for noroviruses recommended in CDC guidance documents? and 2) Are materials written at the recommended grade level for health-related documents (5^th grade level)? Our study findings can be used to inform the revision or creation of consumer education materials that focus on preventing a norovirus infection.

METHODS

To be included in the sample, eligible educational materials (referred to as artifacts hereafter) were published between January 1, 2011 and May 23, 2013 and: 1) target adult consumers, 2) pertain to preparing food in the home, and 3) be available via Advanced Search of Google.com. The start date of our search was chosen because in January 2011, Scallan et al. (2011) reported that Noroviruses causes 58% of foodborne disease in the U.S. We excluded artifacts directed toward children and those formatted as blogs, news articles, theses, dissertations, research articles, Wikipedia entries, question/answer sites, forums, and continuing education training materials. We conducted a Google.com Advanced Search using the following search string: "Norovirus" AND "food handling" AND "food preparation." Food-related terms were chosen as we assumed this would narrow our findings to artifacts that address preventing Norovirus infections related to food preparation in the home. Artifacts were sorted by relevance, and two team members screened each artifact based on inclusion criteria. All eligible artifacts were downloaded as Portable Document Format (PDF) files or HyperText Markup Language (HTML) files.

A coding manual comprised of 71 items divided into four topic areas was created. The topic areas included: 1) identifying information, 2) format (including readability statistics), 3) content (including food safety hazards), and 4) prevention strategies. The items in Table 1 assess three prevention strategies outlined in the Centers for Disease Control and Prevention (CDC) guidelines for preventing and managing a norovirus infection: 1) hand hygiene, 2) exclusion and isolation of sick individuals, and 3) environmental sanitation (Hall et al., 2011). While these guidelines were designed for use in health care and not home settings, at the time of the analysis we believed and continue to believe they were the best source of evidence-based information to prevent norovirus infections.

A corresponding SurveyMonkey^® (Survey Monkey Inc., Paolo Alto, CA) instrument served as the coding sheet. Four trained coders independently conducted a pilot-test of the coding manual. Inconsistencies or ambiguities found in the manual during piloting were corrected before analysis began. Two trained coders then independently reviewed each artifact. Responses were entered into the SurveyMonkey^® instrument. All responses were exported to an Excel spreadsheet. A third coder reconciled disagreements between coders.

Based on alignment, each artifact was assigned a total score, and sub-scores across the three disease management guidance documents were assessed: hand hygiene, isolation and exclusion of sick persons, and environmental sanitation. The maximum possible quality scores for each topic area were 7, 2, and 5, respectively. Response frequencies as well as mean scores, standard deviations, and ranges were calculated using SAS 9.3 (SAS Institute, Inc., Cary, NC).

To assess readability, we used two formulas - Flesch Reading Ease and Flesch-Kincaid Grade Level - both available in Microsoft Word. Flesch Reading Ease scores range from 0-100, with a lower score indicating that a document is more difficult to read than one with a higher score. A score of 70 or above is classified as 'easy' and is written at the grade school level. A score of 60 to 70 is 'standard' and is written at the high school level. A score of 60 or below is 'difficult.' The Flesch-Kincaid Grade Level uses mean sentence and word length to determine grade level between grades 3 and 12 (D'Alessandro, Kingsley, & Johnson-West, 2001).

RESULTS

Our Google Advanced Search yielded 826 results. After opening and viewing the first 292 results (sorted by relevance) the following was displayed on the results listing screen: "In order to show you the most relevant results, we have omitted some entries very similar to the 292 already displayed. If you like, you can repeat the search with the omitted results included." We repeated the search and determined all results were duplicates of those already viewed. These 292 results were screened using our three eligibility criteria and yielded 74 artifacts which were then downloaded. After a second screening, 14 artifacts were removed due to page unavailability, duplicity, or wrong target audience. A total of 60 artifacts were included and analyzed.

Research Question 1: Alignment with Prevention Strategies

The total mean score across all three prevention strategies was low - 5.2 of 14 points - suggesting artifacts were not aligned with CDC guidance documents (Table 2). All (N=60; 100%) artifacts mentioned at least one of the seven components of hand hygiene we assessed but the mean sub-score for hand hygiene was low, 2.3 of 7 points (SD=0.77). Most artifacts (83%) did not address length of handwashing, type of drying devices (92%), avoidance of bare-hand contact with ready-to-eat foods (97%), or use of hand sanitizers (83%). One artifact stated that hand sanitizers were an acceptable alternative to hand washing. Of the artifacts that mentioned exclusion and isolation of sick individuals (78%), the mean score was 1.2 of 2. In nearly all artifacts (72%) sick individuals were discouraged from preparing food for others, but 48% recommended minimizing contact with sick persons. Less than half (48%) addressed at least one aspect of environmental sanitation with the mean quality score 1.7 of 5.

Research Question 2: Readability

The mean Flesch Reading Ease for our sample was 47.6, indicating artifacts were difficult to read as the score is <60 (D'Alessandro, Kingsley, and Johnson-West, 2001). The mean Flesch-Kincaid Grade Level was 10.4, which is more than five grade levels higher than the recommended target grade level (5^th grade) for health-related materials (D'Alessandro, Kingsley, & Johnson-West, 2001).

DISCUSSION

Knowledge is an underlying construct in many health behavior models necessary for one to take informed action, such as implementing strategies to prevent a norovirus infection. So it is reasonable to assert that if one does not have accurate and complete information about these strategies, then one cannot engage in necessary behaviors, defeating the purpose of consumer education. In addition to providing accurate and complete information, the information must also be practical and presented in a manner that is easy to read to increase the likelihood that the recommendations will be implemented. For these reasons, we analyzed consumer-targeted materials available on the Web to determine if existing materials need to be revised or new materials created as we believe consumer education is critical to preventing norovirus infections.

In our analysis, we found information was omitted and/or inconsistent with the CDC guidelines. For example, many artifacts did not address length of hand washing, use of hand sanitizers, type of drying devices, and avoidance of bare-hand contact with ready-to-eat foods. While many discouraged preparing food, less than half recommended minimize contact with sick persons. Furthermore, disinfection was frequently mentioned but the recommended concentration of sodium hypochlorite (bleach) solution to be used to disinfect surfaces contaminated with noroviruses was not. And, none addressed the clean up of vomit and fecal matter. It is through the proper implementation of the three prevention strategies (hand hygiene, exclusion and isolation of sick individuals, and environmental sanitation) that we begin to reduce the burden of illness attributed to noroviruses.

One obvious reason for the incompleteness and/or omission of information is the creators of materials in our sample might not have known about the CDC prevention guidelines. The guidelines were published in March 2011 in Morbidity Mortality Weekly Report (MMWR) which is in the public domain, but MMWR might not be a known or commonly used source of information for those who author consumer education materials. As a result, information to guide development of materials came from other sources, for which the evidence base might be unknown or possibly incorrect. To illustrate, it is well known that an effective hand wash can disrupt transmission of noroviruses with effective hand wash defined as applying hand soap to hands then using friction (rubbing hands together) for a sufficient time (Michaels et al., 2002). While a scrub time of 10-20 seconds is commonly recommended as sufficient by the CDC and the U.S. Food and Drug Administration (FDA, 2013; Hall et al., 2011), three laboratory studies report longer wash times are needed to remove microorganisms (Ojajärvi, 1980; Fuls et al., 2008; Jensen, Danyluk, Harris, and Schaffner, 2015). To further complicate matters, some experts question the evidence base for scrubbing hands for 10-20 seconds suggesting an even shorter time (Bloomfield et al, FILL IN).

These various "positions" on length of a handwash illustrates the importance of creating (and subsequently using) evidence-based guidelines to inform development of consumer education materials as recommendations cannot be based on one or two published studies. In health care, clinicians commonly use clinical evidence-based practice guidelines, recommendations systematically developed by panels of experts who have access to the available evidence and an understanding of the clinical problem and research methods as these clinicians, just like food safety educators, might not have the expertise to consider the quality of the evidence. Thus, evidence-based guidelines such as those prepared by the CDC should underpin consumer education efforts about preventing norovirus infections.

However, these guidelines should be routinely reviewed and updated as even within the CDC guidelines there are gaps in information as the present evidence base to inform norovirus prevention strategies is incomplete. When educators are faced with incomplete information they might use whatever information is available, even if it is not grounded in science, or and simply ignore omit details (Kardes et al., 2008; Sanbonmatsu, Kardes, & Herr, 1992; Simmons & Lynch, 1991). One clear example of this centers around vomit and fecal matter clean up, which none of the reviewed materials addressed. To disrupt the transmission of noroviruses, vomitus and feces must be properly cleaned up. But, how far out one needs to disinfect an area to eliminate noroviruses is unknown as the evidence to support a geographic radius is inconclusive. The only published evidence we have suggests that vomit can be deposited up to 25 feet from its point source but these findings are based on one epidemiological study and not a well-designed laboratory-controlled study (Booth, 2014).

Moreover, even when the evidence base is good, sometimes information that is presented is not complete. Case in point, many materials recommended using sodium hypochlorite (bleach) solutions to treat surfaces, which is correct, but most did not state a concentration of the disinfection solution. Norovirus is a hardy pathogen that resists most disinfection strategies and persists in the environment for long periods, possibly weeks (give citation) illustrating the importance of using a chlorine solution at the proper concentration (Doultree, Druce, Birch, & Marshall, 1999; Gulati, Allwood, Hedberg, & Goyal, 2001; Jimenez & Chiang 2006; Belliot, Lavaux, Souihel, Agnello, & Pothier, 2008). One explanation for this lack of detail could be that some authors presented motivational messages, which tend to be briefer than procedural messages. While motivation is important, alone it is not sufficient because one can be motivated to enact a behavior but do it wrong which is why consumers must have accurate as well as complete information. It is important to note that we did not evaluate the messaging approach as this was beyond the scope of this study but an analysis of messaging approach should be considered in future analyses of materials.

Another potential problem is that in the absence of consumer-targeted prevention strategies, government regulations are often used as a proxy. This appears to be a sensible approach as the presumption is that most regulations are grounded in science. However, one must exercise caution as regulations might not always based on the most current evidence given the often lengthy adoption process. Take for example the use of alcohol-based hand sanitizers as an alternative for hand washing, which the U.S. Food Code prohibits, and which most artifacts in our sample did not mention. In 2002 the CDC released Guidelines for Hand Hygiene in Healthcare but in May 2003 the FDA prepared a written response, clearly stating the CDC guidelines could not be applied to foodservice establishments (FDA, 2003). The underlying logic was that (1) pathogens commonly transmitted by hands in health-care settings differ from those transmitted in foodservice settings; (2) the use of alcohol-based hand rubs in place of hand-washing has not been shown to reduce important foodborne pathogens on food worker hands; and (3) the types and levels of soil on the hands of health care workers differ from that on the hands of foodservice and retail food handlers. Another concern was that while alcohol-based hand sanitizers have been demonstrated to be effective for many pathogens, their efficacy has not been consistently demonstrated for norovirus. It is unclear how much published evidence has been used to support these recommendations as much has been learned about hand hygiene in the last 20 years. In reality, information suggests that some hand sanitizer formulations may be somewhat effective while others are not. Obviously, this situation is complex, and in the interest of providing a clear message, it may be unclear whether to include recommendations regarding alcohol-based hand sanitizers or to rely on government regulations. The conservative approach would be to omit the recommendation; whereas, some authors may decide to provide the knowledge under the impression that recommending some hand cleaning behavior is better than no recommendation.

Some guidelines might be viewed by educators as impractical or unnecessary to implement in a home setting. An example of this is not handling food when sick, which was mentioned but minimizing contact with sick persons was addressed less frequently. This is a significant oversight as many people do not realize that noroviruses are transferred via the fecal-oral route (i.e., associated with contacting persons, contaminated surfaces, and food) (CATES ARTICLE). In a regulated food facility, workers cannot work while ill to minimize sickening others as well as contaminating surfaces and food. This recommendation might not be possible in a home setting as one probably cannot isolate themselves from others in the household as well as they might be the primary food preparer. This illustrates why it is important to analyze not only the accuracy and completeness of information but also its practicality.

Along with lack of alignment, the artifacts in our sample were also not easy to read -- written at a high school grade level and not at the 5^th grade level as suggested. Our study findings are consistent with a study conducted by Badarudeen and Sanjeev (2010) who concluded that in a health-care setting patient education materials were often too difficult to read and could not be comprehended by a substantial portion of the adult U.S. population. Therefore, even when information is accurate, if a reader cannot understand it, they cannot act on it. Educational materials are only as beneficial as the knowledge gained as a result of being able to read them.

Limitations

Due to the ever changing nature of the Web, artifacts sampled during the study could change as time progresses so new materials that might be better aligned with CDC guidance documents and that are easy to read might be available. Likewise, knowledge evolves: in late 2014, the CDC published on their website five messages to help consumers prevent a norovirus infections: 1) practice proper hand hygiene; 2) take care in the kitchen; 3) do not prepare food while infected; 4) clean and disinfect contaminated surfaces; and 5) wash laundry thoroughly (CDC, 2015). However, coding and analysis had already been completed so kitchen and laundry practices were not assessed, demonstrating the challenge of evaluating web-based health messages. Further, to analyze the readability of the artifacts in our sample, we used two readability formulas available through Microsoft Word. This method is widely used, but also not comprehensive, as it only considers factors such as word and sentence length so does not take into consideration comprehension. Future research should also be done to assess materials using the CDC Clear Communication Index, which is a set of 20 items that can be used to develop as well as assess public communication products (CDC, 2014a). The 20 items in the Index build on and expand plain language techniques described in the Federal Plain Language Guidance documents (Plain Language Action and Information Network, 2011).

CONCLUSION

At present, there still is a need to revise and possibly create materials that are accurate, complete, practical, and easy to read. Consumer education, if approached properly, could help reduce the burden of illness attributed to noroviruses, the number one cause of acute gastroenteritis and foodborne disease. To be effective, consumer-targeted materials must be aligned with evidence-based guidelines that carefully assess the practicality and necessity of the recommendations in a home environment. While government regulations are sometimes used as a proxy for recommendations to be applied in a home setting, one must use caution in doing so as the regulatory provisions might not be grounded in the most current science or might be impractical under home conditions. Most importantly, evidence-based guidelines also need to be periodically as the evidence base to inform practices continues to grow. Finally, the readability of the materials must be evaluated and REFERENCE THE CDC CLEAR COMMUNICATION INDEX.

ACKNOWLEDGEMENTS

This work was funded through a grant from the U.S. Department of Agriculture, National Institute of Food and Agriculture, Agriculture and Food Research Initiative, "Building Capacity to Control Viral Foodborne Disease: A Translational, Multidisciplinary Approach," Grant No. 2011-68003-30395.

REFERENCES

Abad, F. X., Pintó, R. M., & Bosch, A.(1994). Survival of enteric viruses on environmental fomites. Applied and Environmental Microbiology, 60(10), p. 3704-3710.

Atmar, R. L., Opekun, A. R., Gilger, M. A., Estes, M. K., Crawford, S. E., Neill, F. H., Ramani, S., Hill, H., Ferreira, J., & Graham, D. Y. (2014).Determination of the 50% human infectious dose for Norwalk virus. Journal of Infectious Diseases, 209, 1016-1022.

Badarudeen, S. &Sabharwal, S. (2010). Assessing Readability of Patient Education Materials: Current Role in Orthopaedics. Clinical Orthopedic Related Research, 468(1):2572-2580.

Barker, J., Vipond, I. B., & Bloomfield, S. F. (2004).Effects of cleaning and disinfection in reducing the spread of norovirus contamination via environmental surfaces. Journal of Hospital Infection,58(1), 42-49.

Belliot, G., Lavaux, A., Souihel, D., Agnello, D., &Pothier, P. (2008). Use of murine norovirus as a surrogate to evaluate resistance of huma norovirus to disinfectants. Applied and Environmental Microbiology, 74(10), 3315-3318.

Berland, G. K., Elliott, M. N., Morales, L. S., Algazy, J. I., Kravitz, R. L., Broder, M. S., Kanouse, D. E., Muñoz, J. A., Puyol, J.-A., Lara, M., Watkins, K. E., Yang, H., &McGlynn, E. A. (2001). Health information on the internet: accessibility, quality, and readability in English and Spanish. Journal of the American Medical Association, 285(20), 2612-2621.

Bloomfield, S. F., Aiello, A. E., Cookson, B., O'Boyle, C., & Larson, E. L. (2007). The effectiveness of hand hygiene procedures in reducing the risks of infections in home and community settings including handwashing and alcohol-based hand sanitizers. American Journal of Infection Control, 35(10), S27-S64.

Booth, C. M. (2014). Vomiting Larry: a simulated vomiting system for assessing environmental contamination from projectile vomiting related to norovirus infection. Journal of Infection Prevention, 15(5), 176-180.

Cates, S. C., Kosa, K. M., Brophy, J. E., Hall, A. J., & Fraser, A. (2015). Consumer education needed on norovirus prevention and control: findings from a nationally representative survey of U.S. adults. Journal of Food Protection, 78(3), 484-490.

Centers for Disease Control and Prevention. (2015). Preventing Norovirus Infection. Accessed 29 October 2015. Available at: http://www.cdc.gov/norovirus/preventing-infection.html

Centers for Disease Control and Prevention. (2014a). The CDC clear communication index. Accessed 27 August 2015. Available at: http://www.cdc.gov/ccindex/

Centers for Disease Control and Prevention. (2014b). Norovirus. Accessed 27 August 2015. Available at: http://www.cdc.gov/norovirus/

D'Alessandro, D. M., Kingsley, P., & Johnson-West, J. (2001).The readability of pediatric patient education materials on the World Wide Web. Archives of Pediatrics & Adolescent Medicine, 155(7), 807-812.

DiClemente, R. J., Pies, C. A., Stoller, E. J., Straits, C., Olivia, G. E., Haskin, J., & Rutherford, G. W. (1989). Evaluation of school-based AIDS education curricula in San Francisco.The Journal of Sex Research, 26(2), 188-198.

Doultree, J. C., Druce, J. D., Birch, C. J., Bowden, D. S., & Marshall, J. A. (1999). Inactivation of feline calicivirus, a Norwalk virus surrogate. Journal of Hospital Infection, 41, 51-57.

DuBay, W. H. (2004).The principles of readability. Costa Mesa, CA: Impact Information.

Eysenbach, G., Powell, J., Kuss, O., & Sa, E.-R. (2002). Empirical studies assessing the quality of health information for consumers on the World Wide Web: a systematic review. Journal of the American Medical Association, 287(20), 2691-2700.

Fisher, J. D., & Fisher, W. A. (1992). Changing AIDS risk behavior. Psychological Bulletin, 3, 455-474.

Fuls, J. L., Rodgers, N. D.,Fischler, G. E., Howard, J. M., Patel, M.,Weidner, P. L.,& Duran, M. H. (2008). Alternative hand contamination technique to compare the activities of antimicrobial and non-antimicrobial soaps under different test conditions. Applied and Environmental Microbiology, 74(12), 3739-3744.

Girard, M., Ngazoa, S., Mattison, K., & Jean, J. (2010).Attachment of noroviruses to stainless steel and their inactivation, using household disinfectants. Journal of Food Protection, 73(2), 400-404.

Gulati, B. R., Allwood, P. B., Hedberg, C. W., &Goyal, S. M. (2001). Efficacy of commonly used disinfectants for the inactivation of calicivirus on strawberry, lettuce, and a food-contact surface. Journal of Food Protection, 64(9), 1430-1434.

Hall, A. J., Lopman, B. A., Payne, D. C., Patel, M. M., Gastañaduy, P. A., Vinjé, J., &Parashar, U. D. (2013). Norovirus disease in the United States. Emerging Infectious Diseases, 19(8), 1198-1205.

Hall, A. J., Vinjé, J., Lopman, B., Park, G. W., Yen, C., Gregoricus, N., &Parashar, U. (2011). Updated norovirus outbreak management and disease prevention guidance documents. Morbidity and Mortality Weekly Report, 60(3), 1-15.

Hall, A. J., Wikswo, M. E., Pringle, K., Gould, L. H., &Parashar, U. D. (2014). Vital signs: foodborne norovirus outbreaks- United States, 2009-2012.Morbidity and Mortality Weekly Report, 63(22), 491-495.

Huang, C., Ma, W., & Stack, S. (2012). The hygienic efficacy of different hand-drying methods: a review of the evidence. Mayo Clini