September 30, 2003, YONKERS, N.Y. -- Consumers need better help to navigate the rapidly growing amount of health care information available on the World Wide Web, even though at least two dozen sets of guidelines and criteria for improving health Web site quality have been published. At the same time, newspapers, trade journals and other media should do more to call attention to those attributes that make a health Web site relevant and useful, according to two studies published today by Consumer WebWatch.

The first study, which evaluated existing criteria sets for health sites put forth by medical institutions, professional groups, government organizations and others, is a joint project of the Health Information Institute, Bethesda, Maryland, and Consumer WebWatch, and was authored by Peter Goldschmidt, M.D., PhD., and the institute's founder and president. The second study, analyzing discussions of health Web sites in the medical and popular media, was commissioned by Consumer WebWatch from the Annenberg School for Communication at the University of Pennsylvania, Philadelphia. It was directed and edited by Joseph Turow, Ph.D., Robert Lewis Shayon Professor of Communication, and a member of Consumer WebWatch's advisory board.

Together, the studies illustrate problems facing consumers and health Web site publishers, as the number of health sites grows and as more people use them to seek information. At least 22 sets of criteria exist for evaluating Web site information. Many are duplicative, only a quarter are intended by their authors for consumer use, and no single set has emerged as useful to consumers.

In addition, there is disagreement among the health community and the medical press about what constitutes a "quality" health Web site, what Web sites actually exhibit, what Web sites actually achieve "quality," and how important it is that a Web site be "clinically accurate" — in other words, that it presents information matching accepted medical wisdom on a particular treatment, syndrome, disease or drug. Further, in the popular press, health site "seals-of-approval," accreditations and other means of judging quality have had little to no visibility.

"This is a difficult situation for consumers, and an almost impossible one for health sites," said Beau Brendler, director of Consumer WebWatch. "Who are consumers supposed to listen to, and how can health Web sites be expected to measure up to so many different sets of rules?"

The good news, Brendler said, is that more work has been done on credibility and quality issues among health Web sites than any other type of site. "But there's been a lot of talking and not much leading," Brendler said. "A trusted organization, group or individual needs to come forward with a set of guidelines consumers can use, that make sense to health Web site publishers, and that guide people toward better sites and away from bad ones."

Highlights of the Health Improvement Institute/Consumer WebWatch study:

• None of the existing 22 criteria sets for health site quality met the Institute's evaluation criteria. The most telling failures were: lack of testing, lack of consumer-friendliness, and failure to define criteria in operational terms — in other words, criteria are not useful to create evaluations or ratings.
• There are no criteria sets readily usable by consumers, and none readily usable by professionals to reliably assess health Web sites.
• · One previously published study, found by the Institute, evaluated the quality of Web sites related to asthma. That study's authors used the Health On the Net Foundation (HON, http://www.hon.ch/) code as the basis for their evaluation. They found that 16 percent of the health sites they examined met HON Code principles. Of sites they found that displayed the HON logo, only 38 percent actually complied with the HON principles.

"There are no user-friendly tools for consumers to use, and they cannot rely on existing seals of approval, to assess the credibility or reliability of health websites," said Goldschmidt, who authored the study. "Our findings to date validate the Institute's work and the need for programs such as Consumer WebWatch to draw attention to a significant problem and the lack of adequate solutions, and to work toward enabling consumers to evaluate Web sites and health information."

Highlights of the University of Pennsylvania Annenberg School study:

• In popular media, the Web is overwhelmingly treated positively as a way to search for health information.
• Systematic critiques of Web sites are rare.
• Health Web sites are mentioned largely in passing to quote information or to send people to information.
• Seal-of-approval programs, such as HON, are almost never mentioned.
• Mentions of best practices — indicators of quality that Web sites have and which audiences ought to consider — are rare.
• The "digital divide" — that Web information is most accessible to those who can afford it — is not presented as an issue. Neither is Web literacy.
• Sites of government medical organizations, often among the most respected for objectivity, alter medical information for political, as opposed to medical, reasons.

"Our findings suggest that the popular press, which is filled with health stories, is doing little to help the public understand one of the biggest health stories in decades — the rise of an enormous and often confusing online health information universe," said Professor Turow, who directed the study. "Advocates of better health information ought to work with producers of magazines, newspapers and TV programs to address how their audiences can become critical users of health information online."

In the future, Consumer WebWatch and the Health Improvement Institute plan to develop independent ratings of health Web sites. In the meantime, concerned consumers may want to judge health sites considering these quality characteristics, based on the Institute's research and Consumer WebWatch's guidelines for improving Web site credibility:

• Sponsorship. Who owns and operates the site? Is it a medical institution, a group of doctors in private practice, a company selling nutritional supplements, a pharmaceutical manufacturer?
• Purpose. What is the Web site's intent? What is the scope of information presented? Does it publish positive and negative information about a particular treatment or drug?
• Audience. Who is the Web site intended for? What is the intended use for the information on the site?
• Currency. How often is the site updated? Does the site say how frequently information is updated, and the date of last update?
• Sources and credentials. What is the origin of information cited, and what is the expertise of the person who wrote it? Does the site contain this information?

To view/download evaluation paper

To view complete media release