ASSESING THE QUALITY OF HEALTH INFORMATION ON THE INTERNET

Khaldoun Almhanna, M.D.



Introduction 1,2

The Web can be used as a quick and direct reference source to answer many consumer health questions regarding many medical condition, syndromes, disorders, medical news, rare diseases, health products, and drug information.

In addition, the web is a source of information about health topics in early stages of research. However, information found on the web need to be filtered and many include voluminous misinformation or nonrelevant information. One of the best resources of valuable, reliable consumer health information is a U.S. government-sponsored, quality Web sited such as healthfinder, available at http//healthfinder.gov/ and MEDLINEplus available at http://www.nih.gov/medlineplus/. However, consumer may not be aware of theses web sites or want to distill specific information on a topic quickly, using available search engine.

Finding useful health information quickly on the Internet can challenge both the consumer and the health professional. Though the performance of currently available search engine has been improving continuously with powerful search capabilities of various types, lack of comprehensive coverage, inability to predict the quality of retrieved results, and absence of controlled vocabularies make it difficult for users to use search engine effectively. The use of the internet as a consumer health information source need to be carefully evaluated as no need to be able to provide informative recommendation to their clientele regarding the selection of search engines and effective search strategies, other than that and because the Internet become one of the most widely used communication media.With the availability of web server software, any one can set up a Web site and publish any kind of data, which is then accessible to all.

The problem is therefore no longer finding information but assessing the credibility of the publisher as well as the relevance and accuracy of a document retrieved from the net

In many cases, a given web site provides no appropriate documentation that support given claims.

Why do we need to evaluate what we find on the Internet

We need to evaluate health information on the Internet because

* Anyone can (and probably will) put anything on the Internet, and many times, there is no way to tell if something is reliable, accurate, truthful, meaningful, thoughtful, or researched.

* It is often difficult to tell what something is, where it come from, how it got there and who the author is.

*It is hard to tell if something is original .

* Many things are not “filtered” or reviewed by a refereeing process or by peer or an authority by publication or editor, or by librarian.

Criteria for evaluating a web site:

The large volume of health information resources available on the Internet has great potential to improve health, but it is increasingly difficult to discern which resources are accurate or appropriate for users. Because of the potential for harm from misleading and inaccurate health information, many organization and individuals have published or implemented criteria for evaluating the appropriateness or quality if theses resources.

A literature and world wide web search found that the most frequent cited criteria were those dealing with content, design and aesthetics of site, disclosure of authors, sponsors, or developers, currency of information, authority of sources, and ease to use.

Criteria related to confidentiality and privacy were only cited by couple of authors.

Before talking about the criteria in details I will like to show this table about the frequency of explicit criteria for evaluation of health related web sites by criteria groups 3


 
Criteria groups
Frequency(%)

(n=165)

Content of site(include quality, reliability, accuracy, scope, depth)
30(18)
Design and aesthetics(include layout, interactivity, presentation, appeal, graphics, use of media
22(13)
Disclosure of authors, sponsors,, developers(include identification of purpose, nature of organization, source of support, authorship, origin)
20(12)
Currency of information(include frequency of update, freshness, maintenance of the site)
14(8)
Authority of resource(include reputation of resource, credibility, trustworthiness)
11(7)
Ease to use(include usability, navigability, functionality)
9(5)
Accessibility and availability(include ease to access, stability)
9(5)
Links
5(3)
Attribution and documentation(include presentation of clear references, balanced evidence)
5(3)
Intended audience
3(2)
Contact address or feedback mechanism(include availability of contact information, contact address)
2(1)
User support(include availability of support, documentation for users)
2(1)
Miscellaneous(include criterion that lacked specificity or were unique
33(20)

(N=165) is the number of authors, r articles that set up criteria for evaluating a web site 

Here is some defined, ranked, and evaluated seven major criteria for assessing the quality of Internet health information: 4

 

Credibility

To determine the credibility of Internet health information, one must consider its source, currency, relevance/utility, and editorial review process.

Source:

Overall, The source of medical information is the premier criterion for its credibility and quality.

Credibility is defined as” the quality or power of inspiring belief”, and to be credible is defined as:” offering reasonable grounds for being believed”. To a great extent, the problems associated with distinguishing credible from less credible or even false information relate to the problem of accurately evaluating the source. A site should display the institution’s or organization’s name and logo as well as the name and the title of the authors.

There is a simple method for objectifying the credibility of Internet source.The characteristics of a credible resource, however follow from logical or common sense rules.The trusted authorities in a society are usually easily identified, and information that comes from or can be attributed to a trusted authority is credible. For example, the medical advice provided by physicians relating to an illness is considered credible unless proven otherwise. Individual health providers and organization such a consumer advocacy groups, voluntary health-related organization, public health communities, and patient support organization can also be considered credible resources of information relating to their area of expertise.Indeed, organized group of experts may be considered to have greater authority than individual experts given the likely focusing of collective knowledge of given issue. Similarly, hospitals, large groups practices government health agencies, and other entities that bring together medically knowledgeable professionals have aggregate credibility, University medical schools have the highest degree of medical credibility given that they are expected to represent collection of physician specialists who are working at or extending the leading edge of medical knowledge.Individual healthcare professionals generally gain increased credibility by virtue of an association with theses highly credible groups. 

Currency:

Currency in health-related web sites can be defined as keeping up to date with the present state of medical/clinical knowledge.Currency is expected on the Internet site of a credible resource, but in reality, this may fall short.The initial burst of enthusiasm that promote a medical author to produce a web site may soon be tempered by the realization of the time and effort involved in keeping the site up to date.

The date of the original document and the date of content posting should also be display so the user can judge the timelines of the information. Though the date of posting does not demonstrate that the information provided is correct or current, it does provide an indicator of currency. For example, if there was a recent discovery relating to the treatment of a disease, a site discuss treatment options dated several months ago probably would not include this discovery.Area with rapid development such as AIDS research demands very current information.For this reason, it is best to talk with health care providers regarding the timelines of the information

Relevance/Utility:

Relevance and Utility are attributes that will benefit the user of the site. Relevance to how closely the actual content of a site corresponds to the information it purports to provide.For example, if a site’s heading mentions recipes for diabetic patients, a recipe for non-diabetics may not be relevant.

Utility denotes the usefulness of a site. For example, suppose a user wanted to quit smoking, but did not know how to go about it.A site intended to help people stop smoking would not have much utility if it only discussed the reasons to stop smoking and did not provide tools that would help actually doing so.

Site evaluation:

In an academic community, the peer review process is used to insure the validity and the quality of the information presented in papers and reports. The general public, however, is more likely to understand a “seal of approval” from an individual or group commonly perceived as credible. Sites should indicate whether the information provided has just subjected to review, and if so, describe the process and individuals involved.

Content

The content of health information on the Internet must be accurate and complete. An appropriate disclaimer should also be provided.

Accuracy:

How does this information compare with that in other sources in the field?

Given that accuracy of content is based on evidence and its verification, the site should identify the data that underlies the conclusions presented.

Clinical or scientific evidence that support a position should be clearly stated. The framework of the study should also be described in language the lay person can understand. And users need to be aware that testimonials are not evidence

Quality of evidence:

Although much of the health care information available on the Internet is written at a level of the public to understand, it should still reflect the principles of evidence based medicine, including sound research mad expert opinion.

Clinical or scientific evidence to support a position should be clearly presented.For example, an article about a particular type of cancer therapy should include a discussion of supporting study.The framework of the study should be described in language that lay person could understand.What was the scope and what was the limitation of the study? Was the assignment of patients to treatment randomized? What were the results and the conclusions of the study? Do other studies substantiate the theory? Do other sources (for example peer-reviewed journal articles) support the theory?

The following table depicts the validity of the various types of health research for the consumer to consider to determine the accuracy of medical study. 4


 
Validity of evidence
What to look for
++++ (Best evidence)
Randomized controlled trial
+++
Non-randomized controlled trial
++
Well designed cohort or case-control analysis
+
Opinions of respected authorities, case report, descriptive studies, report of expert committees
No evidence
Misrepresentation, fraud

 

Disclaimer:

A disclaimer describing the limitations, purpose, scope, authority, and currency of the information should be provided. To ensure accuracy and avoid plagiarism and copyright violation, sources of the information should be disclosed. The disclaimer should emphasize as well that the content is general health information, not medical advice.

Completeness:

The discussion of a topic should be comprehensive and balanced. Users should be wary of one-sided views with critical information missing.

Pertinent facts, negative results, and a statement of any information not known about the subject should be included.

Disclosure

Web sites should provide appropriate disclosures, including the purpose of the site, as well as any profiling/collecting of information associated with using the site, so users can understand the intent of the organization or individual in providing the information.

Purpose:

The mission or purpose of the site should be clearly stated, and the information provided should be appropriate to that mission or purpose. To what audience the author is writing?Is this reflecting in the writing style, vocabulary, or tone? Does the material inform? Explain? Persuade? Is there sufficient evidence?

Profiling/collecting of information:

Web sites request and use information for purpose of which the users may be unaware. It is critical that users be informed of the collection, use, and dissemination of any information they may be providing in visiting the site. Only then can they make an informed decision to provide the information and or approve of its eventual use.

 

Links

Especially critical to the quality of an Internet site are its external links-connections to other internal pages or to external sites that form the web-like structure of information searchers within and among sites. There are four criteria for evaluating the quality of links: selection, architecture, content, and back linkage.

Selection:

The selection of links is made at the originating site.The person or group responsible for link selection should have the expertise and credentials to evaluate critically the appropriateness of those links. It is also important that the original and linked sites target a set of readers with similar characteristics.

Architecture:

The Architecture or design of pointers to linked sites is important for ease of navigation: whether there are timely escape mechanisms during side searches, whether the user can easily find his or her way backwards and forwards, and whether the structure is apparent and logical to the reader.

Image-based icons and textual identifiers should be meaningful and consistent.

Content:

The content of the links should be accurate, current, credible, and relevant.The content of the originating site is enhanced if it includes links to high-quality sites; on the other hand, links to poor quality sites indicate a lower-quality originating site.

Users should be alerted when they are about to view an external site. For example, sites could provide information relating to the linked resource before the user clicks to the site, and /or use transition screen.

Back linkages:

Back linkages are links form a web site to another. Many web sites track and publish back linkages for the purpose of enhancing their credibility and marketability.The best way to evaluate back linkages is to examine the context in which they are used, that is, their purpose, relevance, credibility, and authority, as well as any associated bias.

Design

The design or layout of the web site, including graphics and text, as well as links is important to the effective delivery and use of any web-based information, even though it does not affect the quality of the information per se. For example, do essential instructions appear before links and interactive portions? Do all parts work? Is using the site intuitive, or are parts likely to be misunderstood? Can you find your way around and easily locate a particular page from any other page? Is the structure stable or do features disappear between visits? Does it look friendly?The design of web sites can be evaluated in terms of accessibility, logical organization (navigability), and internal search capability.

Access:

Is the web site written in standard html (hyper text markup language), or have proprietary extensions been used? Must you download software to use it? Do part of it take too long to be load? Is it usually possible to reach the site, or is it overloaded?Is it stable or the URL (Uniform Resources locator) keep changing? Does the site use words the average person would try in a search engine? Is it open to everyone on the Internet, or do parts require membership and/or fees?Are there rules for use stated up front? Web sites should be accessible by the lowest level available browser technology. Other features to improve access include option for assessing the information when multimedia browsers are not available, as well as options for enabling use by the people with disability (hearing and seeing impaired……).

Logical organization (navigability):

The best web sites are clearly focused on their purpose and target audience.They are simple, internally consistent, and easy to use. Cross-references are provided to aid the user in comprehending the overall structure of the information.The composition of the information reflects an awareness of reading level, language, and the need for a balance of text and graphics, color and sound.

Internal search capability:

An internal search engine is a highly desirable component for most web sites with depth and breadth of content.The scope and function of the search engine-what it covers and how it works- should be clearly described.The search engine should be capable of searching specified content by keyword or search string and retrieving only relevant materials.It should also have a user interface that is easy to understand and use.

Interactivity 

Interactivity includes three criteria: Mechanism for feed back, chat rooms, and tailoring

Mechanism for feedback:

Reputable journals provide a feedback mechanism for their readers, and so, too, should web sites.The capability for interaction is unique benefit of the Internet.For example, a link to send criticism and comments to the site’s resource should always be included with the original information.Users should be able to comment on the validity and value of the information, and possibly point out areas of omission or obvious bias.A professionally operated web site will endeavor to respond to user feed back within a reasonable amount of time.

Chat rooms:

Chat rooms allow information to be exchange among many individuals, often anonymously.Whether a moderator is present should be posted, along with a warning that the information may not be accurate.If a moderator is present, the individual should be identified, together with his /her expertise and affiliations, and the source of his/her compensation

Tailoring:

In cases where web site provides an interactive service, such as tailoring information to the user based on clinical algorithms, the algorithm should be stated, including its developer and the site’s affiliation with the developer.

Caveats

Consumers should recognize that while use of the Internet for health information can be wonderful and educational experience, it can also have potential drawbacks. In particular, personal information may be easily given, stored, and shared for purpose that users may bot be aware of or agree to those uses.

Additionally, information may be inaccurate; thus, the consumers must be alerted to potential quack medical claims.

Alert:

The consumer must be wary of sound-a-like names or names that seem prestigous.Impressive names, conceived by shrewd marketing strategies, can be quite misleading, the American institute of Drug Analysis may sound like a well-equipped facility with academically credential staff, but it may in fact be a poorly instrumented and staffs laboratory in a garage.There is no legal restriction on the naming of business.

Products and services:

Site that sells products and services in health may be regulated by the FDA as to what they can clam.

Consumers must be highly skeptical of claims of “amazing results”, “earthshaking breakthroughs”, or miracles, as well as “secret cures” knows only to purveyor.Likewise, they should be wary if basic science widely accepted medical principle, or sound public health policies are attacked.

Health on the net foundation HON code: 5

The health on the net foundation code of conduct (HON code) for medical and health Web sites addresses one of the Internet’s main health care issues: the reliability and credibility of information.

The HON code is not an award system, nor does it intend to rate the quality of the information provided by a Web site.It only defines a set of rules and principles to hold web site developers to basic ethical standards in the presentation of information; and help make sure readers always know the source and the purpose of the data they are reading.

How it works?

You can use the HON code to verify a medical or a health site by entering the relevant URL and click GO. And HON’s MedHunt search will display results from its data –base, but you should be aware that if a given web site does not bear the HON code seal and is not listed on MedHunt, is NOT necessarily and indication of poor quality.

This is a very short interactive questionnaire that will help you assess if a Web site follows the ethical principles highlighted by HON code.

·The site provides and/or hosts medical /health information and advice

·The information and advice Is given by medical /health professionals

·A clear statement is made that the information or advice is offered bynon- medical or professionals or organization

·There is no statement readapting authorship.

·The site and its mirrors respect the legal requirements for medical information privacy that apply in the country and state of their location

·yes

·No

·I have no knowledge of these law

·The last modification date of the web site is clearly displays

·Yes

·No

·The site contains information from other resources

·An HTML link is provided to the source data

·A bibliographic reference to the source data is provided

·No reference to the source is made

·The site makes claims relating to thebenefit or performance of a specific medical treatment , commercial product or service

·The claims are supported by clear references to scientific research results and/or published articles

·The claims are based on the personal research or opinion of the site developer.

·All the pages of the web site display the contact e-mail address of the webmaster or a link to it

·Yes

·No, only some of the pages

·No, none of the pages

·No, the site only offers a feedback form.

·The source of the funding of the site is

·Clearly described

·No funding explanation is provided

·The site displays banner advertising, which is a source of income:

·Clearly described

·No funding explanation is provided

And the results of these questionnaires will help you to identify which ethical aspects of the HON code principles your chosen Web site does not respect.

Comparison of health information Megasites, and search engines

Here is a listing of the major search engines and directory services on the world-wide web and their features6:


 
Search engine/ directory
URL
Feature
AltaVista
http://www.altavista.com//
One of the biggest and most powerful search engines on the web. It has a partnership with looksmart who provides it with its directory listing. It also features a directory and online shopping
HotBot
http://www.hotbot.com//
Comprehensive; intuitive; user friendly and fast search engine
Excite
http://www.excite.com//
Useful ability to list result by web sites as well as search for documents that are similar to any given in the results. All results are given a relevancy rating. 
Infoseek
http://www.infoseek.com//
Runs search engine and separate directory. Results are automatically given as both web pages and as reviewed sites from its directory listing. 
Lycos
http://www.lycos.com//
Features ‘top 5%’, which are sites, that reviewers have picked as being the best on the web.Can also search for pictures and sounds using keyword search.
Yahoo
http://www.yahoo.com//
The most popular directory on the web, listing more than 800 000 web sites. It has links to all the major search engines
LookSmart
http://www.looksmart.com//
Contains a directory of 20000 subjects covering 300 000 web sites. Also features smart shopper and Classified advertisement

Comparisons here are made across many famous health information sites (As of April 2000)with specific focus on Target audience, who selects, and last modified, availability of search engine, and availability of help. 7


 
Site name
Site address
Target Audience
Who selects
Last modified
Search engine
Help
Achoo
www.achoo.com
medical community
Team
not stated
available
available
Biosites
www.library.UCSF.edu
Not stated
Librarians
not stated
available
available
Clinic Web
www.web-clinic.com
students and practitioners
not stated
not stated
available
Einet Galaxy
http://galaxy.einet.net
not stated
Information specialists
not stated
available
available
GHN

Global health network

www.astralcom.com/ghn
public health and prevention
Various professionals
within 2 months
HON

Health on the net foundation

www.HON.com
health care professionals and providers
not stated
within 2 weeks
available
Health AtoZ
www.healthAtoZ.com
comprehensive: health and medicine
Medical professionals
not stated
available
Healthfinder
www.healthfinder.com
public
not stated
within one month
available
Healthlinks
www.healthlinks.com
health sciences faculty, student, and staff
Librarians
within 2 weeks
available
HealthWeb
www.healthweb.org
health care professionals
Librarians, information professionals, and health professionals
every 3 months
HSLS, U.

Pittsburgh

Health Science library system

www.hsls.pitt.edu
not stated
not stated
within one month
Martindale
www-sci.lib.uci.edu/ ~ Martindale
not stated
not stated
within one week
MedExplorer
www.MedExplorer.org
not stated
not stated
not stated
available
MDchoice
www.mdchoice.com
Physician and patients
Physician
Daily
Medical matrix
www. Medical matrix.org
USA physicians and health care professionals
Practicing clinicians
Daily
available
Medical world search
www.mwsearch.com
clinical practitioners or consumers
Editorial board
not stated
available
available
MedWeb
www.medweb.emory.edu
not stated
Software(search engine/spider)
not stated
available
MMRLS*
www.MMRLS.org
not stated
not stated
not stated
available
NOAH
www.NOAH.cuny.edu
public, consumers
Librarians and content committee
within one week
available
available
OMNI
www.OMNI.ac.UK
UK and world
Volunteers, information specialists
within one week
available
available
Virtual naval Hospital
www.vnh.com
patients and health care providers
Editorial board
continuously
available
WebDoctor
www.gretmar.com/webdoctor
physicians
Physicians
11/27/97
available
Yahoo
www.yahoo.com
not stated
not stated
not stated
available
available

* Metrowest Massachusetts Regional Library System

Summary

The Internet presents a powerful mechanism for helping users improve their health-care decision making by providing easy and rapid access, exchange, and dissemination of enormous amounts of health information.Yet users must be aware of the potential for misinformation and recognize the critical need to assess the quality of the information provided. Content provider must be encouraged to develop and post high-quality information, and policy makers and health care professionals must be educated on this important health issue.

References:

1.Gang Wu, Jie Li., Comparing web search engine performance in searching consumer health information. Bull. Medical Library Association. 1999:87(4)

2.Finding and judging advice online. The New York Times. March, 6, 2000

3.Paul Kim et al., Published Criteria for evaluating health related web sites: Review. BMJ 1999;318:647-649

4.Helga E. Rippent et al. For assessing the quality of health information on the internet, http://hitiweb.mitretek.org/hswg/

5.Health on The Net Foundation, http://www.hon.com//

6.Downes Pk.Successful Web Searching. British Dental Journal.1998:Oct 24; 185:393-399.

7.http://www.lib.umich.edu/megasites/abstract.htlm