In 2005 the RAND corporation ‘guesstimated’ figures for healthcare IT benefits based on existing, limited data sets (http://www.rand.org/pubs/research_briefs/RB9136/index1.html).
Their methods and results were contested, such as by the Heartland Institute (as here). Recent studies also put the RAND assumptions including those about flawless implementation and acceptance in serious doubt (as here).
Similarly, the often-quoted 2000 Institute of Medicine (IOM) report "To Err is Human" used several small studies to extrapolate a death rate due to medical mistakes at 98,000 per year. That report was challenged by medical informatics pioneer Clement McDonald, MD in a 2000 JAMA article "Deaths Due to Medical Errors Are Exaggerated in Institute of Medicine Report" here, PDF.
However, the refutation didn't stop anyone who could benefit from quoting such numbers from touting them, including those pushing healthcare IT. In fact figures such as these have been used as a bludgeon to affect national policy on healthcare IT.
If the benefits of HIT can be guesstimated, so can and should the dangers.
The true incidence of health IT-related medical errors, injuries and deaths from IT error, poor design, cognitive overload, clinical disruption, etc. is unknown. The Joint Commission in its 2008 Sentinel Events Alert advised that "There is a dearth of data on the incidence of adverse events directly caused by HIT overall" (as here).
As a thought experiment, presented here are some estimated corrective factors that might shed some light on the actual figures for HIT-related injuries and deaths "on the ground" when national, universal HIT deployment is achieved.
The thought experiment:
Jeffrey Shuren, MD, JD of FDA mentioned reports of 44 injuries and 6 deaths over 2 years that are likely the "tip of the iceberg" at the Feb. 25, 2010 HIT Policy Committee Adoption/Certification Workgroup on IT safety (http://hcrenewal.blogspot.com/2010/02/fda-on-health-it-adverse-consequences.html).
Ross Koppel, PhD at the same Feb. 25 IT safety conference opined that "We don't know 99 percent of the medication ordering errors that are made [due to difficulty in recognition, lack of proper studies and other factors - ed.]. If 100 percent of the known errors were reported, that would be 1 percent of the [true] total. But the data suggests that the maximum on voluntary reporting is about 5 percent. So 5 percent of 1 percent that is what we know is reported...."
Now, some estimated corrective factors:
- Due to a severe lack of awareness by physicians and healthcare organizations of FDA as a "go-to" organization to whom to report HIT-related medical errors and harm, let us assume FDA is privy to only 1/100 of the known injuries and deaths due to HIT (mostly reported locally). That's probably a conservative figure. Corrective factor = 100x
- Based upon figures from "Electronic Health Records in Ambulatory Care — A National Survey of Physicians", NEJM 359:50-60, July 3, 2008, let us roughly estimate that 10% of doctors/organizations are using HIT "meaningfully" (or better term, "in good faith") in 2010. Corrective factor for HIT-related errors that cause harm = 10x for universal use nationwide.
- Per Koppel, only a maximum of 5% of medication errors are voluntarily reported at all. Assuming similar figures for [medication and other] HIT-related errors that cause harm, corrective factor = 20x
- In approximating the actual number of HIT-related medication errors based on reported numbers, we would need to multiply reported figures by 100x per Koppel (since only 1% of HIT-related medication errors are recognized as HIT-related). However, from the JC Sentinel events alert figures from the United States Pharmacopeia MEDMARX database from 2006, only 1.25% (i.e., approximately 1%) of medication errors actually cause harm, so reported figures on medication errors require no corrective factor regarding patient harm (corrective factor for HIT-related medication errors that cause harm = unity = 1x).
- Although the learning curve never becomes flat in part due to clinician unfriendliness of the software, newcomers to HIT will for a number of years be more error-prone and experience more HIT-related adverse outcomes than experienced users and organizations. Conservatively assume a corrective factor = 2x.
Overall, that creates a corrective factor of 100 x 10 x 20 x 2 = 40,000
Multiply the FDA figures per year (22 injuries and 3 deaths) by this factor = 880,000 injuries per year, 120,000 deaths when universal HIT use is achieved.
Merely multiply by 4,000 for a guesstimate of current figures at ~10% diffusion.
This assumes no major changes occur with regard to the technology's risks, such as via the accelerated interdisciplinary research in biomedical informatics, computer science, social science, and health care engineering recommended by the National Research Council in their Jan. 2009 report on HIT (http://www8.nationalacademies.org/onpinews/newsitem.aspx?RecordID=12572), and via other studies and improvements that might be made.
The way this industry operates, however, e.g., as in my post "Healthcare IT Corporate Ethics 101", the National Research Council-recommended changes are unlikely in the near future.
While this is a mere thought experiment, the result certainly suggests we need to know the actual rates of HIT-related patient harm, and act to understand and minimize these events.
In setting national healthcare policy, we should not rely on thought experiments - but even more importantly, we should not be relying on guesswork and wishful thinking as we are currently.
