... We conclude that the materiality of adverse event reports cannot be reduced to a bright-line rule ... Because adverse reports can take many forms, assessing their materiality is a fact-specific inquiry, requiring consideration of their source, content, and context.
Wall Street Journal author and "Numbers Guy" Carl Bialik adds to that point in an article today "Making a Stat Less Significant" where he writes:
To determine whether a medical side effect is significant in an experiment requires knowing that every incidence of that side effect is being reported. Researchers can feel confident that is happening in a controlled clinical trial of a drug, but they can't be sure when a drug is being sold to the general public, as was the case with Zicam.
In other words, when one is not sure that every incident of a side effect is being reported, one should not cavalierly dismiss "anecdotal" reports of side effects, especially from reliable reporters.
The practictioners of Medical Informatics, along with the HIT Industry and its customers, appear to have failed in that regard with respect to clinical IT (electronic medical records, CPOE etc.) For years they have argued that these medical devices should not be regulated because that would "stifle innovation" and that reports of device adverse events were "anecdotal." Many in the field still make these arguments.
This view extends all the way up to the Director of the Office of the National Coordinator for Health IT, who glibly stated per the Aug. 2010 Huffington Post Investigative Fund article FDA, Obama Digital Medical Records Team at Odds over Safety Oversight that FDA's own reports of health IT related injuries and deaths were “anecdotal":
ONC director Blumenthal, the point man for the administration, has called the FDA’s injury findings “anecdotal and fragmentary.” He told the Investigative Fund that he believed nothing in the report indicated a need for regulation.
Those "injury findings" appear in an FDA Internal Memo made available by the aforementioned Huffington Post Investigative Fund and archived at the following link:
Internal FDA memorandum on HIT risks (PDF) to Jeffrey Shuren MD JD (Director, Center for Devices and Radiological Health). Health Information Technology (H-IT) Safety Issues. "This is an Internal Document Not Intended for Public Use." Feb. 23, 2010.
(My description/summary of the memorandum is at my Aug. 2010 post "Internal FDA memorandum of Feb. 23, 2010 to Jeffrey Shuren on HIT risks. Smoking gun?")
That memorandum itself emphasizes how FDA's own knowledge of these events is partial due to reporting impediments and lack of knowledge of resources such as FDA's MAUDE database.
The known reports were likely "the tip of the iceberg" according to the Director of FDA’s Center for Devices and Radiological Health (CDRH) Jeffrey Shuren, MD, who also happens to be a lawyer.
As at the aforementioned "tip of the iceberg" link, at an HHS meeting of the HIT Policy Committee's Adoption/Certification Workgroup on February 25, 2010, Shuren testified:
... In the past two years, we have received 260 reports of HIT-related malfunctions with the potential for patient harm – including 44 reported injuries and 6 reported deaths. Because these reports are purely voluntary, they may represent only the tip of the iceberg in terms of the HIT-related problems that exist.
Even within this limited sample, several serious safety concerns have come to light. The reported adverse events have largely fallen into four major categories: (1) errors of commission, such as accessing the wrong patient’s record or overwriting one patient’s information with another’s; (2) errors of omission or transmission, such as the loss or corruption of vital patient data; (3) errors in data analysis, including medication dosing errors of several orders of magnitude; and (4) incompatibility between multi-vendor software applications and systems, which can lead to any of the above.
The problem with ignoring testimony and reports of health IT-related difficulties and dismissing them as "anecdotal" goes back to the issue of "knowing that every incidence of that side effect is being reported."
While FDA itself admits significant doubt about completeness of reporting in its memo, what's worse is that Koppel and Kreda at University of Pennsylvania wrote a paper from which one might conclude that the healthcare and health IT industries are themselves aligned to conceal health IT adverse events reports.
In their remarkable article Health Care Information Technology Vendors' "Hold Harmless" Clause - Implications for Patients and Clinicians, Journal of the American Medical Association, 2009;301(12):1276-1278, we learn that there is little motivation for device safety in the health IT industry:
Healthcare information technology (HIT) vendors enjoy a contractual and legal structure that renders them virtually liability-free—“held harmless” is the term-of-art—even when their proprietary products may be implicated in adverse events involving patients. This contractual and legal device shifts liability and remedial burdens to physicians, nurses, hospitals, and clinics, even when these HIT users are strictly following vendor instructions...HIT vendors are not responsible for errors their systems introduce in patient treatment because physicians, nurses, pharmacists, and healthcare technicians should be able to identify—and correct—any errors generated by software faults. [In other words, they are expected to be clairvoyant when presented with erroneous or missing data - ed.]
We additionally learn that:
The significant disparity between buyers and sellers in knowledge and resources [about healthcare IT problems] is profound and consequential. Vendors retain company confidential knowledge about designs, faults, software-operations, and glitches. Their counsel have crafted contractual terms that absolve them of liability and other punitive strictures while compelling users’ non-disclosure of their systems’ problematic, or even disastrous, software faults.In other words, health IT customers and users have a gag order imposed on them regarding software faults and defects.
I think any reasonable person would conclude there is great doubt as to whether "every incidence of [HIT side effects] is being reported."
I also pointed out in JAMA (link) and on my Drexel website (link) how agreeing to these terms caused hospital executives to violate both their fiduciary duties to their organization's workers as well as Joint Commission safety standards obligations.
(I've personally reported health IT defects I'd observed in hospitals where my relatives were patients to FDA's MAUDE database, discovering that the institution itself, whose officials I alerted to the problems, did not. An example is here.)
The above is all common sense.
Thus, the dismissal of reports of health IT-related patient injury, deaths, and "near misses" represents a failure of common sense, as well as a massive abrogation of fiduciary responsibilities and legal and ethical obligations among the Medical Informatics, health IT vendor, healthcare delivery, and healthcare regulatory sectors.
One end result is that it permits software like this to be mandated by state governments on hundreds of hospitals. One can only imagine the public, press and legal reactions if mission-critical software issues of this magnitude were brought forth after an aviation or nuclear power plant disaster.
The cavalier dismissals of HIT mishap reports clearly fall into the "knew, or should have known" category of negligence.
Plaintiff attorneys for patients injured or killed via HIT-related mishaps should take note.
-- SS
Note: my WSJ comment on this issue appears here.
