Those Who Dismiss Healthcare (and Healthcare IT) Adverse Events Reports as Mere "Anecdotes" Have Lost - Supreme Court-Style

At my Sept. 2010 post "The Dangers of Critical Thinking in A Politicized, Irrational Culture" I wrote:

... It's the EMR "anecdotalists" (as opposed to the "Markopolists") who say that "anecdotes" of HIT-related injury are meaningless. They deem reports of safety issues and HIT-related misadventures and risk as simply "anecdotal", and that "anecdotes don't make evidence" (or "anecdotes don't make data").

For "anecdotes" of patient harm due to medical devices even from the most reliable of sources to be counted as "evidence" of device risk, apparently, the stories need to be blessed with Statistical Holy Water. The Holy Water must also be of a brand approved by the academic pundits.

For me, this is no longer merely a professional debate. My elderly relative became one of those "anecdotes" in May last year.

I address the casual, Dogbert-style, waving-of-the-hand "Bah!" dismissal of health IT harm "anecdotes" at numerous other posts as well, such as "EHR Problems? No, They're Merely Anecdotal" and "Health IT: On Anecdotalism and Totalitarianism".

Bah! Your Health IT adverse events reports are anecdotes, and anecdotes don't make data!

In those posts I also mention how Australian informatics professor Dr. Jon Patrick had essentially hit the flaws of this argument out of the Southern hemisphere with a short editorial in the journal "Applied Clinical Informatics" entitled "The Validity of Personal Experiences in Evaluating HIT." That essay is free at the link and is worth reading.

Interestingly and thankfully, the "anecdotes are meaningless" crowd have now lost, and lost big - Supreme Court style. In fact, the U.S. Supreme Court has shown far more common sense than many esteemed academics and industry pundits.

As noted in this post at Derek Lowe's pharmaceutical industry "In the Pipeline" blog, the company that made "Zicam", a zinc-based over-the-counter cold remedy, tried to defend shareholder suits that the company withheld case reports of Zicam causing permanent loss of smell via arguing that such reports "did not reach a level of statistical significance", i.e., were "anecdotal." The case went to the U.S. Supreme Court.

The Supreme Court would have none of that argument:

"Matrixx’s [Zicam's manufacturer - ed.] premise that statistical significance is the only reliable indication of causation is flawed. Both medical experts and the Food and Drug Administration rely on evidence other than statistically significant data to establish an inference of causation. It thus stands to reason that reasonable investors would act on such evidence.

The full court decision is at this link: http://www.supremecourt.gov/opinions/10pdf/09-1156.pdf (PDF file), but a passage I consider key to this issue is as follows:

... We conclude that the materiality of adverse event reports cannot be reduced to a bright-line rule ... Because adverse reports can take many forms, assessing their materiality is a fact-specific inquiry, requiring consideration of their source, content, and context.

This is common sense incarnate. It applies not just to drugs, but to medical devices, to health IT, and to other domains as well.

In essence, it is saying that adverse events reports, especially repeated ones, from trustworthy sources are not to be lightly dismissed, but should serve at the very least as red flags that there may be a systemic problem requiring further investigation.

One wonders how and if public healthcare IT vendors will begin disclosing "anecdotal" reports of their products causing patient harm to their own stockholders.

One also wonders if the academic anecdotalists (up to the level of the chair of the Office of the National Coordinator for Health Information Technology at HHS) will cease their unfettered dismissal of health IT AE reports as mere "anecdotes" and therefore let's roll out this 100% beneficent technology nationwide ASAP:

"Nothing [ONC has] found would give them any pause that a policy of introducing EMR's could impede patient safety." - David Blumenthal

That sounds a bit like the refrain of the makers of Zicam.

One might also wonder if the anecdotalists merely lack common sense, or are using this form of epistemological dementia to obscure conflict of interest.

On a final note, my favorite comment at the aforementioned "In the Pipeline" blog story is this by anonymous commenter "Still Scared of Dinosaurs":

One of the most important ideas real statisticians must get into their heads is "Thou shalt not worship the 0.05 threshold". The whole concept of "statistical significance" for AEs is idiotic and the fact that Matrixx based any part of their defense on it indicates that their stupidity did not end when they named the company.

Perhaps this Dilbert cartoon is apropos to the Supreme Court decision:


-SS

Addendum:

I thought it appropriate to share these thoughts with the leadership of the Joint Commission, the organization that accredits healthcare organizations in the United States:

From: Scot Silverstein
Sent: Sunday, March 27, 2011 10:41 AM
To: MGiuntoli, Anita; Chassin, Mark; Schyve, Paul; Legaspi, Shirley
Cc: Ross Koppel; 'David Kreda'
Subject: Re: MATRIXX INITIATIVES, INC., ET AL. v. SIRACUSANO ET AL.

Not a complaint this time [about health IT failure - ed.], but an observation.

The JC has noted health IT risks in the Sentinel Events Alert "Safely implementing health information and converging technologies" of 2008.

The company that made "Zicam", a zinc-based over-the-counter cold remedy, tried to defend shareholder suits that the company withheld case reports of Zicam causing permanent loss of smell via arguing that such reports "did not reach a level of statistical significance", i.e., were "anecdotal." The case went to the U.S. Supreme Court.

The Supreme Court would have none of that argument:

"Matrixx’s [Zicam's manufacturer] premise that statistical significance is the only reliable indication of causation is flawed. Both medical experts and the Food and Drug Administration rely on evidence other than statistically significant data to establish an inference of causation. It thus stands to reason that reasonable investors would act on such evidence.

The full court decision is at this link: http://www.supremecourt.gov/opinions/10pdf/09-1156.pdf (PDF file), but a passage I consider key to this issue is as follows:

... Because adverse reports can take many forms, assessing their materiality is a fact-specific inquiry, requiring consideration of their source, content, and context.

This is common sense incarnate. It applies not just to drugs, but to medical devices, to health IT, and to other domains as well.

I believe JC should start to pay serious attention to "anecdotal reports" of health IT-caused patient injury, and consider reliable reporting of these events as an Accreditation standard.

As I noted in my July 2009 JAMA letter to the editor "Health Care Information Technology, Hospital Responsibilities, and Joint Commission Standards" in response to Koppel and Kreda's JAMA article on HIT industry practices, "hold harmless" and "gag" clauses must go, and be replaced with proactive reporting of healthcare IT-related "events."

Scot Silverstein

-- SS

A Simple Lesson for the Health IT Industry

From an Op Ed "Living with the Electronic Car" in today's Wall Street Journal:

"A Toyota executive recently explained to a Congressional committee investigating claims of uncontrolled acceleration: "We need to reduce the number of things we ask our customers to do correctly." In fact, the exec was describing the essence of responsible engineering - though perhaps the balance in auto design has gotten out of whack."

Considering the feedback from physicians on the needless complexity of electronic medical records and other computerized medical devices for example at "An Honest Physician Survey on EHR's", it seems the healthcare IT industry has yet to learn this simple lesson.

I'm frankly not convinced there's "anyone home" in this complexity-loving industry who could fathom such advice as a good business practice.

It also seems that industry may not give a damn about such lessons, even in the most safety critical of environments, the intensive care unit, as long as profits are maintained.

See for example "A Lawsuit Over Healthcare IT Whistleblowing and Wrongful Discharge: Malin v. Siemens Healthcare." Also see "Third-Party Reviews of Medical Devices Come Under Scrutiny at the FDA - Except Healthcare IT Medical Devices, Which Get Special Accommodation" on political maneuvering by this industry to avoid the federal regulation other healthcare drug and device sectors have been subject to for decades (largely as a result of public health disasters such as this and this, I might add).

The interesting aspect of these issues is that the executives and officials behind these decisions and machinations are setting themselves up as near-indefensible defendants in future litigation by patients (and their estates) harmed or killed by healthcare IT-related problems.

For instance, I had communications with the Joint Commission leadership over issues I raised in my July 22, 2009 JAMA letter to the editor "Health Care Information Technology, Hospital Responsibilities, and Joint Commission Standards." Namely, on how hospital executives were violating their fiduciary and Joint Commission safety standards obligations, and jeapardizing patient safety, in signing the traditional health IT contract calling for confidentiality about health IT malfunctions and defects. The JC Leadership is quite well aware of this letter and my more thorough essay here.

Nothing has been heard from the Joint Commission on these issues since.

Perhaps never before have the malpractice lawyers been provided a better scenario for taking the houses and personal property from irresponsible and/or conflicted healthcare and health IT regulators and other officials via lawsuits as health IT diffusion increases, with its "tip of the iceberg" injury and death occurrences now firmly established.

-- SS