Malfunctions are common with clinical decision support systems
Software upgrades and changes in data codes often cause CDS errors, and 93 percent of CMIOs surveyed reported at least one malfunction, which are often difficult to detect.
Problems and malfunctions involving clinical decision support systems are widespread at U.S. hospitals, and those issues may have potentially negative effects on patient safety, according to results of a new study.
The study identified and investigated several malfunctions in clinical decision support (CDS) systems at Brigham and Women’s Hospital in Boston and also included a preliminary survey of chief medical information officers to assess the frequency of such malfunctions.
Researchers found that Some 93 percent of the 29 CMIOs surveyed had experienced at least one CDS system malfunction, while two-thirds experienced malfunctions at least annually, according to researchers, who published their findings in the Journal of the American Medical Informatics Association. Among the most common contributing factors to malfunctions reported by the CMIOs, 18 reported malfunctions occurring at the time of an upgrade of EHR software, and 18 reported issues related to changes in data codes or data fields—both of which are known high-risk events for malfunctions.
“I don’t think we have a robust handle on how widespread these malfunctions are,” says David Bates, MD, chief innovation officer for Brigham and Women’s Hospital. “But we do know that they occur at nearly every institution that we talked to.”
Four kinds of CDS system malfunctions were identified at Brigham and Women’s Hospital:
Beyond these issues, Bates and his fellow researchers worry that CDS system malfunctions often go undetected and that malfunction rates are much higher. “This is a tip of the iceberg sort of situation,” he contends.
It’s particularly difficult to detect when alerts fail to fire, researchers found. “A range of causes, including changes in codes and fields, software upgrades, inadvertent disabling or editing of rules, and malfunctions of external systems commonly contribute to CDS system malfunctions, and current approaches for preventing and detecting such malfunctions are inadequate,” state researchers. They say better methods are needed to prevent and detect these malfunctions.
Much of the promise of EHRs to improve patient care rests on clinical decision support mechanisms, and that a particularly significant type of health IT-related adverse event is the malfunction of these systems, the researchers say.
Although Bates believes that clinical decision support clearly improves patient safety, “now that we have more of it, I think what this suggests is that we also need tools to monitor how well it’s working.” He recommends that healthcare organizations put in place “approaches for detecting and measuring how their decision support is working and, when they find issues, for intervening.”
Bates says the issues are largely on the provider side, and he does not blame vendors for the issues encountered in the study.
“Often, the problems involve the fact that some internal identifier was changed or there was a rule inside the system on the provider side that got edited,” he observes. “There are issues when there are upgrades and you have to look for example after an upgrade to make sure that everything is working well, because often it will not be. I don’t think it’s possible for vendors to find all issues before they deliver upgrades. Obviously, it’s in their court to do as much as they can.”
Bradley Merrill Thompson, general counsel for the Clinical Decision Support Coalition, argues that “such software does much good in that it can help professionals think more comprehensively about diagnostic possibilities and treatment options, and stay current with medical advancements.”
At the same time, Thompson concedes that clinical decision support capabilities can malfunction, like all software can. “That is why it is important for the doctors and others who use it to stay vigilant and engaged—we will not be replacing doctors anytime soon,” he says, adding that “alarming and monitoring software are typically regulated by the [Food and Drug Administration].”
Part of the problem, Thompson asserts, is that the FDA has yet to deliver guidance on CDS, which it has promised since 2011, when the agency held a regulatory hearing on the topic.
Stakeholders were hoping a 2014 FDASIA-mandated report to Congress on a risk-based regulatory framework for health IT from the FDA, Federal Communications Commission and Office of the National Coordinator for HIT would provide additional clarity on CDS software. But, that didn’t shed much light on the FDA’s oversight, according to Thompson.
“In 2015, it was the agency’s highest priority, but we still haven’t seen it yet,” Thompson says. “Someone—whether FDA or Congress—needs to define the portion of CDS that FDA will regulate.”
However, Bates does not believe that FDA regulations would address CDS system malfunctions. “Regulation would mostly affect what vendors are delivering, and most of these issues are really not at that level,” he argues. Malfunctions “are mostly post-implementation issues. More regulation around this would not be helpful and would probably stifle innovation.”
Nonetheless, Thompson insists that such guidance “would add clarity at least” to the uncertainty that surrounds this regulatory gray area. A survey of CDS developers released in late February by the Clinical Decision Support Coalition showed that one-third reported having to abandon product development efforts because of FDA regulatory uncertainty. Of those who pressed ahead, almost two-thirds encountered significant delays in development—greater than one year in 20 percent of those cases.
“If you asked two different people at FDA, you’re likely to get two different answers,” Thompson contends. “There is very little consensus within the agency regarding the approach that they will take.”
The study identified and investigated several malfunctions in clinical decision support (CDS) systems at Brigham and Women’s Hospital in Boston and also included a preliminary survey of chief medical information officers to assess the frequency of such malfunctions.
Researchers found that Some 93 percent of the 29 CMIOs surveyed had experienced at least one CDS system malfunction, while two-thirds experienced malfunctions at least annually, according to researchers, who published their findings in the Journal of the American Medical Informatics Association. Among the most common contributing factors to malfunctions reported by the CMIOs, 18 reported malfunctions occurring at the time of an upgrade of EHR software, and 18 reported issues related to changes in data codes or data fields—both of which are known high-risk events for malfunctions.
“I don’t think we have a robust handle on how widespread these malfunctions are,” says David Bates, MD, chief innovation officer for Brigham and Women’s Hospital. “But we do know that they occur at nearly every institution that we talked to.”
Four kinds of CDS system malfunctions were identified at Brigham and Women’s Hospital:
- An alert for monitoring thyroid function in patients receiving amiodarone stopped working when an internal identifier for amiodarone was changed in another system.
- An alert for lead screening for children stopped working when the rule was inadvertently edited.
- A software upgrade of the EHR software caused numerous spurious alerts to fire.
- A malfunction in an external drug classification system caused an alert to inappropriately suggest antiplatelet drugs, such as aspirin, for patients already taking one.
Beyond these issues, Bates and his fellow researchers worry that CDS system malfunctions often go undetected and that malfunction rates are much higher. “This is a tip of the iceberg sort of situation,” he contends.
It’s particularly difficult to detect when alerts fail to fire, researchers found. “A range of causes, including changes in codes and fields, software upgrades, inadvertent disabling or editing of rules, and malfunctions of external systems commonly contribute to CDS system malfunctions, and current approaches for preventing and detecting such malfunctions are inadequate,” state researchers. They say better methods are needed to prevent and detect these malfunctions.Much of the promise of EHRs to improve patient care rests on clinical decision support mechanisms, and that a particularly significant type of health IT-related adverse event is the malfunction of these systems, the researchers say.
Although Bates believes that clinical decision support clearly improves patient safety, “now that we have more of it, I think what this suggests is that we also need tools to monitor how well it’s working.” He recommends that healthcare organizations put in place “approaches for detecting and measuring how their decision support is working and, when they find issues, for intervening.”
Bates says the issues are largely on the provider side, and he does not blame vendors for the issues encountered in the study.
“Often, the problems involve the fact that some internal identifier was changed or there was a rule inside the system on the provider side that got edited,” he observes. “There are issues when there are upgrades and you have to look for example after an upgrade to make sure that everything is working well, because often it will not be. I don’t think it’s possible for vendors to find all issues before they deliver upgrades. Obviously, it’s in their court to do as much as they can.”
Bradley Merrill Thompson, general counsel for the Clinical Decision Support Coalition, argues that “such software does much good in that it can help professionals think more comprehensively about diagnostic possibilities and treatment options, and stay current with medical advancements.”
At the same time, Thompson concedes that clinical decision support capabilities can malfunction, like all software can. “That is why it is important for the doctors and others who use it to stay vigilant and engaged—we will not be replacing doctors anytime soon,” he says, adding that “alarming and monitoring software are typically regulated by the [Food and Drug Administration].”
Part of the problem, Thompson asserts, is that the FDA has yet to deliver guidance on CDS, which it has promised since 2011, when the agency held a regulatory hearing on the topic.
Stakeholders were hoping a 2014 FDASIA-mandated report to Congress on a risk-based regulatory framework for health IT from the FDA, Federal Communications Commission and Office of the National Coordinator for HIT would provide additional clarity on CDS software. But, that didn’t shed much light on the FDA’s oversight, according to Thompson.
“In 2015, it was the agency’s highest priority, but we still haven’t seen it yet,” Thompson says. “Someone—whether FDA or Congress—needs to define the portion of CDS that FDA will regulate.”
However, Bates does not believe that FDA regulations would address CDS system malfunctions. “Regulation would mostly affect what vendors are delivering, and most of these issues are really not at that level,” he argues. Malfunctions “are mostly post-implementation issues. More regulation around this would not be helpful and would probably stifle innovation.”
Nonetheless, Thompson insists that such guidance “would add clarity at least” to the uncertainty that surrounds this regulatory gray area. A survey of CDS developers released in late February by the Clinical Decision Support Coalition showed that one-third reported having to abandon product development efforts because of FDA regulatory uncertainty. Of those who pressed ahead, almost two-thirds encountered significant delays in development—greater than one year in 20 percent of those cases.
“If you asked two different people at FDA, you’re likely to get two different answers,” Thompson contends. “There is very little consensus within the agency regarding the approach that they will take.”
More for you
Loading data for hdm_tax_topic #better-outcomes...