DoD rollout of Cerner EHR deemed not operationally effective or suitable
The system does not demonstrate enough workable functionality to manage and document patient care, contends the military’s director of operational test and evaluation.
While the Department of Defense contends that its initial deployment last year of MHS GENESIS—a new Cerner electronic health record system—at four military sites in the Pacific Northwest was a success, the EHR is “neither operationally effective nor operationally suitable,” according to a new report from DoD’s director of operational test and evaluation.
DOT&E’s initial test and evaluation report, dated April 30 and released by DoD on Friday, is based on an assessment of three of four military sites in Washington State that were part of the rollout.
“MHS GENESIS is not operationally effective because it does not demonstrate enough workable functionality to manage and document patient care,” wrote Robert Behler, director of operational test and evaluation, in a letter to senior Pentagon officials accompanying his report. “Users successfully performed only 56 percent of the 197 tasks used as measures of performance. MHS GENESIS is not operationally suitable because of poor system usability, insufficient training and inadequate help desk support. Survivability is undetermined because cybersecurity testing is ongoing.”
Also See: DoD initial rollout of Cerner EHR comes under fire from senator
In addition, Behler reported that users of the Cerner EHR gave its usability “an average score of only 37 out of 100 on the System Usability Scale (SUS), well below the threshold of 70 that indicates acceptable usability,” while “training was insufficient to overcome usability problems, and a lack of documentation forced users to develop their own operational workarounds.”
As a result of the findings, Behler recommended that the Under Secretary of Defense for Acquisition and Sustainment delay further fielding of the Cerner EHR until the Joint Interoperability Test Command completes the initial operational test and evaluation and the Program Management Office “corrects any outstanding deficiencies.”
According to Behler, the initial operational test and evaluation was conducted at Fairchild Air Force Base and Naval Health Clinic Oak Harbor from September 25 to October 6, 2017, and Naval Hospital Bremerton from December 4 to 15, 2017. However, he said IOT&E at Madigan Army Medical Center was postponed to “remediate significant problems discovered at the first three sites” and will be rescheduled later this year. After the review at that site is completed, he said his office will release its final report.
Last year, DoD completed full deployment of the Cerner system to all four initial operational capability sites, culminating with Madigan Army Medical Center.
Stacy Cummings, program executive officer for Defense Healthcare Management Systems, told members of the press on Friday that despite the findings of the DOT&E report, DoD still plans to continue deployment of MHS GENESIS beyond the Pacific Northwest beginning in 2019, and that the system continues to be on track for full deployment by 2022.
Cummings noted that DoD conducted an eight-week “optimization period” starting in January and that the military has “made significant progress in adoption of workflows, roles and responsibilities, issue resolution, and training and change management activities.”
She also said DoD “engaged HIMSS Analytics—widely recognized as the industry standard for measuring electronic health record adoption—to conduct an independent measure of the progress and effectiveness of MHS GENESIS at our four initial sites.” According to Cummings, prior to MHS GENESIS deployment, the average score for its legacy systems was Stage 2 EMRAM out of a possible Stage 0 to Stage 7 scoring on the HIMSS Analytics scale. However, post deployment, the sites scored at or above Stage 5—which is “well above the national average,” she added.
Nonetheless, according to DOT&E’s initial operational test and evaluation report, users of MHS GENESIS rated the training they received as “poor,” with most users (67 percent) indicating that they needed more training. In response, Cummings acknowledged that the MHS GENESIS “training strategy really needs to change as we move forward,” and “one of the things that we learned is that we need to do a better job of preparing our sites from a technology as well as a workflow readiness perspective.”
The DOT&E report also reported that users of the Cerner system “often suffered from unacceptably long MHS GENESIS login times.” However, Cummings countered that as a result of a software upgrade that took place last month, DoD has been able to reduce login times by 50 percent “and our reliability has gone up significantly.”
When it comes to scalability, the DOT&E report observed that although the system’s scalability “has not been formally tested,” there are “indications that MHS GENESIS may not be scalable.”
In response, Travis Dalton, Cerner’s federal general manager, told reporters that the EHR vendor is “very confident” in the system’s ability to scale, citing the fact that the company has customers that are “exponentially larger” than the current size of DoD’s MHS GENESIS deployment.
“The systems and the code that we’re utilizing here for DoD are the same set of code and systems that are live at thousands of commercial sites around the globe,” added Dalton. “We’re entirely confident in the safety, the quality and the capability of our solutions, as evidenced by our position in the marketplace and their use globally.”
DoD awarded a $4.3 billion contract in 2015 to prime contractor Leidos to modernize the military’s EHR system, which will be deployed worldwide to support healthcare delivery for 9.4 million beneficiaries. The Leidos-led team includes consultancy Accenture, dental software vendor Henry Schein and Cerner, which provides the core Millennium capability.
“While we still have challenges to overcome, we are clearly making significant progress and are on the right track to deliver a modern, secure and connected electronic health record to the Military Health System,” concluded Cummings. “And, we continue to be—and have been—open and transparent.”
Cerner’s Dalton added that there were no “surprises” for the EHR vendor in terms of the findings of the DOT&E report. “We expected there to be things to work through, and we’re working through them in a consistent manner,” he said. “It’s consistent with what we might see in a commercial environment.”
DOT&E’s initial test and evaluation report, dated April 30 and released by DoD on Friday, is based on an assessment of three of four military sites in Washington State that were part of the rollout.
“MHS GENESIS is not operationally effective because it does not demonstrate enough workable functionality to manage and document patient care,” wrote Robert Behler, director of operational test and evaluation, in a letter to senior Pentagon officials accompanying his report. “Users successfully performed only 56 percent of the 197 tasks used as measures of performance. MHS GENESIS is not operationally suitable because of poor system usability, insufficient training and inadequate help desk support. Survivability is undetermined because cybersecurity testing is ongoing.”
Also See: DoD initial rollout of Cerner EHR comes under fire from senatorIn addition, Behler reported that users of the Cerner EHR gave its usability “an average score of only 37 out of 100 on the System Usability Scale (SUS), well below the threshold of 70 that indicates acceptable usability,” while “training was insufficient to overcome usability problems, and a lack of documentation forced users to develop their own operational workarounds.”
As a result of the findings, Behler recommended that the Under Secretary of Defense for Acquisition and Sustainment delay further fielding of the Cerner EHR until the Joint Interoperability Test Command completes the initial operational test and evaluation and the Program Management Office “corrects any outstanding deficiencies.”
According to Behler, the initial operational test and evaluation was conducted at Fairchild Air Force Base and Naval Health Clinic Oak Harbor from September 25 to October 6, 2017, and Naval Hospital Bremerton from December 4 to 15, 2017. However, he said IOT&E at Madigan Army Medical Center was postponed to “remediate significant problems discovered at the first three sites” and will be rescheduled later this year. After the review at that site is completed, he said his office will release its final report.
Last year, DoD completed full deployment of the Cerner system to all four initial operational capability sites, culminating with Madigan Army Medical Center.
Stacy Cummings, program executive officer for Defense Healthcare Management Systems, told members of the press on Friday that despite the findings of the DOT&E report, DoD still plans to continue deployment of MHS GENESIS beyond the Pacific Northwest beginning in 2019, and that the system continues to be on track for full deployment by 2022.
Cummings noted that DoD conducted an eight-week “optimization period” starting in January and that the military has “made significant progress in adoption of workflows, roles and responsibilities, issue resolution, and training and change management activities.”
She also said DoD “engaged HIMSS Analytics—widely recognized as the industry standard for measuring electronic health record adoption—to conduct an independent measure of the progress and effectiveness of MHS GENESIS at our four initial sites.” According to Cummings, prior to MHS GENESIS deployment, the average score for its legacy systems was Stage 2 EMRAM out of a possible Stage 0 to Stage 7 scoring on the HIMSS Analytics scale. However, post deployment, the sites scored at or above Stage 5—which is “well above the national average,” she added.
Nonetheless, according to DOT&E’s initial operational test and evaluation report, users of MHS GENESIS rated the training they received as “poor,” with most users (67 percent) indicating that they needed more training. In response, Cummings acknowledged that the MHS GENESIS “training strategy really needs to change as we move forward,” and “one of the things that we learned is that we need to do a better job of preparing our sites from a technology as well as a workflow readiness perspective.”
The DOT&E report also reported that users of the Cerner system “often suffered from unacceptably long MHS GENESIS login times.” However, Cummings countered that as a result of a software upgrade that took place last month, DoD has been able to reduce login times by 50 percent “and our reliability has gone up significantly.”
When it comes to scalability, the DOT&E report observed that although the system’s scalability “has not been formally tested,” there are “indications that MHS GENESIS may not be scalable.”
In response, Travis Dalton, Cerner’s federal general manager, told reporters that the EHR vendor is “very confident” in the system’s ability to scale, citing the fact that the company has customers that are “exponentially larger” than the current size of DoD’s MHS GENESIS deployment.
“The systems and the code that we’re utilizing here for DoD are the same set of code and systems that are live at thousands of commercial sites around the globe,” added Dalton. “We’re entirely confident in the safety, the quality and the capability of our solutions, as evidenced by our position in the marketplace and their use globally.”
DoD awarded a $4.3 billion contract in 2015 to prime contractor Leidos to modernize the military’s EHR system, which will be deployed worldwide to support healthcare delivery for 9.4 million beneficiaries. The Leidos-led team includes consultancy Accenture, dental software vendor Henry Schein and Cerner, which provides the core Millennium capability.
“While we still have challenges to overcome, we are clearly making significant progress and are on the right track to deliver a modern, secure and connected electronic health record to the Military Health System,” concluded Cummings. “And, we continue to be—and have been—open and transparent.”
Cerner’s Dalton added that there were no “surprises” for the EHR vendor in terms of the findings of the DOT&E report. “We expected there to be things to work through, and we’re working through them in a consistent manner,” he said. “It’s consistent with what we might see in a commercial environment.”
More for you
Loading data for hdm_tax_topic #better-outcomes...