Key lessons on legal liability and the use of algorithms
The veto of an AI-related bill in California shines a light on the liability complexity currently facing health systems.
Something significant just happened in California, but because of the complexity involved, it hasn’t received the national-level coverage it deserves.
It’s all about artificial intelligence and the emerging legal liability dangers facing physicians, nurses and other clinicians in practice, as AI algorithms come to be very widely adopted by hospitals and health systems — and patient care liability issues inevitably arise.
On September 30, California Governor Gavin Newsom vetoed Assembly Bill 2575. Authored by Assemblymember Liz Ortega (D-Hayward), AB 2575 would have protected healthcare workers from employer retaliation solely based on their overriding an AI system within their scope of practice.
Additionally, it would bar AI vendors from asserting any defense targeting individual clinicians for failure to override an algorithm, in the event of patient harm in a case.
Ortega had introduced the bill in February, in her role as chair of the Labor and Employment Committee and a member of the Privacy and Consumer Protection Committee, after hearing concerns from clinicians in practice. It was co-sponsored by the California Nurses Association and the California Federation of Labor Unions.
A total of 51 Assembly members voted for it (50 Democrats and one Republican), while 16 voted against it (15 Republicans and one Democrat), while 12 were absent or did not vote.
But, days before the bill passed, and speaking for the California Hospital Association, Carmela Coyle, the association’s president and CEO, stated that, “If signed into law by the Governor, Assembly Bill 2575 would create a new protected class for health care workers who override a clinical support system even when a patient is harmed,” and she urged Newsom to veto the bill.
In vetoing the bill, Newsom stated that, “While I commend the author’s commitment to ensure that artificial intelligence technologies deployed in clinical settings support safe patient care and preserve professional standards, including the ability of healthcare workers to exercise their professional judgment, it is essential that new laws in this area be practical and enforceable in their implementation to achieve their intended goals.
“This bill’s anti-retaliation provisions do not protect workers as intended,” he continued. “By prohibiting adverse actions based solely on the employee’s override of, or reliance on, a clinical decision support system, this bill ties the Labor Commissioner’s hands in establishing a violation — a higher bar than other retaliation protections, which simply prohibit action because of a protected activity.
“More importantly, linking anti-retaliation protections to a scope-of-practice determination puts the Labor Commissioner in the position of adjudicating the standard of care for patients — something the Labor Commissioner does not have the medical expertise or skill to do.”
In a statement released after the veto, Sandy Reding, president of the California Nurses Association, denounced the governor’s action, saying that it would empower hospitals and health systems to discipline nurses who override AI algorithms to protect their patients.
"The governor knows AI should never replace a nurse's clinical judgment, but he vetoed the bill that would have protected us for using our judgment," Reding said. "That makes no sense to any nurse at the bedside. Today, a nurse who sees that the algorithm is wrong and acts to save her patient can still be disciplined or fired by her employer.”
In the weeds with AI
Setting aside the partisan issues involved, what’s clear is that no one can yet figure out any kind of “perfect solution” to how to accomplish several key goals at the same time — ensure patient safety, enable clinicians through the use of efficient and effective AI algorithms, ensure that clinicians remain free to use their best clinical judgment in all situations and keep clinical operations running at maximum efficiency.
In other words, we’ve reached the moment at which the deployment of AI algorithms has met real-world complexity. Nurses are potentially even more vulnerable than physicians are, to being caught in a no-win situation, as they have less clinical authority than physicians do and could potentially be subject to professional discipline in certain circumstances.
There’s no perfect answer. But what has happened with this particular state legislative bill illustrates why AI governance — involving all categories of clinicians as well as all the health IT and data analytics professionals, and senior leaders from all disciplines and relevant sectors within hospitals and health systems — must be involved and engaged from the very beginning as AI algorithms are deployed and implemented.
It’s crunch time now, and there’s never been a more crucial moment for optimal governance and decision-making around all elements of AI. Patients and communities deserve no less.
Mark Hagland is a contributor to Health Data Management and is a Fellow of the American College of Health Data Management.