AI Can Read The Scan—It Can’t Hold The Hand

This voice experience is generated by AI. Learn more.This voice experience is generated by AI. Learn more.Paula Ferrada is the Chair, Department of Surgery – IFMC and System Chief for Trauma and Acute Care Surgery at Inova Healthcare System.

gettyEvery hospital system is running the same experiment right now, whether we call it that or not. We are putting algorithms in front of enormous amounts of patient data and asking them to help us diagnose faster, predict deterioration earlier, reduce errors and make an overwhelmed healthcare system work better.

What concerns me most is not the technology. It is that AI is moving faster than healthcare leadership is learning how to govern it. We cannot afford to be spectators.

AI in medicine is not a strategy. It is a tool. A scalpel does not decide when to operate, and an algorithm should not decide how we care for a human being. Someone with clinical judgment and organizational authority must decide where AI belongs, where it does not and ultimately be accountable for what happens next.

That is why AI cannot simply be handed to IT or innovation teams. Clinical and executive leaders need to be in the room while these systems are being tested and embedded into patient care. We need to ask the difficult questions: Is this making patients safer? Who could be harmed? What bias exists in the data? What happens when the algorithm is wrong? Can clinicians challenge it? Who is accountable when human judgment and the machine disagree?

These are not technology questions. They are leadership questions.

AI can analyze imaging, recognize patterns and flag findings that deserve attention. In the ICU, predictive models may recognize deterioration before it becomes obvious at the bedside. Earlier recognition is not simply about efficiency. It can change what happens to a patient.

AI can also take on some of the administrative work that has slowly consumed medicine. Ambient AI scribes may be one of the least glamorous but most meaningful examples. Physicians spend far too many evenings finishing documentation. If AI can give clinicians some of that time back, that matters for burnout and retention (and ultimately patients).

The best use of AI is not replacing the physician. It is removing some of the noise so we have more time to think, listen, connect and care.

AI is not neutral simply because it is mathematical. An algorithm trained on incomplete or unrepresentative data can reproduce inequities already present in healthcare, except now those decisions arrive wrapped in the appearance of objectivity.

That may make bias more dangerous. We question human judgment. We may be less likely to question a number on a screen. There is also the black box. If an AI system tells me my patient is at risk but cannot help me understand why, what am I supposed to do with that information? Clinicians cannot replace judgment with faith in an algorithm they cannot understand or challenge.

And AI cannot repair a broken system by itself. Saving a physician ten minutes does not automatically create ten more minutes with a patient. Those minutes can simply disappear into the next bottleneck.

Healthcare organizations are moving quickly while governance, accountability and clinical oversight struggle to keep pace. Who is responsible when an algorithm is wrong? Who monitors for bias after deployment? Who decides when a tool should be removed? What happens when the clinician and the algorithm disagree?

If nobody can answer those questions clearly, the technology is not ready for patient care. Buying an AI platform is easy. Calling ourselves “AI-enabled” is easy. Transforming care responsibly is much harder.

That requires leadership. Leaders need to define the problem before buying the technology. Clinicians need to be involved in design and implementation. We need measurable outcomes, clear accountability, transparency about limitations and the courage to stop using a tool when it does not make care better.

Deploying AI is not a strategy. Leading it is.

There are parts of medicine AI may eventually do better than we do. It will recognize patterns faster and process more information. I welcome that. I want every tool that helps me take better care of a patient.

But medicine is more than processing information. A model has never sat with a family at 2 a.m. and told them their father is not going to survive. It has never stood in a trauma bay with seconds to decide whether to open a patient’s chest. It has never looked into the eyes of someone who is terrified and said, “I’ve got you,” knowing the weight of that promise.

That is not pattern recognition. That is judgment, trust, compassion and accountability. That is medicine. AI can inform those moments. It cannot own them. The organizations that get AI right will have leaders who remember why the technology is there in the first place.

AI can help us make better decisions. But it cannot hold a patient’s hand. And it cannot carry the responsibility for what happens next.

Forbes Business Council is the foremost growth and networking organization for business owners and leaders. Do I qualify?