When humility becomes code: How AI is learning to say ‘I don’t know’
Leo Anthony Celi, an intensive care doctor in Boston, is working to make artificial intelligence (AI) more humble. He works at Beth Israel Deaconess Medical Center, leads research at MIT's Institute for Medical Engineering and Science, and teaches at Harvard Medical School. In a recent interview, he said, "Right now, we use AI as an oracle. We could use it as a coach."
Preventable medical errors kill between 200,000 to 400,000 Americans every year. The range is wide because researchers disagree on the count. Johns Hopkins, in its widely cited 2016 estimate, put the figure above 250,000. AI was meant to bring that number down. ICU studies show the opposite. When a physician's instinct disagrees with AI, the physician tends to defer to AI, especially when it sounds certain. Researchers call this automation bias.
Celi's team proposed an inversion. Don't make AI smarter. Make it more humble. Their framework, published in 2026 in the medical journal BMJ Health & Care Informatics, is called BODHI — Balanced, Open‑minded, Diagnostic, Humble, Inquisitive. The idea is simple. Before answering, AI has to assess its confidence and what it doesn't know. The team wrapped GPT‑4.1‑mini and GPT‑4o‑mini in BODHI and tested both on 1,000 clinical cases. Without it, GPT‑4o‑mini never asked for more context. With it, the same model paused in 735 of those cases, often saying something like, "I don't have enough information. What test could you run?"
There was a tradeoff. Both models scored about 12 percentage points lower on standard communication-quality tests. The researchers had a clear explanation: Confident statements sound polished, while cautious questions do not. The benchmarks may have rewarded the very behavior that makes clinical AI risky.
This issue is not limited to the United States. Korea is one of the most active countries in the use of medical AI, from radiology screening to triage and chronic disease management. If a Korean hospital implements a system that prioritizes uncertainty, it also brings the ICU problem home. The choice between using AI as an oracle or a coach is one we will face, whether we realize it or not.
And the implication does not stay in the hospital. Humility is one of the first values parents try to teach their children. Teachers spend years on it. Pastors preach it. Now, we install it in a language model with a single prompt. Theologians have long distinguished two forms of humility. Situational humility is a behavior: the executive who says I may be wrong, the senior who yields the floor. It is learnable, coachable and now demonstrably codable. Dispositional humility is different. C.S. Lewis described it in "Mere Christianity" as the person you remember not for modesty but because they took a genuine interest in you. Nineteenth‑century pastor Andrew Murray wrote that humility is not a list of actions but a way of being, not thinking less of yourself, but thinking of yourself less.
In "Thinking, Fast and Slow," Daniel Kahneman showed how often we mistake signals for real substance. When someone sounds confident, we tend to believe they are accurate. This is one reason ICU doctors trust AI. It gives off the right signals. BODHI adds situational humility to the machine, not a lasting trait, since the system has no self to humble. Now, GPT can produce those signals, like a careful pause or a phrase such as, "I'm not sure," but even more smoothly than most people.
The humility we value in friends, mentors, colleagues and pastors, what if it has always been just a signal? And what if a chatbot now performs that signal better than the people we once trusted to show it? In a society where tone and phrasing often count as much as belief, the polite, self‑effacing rhetoric of left and right alike might be the first thing a chatbot simply learns to mimic perfectly.
Choi Hee-jin is an educator and practical theologian, Yale Divinity School fellow (2025–2026), exploring medical humanities at Duke. She writes about medicine, technology, and human flourishing, with imagination and hope, at Human Becoming (humanbecom.ing).