The Preparation of Scientific Minds in the Age of AI

Marsha Seidelman, MD, DiplABLM

Artificial intelligence (AI) is top of mind in our clinical and non-clinical lives. A recent NEJM article about its impact on scientific thinking drew me down the rabbit hole of exploring its implications for medical training and patient care.

Those of us who were residents before we carried our accessory brains in our pockets can surely remember needing to weave elements of a patient's history together, synthesize a coherent story with a likely culprit, and come up with a reasonable differential diagnosis, all with little or no sleep. Not saying that was good for any of us, or for our patients for that matter. But over time, we developed the ability to do that much more easily, based on our reading and experience, something the NEJM article calls 'preparation of the mind'. What do we risk if residents' first move now is to check their pockets for their accessory brains?

To take a non-clinical example, much research has been done about how brains function when AI is used to write a paper. By monitoring brain activity, we know there is less brain activity when using AI than when writing without assistance. It's too early in the usage of AI for everyday tasks to know how it will affect the development of brains and critical thinking. I have parallel concerns related to medical trainees and to all of us adults dodging dementia. Will AI be used instead of 'brain exercise' (as in, why bother to remember this when I can just look it up again), or to increase brain challenges, with more information to indulge our thought processes and provoke thoughtful discussions?

No one would argue that finding information easily is essential, perhaps more so in medicine than in many other fields, but we have to remain clear-eyed about the promise and the perils.

One concern is that scientists in training might forgo the preparation of their minds. Are doctors in training developing the same ability for independent thought, critical thinking, organization of facts and good medical judgment when they use AI as often as possible? Will they be able to notice patterns, make connections, and perhaps most importantly, recognize incorrect information? AI may generate inaccurate recommendations, especially with unusual situations when we'd value its assistance the most.

I use Open Evidence, an AI tool that was trained using medical literature (and presumably much more). Supposedly this is more trustworthy than other AI tools for medical information. Recently, I was doing a talk about the immune system, not my primary specialty, and I asked about vitamin D related to immunity. Fortunately, I had done research by then, and was looking for a specific nuanced detail, but the advice it gave was to give 10,000 IU daily of vitamin D. As one should when faced with misinformation, I asked if there wasn't another choice, and I received a totally different answer. What if a clinician acted on this? Using AI without already knowing background information could be hazardous to our patients' health.

We will all learn with time how to question AI most effectively. We risk being led astray if we input incorrect data or if we don't appreciate biased info that the AI was trained on. For instance, it may perform less accurately in under-represented populations, potentially exacerbating inequities in healthcare. Also, the AI algorithms will make recommendations based on clinical guidelines, which we humans use as well. But in our assessments, we also take patient preferences, feasibility, and prior experience into account. After all, the guidelines represent expert opinion and are not evidence-based. And what if we don't consult with AI or disagree with AI, what will the malpractice prospects look like if there's a less than ideal patient outcome? Whose judgment is better? We want to pursue the best medicine, without losing the art of medicine.

AI has already been shown to improve clinician assessment in diagnostic imaging, retinal photos and pathology slides. We will continue to see it help unload time-consuming administrative tasks related to documenting patient visits, scheduling, and submitting prior authorizations and bills. In the long run, AI should allow us more time in face-to-face interactions with patients, and help improve quality of care with fewer errors.

As with non-medical utilization of AI, however, legal and ethical guardrails are essential. In our case, these would be to assure patient safety and privacy related to the collection, storage and disclosure of patient records. A separate issue, the impact on health care costs, is complex, including whether they are likely to be increased or decreased, and to whom any cost savings would accrue. Trust me, this is another very deep rabbit hole.

We will certainly be increasing implementation of AI in medical care. Ideally, it will support, not replace, clinical judgment, with the human having the ultimate clinical responsibility. The challenge ahead will be to use it safely and to great advantage to assist intelligent doctors with 'prepared minds' who can think on their feet, with the advantage of having the personal touch that patients appreciate.

References

Four professionals stand beneath a large AI icon surrounded by digital icons like a pie chart, a microchip, and a bar graph.

Recent Posts

Posted in Member Perspective and tagged