In California and Wisconsin for instance, OpenAI’s “GPT” generative artificial intelligence is reading patient messages and drafting responses from their doctors. The operation is part of a pilot program in which three health systems test if the AI will cut the time that medical staff spend replying to patients’ online inquiries. 

UC San Diego Health and UW Health began testing the tool in April. Stanford Health Care aims to join the rollout early next week. Altogether, about two dozen healthcare staff are piloting this tool. Marlene Millen, a primary care physician at UC San Diego Health who is helping lead the AI test, has been testing GPT in her inbox for about a week. Early AI-generated responses needed heavy editing, she said, and her team has been working to improve the replies. 

They are also adding a kind of bedside manner: If a patient mentioned returning from a trip, the draft could include a line that asked if their travels went well. “It gives the human touch that we would,” Dr. Millen said. There is preliminary data that suggests AI could add value. ChatGPT scored better than real doctors at responding to patient queries posted online, according to a study published Friday in the journal JAMA Internal Medicine, in which a panel of doctors did blind evaluations of posts.

As many industries test ChatGPT as a business tool, hospital administrators and doctors are hopeful that the AI-assist will ease burnout among their staff, a problem that skyrocketed during the pandemic. The crush of messages and health-records management is a contributor, among administrative tasks, according to the American Medical Association. 

Doctors are increasingly using AI to answer patient questions.
Epic, the company based in Verona, Wis., that built the “MyChart” tool through which patients can message their healthcare providers, saw logins more than double from 106 million in the first quarter of 2020 to 260 million in the first quarter of 2023. Epic’s software enables hospitals to store patient records electronically.   

Earlier this month, Epic and Microsoft announced that health systems would have access to OpenAI’s GPT through Epic’s software and Microsoft’s Azure cloud service. Microsoft has invested in OpenAI and is building artificial intelligence tools into its products. Hospitals are piloting GPT-3, a version of the large language model that is powering ChatGPT.

ChatGPT has mystified computer scientists for its skill in responding to medical queries—though it is known to make things up—including its ability to pass the U.S. Medical Licensing Exam. OpenAI’s language models haven’t been specifically trained on medical data sets, according to Eric Boyd, Microsoft’s corporate vice president of AI Platform, though medical studies and medical information were included in the vast data set that taught it to spot patterns. 

“Doctors working with ChatGPT may be the best messenger,” said John Ayers, a computational epidemiologist at the University of California, San Diego, and an author of the JAMA study.  The AI pilot has some healthcare staff buzzing, said Dr. Millen. “Doctors are so burnt out that they are looking for any kind of hope.” 

That hospital system saw patient messages jump from 50,000 messages a month before the pandemic, to over 80,000 a month after, with more than 140,000 messages in some pandemic months, Dr. Millen said. Doctors and their teams are struggling to accommodate the extra load, she said. “I don’t get time on my schedule. My staff is really busy, too.”  

Now when Dr. Millen clicks on a message from a patient, the AI instantly displays a draft reply. In doing so, the AI consults information in the patient’s message as well as an abbreviated version of their electronic medical history, said Seth Hain, senior vice president of research and development at Epic. Medical data is protected in compliance with federal laws requiring patient privacy, he added. 

There is an option to start with the draft—and edit or send the message as-is, if it is correct—or start with a blank reply. The AI drafts reference the patient’s medical record when it proposes a response, for example, mentioning their existing medication or the last time they were seen by their physician. “It is helping us get it started,” she said, saving several seconds that would be spent pulling up the patient’s chart. 

For now, the San Diego team has stopped the AI from answering any query that seeks medical advice. Similarly in Wisconsin, the 10 doctors at UW Health have enabled AI responses to a limited set of patient questions, including prescription requests and asks for documentation or paperwork, according to Chero Goswami, chief information officer at UW Health. 

Administrators and doctors say the tool could be transformative—but only if it works. If the drafts require too much fact-checking or modification or demand too much time, then doctors will lose trust in it, said Patricia Garcia, a gastroenterologist at Stanford Health Care who is part of the team that aims to begin trialing GPT for messages next week. “They are only going to use this as long as it is making their life easier.”  

There is an option to start with the draft—and edit or send the message as-is, if it is correct—or start with a blank reply. The AI drafts reference the patient’s medical record when it proposes a response, for example, mentioning their existing medication or the last time they were seen by their physician. “It is helping us get it started,” she said, saving several seconds that would be spent pulling up the patient’s chart. 

For now, the San Diego team has stopped the AI from answering any query that seeks medical advice. Similarly in Wisconsin, the 10 doctors at UW Health have enabled AI responses to a limited set of patient questions, including prescription requests and asks for documentation or paperwork, according to Chero Goswami, chief information officer at UW Health. 

Administrators and doctors say the tool could be transformative—but only if it works. If the drafts require too much fact-checking or modification or demand too much time, then doctors will lose trust in it, said Patricia Garcia, a gastroenterologist at Stanford Health Care who is part of the team that aims to begin trialing GPT for messages next week. “They are only going to use this as long as it is making their life easier.”