Have a Thorny Medical Question? Your Doctor May Be Using A.I. for That.

Dr. Nicholas Gavin, an emergency medicine doctor at Mount Sinai in New York City, was working an overnight shift last summer when a patient arrived with a perplexing set of symptoms. Within moments, his three younger colleagues—two medical students and a resident—were consulting a free artificial-intelligence-powered app for physicians, OpenEvidence.

Dr. Gavin soon discovered that they were not alone. A third of Mount Sinai’s 9,000 doctors were already regular users of OpenEvidence, as revealed in a meeting last year with the start-up’s leaders.

“That was an ‘aha’ moment for our leadership,” said Dr. Gavin, who also serves as the system’s chief clinical innovation officer.

OpenEvidence’s A.I. app, essentially a chatbot for medicine, has gained immense popularity among physicians. If you speak to a doctor, there’s a good chance they use the app to ask specific medical questions or engage in diagnostic discussions.

More than half of the nation’s physicians are regular users of this innovative tool. Last month alone, they posed 30 million questions and consultations through the app—nearly double the volume from six months prior. A separate survey last year of 1,000 physicians indicated that 45 percent utilized the app, nearly three times the percentage that used ChatGPT, according to Offcall, a career information service for doctors.

This rapid growth propelled the start-up to a $12 billion valuation in January, a significant leap from $3.5 billion just six months earlier.

However, the swift adoption of the app since its launch in 2024—one of several A.I.-enhanced programs vying for physician attention—has raised concerns about its application in critical situations. In a high-stakes field like medicine, healthcare systems are grappling with issues of patient privacy, safety, and trust, alongside the limitations of the technology itself.

“It’s not an oracle; it’s a tool,” stated Daniel Nadler, founder and CEO of OpenEvidence. “Knowledge and knowledge workers still matter.”

The doctor’s office has long been a target for computer-assisted decision-making, but success has been limited until the recent advancements in A.I.

The initial wave of A.I. in medicine focused on alleviating the heavy burden of documentation that contributes to physician burnout, utilizing A.I. scribe software for transcriptions and summaries of patient visits. The second wave, which is just beginning, aims to assist doctors with reliable information and advice to guide diagnosis and treatment at the patient’s bedside.

Competition has intensified recently. UpToDate, a well-known electronic reference for doctors, has revamped its service with an A.I. chatbot interface. Doximity, an online professional network for physicians, acquired an A.I. start-up that mines medical literature and generates summaries. Abridge, a rapidly growing A.I. scribe maker, is also adding decision-support tools. Last month, OpenAI introduced ChatGPT for Clinicians.

OpenEvidence has emerged as a front-runner by exclusively using medical journals and high-quality research to train its A.I. models. Physicians can ask the app specific questions or input patient characteristics and symptoms to receive potential explanations. The app complies with federal laws protecting patient health information, and users are advised against entering any personally identifiable information.

OpenEvidence provides a summary of the most likely diagnoses, along with other “most important not to miss diagnoses,” each linked to the research articles that inform the summaries.

“A.I. is solving some of the problems that have long plagued the practice of medicine,” remarked Dr. Raja-Elie Abdulnour, chief clinical innovation officer at NEJM Group, which publishes The New England Journal of Medicine. “These tools just didn’t exist before, and that’s why people are so excited about them now.”

Despite the early enthusiasm, medical experts urge caution. The research conducted so far into the benefits and shortcomings of A.I. in medicine presents mixed results.

While A.I. has excelled in standard licensing exams and outperformed human doctors in diagnosing certain cases, it has also faltered, failing to accurately summarize research papers or providing incorrect answers to diagnostic queries. Experts agree that it won’t replace human practitioners anytime soon.

“The potential for A.I. is great, but we’re not there yet,” stated Dr. Eric Topol, a cardiologist and executive vice president at Scripps Research in San Diego. “It hasn’t really been tested and demonstrated in the messy, real world of medicine.”

Dr. Topol co-authored a recent paper, “The Illusion of Readiness in Health A.I.,” which identified “significant competency gaps” in the capabilities of large A.I. systems when applied to healthcare.

Evaluations thus far have primarily focused on the performance of large language models from tech giants like OpenAI and Google, which are trained on data from the open internet.

OpenEvidence, founded in 2022, adopted a more focused approach. It posited that smaller A.I. software models trained on specialized data could outperform larger models in a specific, information-rich field like medicine. The start-up initially trained its software on publicly available medical data from sources like the National Library of Medicine.

Subsequently, the company secured content licensing agreements with The New England Journal of Medicine, The Journal of the American Medical Association, and other publishers of peer-reviewed medical literature.

OpenEvidence is accessible to any government-verified physician in the United States as a free, downloadable app.

“We treated physicians like consumers,” Mr. Nadler explained. Users encounter advertisements—many from pharmaceutical companies—during the brief wait for the A.I. to respond. Physicians see ads on only 5 percent of their inquiries, according to the company.

Bypassing traditional gatekeepers in hospital technology departments has raised some concerns. OpenEvidence has relied on “shadow A.I.,” where employees use such tools without their employers’ knowledge or oversight.

Some health systems are now working to integrate OpenEvidence into their institutional frameworks. Mount Sinai announced in March that it would provide a link to OpenEvidence directly from a patient’s electronic health record.

However, this agreement does not grant the start-up access to the medical center’s patient data. Dr. Gavin indicated that such integration could occur later, but only after thorough testing and controls.

Ensuring patient privacy and safety will be “paramount,” he emphasized, adding that “we’re not going to just throw a patient’s data over the wall to a private company.”

Doctors in smaller practices across the country, particularly in rural areas, have expressed their support for the technology.

In Corinth, Mississippi, Dr. Ben Long identifies as an A.I. skeptic. However, he feels reassured knowing that OpenEvidence generates answers based solely on high-quality, peer-reviewed information.

Initially, Dr. Long used the app mainly as a reference tool for factual questions. Now, he views it as “a consultant, a thought partner” with which he engages in dialogue.

“A.I. forces you to think more deeply about your own thinking, challenging your assumptions and why you might be wrong,” Dr. Long noted.

A.I. also enables doctors to access expertise that would typically belong to specialists.

Dr. Barbara Creighton frequently diagnoses and treats complex cases at a community hospital in Fairbanks, Alaska, which can involve multiple conditions and failing organs. In larger medical centers, a team of specialists might be consulted—such as an infectious disease expert, a pulmonologist, and a gastroenterologist.

Dr. Creighton’s smaller hospital lacks such extensive staffing. While they do have an arrangement with a larger medical center for specialist consultations, she increasingly relies on OpenEvidence to answer many questions, saving her time and her hospital money.

“It’s like having a bunch of specialists in your pocket,” Dr. Creighton remarked.

At Mount Sinai, Dr. Gavin views A.I. technology as a powerful tool to help fulfill the promise of precision medicine, tailoring treatments to individual patients.

Progress will require a “patchwork of solutions” from hospitals, medical schools, and private companies, he noted. Whether OpenEvidence will thrive and play a role in that long-term future remains uncertain.

“But it represents a step in that direction,” Dr. Gavin concluded.

Dr. Nicholas Gavin, an emergency medicine doctor at Mount Sinai in New York City, was working an overnight shift last summer when a patient arrived with a perplexing set of symptoms. Within moments, his three younger colleagues—two medical students and a resident—were consulting a free artificial-intelligence-powered app for physicians, OpenEvidence.

Dr. Gavin soon discovered that they were not alone. A third of Mount Sinai’s 9,000 doctors were already regular users of OpenEvidence, as revealed in a meeting last year with the start-up’s leaders.

“That was an ‘aha’ moment for our leadership,” said Dr. Gavin, who also serves as the system’s chief clinical innovation officer.

OpenEvidence’s A.I. app, essentially a chatbot for medicine, has gained immense popularity among physicians. If you speak to a doctor, there’s a good chance they use the app to ask specific medical questions or engage in diagnostic discussions.

More than half of the nation’s physicians are regular users of this innovative tool. Last month alone, they posed 30 million questions and consultations through the app—nearly double the volume from six months prior. A separate survey last year of 1,000 physicians indicated that 45 percent utilized the app, nearly three times the percentage that used ChatGPT, according to Offcall, a career information service for doctors.

This rapid growth propelled the start-up to a $12 billion valuation in January, a significant leap from $3.5 billion just six months earlier.

However, the swift adoption of the app since its launch in 2024—one of several A.I.-enhanced programs vying for physician attention—has raised concerns about its application in critical situations. In a high-stakes field like medicine, healthcare systems are grappling with issues of patient privacy, safety, and trust, alongside the limitations of the technology itself.

“It’s not an oracle; it’s a tool,” stated Daniel Nadler, founder and CEO of OpenEvidence. “Knowledge and knowledge workers still matter.”

The doctor’s office has long been a target for computer-assisted decision-making, but success has been limited until the recent advancements in A.I.

The initial wave of A.I. in medicine focused on alleviating the heavy burden of documentation that contributes to physician burnout, utilizing A.I. scribe software for transcriptions and summaries of patient visits. The second wave, which is just beginning, aims to assist doctors with reliable information and advice to guide diagnosis and treatment at the patient’s bedside.

Competition has intensified recently. UpToDate, a well-known electronic reference for doctors, has revamped its service with an A.I. chatbot interface. Doximity, an online professional network for physicians, acquired an A.I. start-up that mines medical literature and generates summaries. Abridge, a rapidly growing A.I. scribe maker, is also adding decision-support tools. Last month, OpenAI introduced ChatGPT for Clinicians.

OpenEvidence has emerged as a front-runner by exclusively using medical journals and high-quality research to train its A.I. models. Physicians can ask the app specific questions or input patient characteristics and symptoms to receive potential explanations. The app complies with federal laws protecting patient health information, and users are advised against entering any personally identifiable information.

OpenEvidence provides a summary of the most likely diagnoses, along with other “most important not to miss diagnoses,” each linked to the research articles that inform the summaries.

“A.I. is solving some of the problems that have long plagued the practice of medicine,” remarked Dr. Raja-Elie Abdulnour, chief clinical innovation officer at NEJM Group, which publishes The New England Journal of Medicine. “These tools just didn’t exist before, and that’s why people are so excited about them now.”

Despite the early enthusiasm, medical experts urge caution. The research conducted so far into the benefits and shortcomings of A.I. in medicine presents mixed results.

While A.I. has excelled in standard licensing exams and outperformed human doctors in diagnosing certain cases, it has also faltered, failing to accurately summarize research papers or providing incorrect answers to diagnostic queries. Experts agree that it won’t replace human practitioners anytime soon.

“The potential for A.I. is great, but we’re not there yet,” stated Dr. Eric Topol, a cardiologist and executive vice president at Scripps Research in San Diego. “It hasn’t really been tested and demonstrated in the messy, real world of medicine.”

Dr. Topol co-authored a recent paper, “The Illusion of Readiness in Health A.I.,” which identified “significant competency gaps” in the capabilities of large A.I. systems when applied to healthcare.

Evaluations thus far have primarily focused on the performance of large language models from tech giants like OpenAI and Google, which are trained on data from the open internet.

OpenEvidence, founded in 2022, adopted a more focused approach. It posited that smaller A.I. software models trained on specialized data could outperform larger models in a specific, information-rich field like medicine. The start-up initially trained its software on publicly available medical data from sources like the National Library of Medicine.

Subsequently, the company secured content licensing agreements with The New England Journal of Medicine, The Journal of the American Medical Association, and other publishers of peer-reviewed medical literature.

OpenEvidence is accessible to any government-verified physician in the United States as a free, downloadable app.

“We treated physicians like consumers,” Mr. Nadler explained. Users encounter advertisements—many from pharmaceutical companies—during the brief wait for the A.I. to respond. Physicians see ads on only 5 percent of their inquiries, according to the company.

Bypassing traditional gatekeepers in hospital technology departments has raised some concerns. OpenEvidence has relied on “shadow A.I.,” where employees use such tools without their employers’ knowledge or oversight.

Some health systems are now working to integrate OpenEvidence into their institutional frameworks. Mount Sinai announced in March that it would provide a link to OpenEvidence directly from a patient’s electronic health record.

However, this agreement does not grant the start-up access to the medical center’s patient data. Dr. Gavin indicated that such integration could occur later, but only after thorough testing and controls.

Ensuring patient privacy and safety will be “paramount,” he emphasized, adding that “we’re not going to just throw a patient’s data over the wall to a private company.”

Doctors in smaller practices across the country, particularly in rural areas, have expressed their support for the technology.

In Corinth, Mississippi, Dr. Ben Long identifies as an A.I. skeptic. However, he feels reassured knowing that OpenEvidence generates answers based solely on high-quality, peer-reviewed information.

Initially, Dr. Long used the app mainly as a reference tool for factual questions. Now, he views it as “a consultant, a thought partner” with which he engages in dialogue.

“A.I. forces you to think more deeply about your own thinking, challenging your assumptions and why you might be wrong,” Dr. Long noted.

A.I. also enables doctors to access expertise that would typically belong to specialists.

Dr. Barbara Creighton frequently diagnoses and treats complex cases at a community hospital in Fairbanks, Alaska, which can involve multiple conditions and failing organs. In larger medical centers, a team of specialists might be consulted—such as an infectious disease expert, a pulmonologist, and a gastroenterologist.

Dr. Creighton’s smaller hospital lacks such extensive staffing. While they do have an arrangement with a larger medical center for specialist consultations, she increasingly relies on OpenEvidence to answer many questions, saving her time and her hospital money.

“It’s like having a bunch of specialists in your pocket,” Dr. Creighton remarked.

At Mount Sinai, Dr. Gavin views A.I. technology as a powerful tool to help fulfill the promise of precision medicine, tailoring treatments to individual patients.

Progress will require a “patchwork of solutions” from hospitals, medical schools, and private companies, he noted. Whether OpenEvidence will thrive and play a role in that long-term future remains uncertain.

“But it represents a step in that direction,” Dr. Gavin concluded.