ne of the limitations of large language models is that their training is frozen in time. If you ask OpenAI’s viral chatbot ChatGPT if Covid vaccines work against the most common variant circulating in 2023, it responds: “As an AI language model, I don't have access to real-time data or information beyond my last update in September 2021.”
During the Covid pandemic, as the number of scientific studies about Covid-19 ballooned from zero to tens of thousands in the span of a few months, Nadler saw that healthcare providers were facing a similar problem to traders: how to separate credible, actionable information from the noise.
Nadler says OpenEvidence is trying to take on the big incumbent database used by two million healthcare workers worldwide called UpToDate from the Netherlands-based global data company Wolters Kluwer. The clinical solutions in Wolters Kluwer’s health division, which includes UpToDate, generated more than $900 million in revenue in 2022.
Every large language model behaves differently, but the general idea is that they compose answers by predicting the next most likely word in a sentence. When the models tend to get an answer wrong is when “many different completions [are] equally probable,” says Uri Alon, a postdoctoral researcher at the Language Technologies Institute at Carnegie Mellon University, who is not affiliated with OpenEvidence.