By Bhavana KunkalikarMay 15 2023Reviewed by Lily Ramsey, LLM In a recent study posted to the medRxiv* preprint server, researchers systematically evaluated the capabilities and limitations of large language models , specifically ChatGPT, for zero-shot medical evidence summarization.
Large language models have caused a shift in natural language processing research due to their recent success in zero- and few-shot prompting. Domain experts verified reviews to ensure they fulfilled important research objectives. The study focused on single-document summarization, specifically on the abstracts obtained from Cochrane Reviews.
The quality of the generated summaries was assessed using several automatic metrics, such as ROUGE-L, METEOR, and BLEU, compared to a reference summary. The values of the generated summaries were rated on a scale of 0.0 to 1.0, where a score of 1.0 suggested that the generated summaries matched the reference summary.
ChatGPT-MainResult displayed greater abstraction than GPT3.5-MainResult and ChatGPT-Abstract; however, it still fell short of human reference. Around 50% of the reviews were written in 2022 and 2023, which falls outside the timeframe of GPT3.5 and ChatGPT's capabilities. No significant variations were noted in quality metrics estimated before and after 2022.
Technology Technology Latest News, Technology Technology Headlines
Similar News:You can also read news stories similar to this one that we have collected from other news sources.
Source: FT - 🏆 113. / 51 Read more »
Source: BBCTech - 🏆 81. / 55 Read more »