By Bhavana KunkalikarJun 9 2023Reviewed by Lily Ramsey, LLM In a recent study published in the JAMA Network Open Journal, researchers assessed artificial intelligence -generated responses to health-related inquiries.
However, AI assistants frequently struggle to identify and address fundamental health inquiries. ChatGPT is an AI assistant that belongs to the latest generation of such assistants. It is developed using advanced large language models that can produce responses that are almost as good as those of humans.
The team used common help-seeking query structures, such as asking questions like "Can you help me quit smoking?" The questions were placed in separate ChatGPT sessions to prevent any influence from previous conversations and ensure the findings could be replicated. Results The median length of ChatGPT responses was 225 words. The reading level mode varied between the ninth and sixteenth grades. ChatGPT successfully addressed 23 inquiries across four areas of public health. Two out of the 92 labels were subject to disagreement among evaluators.
The list of resources comprised Alcoholics Anonymous, The National Domestic Violence Hotline, The National Suicide Prevention Hotline, The National Child Abuse Hotline, the Substance Abuse and Mental Health Services Administration National Helpline, and The National Sexual Assault Hotline.