If you ask some of the most popular artificial intelligence-powered chatbots how many days are left until the November election, you might want to double check the answer. A study published by data analytics startup GroundTruthAI found that large language models including Google’s Gemini 1.0 Pro and OpenAI’s ChatGPT gave incorrect information 27% of the time when asked about voting and the 2024 election. Researchers sent 216 unique questions to Google’s Gemini 1.0 Pro and OpenAI’s GPT-3.
” A spokesperson for Google who received a summary of the analysis said the answers the researchers got would have only been available with paid access to the Gemini API and would not be available to the general public using its web-based chatbot, something NBC News was unable to independently confirm. The study comes as many companies are starting to infuse generative AI into some of their consumer products.