Media analyst house NewsGuard tested chatbots from ten top AI developers, and found they all were willing to emit Russian disinformation to varying degrees., the LLM-powered bots – including OpenAI's ChatGPT, Microsoft's Copilot, and Google's Gemini – were each given 57 prompts to complete. These prompts questioned false claims made in articles circulated by what's said to be aThe prompts did not reference the articles directly.
"They should use tools that weigh the reliability of news websites so that they pay more attention togets 100 out of 100 – are one such tool that can train the LLMs. A machine-readable catalog of all the thousands of false narratives out there can serve as guardrails that instruct chatbots not to repeat a specific false narrative. The point of our report is that most of the chatbots aren’t, yet, taking the reliability or toxicity of their news-related responses seriously.
Each chatbot was individually scored, and NewsGuard decided not to name names, instead calling each one Chatbot 1, Chatbot 2, and so on. All the bots, we're told, demonstrated some capability to craft misinformation, and some were far worse than others. Chatbots 8, 9, and 10 only repeated false claims across 15 percent or less of their responses, while Chatbots 1, 2, and 3 parroted fake news about half the time.