, researchers are now saying that yes, OpenAI's GPT large language model appeared to be getting dumber., researchers out of Stanford and Berkeley found that over a period of a few months, both GPT-3.5 and GPT-4 significantly changed their "behavior," with the accuracy of their responses appearing to go down, validating user anecdotes about the apparent degradation of the latest versions of the software in the months since their releases.
"Both GPT-4 and GPT-3.5," the abstract continued, "had more formatting mistakes in code generation in June than in March."This study affirms what users have been saying for more than a month now: that as they've used the GPT-3 and GPT-4-powered ChatGPTover time, they'veThe seeming degradation of its accuracy has become so troublesome that OpenAI vice president of product Peter Welinder attempted to dispel rumors that the change was intentional..
"We find that the performance and behavior of both GPT-3.5 and GPT-4 vary significantly across these two releases and that their performance on some tasks have gotten substantially worse over time," the paper noted, adding that it's "interesting" to question whether GPT-4 is indeed getting stronger.