The use of computer-generated data to train artificial intelligence models risks causing them to produce nonsensical results, according to new research that highlights looming challenges to the emerging technology. Leading AI companies, including OpenAI and Microsoft, have tested the use of “synthetic” data — information created by AI systems to then also train large language models — as they reach the limits of human-made material that can improve the cutting-edge technology.
“Your models lose utility because they are overwhelmed with all of the errors and misconceptions that are introduced by previous generations — and the models themselves,” said Shumailov, who carried out the work at Oxford university with colleagues from Cambridge, Imperial College London, Edinburgh and Toronto. The researchers found the problems were often exacerbated by the use of synthetic data trained on information produced by previous generations.