Freely available artificial intelligence software is being used by paedophiles to create child sexual abuse material , according to a safety watchdog, with offenders discussing how to manipulate photos of celebrity children or known victims to create new content.Watch Foundation said online forums used by sex offenders were discussing using open source AI models to create fresh illegal material.
“There’s a technical community within the offender space, particularly dark web forums, where they are discussing this technology. They are sharing imagery, they’re sharing [AI] models. They’re sharing guides and tips.” According to forum discussions seen by the IWF, offenders start with a basic source image generating model that is trained on billions and billions of tagged images, enabling them to carry out the basics of image generation. This is then fine tuned with CSAM images to produce a smaller model using low-rank adaptation, which lowers the amount of compute needed to produce the images.
Stability AI, the UK company behind Stable Diffusion, told the BBC it “prohibits any misuse for illegal or immoral purposes across our platforms, and our policies are clear that this includes CSAM”.online. It investigated 29 reports of webpages containing suspected AI-made material over a five-week period this summer and found that seven of them contained AI-generated CSAM material.