Using AI in science promises to add to problems in reproducing important results, the UK's highly prestigious Royal Society has warned.
"A growing body of irreproducible studies are raising concerns regarding the robustness of AI-based discoveries," she declared."Barriers such as insufficient documentation, limited access to essential infrastructures and a lack of understanding of how AI tools reach their conclusions make it difficult for independent researchers to scrutinise, verify and replicate experiments," the paper explains.
The Royal Society was founded in 1660 and scientists including Isaac Newton, chemist Humphry Davy, and Ernest Rutherford, discoverer of the atomic nucleus, are among its past presidents. Models developed in a commercial setting can add to the problem."For instance, most leading LLMs are developed by large technology companies like Google, Microsoft, Meta, and OpenAI. These models are proprietary systems, and as such, reveal limited information about their model architecture, training data, and the decision-making processes that would enhance understanding.