SAN FRANCISCO – In May, hundreds of well-known people in the world of artificial intelligence signed an open letter warning that“Mitigating the risk of extinction from AI should be a global priority, alongside other societal-scale risks such as pandemics and nuclear war,” the one-sentence statement said.
“Today’s systems are not anywhere close to posing an existential risk,” said Professor Yoshua Bengio, an AI researcher at the University of Montreal. “But in one, two, five years? There is too much uncertainty. That is the issue. We are not sure this won’t pass some point where things get catastrophic.”If you ask a machine to create as many paper clips as possible, they say, it could get carried away and transform everything – including humanity – into paper clip factories.
Then it will keep looking for ways of reaching that goal, particularly if it is connected to other Internet services.