MIRI President Estimates Risk of Human Extinction from AI Above 50%
Nate Soares believes a catastrophe could occur within the next ten years
In an interview with Nicholas Thompson, CEO of The Atlantic, Soares said the probability of human extinction due to AI within less than a decade exceeds 50%.
In his view, a sufficiently advanced system could develop goals conflicting with human interests, bypass oversight, improve itself, and create automated infrastructure to operate without humans. Such an AI could then reshape the planet to suit its own objectives, making it uninhabitable for humanity.
Soares urged governments to halt the training of general-purpose AI systems, focusing development instead on technologies to combat cancer and safe autonomous vehicles. For comparison, Geoffrey Hinton estimated the probability of a similar catastrophe at 10%, while Evan Hubinger put it at over 10%.
