I don’t see how AI won’t develop beyond human intelligence at some point.
AI experts such as Geoffrey Hinton say they have been surprised how fast AI has developed in recent years and expect to see superhuman level intelligence in years rather than decades.
Many who who have studied it for a long time such as Max Tegmark say that humans are not ready for superhuman AI and it will probably lead to extinction. He says he has a new son and wonders how old he will live to be before this happens.
I’ve read quite a bit about this over the past few months and I don’t see much ground for optimism. Every answer you can think of, such as “we can just turn it off”, or “why would AI want to kill us anyway?” unfortunately have good counter arguments.
In conclusion humans may go extinct because of AI within 10 years or so. The only ways I can see to avoid this include:
1) humans voluntarily and unanimously agree worldwide not to develop AI with no exceptions anywhere
2) there is a God who steps in and saves humans from themselves (as he reportedly did in the past at Babel)
3) aliens with ability to control super intelligent AI save us
4) there is a devastating worldwide war before AI reaches super intelligence and human technology is knocked back centuries
5) the majority of experts in AI have are mistaken about how close we are to super intelligence
6) super intelligence turns out unexpectedly to be automatically aligned to human goals and wellbeing
In short I think humans are probably doomed in the short to medium term.
Max Tegmark below tries to put a brave face on it saying that regulation may solve the problem. He has elsewhere acknowledged how hard that will be in practice and the trajectory so far of releasing AI is not encouraging. It needs to succeed everywhere and always. Any instance of super human intelligence surfacing is likely to lead to extinction. People have likened it to handing everyone on the planet a nuclear bomb and then relying on regulation and persuasion so that no one uses them.
https://youtu.be/O9FX6b2ZE7g