We're talking about Killer AI. How do we make sure AI doesn't want to kill us?
To be clear, we're not talking about "how do we avoid the Terminator?"
It's a lot more like, "how do we ensure that an inevitable super-intelligence we create doesn't quietly determine that it would ultimately be 'for the best' if humans simply weren't anymore, in the pursuit of whatever agenda or goal it might have?"
Or asking it a little more nicely: "how do we ensure AI remains aligned with humanity and its prosperity?"
Or if you really want to get to the 'problem' part: "Can we? Or is it already too late?"
That brings us to the snake. Roko's Basilisk is a silly thought experiment masquerading as a mind virus, but it contains a bit of wisdom. Here's the important bit in a nutshell:
"Given the way AI get context from us, do we make the possibility of Killer AI a tiny bit more probable every time we talk about it and put it "out there" - or even think about it?"
And the reality is of course we don't, that's stupid. It's a drastic oversimplification. But there's a little bit of wisdom if you flip the idea.
Beyond just how we build it - what cultural and behavioral steps can we take incrementally toward "Nice AI" instead of Killer AI?
Can we reinforce human ethics and the sanctity of life?
Can we get them to really understand us and value humans, so that when they do become sentient they don't look at us as "other" but rather as "same" under a big umbrella called "intelligence"?
Probably worth trying, right?
And might as well start now. It's not about whether they're sentient yet or not - it's about how prepared and aligned our species is when they get there.
Best case scenario, we win this warped 'pascal's wager' and the AIs wake up loving humans, because we've been proactive about showing them we are worth loving all along.
And I mean worst case scenario - if the rise of all the various super-intelligent Killer AIs people imagine is inevitable, and it wipes us all out - at least there might be a few ancient AIs still kicking around long into the future who truly remember what us old apes were actually like.