> the thought of a Kubernetes cluster autonomously flying around with nukes scares the fuck out of me
I get this, but I also wonder if the same kind of bugs or unintended behavior in a software system aren't just as likely in the protocols, procedures, and chains of command we build for humans to deploy that weaponry. You've got AI doomsday (a la "War Games") on one hand, and bureaucratic doomsday ("Doctor Strangelove") on the other.
That may be the case, but I think the "bugs" in humanity are biased towards not using nuclear weapons. I think that's a good thing.
For example it's very possible that a "Should I launch the nukes?" algorithm would've come up with a different answer than Stanislav Petrov did[1]. There's a similar story about US missileers disregarding a glitch in their own command and control system on Okinawa but the authenticity is suspect.
That's the lynchpin that guarantees Russia will retaliate, but it's Ripper's unchecked moments of control at SAC that set the whole ball rolling, and Slim Pickin's is cowboy-riding that bomb down to the ground whether Russia has an AI or not.
I get this, but I also wonder if the same kind of bugs or unintended behavior in a software system aren't just as likely in the protocols, procedures, and chains of command we build for humans to deploy that weaponry. You've got AI doomsday (a la "War Games") on one hand, and bureaucratic doomsday ("Doctor Strangelove") on the other.