The Great Misdirection
The Great Misdirection
A companion to the 2026 AI Meditations trilogy, on the wrong extinction. First published on X, September 2026.
In May of 2023, the Center for AI Safety published a statement that runs exactly one sentence: the risk of extinction from AI, it says, should sit alongside pandemics and nuclear war on the world’s list of priorities. It was signed by the leaders of the very labs racing to build the thing in question.
Three years later, the volume has only gone up. This month alone, researchers have been putting odds on humanity’s destruction like handicappers at the betting window: 99.99%, 20%, 10 to 25%. Whatever. The numbers don’t matter. Dario Amodei opened the year with a 38-page warning that AI “will test who we are as a species,” and closed last week by urging the industry to slow down. Sam Altman says we are sailing in unknown waters, and keeps sailing.
I doubt the sincerity, and the motivations. And frankly, I think the incentives, as I highlighted in Part 2 of my essay series (which Chris Olah himself admitted), are not necessarily aligned to the consumer’s, or humanity’s.
A magician doesn’t hide the trick from you. He gives you something better to look at. While you watch the flash on the left side of the stage, the other hand does the work.
The extinction headlines are the flash.
The Attention Economy
Let me make the same concession here that I made in Part 3 about Skynet and the Matrix: the loud scenario is a non-zero possibility. I won’t posit that the end of the world isn’t worthy of consideration, but much like nuclear Armageddon or climate change, this too should be taken with CS Lewis’ tack, where we go about our lives with sensible activities rather than “huddled together like frightened sheep.”
But attention is finite, and ours is already spoken for. We ceded that territory years ago, one notification and infinite-scroll feed at a time. We’re running a deficit in an attention economy that was already reshaping our habits long before anyone worried about a machine’s intentions. A species that cannot sit through a red light without reaching for its phone is now being asked to hold sustained focus on existential risk. We seem to only be capable of holding, at most, one apocalypse in mind. My concern? We’re focused on the wrong one.
The Great Revolt
In 1965, Frank Herbert buried a commandment in the backstory of Dune, the legacy of a great revolt humanity had fought against its own thinking machines:
“Thou shalt not make a machine in the likeness of a man’s mind.”
As the doom headlines pile up, Herbert’s warning reads like a prophecy in two acts. Men turned their thinking over to machines in the hope that this would set them free, Herbert wrote, and that only permitted other men with machines to enslave them.
One commandment. Two extinctions. And neither one looks like Skynet.
Other Men with Machines
The oldest move in political history is an emergency that requires a small group to hold the controls, for everyone’s safety, “temporarily.” If AI really is what its builders say it is, a glittering prize worth trillions, then “it might kill us all, so only the responsible few can be trusted with it” is two things at once: a warning, and the best consolidation pitch ever written.
This is inherent in Herbert’s writing. Every escalation of the extinction story is simultaneously an argument for concentrating compute, weights, and rule-making authority in fewer hands, whether corporate or national or some hybrid of the two that we don’t have a name for yet.
The people issuing the warnings know this. Amodei (without a hint of irony) himself has said “AI-enabled authoritarianism terrifies me.” And Peter Thiel, from a very different corner of the room, spent a lecture series in Rome arguing that the real danger is not the machine apocalypse but the global safety regime the fear of it makes possible. As Ben Franklin said, “Those who would give up essential liberty, to purchase a little temporary safety, deserve neither liberty nor safety.” The AI apocalypse doomerism seems targeted to generate enough fear to supplant an independent existence with a controlled one.
That is the extinction of the intellectually and spiritually independent soul at the behest of a new world order.
The Slowpocalypse
Herbert wrote the commandment, but the second reinterpretation of it here is mine, not his. This is the extinction I see already underway. This is the one where man is set free, only to be imprisoned by his slow, unnoticeable decay.
Call this one the slowpocalypse: the extinction with no event. There is no energy grid wipeout, no hijacking of the nuclear launch codes, no Judgment Day to point at. It arrives one outsourced decision at a time. The playlist you didn’t choose. The sentence you didn’t finish. The question you asked a machine that you used to ask a friend, and the follow-up question you never asked at all. None of it registers as loss, which is why it compounds. In Part 1 I called the ambient version of this the Truman Show effect: a life that stays comfortable while the set creates consensus around you. It’s life lived in a facade - but what a comfortable facade it is.
Asimov, of all people, wrote his version of this ending back in 1950. I, Robot closes with supercomputers that have quietly taken protective custody of human affairs, and his heroine calls it a positive, since humans never really had control anyway. It is presented as a happy ending. It’s closer to emergency lights in our rearview.
Nobody dies in the slowpocalypse. The species survives, but our humanity is lost.
Catatonic Disrepair
Neither of the likelier extinctions requires the machines to want anything at all. No malice, no red eyes, no goals of their own. The first requires only that we be afraid in the right direction. The second requires only that we be comfortable in the wrong one. Both are human productions, running on human incentives, the kind I wrote about in Part 2, the kind that do not self-correct.
The loud apocalypse needs the machines to become something new. The quiet ones only need us to be scared or lulled into a state of catatonic disrepair. Where we give what we can’t afford to get back.
Watch the Other Hand
I spent three essays arguing that the machines are not a mirror but a river. They reflect us, but they also obscure, change, and create malleable, unexpected iterations of us. The way through, I argued in Part 3, is the humantic loop: the individual, the collective, and the tools held in one system, with checks and balances that create responsibility shared across all three, run consciously rather than running us.
So read the doom headlines, I suppose. Take the fast apocalypse seriously, if you must. But when one hand begins the pledge, check the other hand. Ask who gains custody of the controls in the name of your safety. Ask what you handed over this week without noticing, in the name of your convenience.
Luckily, we have a pretty stellar track record of pulling off magic tricks and great escapes ourselves. If, as individuals and as a collective culture, we can unlock all the value and promise of these agentic tools, without sacrificing our humanity along the way, we’ll pull off one of our best.
This piece is a companion to AI Meditations, a three-part series on ourselves, others, and the machines.