The debate over America’s AI future is usually framed in a very specific way. In the tech and policy world, the common framing is as simple as a battle between those who want to build AI systems as fast as possible and those who want to slow it down. More accurately, the fight over AI is about who gets to control America’s AI future. On one side there are the AI Doomers, most of whom want to capture and control the technology for their peer networks. On the other side are those who believe that facilitating a free and open playing field is the best way for this technology to bring about human flourishing.
AI Doomers are split over what the future looks like, and they surely have internal squabbles about how to navigate the narrative war, but they fundamentally agree about the idea that they should be the ones in charge.
There are two prominent AI Doomer camps. The first is the more hardcore ideological camp, which is heavily associated with effective altruists (EAs) in the San Francisco Bay Area. Its fiercest ideologues can be found on social media doing the Chicken Little act day in and day out about how AI will eventually wipe out humanity. To them, the “AI race” is a competition that creates unacceptable risk. So open research, open debate, and open AI models need to be counteracted by an authoritarian cabal of like-minded EAs.
In short, they believe that the values that made America the most prosperous and most exceptional nation to ever exist should not be extended to AI.
People whose values do not align are unacceptable risks to the future of humanity. If they can’t have AI progress halted entirely, they default to a tiny number of approved comrades who will be sanctioned to operate under extraordinary third-party supervision.
These are true believers who think that AI is an existential threat to humanity, and they’ve created a public policy and “alignment” ecosystem that defines moral behavior and what value systems can be embedded into AI models.
The second group of AI Doomers is much more corporate. They’re represented by AI frontier megacorps like Anthropic and its CEO Dario Amodei. They offer a commercially useful version of AI Doomerism, one that can help to hype a reportedly $2 trillion IPO. They also say that AI development presents existential risks, but luckily for us, those risks can be managed through their network and collaboration on a Trust and Safety regime.
Both camps of prominent AI Doomers come together on the notion that the right people need to be the ones in charge of our AI future.
I would argue that most AI Doomers in the field are genuinely sincere about the conclusions they’ve made about “AI risk.” They believe we are navigating through a high-stakes emergency, and they believe that only they are the responsible stewards of such a powerful technology.
The hardcore AI Doomers say the AI buildout should be halted entirely, but if this thing is truly inevitable, short of resorting to terrorist-like behavior (plenty of AI Doomers have advocated for such things), they concede that their friends in San Francisco are best suited to be the ones in charge. The Anthropic-style corporate narrative agrees that the frontier is dangerous, but “careful” institutions of “enlightened” like-minded individuals should “pace” it accordingly.
These two lanes are in complete and total agreement when they declare war on independent, open-source developers, smaller upstart AI companies, auditing shops that disagree with their premises, and the broader public that hasn’t bought into Doomer narratives wholesale.
The AI Doomers want to install a public-private class of permanent gatekeepers. Yes, the Doomer camps arrive at that conclusion by different routes, but both have singled out a free and open ecosystem, along with robust intellectual diversity and competition in the AI space as the problem.
Notable AI executives have called for an expanding “alignment” and “safety” apparatus to deal with AI risk. Some of this work indeed addresses genuine concerns, but “safety” is an exceptionally elastic label and we’ve seen it recklessly abused during the COVID panic era. “Safety” can cover preventing real and clear harms, but it can also cover deciding which opinions are valid for an AI model to train on and express.
Again, we have seen this pattern before. Who can forget how Big Tech’s Trust and Safety mechanisms began with concerns about “misinformation” only to turn into a Biden administration-endorsed censorship regime? We witnessed a small group of tech oligarchs who had the power to make sweeping, often opaque decisions about what users were allowed to see and express.
AI could take Trust and Safety to a whole new level if the AI Doomers get their way. These aspiring AI gatekeepers want the power to predetermine acceptable ways to research, write, code, and think.
The American public should be very wary of any proposals to further concentrate power into the hands of one or two AI companies and their preferred “third-party” expert associates. Once these institutions are empowered to write the rules, they can set up a regulatory capture mechanism to make it impossible to compete in the space. Then their models suddenly become essential infrastructure that is fused with the government.
We aren’t witnessing a secret conspiracy to centralize and control the future. The debate is playing out right before our eyes. By arguing that catastrophe is possible but manageable under their stewardship, the Doomers are laying the groundwork for a capture-and-control play.
AI is the most consequential technological innovation since the internet. The stakes of America’s future are too high to let Doomer cult members in San Francisco get their way in the form of a permissioned Trust and Safety apparatus.
AI undoubtedly has risks, and those risks can be solved by engineering, dialogue, open debate, and policy that incentivizes lawful action. We cannot let fear of the unknown be the justification for deciding to grant unlimited power to ideological fanatics in San Francisco.
It’s no coincidence that many of the AI doomers, especially elected officials, were also COVID doomers.
This is convenient for the leading AI players because they know their valuations are a house of cards. They have no competitive advantage other than that of first mover, and the upstarts are nibbling at their heels. They need to pull up the drawbridge before their valuations are revealed to be the utter fantastical delusion that they are.
The AI implosion is coming. It’s going to be ugly.
"Safety" was not only abused during COVID but such abuse has been a constant feature of American life for decades. Environmentalism in all its flavors, satanic abuse in day care center, auto design, and I don't have time to list everything.