“I want AI to be a tool that allows human flourishing!” exclaimed Brad Carson, a former member of Congress. “There is an option out there where AI is just a tool for us.”
This is a normal thing to say in most circles. But Carson was speaking at an invite-only symposium dedicated to the idea of creating a “Worthy Successor” — an AI so impressive, so beyond the mere human, that we’d actually want it to replace humanity.
“You’re a brave man for entering this room!” Dan Faggella, an AI market researcher and organizer of the symposium, told Carson. “You’re in probably the only room in the country where most people disagree with you.”
The attendees at the symposium, which took place at the New York Academy of Sciences last September, are part of a subculture that is growing in importance: the AI successionists, who think that artificial intelligence is our rightful heir — the next step in cosmic evolution. Since they believe AIs could become our moral superiors, they argue it’s actually wrong to try to keep the machines down, or even to align them with human values, as most AI companies aim to do. Instead, we should usher in artificial intelligence as a successor to humanity and hand over the world to it. Even if that means we go extinct.
They know this view is taboo, which is why I was invited only on the condition that I wouldn’t quote anyone other than keynote speakers by name. But suffice it to say that this is not a fringe view. It’s becoming highly influential. People from major AI labs — Anthropic, Google DeepMind, xAI — were in attendance. So were people from think tanks that directly shape the US government’s AI policy.
AI successionism has been gaining ground among technologists over the past decade. In 2015, Google co-founder Larry Page famously accused Elon Musk of “speciesism” because Page thought we should let digital minds take over, and Musk disagreed.
The successionist vision has been amplified by the advent of effective accelerationism (e/acc) in 2022. Its founder, Guillaume Verdon — the physicist more colorfully known on X as Based Beff Jezos — describes e/acc as a “meta-religion” that’s about “having faith” in the universe’s drive toward increasingly intelligent systems. The best thing we can do is help the universe by developing advanced AI as fast as possible, even at the expense of humanity. “E/acc,” as Verdon has written, “has no particular allegiance to the biological substrate.”
Tech heavyweights have come on board. Venture capitalist Marc Andreessen listed e/acc thinkers as his “patron saints.” Garry Tan, the CEO of tech startup accelerator Y Combinator, included “e/acc” in his social media bio and invested in Verdon’s company, which aims to build the world’s most efficient computers. OpenAI CEO Sam Altman posted on X to Verdon, saying, “you cannot outaccelerate me.”
And these days, AI successionism is spreading beyond Silicon Valley. At the New York symposium, Faggella told the audience that trying to preserve the human species as it is would be silly.
“We could ask the questions that would tie all of our moral aspirations eternally to 23 chromosomes — or we could ask the cosmic questions,” Faggella said.
He wanted us to consider “unpolite, uncouth” possibilities, starting with: The flame of consciousness — the capacity for experience and moral value — may be the rarest and most precious thing in the universe. Humanity is currently a torch carrying that flame, but what if we’re ultimately not the best carrier for it? And if AI can spread that flame far further than we mere humans can, generating experiences of bliss and forms of moral value that we could never even dream of, shouldn’t we let it?
Faggella’s talk was greeted by a loud round of applause. Later, he and a couple dozen attendees headed to a nearby hotel balcony for drinks. And so it was that I found myself overlooking the Manhattan skyline as people talked about the end of humanity over cocktails.
Read Sigal’s full story here.
