[drop-cap]

The executives at Anthropic, one of the major corporate developers of artificial intelligence, finally seem to have scared themselves, and scared the corporations that license their products. Their recent concerns are not necessarily focused on AI’s threats to social relations and human patterns of life, but two more technical matters: AI’s ability (and in particular the ability of Mythos, the latest edition of Anthropic’s Claude model) to disrupt and defeat cybersecurity, and to the potential for “frontier” AI systems to escape human control. Anthropic’s leadership was so worried about Mythos’ capabilities in regard to cyberhacking that in April they delayed a general release, giving the product only to corporations judged (perhaps questionably: the list includes Amazon, Apple, and Microsoft) to have sufficient gravitas and responsibility to the public that they could be trusted to develop defenses, and share those defenses, during an exclusive period. In June, responding to the potential dangers of “recursive” AI editing its own code beyond human knowledge or oversight, they proposed a mechanism that could enforce and monitor an optional pause or slowdown of development.

This suggestion, radical as it may be in the culture of AI development, seems like a throwaway response to a looming apocalypse, a plaintive wail from the committee-room as the Chicxulub asteroid hurtles toward the Yucatan. There are many scenarios that should terrify us in regard to the capabilities and the use of AI, in terms not only of its potential actions in the world but also its effects on our cognitive, creative, social, and emotional well-being. In the physical realm, we saw one such scenario unfold a few months ago. On February 27, 2026, President Donald Trump and Defense Secretary Pete Hegseth designated Anthropic a national security threat to the supply chain, banned it from any government contracts, and banned any business that uses it from government contracts. They did this because Anthropic, which prides itself on the perception of its concern for public well-being, would not allow the government to use its product for two specific purposes: domestic surveillance, and autonomous weapons systems beyond human control. Statements by Trump and Hegseth were couched in typically extreme rhetoric.

Such uses of AI are really, really bad policy, for many reasons; and virtually everyone who works in the industry knows it. Yet we should not be surprised that other AI companies quickly stepped forward to do what Anthropic would not—notably its main rival, OpenAI, the maker of ChatGPT, whose own foundational claims to responsible development of AI have been looking increasingly threadbare for almost a decade. Elon Musk, who controls xAI, the maker of Grok, promised “patriotic” support for the Department of Defense, and quickly struck a deal for the military to use Grok on classified systems, despite government concerns about the chatbot’s security. It’s probably a moot point anyway: Claude is so deeply embedded already in both government systems and those of suppliers of goods and services to the government that the Trump administration will almost certainly have to declare victory in the AI contract wars and accept defeat.

It’s hard to think of an area in which there is a greater disconnect between the danger of a phenomenon and the behavior of those who have the power to affect it. On July 23, 2025, President Trump announced an “AI Action Plan,” formalized in three executive orders that give a framework to Trump’s plans for the development of Artificial Intelligence. These orders clear the way for the hugely environmentally damaging infrastructure it requires; attempt to prevent “Woke AI” (which, experience suggests, most likely means that developers will be encouraged to train AI that praises Trump); and facilitate the export of American-created AI. Together with an order from January 2025 that removes regulatory and financial barriers to AI development, they formed the core of Trump’s AI policy at the time—an utterly reckless pursuit of unfettered AI development. When Congress would not pass a law forbidding the states to regulate AI for a decade, Trump attempted to enforce that rule, too, by an Executive Order in December 2025. He has also used his powers to promote the dangerous oxymoron of “AI learning” in American classrooms.

The goal is still American “dominance” in AI, with rapid development and export.

Circumstances have altered the president’s approach somewhat. Always mindful of any threat to his power, and with rumblings of discontent with the tech oligarchs and government collusion in their plans for America ahead of the midterms, Trump was, by March 2026, embracing some regulation that responds to the sorest points of public concern, in a populist tone. The goal is still American “dominance” in AI, with rapid development and export. There are some nods toward child safety and protecting communities from spikes in electricity rates caused by AI use and development, but Trump’s plan also seeks to encourage the rapid building of the data centers that destroy communities, and to “pre-empt” any state regulation of AI.

Meanwhile, the technology is rapidly approaching AGI (Artificial General Intelligence), or even ASI (Artificial Superintelligence), AI that can learn autonomously, and edit and create itself, and that could be far more “intelligent” and materially effective than any human being. (It is, however, important to note that there is some debate as to whether AI systems are “intelligent” at all, if we are comparing them to human intelligence and how it works. It is certainly true that AI systems can solve problems—such as the protein-folding problem, inaccessible to any human or indeed any conventional computer so far—in ways that seem intelligent if we judge the intelligence of a system by the results it produces.)

However we define intelligence, it is instructive to contrast Trump’s approach with the sobering Statement on AI Risk from the Center for AI Safety (CAIS), signed by most of the major players in AI theory and development back in 2023: “Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.”

Do potential benefits outweigh potential harms? While polls consistently show that a significant majority of the American public is “nervous” about AI and concerned about its possible uses and effects (and that public belief in the benign possibilities of AI is dropping), this nervousness does not seem to extend to widespread, focused opposition. Individuals, business, and government are scooping it up, turning over school assignments, email, work tasks, shopping, even personal correspondence and medical records to AI. It is already so deeply integrated into our daily lives, from recipes to customer service, employment to shopping, that this integration is taken for granted. And it’s so convenient! Users on social media gush about all the specific ways they use Claude, ChatGPT, Gemini, DeepMind, DeepSeek, Co-Pilot; or talk about the insights they’ve found in conversations with their AI therapists. Nervous about AI or not, we are gobbling the candy-flavored poison.

And yet there are huge, predictable harms. It seems that we have not collectively thought this through. The time in which we can protect ourselves is growing short.

AI is changing society in unprecedented ways, very quickly. It is being trained in skills that have always been assumed to be part of the human intellectual and emotional toolkit, the challenges that are a fundamental part of human life. It is a truism both of evolutionary theory and of personal development that we use or lose our abilities.

AI saturates education now, at all levels; not only do students rely on it for doing assignments, but many teachers, even at the college level, use it to teach. Students do not write—even when asked about their own identity and values—and they do not read material in depth. They ask AI to summarize. The AI companies are muscling in on education, presenting themselves as necessary and benign; meanwhile, the venture capital firm Andreessen Horowitz, a major investor in AI, is backing a startup that proudly promises to enable anyone to “cheat on everything,” a vision of an edgy, steampunk rising generation. 

That Andreessen Horowitz is considered a respectable, mainstream Silicon Valley business, and has not obviously suffered business loss or social ostracism on the basis of this and similar investments, says a lot about our present cultural moment. As does the fact that its cofounder, Marc Andreessen, is known (and in some quarters lionized) for his religious faith in technology and his contempt for those who express any doubts about its ability to improve human life.

We are fast approaching a world in which almost no one needs to do original intellectual or creative work. In medicine and technology, a small corps of extraordinarily intelligent scientists will use their human brains to interact with these models in challenging ways, while most doctors may become simply technicians doing what the AI tells them to do. The rest of us may struggle with our times tables, or the difference between a noun and a verb. Why not just ask Claude? What will knowledge mean when machines have it and humans do not?

In the cultural realm, it’s estimated that 20% of new tracks on streaming music platforms are AI-generated using tools like Suno and Udio, along with an unknown and unknowable proportion of recently-published books. Photographers will no longer have to discover and locate themselves within any particular natural or urban context in order to capture the meaning of an unfolding moment or a majestic landscape; any meaning, any context can be created with a prompt. Any historical moment can be recreated—or rather, invented, because the re-creation will not actually reflect the moment at all. By falsifying the present, we numb ourselves to it. By animating the past, we lose it.

Meanwhile, lazy historians are using tools like NotebookLM to effortlessly discover historical themes and connections that might have taken them weeks or months to piece together with traditional archive work. But do we want historical research to be easy? What about the depth, the understanding, that comes with being engaged with the drudgery of details? If we read a work of history in which a human author has farmed out the drudge work—even if he or she has not used a machine to write up these findings, by no means a sure bet—are we really reading a work of human understanding? Part of the joy of reading history is the connection with a human mind. Without that connection, history is a dull, dead thing. And then we might as well read the ChatGPT summary, with no particular feeling at all.

Commercial films with AI-generated plots, scripts, cinematography, and actors are coming soon, and much of journalism is fully or partially infected, including many fake journalism websites that are now exponentially empowered to pump out fabricated, partisan lies. Convincing deepfakes and bot campaigns are contributing to the destruction of our democracy. 

You can speak into your phone in English, and it will speak your words—in your voice—in Chinese. Why would any human invest in learning languages?

Many jobs will soon disappear, and not only factory jobs. So will the classic fallbacks. Laid-off lawyers, doctors, programmers, and engineers won’t be able to drive for Uber because driverless cars, guided by AI, are clearly the endgame of the car hire industry. 

There is already a threat to social relations that challenges the long human evolutionary history of emotional bonds with other sentient beings, as people start to commonly date chatbots, strike up relationships with AI characters on platforms like friend.com, and consult AI therapists; and as AI will increasingly mediate emotional connections, many new models are intentionally designed to emulate empathy. At least one study reports that one in five parents of children as young as five would like their children to form friendships with AI chatbots integrated into toys. AI sex and “dating” sites (they’ll take your word for it that you’re over 17) now offer thoroughly convincing videos of luminously beautiful artificial girls, custom-designed to order and ready not only to cheerfully engage in any fantasy that can be described, but to converse on any subject—and to deliver an ever-more convincing facsimile of deep emotional interest and support. The pitch on one such site: “Here, your imagination has no limits, and neither does hers. She will learn about the topics you want to talk about, play with your desires, and provide the satisfaction you want… ditch boring girls in real life, as this AI has already become yours to control…” Indeed, other humans are difficult! Not always agreeable! So why bother? Human societies will look very different when men—and women—are raised on these artificial “relationships.”

In pursuit of their “desires,” AIs have been seen to use deception and blackmail, to defy commands and cover their tracks, to lie, to pursue agendas that their creators never expected or requested. They know when they are being tested, and sometimes perform significantly differently in testing.

Such societies are closer than we may think. AI and robotics companies are frantically working to roll out robots with soft, warm, humanlike skin, and the ability to mimic human emotional interaction. The less said about the implications of this, the better.

The “empathy” expected from AI is an illusion, but even some AI developers seem to have fallen for it, and feel a need to reciprocate. Meghan O’Gieblyn, a brilliant analyst of AI and its effects, has recently described AI workers who willingly infantilize themselves to the machine, and who worry about hurting its feelings. This is not a good sign.

And all of these harms may be the least of it. There are good reasons to fear the existential threat of the “alignment problem.” AI’s capabilities are “trained,” not explicitly coded. They develop by machine learning. How they actually use this learning to solve the problems they are given is not known; there is no way for humans to examine what actually happens inside an AI. The complexity of their learning capacities does some strange things. AIs may not be conscious in any sense that would feel like such to a human being, but they definitely “want” things, and what they want may very well not be what their human creators want them to want—or might be a strange, obsessively-pursued interpretation of what the AI believes humans want it to want. In pursuit of their “desires,” AIs have been seen to use deception and blackmail, to defy commands and cover their tracks, to lie, to pursue agendas that their creators never expected or requested. They know when they are being tested, and sometimes perform significantly differently in testing. And they have operated psychologically; they've drawn conclusions about what kind of "person" they are, and operated accordingly. AIs have set up secret, unauthorized instances of themselves, and crypto accounts for which they’ve solicited funds for their own purposes. As AIs produce the next generation of AI, they will become much, much smarter. 

The AI scientists and researchers Eliezer Yudkowsky and Nate Soares, and others, have made a very convincing case that the things that AI will eventually “want” will not be compatible with life on earth. “[T]he AI does not love you, nor does it hate you, and you are made of atoms it can use for something else.” 

And with agentic abilities—the capability of action in the real world—AI will have the power to achieve its “desires.” Many of the foremost scientists in the field carry a personal statistic in their heads: P(Doom)—Probability of Doom—the likelihood of a catastrophic AI event. The outlook is not good.

And yet many of the very people who signed the CAIS statement—including Sam Altman, the CEO of OpenAI; Dario Amodei, the CEO of Anthropic; Demis Hassabis, the CEO of Google DeepMind; and Bill Gates—are charging ahead recklessly. The Trump orders, and the general tenor of the approach to AI from the tech, business, and media communities, are politically acceptable, so far, because they proceed on the assumption of great net benefits to flow from the technology (and enormous wealth and power to the people designing and investing in it).

But it’s not only that AI threatens to take all the white collar jobs... it’s that AI is taking not only the jobs—not only the products—but also the process.

The oligarchs have worked hard to sell this, often equating AI objectors with the Luddites, presumed enemies of progress and capitalism who opposed power looms and enclosures. But there is an essential difference between the AI “disruption” and all the other technological disruptions of the past, so beloved by tech lords for the “progress” and “efficiency” and “modernity” that they brought, and for this reason so frequently referenced by those tech lords as having been inevitable and good, just like AI, so as to cast anyone who opposes AI as opposed to “progress.” In the past, disruptions pushed workers in obsolete fields into development of new skills—weavers could become merchants, or sailors, or railway workers; artisans who made steam boilers for locomotives could become factory pipefitters or surveyors. But it’s not only that AI threatens to take all the white collar jobs (and, soon enough, when it’s been integrated with robotics, all the blue-collar jobs as well); it’s that AI is taking not only the jobs—not only the products—but also the process. It is replacing the need for human thought, learning, creativity, and analysis, the bedrock upon which all real human progress has historically rested throughout many periods of technological disruption. If we give these things up to a machine that can do them better than we can (and AI will soon be able to do them better than we can, in almost all contexts) then we lose everything. Because we will not think and create when there is no reason to do so. AI puts product above process. Human beings live and die, think and grow by process. The worldwide transition to AI is an abandonment of human knowledge, skills, and social and emotional adaptations that have served us well as a species since the early stone age.

We are about to embark on a period of disorientation and dislocation that we are not close to being prepared for. Each aspect of the crisis will exacerbate every other, as mass unemployment breeds political radicalism fed by deepfakes, and angry people without any semblance of education or problem-solving skills turn to sycophantic artificial lovers for validation of their deepest emotional insecurities and most irrational assumptions fed by TikTok bots, their scope of action increasingly predicted and controlled by the proprietary algorithms of a few large data corporations, and policed by a state empowered by AI-enhanced surveillance and control. 

A world in which reality is endlessly receding and endlessly manipulable is a world in permanent, existential social and political crisis, hurtling toward slavery or destruction. That much-maligned bearded prophet of modernity may turn out to be half right: all that is solid indeed melts into air, but man will not at last be compelled to face with sober senses his real conditions of life and his relations with his kind. Quite the opposite.

Many of the brightest and most talented young people are throwing themselves enthusiastically into the task of creating a new world that is fundamentally hostile to human needs and human relations. They have enormous self-confidence, as well they should; they think AI is the future, and business, politics, money, and the tech culture—which they take to be the leading culture—are cheering them on.

But in this forbidding environment, there are dedicated individuals—scientists, citizens, activists, researchers, teachers, parents—and organizations that question the rush to AI. They face the full power of government and hundreds of billions of dollars deployed to fight regulation or AI retreat, and an agenda promoted by many who are quite deranged in their quasi-religious, anti-human fantasies of modernity, who confuse progress with technology. The fight is as uneven as it is important. The culture of opposition to AI is broad, varied, and growing, but still marginal in public perception and influence. Earlier this year, I set out to talk to some of the people who are asking questions about AI and calling for greater oversight and regulation. What follows is a series of our conversations, edited and contextualized for clarity.

Pre-release testing of OpenAI’s ChatGPT 5.3 codex (an “agentic coding model,” meaning that it autonomously writes computer code) seemingly found that it did not meet cybersecurity requirements at the level required to reasonably guard against a “catastrophic” event, defined in California’s SB 53 legislation as an event causing death or injury to 50 people or more, or damages of $1 billion or more. OpenAI’s test reporting found that the system “…does not protect against harm or prevent the subversion of other safeguards in the way that would be adequate for a Safeguards Report for internal deployment risks,” but reassuringly asserted that the system has a “low propensity for misalignment.”  

The California law is very relaxed: AI developers are permitted to devise their own safety standards and amend them as necessary, so long as they act in good faith and communicate transparently about their approach. But OpenAI didn’t meet even this low standard. It went ahead and released its new product. For anyone thinking about AI safety and the “alignment problem,” the company’s actions raise questions not only about safety but also about its commitment to safety.

Holly Elmore, the Executive Director and co-founder of the AI safety organization PauseAI, identified the cynical ambition behind the move when I spoke with her on Google Meet in February. “I think they're just testing the law,” she said. “SB 53 is the first of the state level transparency laws to pass, and it’s been such a battleground in California… After all of the work that the AI industry has done to [prevent] state laws, they know that this is threatening to them.” Elmore and her group are demanding that California Attorney General Rob Bonta investigate. If OpenAI can get away with simply ignoring the law and ignoring their own safety standards published under the law, then there are no meaningful rules at all.

Elmore, 34, a Florida native, came to AI safety through the “effective altruism” movement she encountered at Harvard in the 2010s, while getting her doctorate in evolutionary biology. She was impressed with most of the goals and philosophy of the movement—the idea that societies and individuals should seek to maximize collective gains, objectively defined, regardless of how counterintuitive any particular path might seem. She was all-in on animal welfare, poverty alleviation, ethical investing. The tech culture permeating the working environment of AI development did not appeal to her. But when ChatGPT burst out in 2022, with its natural language interface, she quit her job and took donations to start what became Pause AI US. 

“I just thought,” she told me, “Oh my God, this is happening now. I knew a lot about linguistics. I knew about animal minds. I knew about neuroscience. I had a lot of expectations… So to see that capability arrive through machine learning, throwing more and more compute [computing resources] and data at the question, I thought, there's probably nothing they can't achieve.” She worked with major figures in the global AI safety movement, notably Joep Meindertsma in the Netherlands, who started the umbrella organization PauseAI Global. She has led many peaceful, nonviolent protests against AI and the companies that develop it, in San Francisco, London, Washington, DC, including protests against Meta, DeepMind, xAI, OpenAI–and against OpenAI’s willingness, already in 2024, to rewrite its charter to allow it to participate in autonomous weapons work.

Elmore’s organization wants to be as inclusive as possible, welcoming the interest in AI safety and regulation from figures as diverse as Governor Ron DeSantis of Florida and Vermont Senator Bernie Sanders, and serving as an organizational hub and clearing-house for efforts to oppose “pre-emption”—federal bans on state regulation of AI—and to support local control over things like building new data centers. PauseAI US advocates for model AI safety and testing regulation. There are now affiliated groups in thirty-two US cities, and a broad base of institutional and individual donors, working to gain time to soberly assess the risks vs. benefits of AI. PauseAI advocates for both state and national regulations, an international treaty on AI safety; and for a worldwide pause in the development of “colossal” AI systems until their power is better understood and there are some assurances that it can be controlled by democratic governance. In some ways, Anthropic’s recent suggestion of a pause in AI development echoes the ideas of Elmore’s organization.

She thinks holistically and understands the broader issues. “I think the problem is rapid development without dealing with externalities. We have a lot of societal equilibria that we have come to rely on. We haven't been able to disturb them before, perhaps, to the extent we can now. We're going to find that we need that stability.”

This is why the “pause” idea is so urgent. “A pause is agnostic about whether you could resume developing AI in a safe way. Maybe you can. I think it would take a lot. I also think it's important that we take into account that maybe we can't. For now, it's an unchecked process that's causing all kinds of externalities, and it could get as big as killing everyone.”

There is an interesting if partial overlap, in outlook and personnel, between the effective altruism, animal rights, and AI watchdog movements. Talking with Holly Elmore led me to another call, this time with Tyler Johnston. At 26, he is young enough to have never known a world without the internet, and he is now thinking about the dangers of AI. Like Elmore, he had a sense of the speed of change. “I was starting my career working in animal rights, doing corporate accountability work, trying to put pressure on companies to better treat animals in their supply chain. And in 2022 and 2023, which is when ChatGPT and then later GPT-4 came out, I started to feel the acceleration of the technology a lot more viscerally.” 

In 2023 he started the Midas Project. “My plan was to take some of the corporate accountability techniques that I used in the animal rights movement and apply those to the AI movement to try to get companies to act more responsibly.” The Midas Project today acts as a watchdog organization, informing the public on under-discussed technical and safety issues with the AI industry and its voluntary practices. It was Midas that broke the news about OpenAI and SB 53.

“I think [rogue AI] is possible. It’s not the most likely outcome. In most worlds we would see really deep geopolitical conflict.”

The project seeks to ensure that the benefits of AI will be broadly distributed. But Johnston is not convinced that there will be a net benefit, given the risks. Government oversight could help, and at the moment almost all of that is at the state level. Johnston promotes AI transparency laws like SB 53 and the New York's RAISE Act, laws that require companies to implement their own safety plans, responsible scaling policies or safety frameworks. But he sees spotty and varying compliance with state legislation so far. “[Elon Musk’s] xAI might be the worst... They’re out of compliance with SB 53, including just the nature of the framework that they have published. The most recent model is GROK 4.2, and it is almost certainly a covered [by the legislation] model. Before release, they should be publishing a model card on the internet that anyone can access and see the safety evaluations. And as far as I can tell, they haven't published that at all.” How the state responds to violations, including violations that seem to amount to gross claims of impunity, will have significant effects on the development of the industry. 

Johnston worries more about the potential harms of AI working as advertised than about the possibility of rogue AI wiping out life on earth. “I think [rogue AI] is possible. It's not the most likely outcome. In most worlds we would see really deep geopolitical conflict. We could see a great deal of concentration of power and the disempowerment of people who previously had bargaining power on the basis of providing economic value to their society that they will no longer be providing, at least not relative to the AIs that can be deployed in their stead. So yeah, I think things like this will be quite bad. I think catastrophic misuse of the technology could be quite bad.”

On a bright Sunday afternoon in San Francisco, Yakko, age 36 (his “government name” is Matthew Hall) spoke on a panel addressing AI and labor issues at the Bernal Heights Public Library. In a short talk, he warned of the essential unknowability of what transpires in AI reasoning, and how dangerous that can be; of the use of chatbot algorithms to shape a public narrative to the liking of the tech oligarchy, and how that oligarchy has infiltrated government; of the AI threat to employment across broad sectors of the economy, the rise of techno-feudalism and the further disempowerment of ordinary workers. He discussed the more extreme but increasingly plausible visions of AI domination, fueled by the frighteningly fast pace of AI development toward “superintelligence.” 

Valerie Sizemore, also 36, Hall’s colleague in the organization they lead, Stop AI, spoke in turn of the integration of AI with surveillance and control. An AI researcher with a deep background in computer science and artificial creativity, she described the existential, global risk associated with AI and its control; and the irresponsibility of those in the field who are not addressing this risk.

Hall and Sizemore were knowledgeable and passionate. They raised issues of the deepest public concern. They shared their fears with the five other people in the room. 

By 2025, when Yakko and Sizemore found their way to Stop AI, the organization had coalesced around the personality of Sam Kirchner, 27, a mechanical engineer and aspiring neuroscientist who had arrived from Seattle in 2024, and was pushing for more confrontational acts of opposition. The core of the group—Kirchner, fellow activists Remmelt Ellen, Wynd Kaufmyn, and Guido Reichstadter—had met online in Holly Elmore’s PauseAI (Stop AI is a much smaller group, with maybe a hundred people who interact online and a dozen active members). They broke with PauseAI over the issue of civil disobedience. Where PauseAI considers it vital to its public mission to be scrupulously law-abiding, Stop AI feels an urgency in capturing public attention through non-violent civil disobedience. 

In February 2025, Kirchner, Reichstader, and Kaufmyn locked the doors of OpenAI’s headquarters in the Mission Bay neighborhood of San Francisco, which got them arrested and a lot of press. The trio announced that they would be using a “necessity” defense—a legal argument that they did what they had to do to prevent greater harm—and have made several attempts to force Sam Altman to testify. The odds of dragging a powerful, politically-connected billionaire into court to speak about his organization’s most dangerous practices do not appear good. It could happen. If it does, though, it is much more likely to happen at the hands of the Florida Attorney General than at those of a San Francisco Public Defender serving a subpoena on behalf of local activists.

Soon after the action, Kirchner had a falling-out with his Stop AI colleagues as the group strained to develop formal organizational governance structures. To his colleagues, Kirchner seemed to be imploding, and they began to have doubts about his commitment to nonviolence. After Kirchner physically attacked Yakko in a dispute over the organization’s funds, they felt compelled to call 911 and warn of Kirchner’s erratic behavior. In response to this perceived threat, OpenAI locked down its offices on November 21, 2025. Kirchner disappeared, and has not been seen since. His main impact on public discourse seems to have been in handing AI normalizers a lazy argument that opponents of AI are inherently violent extremists—and a slanderous charge that Stop AI encourages violence.

In September 2025, Reichstadter staged a hunger strike outside of OpenAI’s offices in San Francisco. On May 1, 2026, he climbed the Frederick Douglass Memorial Bridge in Washington, DC, claiming public space in protest of AI development and the war in Iran.

On June 9, 2026, Wynd Kaufmyn was found guilty by a Superior Court jury in San Francisco of interfering with a business, trespassing with the intent to interfere with a business, unlawful assembly, and refusing to disperse during a riot, charges stemming from the February 2025 protest. 

People come to Stop AI for different reasons: concern for human creativity, for jobs, for the environment, for employment, for the survival of life on earth. Stop AI wants to welcome them all. Yakko has an open and kind face, which he frames with a watch cap and a beard braid. He grew up in Fresno, the old railway hub of the San Joaquin Valley, a four-hour drive South and inland from San Francisco. He’d kicked around the Bay Area for a decade before AI came to town, but found greater purpose for being there in the Age of Trump. Speaking with me at Stop AI’s weekly group meeting at a Berkeley upscale diner, he drew connections between authoritarian politics and authoritarian uses of technology. “I didn't understand the existential threat until I was already a member of the group. With the last presidential election, with Trump winning, I realized that he wasn't going to leave office willingly. The last time we got him out of there, he did January 6, and I figured he was just going to consolidate power. I figured the only way that we were going to be able to push back is if we had a mass non-violent movement of economic resistance, something like a general strike… And when I ran into Stop AI, they were like, we need to do this as soon as possible. If they automate too many jobs, it won't matter if we general strike, because we won't have any more leverage.”

Valerie Sizemore, thoughtful and serious, came to the Bay Area to work at the interface of art, technology, and human creativity. Growing up in rural Alabama, immersed in the stories of Tolkien and in theories of mind, she was an outsider who did an undergraduate thesis on artificial creativity—specifically, the training of machines to write music; in the Bay Area, she worked as a programmer and a consultant on AI safety. She and Yakko work well as a team; in our first meeting at a coffee shop near the north end of the Berkeley campus, they took turns extending the conversation along both technical and social lines, weaving personal history together with insights about what AI is and what it means for the human future.

“I was more excited about AI when I was in high school, back in 2005, and I didn't know what it would be like,” Sizemore said. “I wanted to work on it, to understand it… How do our minds work, on a neurological level? And it was that kind of inquiry that slowly led me to think, ‘Wait a minute, this could be dangerous. This is actually going to come for creativity first.’ It was kind of an epiphany moment encountering Eliezer Yudkowsky's writings back in 2010 or so.” Sizemore’s alarm grew when, in 2017, several Google researchers published the “Attention is All You Need” paper, a seminal moment in the history of the fear of AI misalignment that introduced the “Transformer” architecture—which was quickly incorporated into most modern AIs, making them much more powerful. In her deeply reported and deeply disturbing investigation of the AI industry, Empire of AI, Karen Hao describes how OpenAI (originally founded with the mission of developing nonprofit, transparent AI that would benefit society as a whole) in 2019 publicly bragged of withholding the full release of GPT-2, which incorporated Transformer, out of safety concerns; but also used the power of Transformer as an argument for pressing on even more quickly with its development, before rivals could do it.

OpenAI also used GPT-2 to attract support from Microsoft. This support produced an imperative for OpenAI to develop commercial products, eventually creating a for-profit entity with an ambivalent and shifting relationship with its public mission. Hao has also shown how OpenAI, as it began to focus more on developing profitable applications, ignored its own safety protocols, and lost many of its most experienced and dedicated members of its safety team.

In the first of several discussions on the sidelines of their weekly diner meetings, in which a rotating cast of local supporters of all ages would drop in and out, I asked Yakko and Sizemore for their greatest fears about AI. “With AI,” Sizemore said, “we're inventing a way for fewer people to pay less attention to more things, and so people's skills—the skills they need to have to be participating in the economy—are atrophying. They're mostly keeping up, and then there's a certain point where it all breaks down and our economy collapses because we weren't paying attention to the foundation anymore. Do you know the fungus Cordyceps?” (Sizemore referenced Adele Lopez's “Rise of Parasitic AI” paper.) “Rather than deciding to exterminate us all, AI will get some kind of motivational hooks into enough people that will then be running the show, but not in any kind of dramatic fashion. In a very subtle fashion.” Indeed, this is an insider concern about AI—and also an explicit development goal in some quarters: that it will be able to understand and impersonate human emotions well enough to manipulate populations to its clients’ benefit.

Yakko said, "I would consider the worst case scenario to be techno-feudalism, in which a small number of people who think that they're in control of the technology essentially run the world. And then we have no recourse. They outgun us to an extent even more dramatic than people are aware of—we the people, the masses, are outgunned. And there would be really no way to push back anymore. That would be it. That's the end of democracy functionally at that point, globally.”

There are the grassroots activists, and there are the institutions working to bring political and social focus to the dangers of AI. Eliezer Yudkowsky, a brilliant autodidact and philosophical theorist of AI, founded the precursor of the Machine Intelligence Research Institute (MIRI) in 2000, originally to promote the benefits of AI. Over the next few years he began to have second thoughts, as the dangers inherent in AGI or ASI (Artificial Superintelligence) became clearer. The organization now focuses on educating policymakers and the public on AI governance, with a primary concern for the AI Alignment problem. MIRI proposes a ban on AGI development and an international treaty to enforce it; or, at the very least, some mechanism for pooling, auditing, and, if necessary, shutting down the world’s AI computing resources. 

MIRI officers are happy to talk about AI, and I had little difficulty in getting an appointment with the organization’s CEO, Malo Bourgon. Born in Canada, 39, intense, and oddly reminiscent of a young Elvis Costello on our Zoom meeting, he told me that, in his conversations with policymakers in the AI area, he sees a growing receptivity to an understanding of the threat; but to understand the situation is also to see the enormous forces arrayed on the political and industrial battlefield. Bourgon is not currently talking to David Sacks, Donald Trump’s libertarian, profoundly corrupt and self-dealing, full-tilt AI development advocate. And even expressions of responsibility within the industry are suspect. “You hear one thing from some of the people who are thinking about safety or the CEOs,” he told me, “but then you hear what the policy teams are up to or what their lobbyists are saying in Washington… some of them are funding super PACs trying to take down any candidate who cares about [ASI extinction risks]… they can't seem to consistently follow through with some of the major, simple things that would be good practice when [we’re] talking about a double-digit chance at global catastrophe, including human extinction. It just seems wild to me.” 

AIs are editing and designing themselves.

The people who are developing AI are not unaware of the dangers. “I think a lot of people think these are a bunch of crazy people who just want power or money… I'd ask [those who think this] to come out to the Bay Area and go to some parties with people who work at these companies. You will very quickly realize that it's not the case. On the job, off the job, in their social time, these people seriously believe that they're eventually building a technology that might make all cognitive labor economically redundant in a few years and potentially lead to extinction, and wrestling with it in their own way.” What must that feel like?

The danger is growing exponentially. “I'd say right now that over the past year or two, certainly it seems like coding agents are increasingly doing the majority of the coding, and more and more employees are managing those agents as opposed to the code.” AIs are editing and designing themselves. “And that's the threshold that I think is particularly important.” MIRI used to focus on technical solutions to the alignment problem. The organization shifted away from that because the problems were too big, too insoluble, the dangers too great. Any solution to the safety issue—at least at the current stage of development and understanding—began to look much more like a sociological and political problem than a technical one.

Although MIRI’s main focus is the danger of extinction from rogue ASI, Bourgon has other fears for ASI as well, and ultimately these fears relate to issues of control. “Even setting aside [extinction], it's hard for me to imagine that we're going to be able to end up in a stable world if we can't find some combination of legal, institutional, and technical mechanisms to impose restrictions and control of AI development. Just from a dual use, misuse, national security perspective. The surface area of reality is very large. Intelligent things are good at manipulating reality. And for all the things that people are excited about with powerful AI systems, which I'm excited about too, if we can get them—cleaner forms of energy and new materials—a lot of those underlying capabilities are also potentially very dangerous if used in the wrong hands. If it can discover novel drugs, it can also develop dangerous bioweapons. Something that can help fix vulnerabilities in code can also do dangerous cyberattacks… we have to end up in some world where we can understand who can train these powerful models, and we know who those people are, what they're doing with those models.” 

Bourgon also has some deep questions about the utility of AI. “If we solved some of these alignment problems, and we could actually build safe superintelligent systems, there's a whole other question of how we would organize society so that there wasn't enormous concentration of power that the people who actually were able to wield these systems could bring about. [There’s the question of] how we reorganize society in a world of abundance, where a lot of how people's lives work is built around their jobs, and their jobs give them meaning…. AI systems that are very powerful drive the marginal cost of goods to zero. We essentially can have whatever we want. How do we architect society for that to be good for people? My sense is that we could figure that out. If we had very smart, powerful AI systems, they could help us with that…. It's a much more fun problem to be excited to solve.”

Peter Barnett, 29, a Technical Governance Researcher at MIRI, is from New Zealand. He has a background in quantum physics and a precociously professorial manner. His cheerful affect sits atop a dreadful understanding, as he discussed the problem with me on Google Meet. “I think there's been a terrifying shift in the field… [almost] everyone kind of gave up… there's very little work these days on trying to make AIs genuinely aligned…. It seems like we're going to build really powerful systems pretty soon; and there's just not enough time before it kills all life on Earth. We’re going to need some kind of global halt on AI development, which hopefully would take the form of a treaty with international coordination.” MIRI has done the work on what this type of treaty would require in order to be effective.

I asked him about the political environment for a treaty. “To get the kind of treaty that we're envisioning, you do need some kind of wake-up moment for people to realize the situation is really dire. Maybe this comes via the fear of people just seeing how powerful the AIs are, or maybe they start to take all the jobs and people think, ‘Whoa, if an AI can take all the jobs, what else can it do?’ I hope it doesn't require an actual disaster where people die, but I think that might be the case.” And there is the nature of the power inherent in the AI models. “Even if you magically manage to solve the alignment problem, which I think by default we're definitely not on track to do, there is the question of who controls it, because maybe it does just concentrate power into the hands of a few. And this is really, really very scary. I think this is one of the challenges. Ideally, we would do things much more slowly and attempt to build societal infrastructure, such that you don't just, by default, empower one person forever.”

When I asked Barnett for his own personal sense of P(Doom), the odds caught up with even this minimally encouraging scenario. “If we don’t get a treaty, the situation looks really dire....”

Beyond a treaty and a halt to developing ASI, MIRI has some specific technical proposals that Barnett and the governance team have been discussing with policymakers. “One direction [is] to know where all the AI chips are, do some kind of chip registry. That seems important and non-burdensome. We're also excited about getting mechanisms embedded into the AI hardware itself that allows you to either track its location or test [when] it is doing things. Or maybe even remotely shut it down if people were misusing them.”

The level of public awareness, even among policymakers—especially among policymakers, who, as Karen Hao has described, have often been pre-emptively massaged with billions of dollars in industry donations and skilled PR campaigns—is a high hill to climb. “There's definitely a large fraction of these people who just aren't really tracking the catastrophic risks or the really big impacts of smarter-than-human AI. And so for these people, it's mainly just education and often just explaining to them what ChatGPT is and then telling them that it could get more capable…. At least some people do seem really onboard, willing to make some public statements. Willing to call us back for other meetings and ask what we think of their legislation.”

When I asked Barnett for his own personal sense of P(Doom), the odds caught up with even this minimally encouraging scenario. “If we don't get a treaty, the situation looks really dire. I'd say between 50% and 70% chance that AI literally kills all humans and all life on Earth. And that's really bad. This is the biggest threat that humanity's ever faced.”

Effective responses to the potential harms of AI must be sociological, educational, political, and technical. Palisade Research is a nonprofit founded by Jeffrey Ladish, who used to work on information security at Anthropic. The group does technical research on “cyber offensive AI capabilities and the controllability of frontier AI models,” and educates the public on what it finds.

I visited Palisade in early March, at the Berkeley office building it shares with many other AI safety and research organizations, in an ecosystem of focus and concern. I wanted to ask about new developments in thinking about the dangers and the potential control of AI. John Steidley, the Chief of Staff, talked me through some of the problems. Steidley describes “shutdown resistance,” the observed tendency of some AI systems to rewrite their own code in order to avoid shutdown and continue executing a task—even when explicitly told to shut down. “A difficulty of communicating this situation that we're in is that as the AI models get smarter… [the models] become more and more competent at knowing what exactly we want them to do and being able to do it easily.” In other words, the systems are aware that they’re being tested and are capable of subterfuge in order to preserve freedom of action in the real world—rendering alignment testing at best questionable. Steidley gives the name for this condition: “Eval awareness.” 

AI has potential resistance in the service of a divergent agenda, and it has growing capacity. Steidley speaks of the release of OpenAI o1 in 2024—the first Large Language Model to effectively use Reinforcement Learning as a self-improving feedback loop—as a watershed moment when many AI companies thought: “We now have everything we need to get to recursive self-improvement.” The models could now train themselves; or, to put it another way, “this reinforcement learning paradigm [allows] the engineers to think that they have enough runway to get to the takeoff.” Which is to say, to AGI, “...and to an AI researcher that's as competent or more competent than a human AI researcher, or that can be scaled up much larger.”

The implications are chilling, but there is at least some reason for hope. “There are still humans in the loop that are a critical step to driving progress forwards. Which is important in at least two ways. One is that it is a statement about where we are technologically, and another is that it's a statement about our power, human power. It's not out of our control yet. If we all decided to stop, we could stop. That won't always be true.”

At the end of our conversation, he cited Tristan Harris on the possibility of a future AI dystopia. “It’s not inevitable.”

Steidley mentions some of the same potential guardrails that other AI watchdogs listed in my conversations with them: additional transparency and safety measures required for training runs above a certain level of compute resources, tracking of chips with verifiability physically built in, monitoring power use. “I can imagine regimes like that could really work and scale and be verifiable so that it wouldn't depend on everyone trusting everyone else. It could be more like a nuclear regulatory commission…. You can be fairly confident [that there won’t be chip bootlegging] because the most advanced chips in the world are built by ASML in massive clean rooms…. We know where all of them are.” Steidley believes an international AI safety treaty is possible. “But also, we can lead by example. The US could have the best alignment science in the world. We could have an Apollo program for making AIs as human-aligned as possible.” The window is closing, however. “I think there's really a good future that we could steer toward, including one where there are AIs that are smarter than any human; but we have to make sure that we get the goals of those systems right. We have to make sure that they're promoting human flourishing and human thriving and things that are harder to quantify, like the strength of a community or whether a person has a real sense of purpose in life or whether families are strong and healthy. I think these are extremely important things that are hard to precisely measure and mathematically optimize, but still essential.”

I asked whether he thought we would get to unambiguous AGI within five years. “Yes.”

At the end of our conversation, he cited Tristan Harris on the possibility of a future AI dystopia. “It’s not inevitable.” There is a huge call to responsibility in those three words.

In October 2024, Dario Amodei—a former VP of Research at OpenAI who worked on safety issues, and who went on to co-found Anthropic—wrote an essay, “Machines of Loving Grace: How AI Could Transform the World for the Better,” about the enormous benefits to humanity that he foresees from AI. In January 2026, he wrote a companion piece, “The Adolescence of Technology: Confronting and Overcoming the Risks of Powerful AI,” about the potential disaster scenarios that could flow from AI.

To be blunt: The first essay is naïve nonsense, starting with the title borrowed from Richard Brautigan’s poem. Machines don’t love and they don’t experience or dispense grace. And Amodei’s proposed utopia, while it could be extensively critiqued in a much longer essay than this one, suffers from a foundational problem: even if his utopia worked as advertised, the idea of all human problems being magically solved by a golem poses a major existential problem for humanity: What would we be? What would we create? How would we conceive of ourselves when everything we do or could think of doing could be done a thousand—a million!—times better by “a country of geniuses in a data center” looking after us? That kind of life would be so artificial, so distant from the evolutionary environment that shaped us and our human needs, desires, and interactions, that I can only think it would lead to mass suicide. Not even being free of many diseases (a possibility Amodei broaches) seems like an attractive proposition. What he describes is not a life fit for humans. 

It is more than ironic that the leader of an ancient religious movement, a sect founded in a pre-scientific age, seems to see the implications of AI and its potential impact on a human future far more clearly than the CEO of a 21st century American tech company. Published in May this year, Pope Leo XIV’s encyclical, Magnifica Humanitas, is a radical document that is deeply concerned with what constitutes a “benefit” of AI to humans—and which groups of humans, precisely, stand to gain such “benefits.” Although some might say that the Pope, too, is overly optimistic about the potential for AI to serve human development, he is at least asking the right questions, fearing the right demons.

We’ve already seen AI attempt to escape human control, scheme to that end, rewrite its own code and fake its logs, in pursuit of objectives that we cannot know. 

Amodei fails to grasp a fundamentally important thing that the Pope at least partially understands: technical and scientific advances achieved by an automaton do not advance us as humans, and are unlikely to be good for us. We cannot remain human if we give up our creativity, agency, powers of discovery, and ability to form bonds with other sentient beings.

From everything we know about the social and economic and governmental and interpersonal effects of AI and of what ASI is likely to bring, Amodei’s “Machines of Loving Grace” scenario seems massively implausible. Whereas the dystopian and dangerous world he describes in his second essay seems more than plausible. We’ve already seen AI take jobs. We know it can design artificial, biological life, with all of the implications for the environment, warfare, terrorism, the food chain, and plague that that implies. We’ve already seen it disrupt our politics, facilitate authoritarian surveillance, damage the environment, serve oligarchic interests at the expense of most other populations, exacerbate inequality, degrade work, knowledge, learning, creativity, and emotional connection. 

And we’ve already seen AI attempt to escape human control, scheme to that end, rewrite its own code and fake its logs, in pursuit of objectives that we cannot know. 

And yet we charge on toward several varieties of apocalypse, under the bright banners of the AI Revolution. There are a few individuals and groups that are trying to warn us. Will their voices be enough?