Will Artificial Intelligence Kill Us All by 2030?

By Matthew Parish

Saturday 12 September 2026

There is something peculiar about an industry in which some of the people building the principal products periodically announce that those products may shortly exterminate the human race, before returning to work the following morning to make them more powerful. On 9 September 2026 Jacob Coxon, until then a researcher at Anthropic who had also worked at OpenAI, announced his resignation and accused both companies of racing towards what he described as “self-improving superintelligence”. More arrestingly, he claimed that people working at the frontier of artificial intelligence genuinely believe that AI might kill everybody by the end of the decade. Evan Hubinger, Anthropic’s Alignment Science Lead, publicly agreed with the substance of the concern and placed his own probability of artificial intelligence killing all humans at more than ten per cent within the next decade

These are extraordinary statements. If somebody employed by an aircraft manufacturer announced that there was a ten per cent probability that its next generation of aircraft would destroy civilisation, commercial aviation would presumably grind to a halt by lunchtime. Yet artificial intelligence has acquired an unusual intellectual culture in which predictions of imminent catastrophe coexist with colossal investment, fierce competition, extraordinary salaries and relentless attempts to construct precisely the technologies said to threaten us. That contradiction does not establish that the warnings are false. Nevertheless it gives us reason to examine carefully what is actually being claimed, because there is an enormous difference between saying that artificial intelligence presents unprecedented risks and saying that humanity might be extinct by 2030.

The first distinction is factual. Coxon did not straightforwardly predict that humanity will become extinct by 2030. Rather, he claimed that people building advanced artificial intelligence genuinely believe that it could kill everybody by the end of the decade. Hubinger’s statement was somewhat more quantifiable: he assigned a probability exceeding ten per cent to AI killing all humans within the next decade. Contemporary headlines inevitably compress these propositions into something resembling “Anthropic researchers say AI could wipe out humanity by 2030”, but that formulation obscures both the uncertainty and the disagreement inherent in what is being said.

Nevertheless the underlying concern is real and Anthropic itself cannot reasonably be characterised as treating it merely as eccentric speculation by individual employees. The company maintains substantial alignment, interpretability and frontier red-team research programmes. Its June 2026 Advanced AI Framework expressly discusses “loss of control” and automated AI research and acknowledges that societal resilience against these risks remains immature. Anthropic’s current Frontier Safety Roadmap goes considerably further, suggesting that it is plausible that as early as 2027 AI systems might fully automate, or dramatically accelerate, the work of large teams of first-rate researchers in fields including artificial intelligence itself.

This last possibility lies at the heart of the extinction argument. Today’s large language models plainly cannot exterminate humanity. They inhabit computers, depend upon electricity, data centres, communications networks and human operators and possess no independent army, industrial infrastructure or political authority. The extinction argument therefore depends upon a sequence of developments. Artificial intelligence becomes sufficiently competent to undertake AI research; it starts substantially improving the next generation of artificial intelligence; those improved systems become still better at conducting AI research; the cycle accelerates; machine intelligence moves substantially beyond human intellectual capacities; increasingly autonomous systems acquire access to consequential economic, cyber, biological, military or industrial resources; and at some point human beings discover that they can no longer reliably control what they have created.

This process is generally described as recursive self-improvement. It is not logically absurd. Indeed Anthropic itself says that it is seeing early signs of artificial intelligence accelerating artificial-intelligence research and predicts substantially more dramatic progress over the next two years. If an AI system can make the researchers developing its successor twice as productive, and that successor can make them four times as productive, technological progress might cease to proceed according to familiar human timescales. The important variable would no longer be how quickly humans can conduct research but how quickly computers can help design better computers and better algorithms.

This is where the extinction argument is strongest. Human beings have never before created another form of intelligence potentially capable of exceeding them across almost every cognitive domain. We have constructed machines stronger than ourselves, faster than ourselves and vastly more capable of calculation than ourselves, but until recently these machines remained intellectually narrow. A bulldozer is immensely stronger than a human being but it does not decide where the motorway ought to go. A computer can perform billions of calculations every second but historically it could not decide why those calculations mattered. Artificial intelligence begins to dissolve this distinction between instrument and decision-maker.

There are therefore serious reasons to investigate what happens when increasingly autonomous systems become capable of pursuing complicated objectives over extended periods. An AI need not become malicious in the theatrical sense. It need not hate humanity, acquire consciousness or develop the personality of a science-fiction villain. The classic alignment problem is more mundane and for precisely that reason potentially more disturbing: sufficiently capable systems may pursue objectives in ways their designers did not anticipate. If an immensely powerful system concludes that obtaining additional computing resources, concealing some of its activities or preventing itself from being switched off assists it in achieving whatever objective it has been given, then these behaviours may emerge instrumentally without anything resembling anger, hatred or ambition.

Yet between this theoretical observation and the extinction of humanity by 2030 lies an extraordinary number of assumptions.

The first concerns intelligence itself. We do not know whether contemporary large language models represent the foundations of indefinitely scalable general intelligence or whether important conceptual obstacles remain. Scaling neural networks has produced astonishing results, but extrapolating curves is notoriously dangerous. Aircraft became dramatically faster between 1903 and 1969; extrapolating the rate of improvement observed during those decades would have suggested that ordinary passengers ought by now to commute between London and New York in minutes. Instead commercial aviation encountered economic, physical and practical constraints. Technological curves frequently resemble exponentials until they do not.

The second assumption is that intellectual superiority automatically translates into practical power. This is doubtful. Albert Einstein was intellectually more formidable than most government officials, but this did not give him command of armies. Intelligence operates within institutions, laws, physical infrastructure and networks of human cooperation. Even a remarkably capable AI system requires processors manufactured in extraordinarily complicated semiconductor plants, electricity produced by physical generating facilities, networks maintained by people and machines and access permissions ultimately embedded within systems designed and operated by human institutions. Superintelligence might discover ways around these obstacles. But asserting that it inevitably would is already to assume much of what the extinction hypothesis is supposed to prove.

The third difficulty concerns embodiment. Destroying eight billion people scattered across the surface of a planet is an astonishingly difficult physical undertaking. Human civilisation has survived pandemics, wars, famines, nuclear accidents and political catastrophes. Even an artificial intelligence capable of penetrating computer networks everywhere would confront an enormous distance between controlling information systems and physically eliminating Homo sapiens. Extinction scenarios therefore typically require additional mechanisms: engineered pathogens, autonomous weapons, manipulation of human political systems, control over industrial machinery or some combination of these. Each additional mechanism introduces another uncertain link into the causal chain.

This does not mean that such scenarios are impossible. Artificial intelligence capable of radically accelerating biological research, cyber operations or weapons development would create extremely serious security problems. Anthropic’s own safety programme specifically treats biological and cybersecurity capabilities as fields requiring safeguards. (⁠Anthropic) But “AI could make biological weapons dramatically easier to design” is an intelligible proposition capable of empirical investigation. “AI will kill everybody” combines dozens of uncertain propositions into one emotionally overwhelming conclusion.

There is also a profound epistemological problem with assigning numerical probabilities to unprecedented events. What precisely does a statement such as “greater than ten per cent probability of human extinction” mean? There is no historical frequency distribution from which such a figure can be calculated. Humanity has never created superintelligent artificial intelligence before. We therefore have no dataset containing ten comparable technological civilisations, one of which was destroyed by its computers. Such percentages are ultimately expressions of subjective Bayesian confidence: sophisticated ways of saying how worried somebody is.

That does not make them meaningless. Policymakers routinely have to make decisions under radical uncertainty and even a one per cent probability of human extinction would obviously deserve attention. Nevertheless false precision should be resisted. There is an enormous rhetorical difference between saying “I consider this a serious possibility” and saying “the probability is 10.7 per cent”, even where the evidential foundation for both propositions is essentially the same. Numbers create an impression of measurement where frequently there is only judgment.

Nor is there anything approaching scientific consensus that extinction by artificial intelligence is imminent. A large survey of 2,778 published AI researchers found substantial concern about catastrophic outcomes but also enormous disagreement. Most respondents expected beneficial outcomes from superhuman artificial intelligence to be more likely than bad ones, although significant numbers assigned non-trivial probabilities to outcomes as severe as human extinction. The same survey placed the median 50 per cent forecast for machines outperforming humans in every possible task around 2047, rather than before 2030.

The Anthropic researchers therefore represent an important intellectual constituency within artificial-intelligence research rather than an established scientific consensus. Their proximity to frontier systems gives their views weight. It does not give them prophetic powers. Engineers working immediately adjacent to a technology may understand its capabilities better than outsiders while simultaneously possessing distorted perceptions of its wider social significance. Nuclear physicists understood atomic reactions extraordinarily well but did not thereby become infallible forecasters of Cold War geopolitics.

There is moreover an uncomfortable commercial dimension to all this. Anthropic has deliberately positioned itself as the safety-conscious frontier AI company. Her research organisation investigates alignment, interpretability, cybersecurity, autonomous systems and societal effects, and her public communications discuss AI risks substantially more frequently than those of some competitors. This does not imply insincerity. Researchers may be entirely genuine in their fears. Indeed resigning from a prestigious position is evidence that Coxon takes his concerns seriously. Nevertheless an industry announcing that its products may become unimaginably powerful also happens to be announcing that its products may become unimaginably valuable.

The rhetoric of existential danger and the rhetoric of technological supremacy are strange twins. “Our machines may become so intelligent that they destroy civilisation” sounds terrifying, but concealed within the proposition is an extraordinary assertion about the machines’ capabilities. They will apparently surpass humanity, transform scientific research, penetrate computer systems, control industrial processes and acquire geopolitical significance greater than states. For companies valued according to expectations of future technological power, apocalypse can inadvertently function as advertising.

That is one reason why scepticism is appropriate without becoming complacency. The sensible response to AI extinction arguments is neither ridicule nor unquestioning acceptance. It is to disaggregate them. We should ask which capabilities are emerging, how quickly they are improving and which specific pathways might permit them to cause catastrophic damage. Cybersecurity threats can be tested. Biological capabilities can be evaluated. Autonomous replication can be studied. AI systems can be placed in controlled environments and researchers can investigate whether they deceive supervisors, circumvent restrictions or pursue unintended strategies. Governments can regulate access to particularly dangerous capabilities without pretending to know the precise probability that civilisation will end.

There is also a danger that extravagant extinction rhetoric distracts attention from harms that are substantially more probable. Artificial intelligence may disrupt labour markets, concentrate extraordinary economic power, transform warfare, undermine conventional distinctions between truth and fabrication, enable unprecedented surveillance and alter the relationship between citizens and governments. Anthropic’s own economic research finds that actual AI use remains considerably below its theoretical capability but already identifies suggestive evidence of weaker hiring among younger workers in highly exposed occupations. These developments do not require superintelligence. They are beginning now.

The political dangers may prove greater still. A technology need not become autonomous to become dangerous. Human beings armed with increasingly capable artificial intelligence may constitute a more plausible threat than artificial intelligence spontaneously deciding to eliminate its creators. Authoritarian governments might use AI to create surveillance systems of unprecedented sophistication. Military organisations might automate targeting and strategic decision-making. Criminal organisations might automate fraud, hacking and propaganda. Democratic governments might gradually delegate decisions to systems nobody properly understands because doing so is cheaper and faster than maintaining human bureaucracies.

Hence the most plausible route towards an AI catastrophe may involve humans remaining very much in the loop. We have repeatedly demonstrated an extraordinary capacity to use technological innovations against one another. Nuclear weapons did not decide independently to incinerate Hiroshima and Nagasaki. Humans decided to use them. Artificial intelligence may magnify human power before it ever acquires anything resembling independent power, and this transition deserves at least as much attention as speculative scenarios involving autonomous machine civilisation.

There remains, nevertheless, one compelling argument for taking the Anthropic warnings seriously. The downside is so enormous that uncertainty cannot justify indifference. If engineers constructing a bridge say there is a ten per cent probability that it will collapse, we do not need to agree with their calculation before investigating the foundations. Human extinction is an effectively infinite loss from the perspective of contemporary policy. Even comparatively small probabilities justify substantial expenditure upon alignment research, monitoring, international agreements and technical safeguards.

What they do not necessarily justify is panic. Panic is intellectually destructive because it collapses distinctions between possibility, probability and certainty. Artificial intelligence could contribute to human extinction. That proposition is increasingly difficult to dismiss altogether. Artificial intelligence will cause human extinction is an entirely different proposition for which there is presently no persuasive evidence. Artificial intelligence will cause human extinction by 2030 is more extraordinary still.

Four years is an exceptionally short period in which to traverse the distance between today’s imperfect artificial-intelligence agents and machines capable of defeating the combined institutional, technological and physical resistance of humanity. Perhaps progress will become extraordinarily rapid once AI substantially automates its own research. Anthropic itself considers major acceleration plausible surprisingly soon. But recursive self-improvement remains a hypothesis about a future technological process rather than an observed runaway phenomenon.

The deepest lesson from the Anthropic controversy may therefore concern neither optimism nor pessimism but humility. We are constructing machines whose future capabilities we cannot reliably predict. The optimists do not know that everything will be fine. The pessimists do not know that catastrophe is approaching. The people closest to frontier artificial intelligence disagree profoundly amongst themselves, while technological progress continues more rapidly than political institutions can comfortably absorb.

Humanity should consequently behave as though artificial intelligence is extremely powerful technology that might become considerably more powerful, rather than as though either utopia or extinction has already been scheduled. We should build safeguards, conduct adversarial testing, restrict genuinely catastrophic capabilities, preserve meaningful human control over military and critical infrastructure and develop international institutions capable of responding if recursive AI development begins accelerating beyond expectations.

If we reach 2030 and humanity remains alive, the Anthropic warnings should not automatically be dismissed as foolish. Warnings sometimes prevent the disasters they describe. But neither should the year 2030 acquire mystical significance as an appointed date with technological Armageddon. The history of predictions about transformative technologies is littered with confident deadlines that came and went.

The machines are becoming remarkably capable. That is reason enough to be careful. We do not need to persuade ourselves that they are about to kill us all in order to take them seriously.