(Researched and co-written with GPT-6 Astra Max and Claude Opus 5.5, with appropriate irony.)
September 2026 has been a disorienting month to follow AI. Dario Amodei argues the industry must "pace the frontier". Former Anthropic researcher Jacob Coxon's warning that superintelligent AI "will be smart enough to kill us" has dominated the news cycle. Sam Altman says OpenAI is open to slowing down in step with its rivals, and has shelved its 2026 IPO. Going by the headlines, the industry is finally braking.
Of course, the accelerating model release cadence and financial pages say the opposite. In the last couple of weeks both GPT-6 and Claude Opus 5.5 shipped only weeks after their predecessors. Investors are pushing to get Anthropic to market at a valuation reportedly around US$2 trillion. Nvidia sits at the centre of a web of circular financing, investing in the customers that buy its chips and underwriting the data centres they lease. Chinese labs are closing on the US frontier at a fraction of the price, cutting into the margins that justify those valuations. Washington, meanwhile, is pushing the other way: Trump has dismissed safety concerns as a "hoax".
The brakes and the accelerator are being pressed at the same time, often by the same people. How does anyone make sense of that?
Applying a game theory lens
I was curious if applying a game theory lens could help to understand this more clearly. (Game theory is the field of mathematics which studies strategic decisions where each player’s best move depends on what the others do. Mathematicians John von Neumann and John Nash developed it last century, and Cold War strategists used it to reason about nuclear deterrence, which is an, er, apt precedent here.)
Its tools include understanding the players, payoffs, incomplete/asymmetric information, signalling, credible commitments, races and repeated games. Theoretically these let us put aside the question of who is sincere and ask instead what each player gains from their (assumed) rational moves.
Game theorists sort games along a few dimensions:
A game is cooperative when players can make binding agreements that someone else enforces, such as a court upholding a contract.
It is non-cooperative when no such enforcer exists, so any deal has to hold on its own, usually because breaking it would trigger a credible threat.
In a symmetric game every player faces the same choices and payoffs, so swapping the players changes nothing.
In an asymmetric game their positions differ, as a frontier lab's does from a regulator's.
In a zero-sum game the prize is fixed, so one player's gain is exactly another's loss.
In a non-zero-sum game, choices can make the total prize bigger or smaller.
The most famous non-zero-sum game is the prisoner's dilemma. Two suspects are questioned separately, and each is better off betraying the other whatever the other does. So both betray, and both end up worse off than if they had stayed silent. Rational choices by each player produce a collectively irrational outcome, and when the game is played only once, cooperation rarely survives.
Games are simultaneous when players move without knowing what the others have done.
They are sequential when earlier moves shape later ones, as in a chess game or a negotiation.
And while most textbook games offer a handful of discrete options, continuous games let players choose any amount: how much to invest, how fast to deploy, how much compute to buy.
The AI safety game sits at the difficult end of most of these. It is mostly non-cooperative, because there's no global enforcer. It is sharply asymmetric. The players treat it as zero-sum even where it isn't. And it plays out as a long sequence of moves, with players setting investment levels along a continuum.
My intuition is that looked at this way, behaviour that looks contradictory often turns out to be rational for each player. In this case, the participants include AI labs, investors, researchers, campaigners, philanthropists, governments and rival states. Here’s a breakdown list of the main players, their utility functions (payoffs sought), a rather illuminating “tacit tactical motivations” column... and the central tensions in their positions (worth considering: which player are you?):1
