AI 2040 Plan A: A Detailed Scenario for Navigating Superintelligence

How do you slow down an AI arms race without losing? That's the central question behind "AI 2040: Plan A," a detailed scenario document from Daniel Kokotajlo and the AI Futures Project that generated over 400 comments on Hacker News this week.

The proposal is ambitious: get the US and China to agree on a managed approach to AI development that delays superintelligence until 2040, gives both nations time to solve alignment, and distributes the benefits broadly. It's the kind of big-picture thinking that either reads as visionary or naive depending on your priors.

What Plan A Actually Proposes#

The document lays out a phased approach to AI governance:

Phase 1 (Now - 2029): Establish a "trustless" US-China accord built on chip tracking. Since roughly 98.5% of AI chips globally flow through a small number of design and manufacturing companies (NVIDIA designs, TSMC fabricates), both nations could theoretically track and control compute deployment without needing to trust each other.

Phase 2 (2030 - 2035): Scale AI to human-expert capability levels while maintaining safety oversight. AI systems would reach roughly "top human genius" level but not beyond.

Phase 3 (2035 - 2040): Strategic pause. Use this time for alignment research while AI capabilities are frozen at sub-superintelligent levels. The analogy is to nuclear non-proliferation - a mutual agreement that neither side builds the doomsday weapon.

Phase 4 (2040+): Deploy aligned superintelligence for governance and problem-solving. The document envisions a "citizen's dividend" funded by AI productivity, potentially reaching $1.6 million per person annually by 2035 (inflation-adjusted).

The name "Plan A" is deliberate. The document contrasts it with Plan B (aggressive China containment), Plan C (limited slowdown), Plan D (status quo racing), and Plan S (complete AI research shutdown).

What HN Is Saying#

The discussion split predictably between AI safety advocates who found the proposal thoughtful, and skeptics who considered it geopolitically naive or technically impossible.

The geopolitical skeptics dominated the thread. The core objection: why would China (or any nation) voluntarily give up a potential lead in the most transformative technology in human history?

If carbon taxes are already a lethal policy for any political campaign, it's absurd to think that fears of ASI will create any real movement around pausing AI.

One commenter drew a historical parallel that cuts both ways:

India was militarily superior to Britain in the 1600s - a gunpowder empire with a million soldiers - but was taken over by it in the 1700s. Britain's edge was small: lighter, more maneuverable cannons, standardized ammunition, better military and political organization... If we slow down on ASI voluntarily we'd be allowing a gap to open up that would make the difference between colonial Europe and colonized Asia/Africa look trivial.

This captures the core tension: unilateral slowdown risks being colonized by whoever doesn't slow down. But racing also risks catastrophe. Game theory without easy solutions.

The AI safety advocates pushed back on the fatalism:

Human cloning, human genome editing, and mirror life seem like one precedent; nuclear weapons and nuclear energy another... Plan A isn't a proposal to never build superintelligence, it's a proposal to build it more cautiously and transparently.

They pointed to the Asilomar Conference in 1975, where scientists established a voluntary moratorium on certain genetic engineering techniques until safety protocols were developed. It worked - at least for a while.

The technical half of that argument has concrete model-level examples too: why Fable 5 refuses certain cybersecurity queries shows how far refusal architecture can carry capability control, and where it starts to leak.

The economic skeptics questioned the math:

$1.6 million per person annually? The entire US GDP is about $30 trillion. That's less than $100k per person. Where does the extra 15x come from?

The Plan A document presumably models massive productivity gains from superintelligent AI, but the comment thread didn't resolve the economic assumptions.

The cynics saw regulatory capture:

Everyone can see that much of this 'safety' conversation is ultimately just a tactic to shut potential competitors out of the market and establish a monopoly/duopoly.

This is a real concern. Anthropic, OpenAI, and other frontier labs have obvious incentives to support regulations that raise barriers to entry. "Safety" arguments can serve both genuine safety goals and competitive moats simultaneously.

Historical Precedents Cut Both Ways#

The thread surfaced several historical analogies:

Japan's gun ban (1543-1879): Japan's warrior class suppressed firearms for centuries because guns threatened the samurai social order. It worked domestically - until Commodore Perry arrived with gunboats in 1853 and Japan had to rapidly modernize. Lesson: voluntary technology suppression works until it doesn't.

Nuclear non-proliferation: The Treaty on the Non-Proliferation of Nuclear Weapons has largely held for 50+ years, despite predictions it would fail. Multiple nations have voluntarily given up nuclear weapons programs (South Africa, Ukraine, Kazakhstan). Lesson: international cooperation on dangerous technology is possible.

Genetic engineering moratoriums: The 1975 Asilomar Conference and subsequent bans on germline editing held until He Jiankui's 2018 experiments in China. After He was prosecuted, China tightened its laws. Lesson: norms can work even without perfect enforcement, and violations can strengthen rather than weaken them.

Drone delivery regulation: Mentioned briefly in the thread as an example of technology being "strangled by regulations." Whether this is good or bad depends on your view of autonomous drones.

The Meta-Question#

Underneath the specific proposals, Plan A raises a meta-question: can humanity coordinate on anything this important?

Climate change suggests maybe not - decades of warnings, clear scientific consensus, and we're still struggling with basic carbon pricing. But nuclear weapons suggest maybe yes - we've avoided nuclear war for 80 years despite multiple close calls and ongoing proliferation concerns.

AI might be different from both. Unlike climate change, the incentives for individual actors align more clearly with global safety (nobody wants a misaligned superintelligence). Unlike nuclear weapons, the technology is harder to contain (you can't easily track GPU cycles the way you track uranium enrichment).

The HN discussion didn't resolve these tensions. It probably can't. But it's useful to have concrete scenarios to argue about rather than abstract doomerism or abstract optimism.

My Take#

Plan A is valuable not because it's likely to happen exactly as written, but because it forces concrete thinking about the path from here to there. Most AI safety discussion is abstract: "we need to solve alignment" or "we need to slow down." Plan A asks: how, specifically? Who agrees to what? What enforcement mechanisms exist?

The geopolitical objections are serious. China agreeing to this kind of regime seems unlikely without extraordinary circumstances. But "unlikely" isn't "impossible," and having a concrete plan ready if a window opens is better than scrambling.

For developers, the interesting parts are the technical assumptions: that compute can be tracked, that AI progress can be staged and paused at specific capability levels, that alignment research can succeed given enough time. Each of these is contestable - the same way refusal directions turn out to be a systems problem rather than a single switch you can flip.

Worth reading the full document at ai-2040.com and forming your own view. The HN thread is also worth a read for the diversity of perspectives.

Continue Reading#

• Why Fable 5 Refuses Your Cybersecurity Queries - what refusal architecture can and cannot guarantee
• Refusal Directions Are a Systems Problem - why single-mechanism thinking about model safety breaks down
• Fable 5's Hidden Guardrails - the trust gap when safety behavior is silent
• Demis Hassabis Wants a Frontier AI Standards Body. Here Is the Plan.
• Anthropic Cuts Fable 5 Biology Fallbacks by 85%: What the Safeguard Tuning Means for Developers

Sources#

• AI 2040: Plan A
• Introducing Plan A - Astral Codex Ten
• Hacker News Discussion