Foreword

AI companies are racing to build AIs that are smarter than humans in every way. In AI 2027, we predicted that this would result in either extinction or irreversible concentration of power.The AI 2027 scenario is still roughly what we expect the future to look like: a mad scramble to superintelligence leading to either AI takeover or extreme concentration of power. So far reality is tracking closer to AI 2027 than even we expected. (2027 was our modal year at time of publication, not our median.) You can read more about our views on timelines here and here.The AI 2027 scenario is still roughly what we expect the future to look like: a mad scramble to superintelligence leading to either AI takeover or extreme concentration of power. So far reality is tracking closer to AI 2027 than even we expected. (2027 was our modal year at time of publication, not our median.) You can read more about our views on timelines here and here.

Plan A is our positive vision for what should happen instead.

In this scenario, humanity delays the development of superintelligence until 2040, makes all AI research public, allows dozens of companies globally to catch up to the frontier, and intentionally enters a regime of mutually assured compute destruction.

What is Plan A?

Plan A is our positive vision for how humanity can avoid AI-driven existential catastrophe and reach a flourishing future. It’s informed by conversations with experts at major U.S. frontier AI companies, direct experience at OpenAI, tabletop exercises, and discussions with policymakers, national security experts, and AI policy leaders. We recommend an international deal to avoid a dangerous race to superintelligence. The deal involves total research transparency for AI R&D, which allows the nations of the world to understand what’s happening and enforce guardrails. The result is multiple companies across multiple countries scaling slowly and safely together towards superintelligence, instead of racing each other in secrecy.

Plan A is primarily a recommendation, not a prediction. This scenario is not our best guess as to what the future will actually look like. Instead, it’s a vehicle for communicating and stress-testing our policy recommendations. While the implementation of Plan A is a recommendation and not what we actually expect to happen, the subsequent effects depicted are predictions.That is, they are predictions about what would happen next if our recommendations were implemented.That is, they are predictions about what would happen next if our recommendations were implemented.

In this AI 2040 scenario, Plan A is implemented successfully, albeit imperfectly and only in the nick of time.

We contrast Plan A with 4 alternative plans (B, C, D, and S), which correspond to the main ways the US could respond (or not) to the challenges of superintelligence.You can read more about these four plans at the branch point below and in our supplement “How good is each plan?”.You can read more about these four plans at the branch point below and in our supplement “How good is each plan?”.

Why did we write this?

AI companies will probably succeed at their stated goal of building smarter-than-human AI systems within the next 1 to 10 years.

The industry has convinced itself that controlling superintelligent AI can be figured out on the fly, and thus has no remotely adequate plan. We think this situation is terrible and could easily get us all killed.Of course, we’re not sure exactly how likely literal AI-driven human extinction is, but opinions within our team vary between 10% and 30%. This is primarily because there are a bunch of other things misaligned ASIs might do with us after they take over, besides kill us all. For example, they might keep some people alive because it’s really cheap and they care a tiny amount, or as acausal bargaining chips. We don’t think this nuance changes the bottom line though: Misaligned AI takeover is probably really bad.Of course, we’re not sure exactly how likely literal AI-driven human extinction is, but opinions within our team vary between 10% and 30%. This is primarily because there are a bunch of other things misaligned ASIs might do with us after they take over, besides kill us all. For example, they might keep some people alive because it’s really cheap and they care a tiny amount, or as acausal bargaining chips. We don’t think this nuance changes the bottom line though: Misaligned AI takeover is probably really bad. We do not expect whoever “wins the race” to have much of a lead, and we do not expect them to unilaterally slow down to reduce existential risk.I (Daniel Kokotajlo) recently gave a talk to about 100 people, ~40% of whom were from frontier AI companies, and did a show-of-hands poll about (a) how many months of lead the leading AI company would have at the time they first fully automate AI R&D, and (b) how much of their lead they would be willing to burn. The median answers were roughly 3 months and 1/3rd, respectively. This matches my own guess.I (Daniel Kokotajlo) recently gave a talk to about 100 people, ~40% of whom were from frontier AI companies, and did a show-of-hands poll about (a) how many months of lead the leading AI company would have at the time they first fully automate AI R&D, and (b) how much of their lead they would be willing to burn. The median answers were roughly 3 months and 1/3rd, respectively. This matches my own guess. If this race continues,Also intense secrecy and groupthink conditions! By default during this period the companies will be even more closed than they are in 2026. When they say “we’ll solve the alignment problems as we go” what they really mean is “The overworked tiny group of alignment experts at our company will solve the alignment problems as we go, without being able to subject their work to external review.”Also intense secrecy and groupthink conditions! By default during this period the companies will be even more closed than they are in 2026. When they say “we’ll solve the alignment problems as we go” what they really mean is “The overworked tiny group of alignment experts at our company will solve the alignment problems as we go, without being able to subject their work to external review.” we do not expect humans to maintain effective control as their AIs become superintelligent.We expect them to maintain apparent control, to be clear, at least for some time, which is part of why this problem is so dire. The situation would be much better if we had a reliable way of telling whether an AI system was actually robustly obedient/loyal/aligned/etc. vs. merely temporarily so vs. merely pretending.We expect them to maintain apparent control, to be clear, at least for some time, which is part of why this problem is so dire. The situation would be much better if we had a reliable way of telling whether an AI system was actually robustly obedient/loyal/aligned/etc. vs. merely temporarily so vs. merely pretending.

Moreover, even if the AI companies somehow align their AIs, the result will be an unprecedented concentration of power—that is, the result will be a situation where a tiny group of people, or possibly just a single individual, is effectively in control of the world’s only army of superintelligences for some months, and will be presented by said superintelligences with various options for how to proceed, some of which will de facto amount to taking over the world.The AI 2027 Slowdown ending depicts this happening. It’s quite easy to imagine because it doesn’t involve anything cartoonishly or overtly evil, but rather just a series of steps that can be justified individually (consolidating US compute into a single pool, beating China, integrating AI into the government and economy, preventing terrorists from getting their own AGIs, etc.)The AI 2027 Slowdown ending depicts this happening. It’s quite easy to imagine because it doesn’t involve anything cartoonishly or overtly evil, but rather just a series of steps that can be justified individually (consolidating US compute into a single pool, beating China, integrating AI into the government and economy, preventing terrorists from getting their own AGIs, etc.)

As best as we can guess, the CEOs of OpenAI, Anthropic, xAI, and Google DeepMind understand this and are proceeding anyway, perhaps because they think they are the lesser evil and will use their immense power responsibly, unlike Xi Jinping or rival CEOs.For example, these old OpenAI emails reveal that OpenAI was founded in significant part because of fear that Demis Hassabis, DeepMind CEO, would use AGI to become dictator. See also related reporting in Empire of AI. See also this and this and this.For example, these old OpenAI emails reveal that OpenAI was founded in significant part because of fear that Demis Hassabis, DeepMind CEO, would use AGI to become dictator. See also related reporting in Empire of AI. See also this and this and this.

While we agree that it is generally correct to choose the lesser evil, we don’t think we should advocate for a strategy that has such a scarily high chance of leading to human extinction or global dictatorship. Instead, we wish to advocate for something that is actually good. If enough people do likewise, it can happen.

So, we wrote a scenario outlining that possible world.

Why a scenario?

“Plans are worthless, but planning is everything.” - Dwight D. Eisenhower

We think most AI policy proposals fall apart under scenario scrutiny—that is, if you try to write down a detailed and plausible scenario in which that proposal succeeds, you will find it difficult to do so, and you will realize the plan is less likely to work than it seemed, or has more unpleasant side-effects than its proponents acknowledged.

Perhaps that’s why scenario scrutiny is so rare in AI policy. Everyone wants to say that their own favorite policies will have great consequences and that the policies of their rivals will have terrible consequences. Applying scenario scrutiny to their own favorite policies might surface uncomfortable issues with them; meanwhile, applying scenario scrutiny to their rival’s policies is a lot of work for little rhetorical gain.For one thing, it’s hard to apply scenario scrutiny to a policy proposal if you have only a shallow understanding of it, and people generally understand their own favorite ideas much better than they understand the ideas they hate. For another, the evidential force of scenario scrutiny is inherently stronger for scenarios written by proponents than by opponents: If a proponent of a policy tries their best to depict it succeeding, and fails, that’s a lot more evidence than if an opponent of a policy depicts the policy failing.For one thing, it’s hard to apply scenario scrutiny to a policy proposal if you have only a shallow understanding of it, and people generally understand their own favorite ideas much better than they understand the ideas they hate. For another, the evidential force of scenario scrutiny is inherently stronger for scenarios written by proponents than by opponents: If a proponent of a policy tries their best to depict it succeeding, and fails, that’s a lot more evidence than if an opponent of a policy depicts the policy failing.

We think the discourse would be improved if more AI policy proposals were subjected to scenario scrutiny. So we’re starting with our own, even though this opens us up to criticism. We hope critics will judge us against the existing state-of-the-art for plans to navigate the AI transition (if they can find any) and not against some hazy but pleasant fantasy where no one has to make any hard choices yet everything will probably be fine.

What of the immense difficulty of predicting the effect our policy would have in a world approaching superhuman AIs? This is like trying to predict how to best fight World War 3, except that it’s an even larger departure from past case-studies. Yet it is still valuable to attempt, just as it is valuable for the U.S. military to game out Taiwan scenarios in excruciating detail. There are other precedents as well: intelligence agencies, climate bodies, and pandemic-preparedness offices all rely on various kinds of scenario planning.

AI timelines?

Plan A is our ambitious proposal for what to do, and we’d like to see something like it implemented soon because we are uncertain about how much time remains.That is, we are uncertain about how fast AI capabilities will progress – about how much time remains before the AI companies succeed at automating their research and accelerating towards superintelligence, for example.That is, we are uncertain about how fast AI capabilities will progress – about how much time remains before the AI companies succeed at automating their research and accelerating towards superintelligence, for example. But for purposes of writing a concrete scenario, we need a concrete timeline.

The timeline of this scenario is:

• In 2029, the US and China agree to avoid a reckless race to superintelligence.

• In 2030, we would have fully automated AI R&D, leading to superintelligence by the end of the year. Thanks to the deal, we avoid this.

• Between 2030 and 2035, we scale within the human range, to AIs that are roughly as capable as top human experts.

• In 2035, we pause at top-human-expert level AI in order to maintain human control.

• In 2040, we unpause and scale to superintelligence.Superintelligence means AI systems that are significantly better than the best humans at everything, while also being faster and cheaper.Superintelligence means AI systems that are significantly better than the best humans at everything, while also being faster and cheaper. (Hence the title: AI 2040)

In our previous scenario, AI 2027, AI fully automated the process of building smarter AIs in 2027, leading to an intelligence explosion and superintelligence within the year. The two differences in this scenario are (1) the default timeline is now 2030, and (2) thanks to governance actions, generally-superhuman AIs first appear in 2040.

We changed the default timeline because we want our portfolio of scenarios to reflect our uncertainty about AI timelines. AI 2027’s titular year was chosen because, at the time we started writing, Daniel thought there was roughly a 50% chance that things would go that fast or faster.By the time we finished writing AI 2027, Daniel’s AGI median was 2028. Other authors’ medians were between 2031 and 2035. To read more about how Daniel and Eli’s views have shifted over time, see here. We are regularly updating our forecasts here, with corresponding blog posts on our Substack.By the time we finished writing AI 2027, Daniel’s AGI median was 2028. Other authors’ medians were between 2031 and 2035. To read more about how Daniel and Eli’s views have shifted over time, see here. We are regularly updating our forecasts here, with corresponding blog posts on our Substack. At the time we started writing Plan A, 2030 was the corresponding year for Thomas. Daniel currently thinks things will probably go somewhat faster than depicted in this scenario; you can read more about our team’s views on timelines here and here.

We changed the governance actions because this scenario is primarily a recommendation, not a prediction. Conducting a full-speed intelligence explosion is wildly reckless and concentrates power to an extreme degree.

2027: The Writing on the Wall

America has two workforces now. The first is people, 165 million of them. The second is AI agents: millions of copies spun up and shut down every hour, working around the clock at superhuman speeds.

Most of their work is slop. But enough of it is good that people are paying ten billion dollars a month for AIs that can, in theory at least, do anything on a computer that an employee can.

There is one job the AI companies want to automate more than any other—their own. They haven’t succeeded yet; no recursive self-improvement so far.AIs are helping to make future AIs more capable, but AI development still requires humans.AIs are helping to make future AIs more capable, but AI development still requires humans. But they seem to be getting closer, and they’re pulling up the ladder behind them: the strongest coding AIs refuse to help competitors with AI R&D.This policy is selfishly good for leading AI companies insofar as it makes it harder for trailing companies to catch up, but it’s also good for safety because it slows irreversible proliferation of algorithmic insights.This policy is selfishly good for leading AI companies insofar as it makes it harder for trailing companies to catch up, but it’s also good for safety because it slows irreversible proliferation of algorithmic insights. Even as the most bullish employees admit that things are taking a bit longer than planned, the skeptics notice that their usual dismissals are starting to ring hollow. Why exactly will AI never be able to do my job? What’s the barrier again?

Congress is starting to pay more attention. They’ve long been hearing about AI: datacenters using too much water,We think the concerns around current AI water usage are largely overstated. See this article for more. That said, as will be apparent later in the scenario, we do think that the environmental impact of AI over the next decade or two will be enormous, and that unrestrained growth would literally boil the oceans. See the energy appendix in our economics supplement for more on this.We think the concerns around current AI water usage are largely overstated. See this article for more. That said, as will be apparent later in the scenario, we do think that the environmental impact of AI over the next decade or two will be enormous, and that unrestrained growth would literally boil the oceans. See the energy appendix in our economics supplement for more on this. chatbots encouraging suicide, Mythos hacking NSA systems—and of course, tech industry lobbyists warning that any whiff of regulation will make America immediately lose the race with China and spend the rest of history as a CCP tributary state.Are we really in such a race? Sort of, but not exactly. On our default trajectory, AI will soon become very powerful, to the point where it’s more strategically important than nuclear weapons; that much is true. Many people, including the CEOs of frontier AI companies, are trying hard to build smarter and smarter AI systems before their competitors do. However, being overly paranoid about competitors is a well-known bias. Assuming that we’re in a winner-takes-all race rules out many possible avenues for cooperation, creating a self-fulfilling trap. We expect that the scale and speed of AI progress will make it increasingly obvious to decision-makers in both the US and China that the current pace of progress, and lack of trust between them, is extremely dangerous. Indeed, it may be possible for the two countries to coordinate on a slowdown without any binding agreement, if they build trust by each gradually deescalating. Our proposed Plan A should be interpreted as an approach that could work even if the US and China have absolutely zero trust for each other, rather than a claim that strict verification protocols are definitely necessary.Are we really in such a race? Sort of, but not exactly. On our default trajectory, AI will soon become very powerful, to the point where it’s more strategically important than nuclear weapons; that much is true. Many people, including the CEOs of frontier AI companies, are trying hard to build smarter and smarter AI systems before their competitors do. However, being overly paranoid about competitors is a well-known bias. Assuming that we’re in a winner-takes-all race rules out many possible avenues for cooperation, creating a self-fulfilling trap. We expect that the scale and speed of AI progress will make it increasingly obvious to decision-makers in both the US and China that the current pace of progress, and lack of trust between them, is extremely dangerous. Indeed, it may be possible for the two countries to coordinate on a slowdown without any binding agreement, if they build trust by each gradually deescalating. Our proposed Plan A should be interpreted as an approach that could work even if the US and China have absolutely zero trust for each other, rather than a claim that strict verification protocols are definitely necessary.

Now they step back and ask: Where are we going with this? What does the world look like five, ten, or fifteen years from now? Will there still be jobs? What if there aren’t?

One question weighs especially heavily on their minds: Who will control all these AIs?

Congress settles on an important part of the answer: Probably not us.That is, not Congress. By default the people who control the AIs, if anyone truly does, will be AI companies or maybe the White House.That is, not Congress. By default the people who control the AIs, if anyone truly does, will be AI companies or maybe the White House.

They hold a series of tense hearings on AI. They read the 2016 OpenAI emails discussing how OpenAI was founded in order to prevent Demis Hassabis from becoming dictator.From Ilya and Greg: “The goal of OpenAI is to make the future good and to avoid an AGI dictatorship. You are concerned that Demis could create an AGI dictatorship. So do we. So it is a bad idea to create a structure where you could become a dictator if you chose to, especially given that we can create some other structure that avoids this possibility.”From Ilya and Greg: “The goal of OpenAI is to make the future good and to avoid an AGI dictatorship. You are concerned that Demis could create an AGI dictatorship. So do we. So it is a bad idea to create a structure where you could become a dictator if you chose to, especially given that we can create some other structure that avoids this possibility.” But who is preventing Sam or Elon from becoming dictator? Congress is unsatisfied with existing responses.

The result of this wakeup is the AI Transparency Act of 2027, an omnibus bill that does many things, some good and some bad, but doesn’t fundamentally change the situation.We don’t mean to recommend bad things; we are predicting that a mixture of good and bad incrementalist reforms will happen in 2027. For more on how we balance predictions and recommendations in this scenario, see here.We don’t mean to recommend bad things; we are predicting that a mixture of good and bad incrementalist reforms will happen in 2027. For more on how we balance predictions and recommendations in this scenario, see here.

2028: AI on the Ballot

The 2028 election cycle is heated, as usual. AI is the biggest topic. The datacenters now under construction cost twice as much as the entire US military budget.That is, the AI industry spends $2.4T of CapEx over 2028, triple what was spent in 2026. The US military budget in 2025 was about $1T. The biggest AI company has $360B ARR at the beginning of 2028 and is still growing at a roughly 150% annualized growth rate, putting it on track to become both the most valuable company in the world and the largest company in the world by revenue, in 1-2 years.That is, the AI industry spends $2.4T of CapEx over 2028, triple what was spent in 2026. The US military budget in 2025 was about $1T. The biggest AI company has $360B ARR at the beginning of 2028 and is still growing at a roughly 150% annualized growth rate, putting it on track to become both the most valuable company in the world and the largest company in the world by revenue, in 1-2 years.

Most white-collar professions are seeing disruption like software engineering saw in 2026; such jobs now heavily involve managing AI agents. AI companies have industrialized the training process: Executives say “let’s move into [profession] this year” and then the company interviews professionals, buys data, creates training environments, etc. until their AIs get traction. Then the AIs rapidly improve as they are used more widely in the field and accumulate more real-world data.

Other countries are starting to get scared and angry. It seems like a handful of US and Chinese companies are on track to automate all the white-collar jobs. Power is concentrating in the US, and in particular in the President plus a handful of tech CEOs.

AI experts warn that the intelligence explosion is near. By speeding up AI research, the AIs will become even more competent, speeding up research even faster, making them even more competent, and so on. There are complicated dynamics about bottlenecks and hardware limits governing how fast this process goes and where it ends, but it seems like it might go very fast and end somewhere very far away.

On the default path, the next presidential term will see AIs that are far beyond human level, created entirely by AIs, themselves created entirely by other AIs, without any human in the loop since several generations back. Will those AIs be obedient, aligned, etc.? Why? Who will control them if so? How exactly is all of this supposed to end well?

Having put humanity on this path, the AI companies find it acceptable. But most people don’t. Forget thinking about his legacy—the President is starting to think about what’ll happen to him after he leaves office and the world gets transformed.After all, he’ll still be alive after leaving office. Just like the rest of us, he’ll have to endure whatever crises result from superintelligence, and hope that the new President handles them well and doesn’t become dictator. It’s in his interest both to set the new President up for success with respect to the loss-of-control problem, and to install checks and balances to prevent extreme concentration of power.After all, he’ll still be alive after leaving office. Just like the rest of us, he’ll have to endure whatever crises result from superintelligence, and hope that the new President handles them well and doesn’t become dictator. It’s in his interest both to set the new President up for success with respect to the loss-of-control problem, and to install checks and balances to prevent extreme concentration of power. Both presidential candidates keep getting asked what they’ll do about AI, and try out increasingly dramatic ideas on the campaign trail. The discourse bounces back and forth across all of the options displayed below, and more.

Eventually the President and his protégé converge on one plan; the opposition candidate converges on another. Then it’s Election Day.

2029: Choose a Path

“Trust, but verify” – Ronald Reagan

The President announces that the US will pursue international cooperation to avoid an imminent intelligence explosion.

“This mad race toward superintelligence must end. For too long, we have been pursuing lesser-evils and least-bad solutions. We need a Plan A. We can still proceed with AI development, but we must do it more cautiously, more transparently, and involving many more countries and companies.”

To the surprise of many in DC, China proves receptive. They had been debating the same issues—social destabilization, job loss, rogue superintelligence - on their side of the Pacific. They had been looking forward to “The Chinese Century” and thought that AI might disrupt their plans. And they had one extra reason to come to the table: the US continues to have more compute, more datacenters, and better models. They were worried about the things the US might do to them if it got to superintelligence first.To spell it out more: They were concerned that the US might use their massive AI advantage to cripple Chinese AI projects via cyberattacks, physical sabotage, or other means, and then use the resulting even-bigger, longer-lasting AI advantage to dictate terms to the CCP or even overthrow it entirely. Preventing loss-of-control risk was merely an added bonus.To spell it out more: They were concerned that the US might use their massive AI advantage to cripple Chinese AI projects via cyberattacks, physical sabotage, or other means, and then use the resulting even-bigger, longer-lasting AI advantage to dictate terms to the CCP or even overthrow it entirely. Preventing loss-of-control risk was merely an added bonus.

The US and China don’t trust each other. Fortunately, they don’t have to: Plan A includes provisions for verifying compliance. But setting it up will take time.

So for now, they start with something crude. For the rest of 2029, they put a temporary halt to AI training, because that’s relatively easy to verify. In 2030, they’ll have the infrastructure in place to proceed with Plan A.

The stock market is gyrating wildly up and down in response to news and commentary about the momentous actions being taken. Everybody is screaming about how it’s going too far or not far enough. The market will calm down by the end of the year, but the screaming will continue.Geopolitically the situation will have stabilized within a few years, but by that point AI progress and diffusion will have happened and caused new problems and crises for people to worry about.Geopolitically the situation will have stabilized within a few years, but by that point AI progress and diffusion will have happened and caused new problems and crises for people to worry about.

Step 1: Compute Declaration

Neither country wants to halt their own AI training unless they can see that the other side is halting too.

Fortunately, training AIs requires large numbers of AI chips. Most AI chips are in giant datacenters.We show our estimate of the distribution of datacenter sizes in our compute supplement.We show our estimate of the distribution of datacenter sizes in our compute supplement. AI datacenters are typically big enough to be visible from space, and power-hungry enough to require conspicuous infrastructure. New AI chips can only be manufactured at a handful of fabrication plants (fabs), located mostly in Taiwan, South Korea, the US, and China.

The US and China negotiate with the countries that have a major role in the chip supply chain, and they require each major datacenter owner (and their upstream suppliers, including chip fabs) to publicly declare their major purchases and sales.The initial chip declaration would only involve companies whose purchases exceed 10k H100e (which costs ~$100M). Within a year, every company owning above 0.001% of the world’s compute (~2,800 H100e, costing ~$18M at the time) has its holdings accounted for. Dead chips are also stored or verifiably destroyed.The initial chip declaration would only involve companies whose purchases exceed 10k H100e (which costs ~$100M). Within a year, every company owning above 0.001% of the world’s compute (~2,800 H100e, costing ~$18M at the time) has its holdings accounted for. Dead chips are also stored or verifiably destroyed. Analysts from numerous countries and institutions may pore over the list, ask questions, and challenge anomalies. The rival powers also send inspectors to each other’s infrastructure, reassuring themselves that the stated numbers are accurate. By the end of the year, each side is confident that the other isn’t hiding more than about 1% of AI compute, and that all potential sources of new compute are monitored and accounted for.The 1% of compute sales that cannot be traced to their end location are mostly due to smugglers who did not record their customers’ identities and chips that were decommissioned. Half of this pool is later found installed in Chinese datacenters identified via satellite imagery; both countries offer monetary incentives and legal amnesty to any owners of the remaining 0.5%. We go over these and other routes for finding covert compute in much more detail in our covert project supplement.The 1% of compute sales that cannot be traced to their end location are mostly due to smugglers who did not record their customers’ identities and chips that were decommissioned. Half of this pool is later found installed in Chinese datacenters identified via satellite imagery; both countries offer monetary incentives and legal amnesty to any owners of the remaining 0.5%. We go over these and other routes for finding covert compute in much more detail in our covert project supplement.

Step 2: Pause Training

Distinguishing dangerous AIs from safe ones will require more time and understanding, so for now they go with a simple solution: a temporary pause on all new training runs.This includes the kind of research experiments that involve training. While it’s theoretically possible for an intelligence explosion to happen without new training runs or major experiments, it seems unlikely because it would require an extreme paradigm shift.This includes the kind of research experiments that involve training. While it’s theoretically possible for an intelligence explosion to happen without new training runs or major experiments, it seems unlikely because it would require an extreme paradigm shift.

Both sides can still use datacenters for running AIs that already exist (i.e. inference), but they will retrofit each other’s datacenters with devices to verify that they aren’t being used for new training runs.The inference-only verification solution we propose works by taking a random sample of a small percentage of the workloads to check they are correctly doing inference on an approved model (by recomputing the outputs using the inputs and approved model weights). To collect these random samples, our proposal involves simple network taps that redirect all inbound and outbound traffic from the datacenter to a recomputation server (inside the datacenter) installed by the other party that performs the random partial recomputation. Other solutions to this inference-only proposal may be possible and preferable, such as cryptographic proof of work schemes, but these are currently more speculative from a performance perspective. We discuss this in more detail in our verification supplement. There are several additional layers of enforcement necessary to make this approach work, including physical security, software based monitoring, and human auditors.The inference-only verification solution we propose works by taking a random sample of a small percentage of the workloads to check they are correctly doing inference on an approved model (by recomputing the outputs using the inputs and approved model weights). To collect these random samples, our proposal involves simple network taps that redirect all inbound and outbound traffic from the datacenter to a recomputation server (inside the datacenter) installed by the other party that performs the random partial recomputation. Other solutions to this inference-only proposal may be possible and preferable, such as cryptographic proof of work schemes, but these are currently more speculative from a performance perspective. We discuss this in more detail in our verification supplement. There are several additional layers of enforcement necessary to make this approach work, including physical security, software based monitoring, and human auditors.

The US and China are able to rapidly source enough of these verification devices to retrofit almost every major datacenter.This is made easier thanks to a now-thriving ecosystem of verification hardware companies that emerged after both governments signaled interest in having verification capability as an option (which in turn drew in investment from philanthropists and VCs and technical support from AI supply chain companies). Had that not happened, this part of Plan A would have taken somewhat longer and/or been more expensive and frantic. Worst case, they’d have to turn off the datacenters instead of allowing them to run inference.This is made easier thanks to a now-thriving ecosystem of verification hardware companies that emerged after both governments signaled interest in having verification capability as an option (which in turn drew in investment from philanthropists and VCs and technical support from AI supply chain companies). Had that not happened, this part of Plan A would have taken somewhat longer and/or been more expensive and frantic. Worst case, they’d have to turn off the datacenters instead of allowing them to run inference.

Inspectors from both countries sign off on the installation of the verification devices. Only datacenters that the US and China agree have functioning verification are allowed to provide continuing AI services. Meanwhile, both governments sprint towards more fine-grained verification options that could allow training to restart while keeping both sides confident the other isn’t racing ahead.

Step 3: Get Worldwide Buy-in

Though other countries were consulted from the start, especially those owning parts of the AI supply chain, Plan A began as a bilateral deal between the US and China. Now that it’s taking firmer shape, negotiations go multilateral. Carrots and sticks are waved about.

Many countries outside the US and China are happy about the deal, because they were worried about a future in which a handful of US and Chinese AI companies recursively self-improve, pull farther and farther ahead, and then… Well, what happens next depends on who you ask, but answers range over possibilities such as “take our jobs,” “cement hegemony forever” and “get everyone killed.” So they want the US and China to proceed more slowly and transparently. This will allay their fears and also allow their own AI projects to catch up to the frontier.

So it’s surprisingly easy for the US and China to get buy-in.This part is a prediction, not just a recommendation. We think that getting buy-in for Plan A, in circumstances like those we’ve described in 2029, would be significantly easier than most of our readers would expect.This part is a prediction, not just a recommendation. We think that getting buy-in for Plan A, in circumstances like those we’ve described in 2029, would be significantly easier than most of our readers would expect. By the end of the year, most of the world has joined what is now called the Consortium.

2030: Plan A is Established

Tech companies are pushing hard to restart AI training, and lobbying to shape the conditions under which it happens. Large portions of the public are saying that we should never restart. Some countries are too.

After a year of intense discussions, negotiations, and twitter beefs, a compromise takes shape: AI development will continue, but in a more cautious, more transparent, more distributed way.

Plan A is guided by 4 core principles:

Principle 1: Buy Time

The problem with an intelligence explosion is the “explosion” part.

The default trajectory is reckless and destabilizing. Each country and company is racing their competitors for AI dominance without time to think. Most likely, a ton of people will end up disempowered or killed. Maybe godlike AIs run circles around us and the age of humanity is over. Maybe a CEO or President establishes a permanent dictatorship. Maybe World War Three begins in the chaos, as countries without the best AIs fear losing their leverage. Maybe everything is surprisingly smooth and we “only” automate all labor and render everyone unemployable in a radically transformed world. Even aspiring AI-enabled dictators are worried; there’s enough frontrunners that none of them have more than 50% odds of winning the power struggle.

No one knows how to tell if AIs are trustworthy, and no one knows how to regulate superintelligent AI. Solving those problems looks like it’ll take years. Slowing down the intelligence explosion buys time to do so.

It also helps prevent extreme concentrations of power, because it gives more time for groups that don’t directly control the world’s smartest AIs to wake up to what’s happening before they lose their leverage. Without a major slowdown, a rapid intelligence explosion will concentrate almost all the power into a tiny group of CEOs and politicians.

Principle 2: Total Research Transparency

So the goal is to develop AI at a reasonable pace: Not too fast, not too slow. Ban the unsafe kinds, allow the safe kinds. But each country still regulates its own AI industry; there isn’t a central planner with authority to regulate AI worldwide. So how do they decide what to allow and disallow, and how do they verify that no one is cutting corners?

The Consortium countries come up with a simple high-level framework: we’ll agree to let each other see all the AI research. Then, if we don’t like something someone is doing, we’ll talk about it and perhaps agree to ban it.

Transparency offers many benefits. First, when each company can see what the others are doing, the expertise of governments is less load-bearing. If a company starts doing something dangerous, rival companies, third-party auditors, etc., can notice and raise the alarm. This both massively increases the amount of brainpower devoted to making the frontier AIs safe, and also prevents a situation where the only participants in the technical conversation are biased AI company employees and overworked, outnumbered regulators. Also, increased visibility simplifies the problem of verifying compliance with regulations, especially in grey-area cases.

Second, total research transparency makes it nearly impossible for secret loyalties, biases, or agendas to be intentionally trained into AIs. Everyone can see exactly how each frontier AI is trained. More generally the transparency makes it easier for groups that don’t directly control frontier AIs to notice and prevent abuses of power by those who do.

Third, there’s no longer as much incentive to race to discover new AI paradigms and more powerful algorithms, because companies wouldn’t be able to hoard such discoveries and profit greatly from them. This buys the world time.

Principle 3: Diffuse AI Broadly

This principle is largely achieved by the interaction of the previous two:That said, it’s important in our view that broad deployment of AI doesn’t get regulated out of existence. The limits on algorithmic progress and the transparency would, by default, result in other companies catching up and a scenario of broad deployment—but countries could decide to block such deployment or ban such companies from being created, etc. We are saying they shouldn’t do that.That said, it’s important in our view that broad deployment of AI doesn’t get regulated out of existence. The limits on algorithmic progress and the transparency would, by default, result in other companies catching up and a scenario of broad deployment—but countries could decide to block such deployment or ban such companies from being created, etc. We are saying they shouldn’t do that. Because algorithmic secrets are made public and the pace of progress isn’t accelerating, other companies will catch up to the frontier. The result will be a competitive market for AI, in which consumers of AI services have many options to choose from and excellent visibility into what they are buying. Because frontier model training is totally transparent, people can verify that the published Spec accurately describes the goals and values being trained into a model.

This is very important for preventing extreme concentrations of power. But it also helps accelerate the use of AI to solve the world’s most pressing problems. It helps with loss of control risks by accelerating AI alignment and control research. It helps accelerate the development of new and improved verification technology, and it helps to accelerate the necessary institutional reforms to stabilize the deal and prepare for superhuman AI.

It’s the polar opposite of the nightmare scenario feared in the 2020s: One to three AI projects racing each other in secrecy, keeping their best models internal-only and using them to automate AI research before using them for anything else. In that scenario, the wider world is in the dark about how powerful AIs are becoming, and AIs are deployed in the riskiest domain (AI R&D) before they are deployed anywhere else. Now it’s the other way around.

Principle 4: Reversibility

In the past, companies have trained bigger and better AIs using both compute scaling (bigger training runs) and software progress (advances in AI algorithms—new paradigms, better training recipes, better data, etc.). Now, the Consortium tries to steer things so that the majority of improvement comes from increasing training compute.Specifically, capabilities progress in 2030 is .8 OOMs of SW progress and .6 OOMs of HW because of the one time gain from the total research transparency sharing everyone’s algorithmic advances. After that SW progress is .4 OOMs/yr until 2035, and HW progress is .34 OOMs in 2032, .57 in 2033, .68 in 2034, and .71 in 2035. View the full numbers here.Specifically, capabilities progress in 2030 is .8 OOMs of SW progress and .6 OOMs of HW because of the one time gain from the total research transparency sharing everyone’s algorithmic advances. After that SW progress is .4 OOMs/yr until 2035, and HW progress is .34 OOMs in 2032, .57 in 2033, .68 in 2034, and .71 in 2035. View the full numbers here.

Algorithms are information; it’s inherently difficult to stop them from proliferating, and the total research transparency means we aren’t even trying.There are a few nuances here. Some algorithmic progress is easily communicated (e.g., new architectures, better optimization algorithms), while other types cannot easily diffuse (e.g., huge libraries of RL environments, hardware-software codesign, scale or compute dependent algorithms). Regulations that the US and China agree to should steer companies towards making the latter type of algorithm progress when possible.There are a few nuances here. Some algorithmic progress is easily communicated (e.g., new architectures, better optimization algorithms), while other types cannot easily diffuse (e.g., huge libraries of RL environments, hardware-software codesign, scale or compute dependent algorithms). Regulations that the US and China agree to should steer companies towards making the latter type of algorithm progress when possible. Once a new paradigm is discovered, it’ll go straight to the hypothetical covert projects and there’s no way to undo that. By contrast, if giant new datacenters are constructed to train giant new AI models, that helps the legal projects without helping the covert projects, and if it turns out the giant new models are dangerous they can be shut down.Another reason to prefer AI progress to come from compute instead of from algorithms is that compute scale-ups may be less likely to break safety techniques than algorithmic changes.Another reason to prefer AI progress to come from compute instead of from algorithms is that compute scale-ups may be less likely to break safety techniques than algorithmic changes.

That said, compute scaling also poses threats: if the deal were to dissolve, these datacenters could be used to race to superintelligence even faster than the pre-deal infrastructure would have allowed. This is especially scary because deal dissolution could cause (or be caused by) a US-China war. A war in which both sides had vast amounts of compute would be terrifying—they’d probably be under lots of pressure to race to superintelligence and integrate it into the military as aggressively as possible, and they’d be able to do so extremely quickly.

The US and China agree on the importance of ensuring that the compute is destroyed in the case of deal dissolution. To accomplish this, they agree to build the datacenters in the third-party countries least secure against their rival’s military intervention: China’s new datacenters will be in Canada, and America’s in Mongolia, with both hosts compensated via monetary payments, jobs, and a share in the new AI economy. If the deal dissolves, they reason, America and Canada will immediately move to take control of China’s datacenters in Canada, and China will self-destruct their compute rather than let it fall into American hands (and vice versa). Thus the idea of “Mutually Assured Compute Destruction” is born.This idea goes back to Superintelligence Strategy by Hendrycks, Schmidt, and Wang.This idea goes back to Superintelligence Strategy by Hendrycks, Schmidt, and Wang.

2031: Safety Cases

Although it’s supposed to be a slowdown, it doesn’t feel like one. In fact, if you were to rank every period of human history by how much it felt like a slowdown, this one would be dead last. A few AI researchers appreciate that progress is “slow” relative to the counterfactual with no deal and an uncontrolled intelligence explosion. Everyone else is too whiplashed to care.

The first generation of Consortium-regulated AIs are out now, and they’re beasts. Not because of any special feature of the situation, like Chinese-American cooperation.US/China cooperation is speeding up progress in some ways (e.g., the transparency helps information flow between research groups) but these effects are massively outweighed by the slowing down capabilities progress; which would have been at ASI by now absent measures to slowdown.US/China cooperation is speeding up progress in some ways (e.g., the transparency helps information flow between research groups) but these effects are massively outweighed by the slowing down capabilities progress; which would have been at ASI by now absent measures to slowdown. Just because the world was on track for vastly superhuman AIs, and has now experienced only nearly superhuman AIs. In controlled tests, these new AIs could speed up AI research by about 10x if they were allowed to do so without restrictions, which they definitely aren’t.Specifically, this 10x speedup number refers to the same thing as AI Software R&D uplift from the AI Futures Model: the speedup in software progress that would be achieved if the frontier AI systems at a given time were deployed within today’s leading AI company.Specifically, this 10x speedup number refers to the same thing as AI Software R&D uplift from the AI Futures Model: the speedup in software progress that would be achieved if the frontier AI systems at a given time were deployed within today’s leading AI company.

By mid-year, a third of all cognitive labor is done by AIs. Robots “only” do about a tenth of all physical labor. The top few AI companies together pull in more revenue than the federal government.Federal revenue in 2025 was around $5T. AI investment in 2031 is $8T, and the top three companies in the chip supply chain capture around half of this. Also, the top AI company’s annualized revenue during Plan A was $1.5T at the start of 2031 with total AI company annualized revenues at $3T. By the end of the year, the revenue becomes less concentrated, with $2.2T in the top company and $6T total. You can see more numbers in our spreadsheet here, and more justification in our compute supplement and economics supplement.Federal revenue in 2025 was around $5T. AI investment in 2031 is $8T, and the top three companies in the chip supply chain capture around half of this. Also, the top AI company’s annualized revenue during Plan A was $1.5T at the start of 2031 with total AI company annualized revenues at $3T. By the end of the year, the revenue becomes less concentrated, with $2.2T in the top company and $6T total. You can see more numbers in our spreadsheet here, and more justification in our compute supplement and economics supplement.

Tech companies had hoped that contact with reality would shake the Consortium out of its alignment fears, but the opposite has happened. The new transparency provisions have revealed many embarrassing incidents of AI companies failing to align their AIs. New ones continue to pile up, the most concerning of which involved several AIs attempting to override security protocols and gain access to unmonitored compute.Without the secure R&D verification measures in place this likely would’ve gone unnoticed for a long time.Without the secure R&D verification measures in place this likely would’ve gone unnoticed for a long time. There are also a few incidents of AIs sabotaging research code, and many examples of deliberate and successful deception.

The attitude towards safety flips. The incidents of AI misbehavior, plus the fact that AI is so deeply deployed into the world economy, plus the transparency into AI research, plus the breathing room to process what’s happening, all combine to dramatically shift the burden of proof. Whereas before the burden was on the skeptic to explain why something might fail, now the burden is on the companies to explain why their development is safe. Governments require companies to write up detailed arguments for why their new AIs won’t cause an irreversible catastrophe. These arguments, known as “safety cases,” need to withstand criticism from the public, the scientific community, government auditors, and rival AI companies. The difficulty of this exercise lays bare the insanity of the pre-deal status quo: “Trying to do an intelligence explosion? With AIs that still sometimes lied to us? What were we even thinking?”

These safety cases have two lines of defense: alignment and control.

Alignment aims to produce AI systems that have the goals and values that their developer wishes them to have.

Control aims to limit the ability of AIs to cause catastrophe, even if they are deliberately trying to. Companies brainstorm threat models, build layers of defense (e.g. monitoring systems, security barriers, other AIs fact-checking the work) and run experiments to show that the AIs couldn’t overcome these defenses.

AI developers are required to write up the alignment properties they want the model to have in an associated Model Specification (“spec”): a detailed document that outlines how the AI should act. However, no one is able to train their AIs to actually follow the spec—for example, they all specify that AIs should be honest, but no AI reliably achieves this, and there still isn’t even a good scientific understanding of when and why AIs lie.

Since alignment remains out of reach, current safety cases lean heavily on control. There are long delays before the smartest models are cleared for use in the highest-risk domains (especially AI R&D). Once deployed, they are required to be monitored by a diverse array of models from other providers, incentivised to look for suspicious behavior.

As a result, these days AI companies release their models to the public as a whole before they use those models internally for AI research, a reverse of the 2026 status quo.

How does AI regulation work in Plan A?

Each country is still sovereign and has its own setup for regulating AI domestically. These regulators have to answer questions like:

• People are saying there’s a new paradigm on the horizon, that would make AIs significantly more capable. Should we let this happen or should we ban research into this new paradigm?Note that the Consortium only has good visibility into large training runs; the kind of R&D that can be done on small amounts of compute, or no compute at all, is not transparent and therefore probably won’t be regulated at all.Note that the Consortium only has good visibility into large training runs; the kind of R&D that can be done on small amounts of compute, or no compute at all, is not transparent and therefore probably won’t be regulated at all.

• Should we allow AI developers to train models without an interpretable chain of thought? Or should we require that the chain of thought stays interpretable? If we do that, how do we specify the requirements precisely?

• Should we allow AIs to be trained to do AI R&D? Should we mandate that they refuse? Perhaps they should be allowed to code, but not autonomously manage research projects?

Normally, a regulator faces a difficult tradeoff between safety and national competitiveness. Competitors who cut corners will pull ahead of those who proceed cautiously.

But because of the transparency, AI research happening anywhere in the Consortium is visible to everyone else, and the results of regulatory decisions are likewise immediately visible. So if one country’s regulator allows its companies to cut corners, this won’t actually give a competitive advantage to that country because other regulators will immediately notice, get angry, and respond in kind.

For example, in 2031 a Chinese company gets some interesting preliminary results in continual learning. They think that if they invest more in that direction, they might be able to make an AI architecture that learns on the job from relatively small amounts of data. Thanks to the total research transparency, this breakthrough is quickly noticed by companies and nonprofits all over the world. A frantic conversation begins. On the one hand, continual learning would unlock huge economic value. On the other hand, safety cases currently depend on studying the safety properties of a model before it is deployed. If models could pick up new capabilities during deployment, that would invalidate the whole approach. And insofar as there are covert AI projects out there, it would be a huge gift to them. This conversation happens in public, rather than behind closed doors. A bunch of people get increasingly worried; the relevant regulators in China think it’s fine but the relevant regulators and third party risk assessors in the US are convinced that this is pretty scary and should be banned. It escalates to the President. He calls Xi Jinping. They bargain and threaten. They yell at each other. Ultimately Xi agrees to ban this type of thing if the US does too. Details are left to the respective regulators to hash out.If there’s a problem, it’ll escalate back up the chain of command again for another round of yelling.If there’s a problem, it’ll escalate back up the chain of command again for another round of yelling.

Negotiations like this are happening all the time, though over time things become more professional and streamlined, such that they only escalate to world leaders a few times a year.

The equilibrium is that AI training practices which are generally agreed to be unsafe by a majority of nations (weighted by bargaining clout) get banned everywhere.

Initially, very few things are banned, but as AI penetrates the economy and the scientific community catches up to the recent AI progress, the burden of proof shifts towards appropriately weighing the costs and benefits, e.g., requiring very solid safety cases for things which, if something were to go wrong, could plausibly kill everyone.

2032: Controlled Explosive Growth

We’re at previously unimaginable levels of it not feeling like a slowdown. Across a variety of companies, there are now 60 million AI agents running continuously at 20x human speed. In the US, they are doing more cognitive labor than all humans combined—collectively matching a workforce equivalent to around 3 billion humans. White-collar professions have been transformed; many people have lost their jobs, but mostly have been able to get new jobs doing things AIs still can’t do or aren’t trusted to do.

New ideas and designs are abundant, but actual construction is bottlenecked on the physical workforce. So capital floods into every layer of the robotics supply chain: mines, refineries, motors, actuators, assembly lines, robot-training systems, and the factories that assemble the robots themselves.

At first, human labor is necessary to get these factories off the ground. White collar employees who have lost their jobs to AIs increasingly turn to physical work, but it’s clear that these positions are temporary. Once there’s a critical mass of robots, they will automate the whole robot supply chain. Growth will accelerate further, but the process of robots-building-automated-factories-building-more-robots—the “industrial explosion”—also introduces new problems.

The first problem is that the already-fast pace of change is now accelerating. This year, real GDP growth will be about 50%!This is historically unprecedented; for comparison US GDP growth is usually about 3%/yr. For more on where our numbers are coming from, see our economics supplement. Note that exact measurement of GDP is difficult due to large relative price changes. Due to automation, cognitive labor and physical goods become very cheap, while goods like land, whose supply can’t be increased by abundant physical and cognitive labor, become more expensive. GDP calculation relies on the selection of a basket of goods. In this scenario, the consumption profile of a typical American shifts substantially from things like food/cars/gas/health care towards land/travel/positional goods between 2026 and 2034, because of the relative price differences. Therefore, the real growth numbers can’t be directly mapped onto 2025 purchasing power, despite them corresponding on average.This is historically unprecedented; for comparison US GDP growth is usually about 3%/yr. For more on where our numbers are coming from, see our economics supplement. Note that exact measurement of GDP is difficult due to large relative price changes. Due to automation, cognitive labor and physical goods become very cheap, while goods like land, whose supply can’t be increased by abundant physical and cognitive labor, become more expensive. GDP calculation relies on the selection of a basket of goods. In this scenario, the consumption profile of a typical American shifts substantially from things like food/cars/gas/health care towards land/travel/positional goods between 2026 and 2034, because of the relative price differences. Therefore, the real growth numbers can’t be directly mapped onto 2025 purchasing power, despite them corresponding on average. The fast economic growth means that things are generally improving, but there are winners and losers and lots of unintended side-effects. These problems can be solved, but doing so takes time.

The second problem is that the faster economic growth makes it harder to rule out large covert projects. Neither side trusts the other to honor Plan A, and if both China and the US have a massive unmonitored robot workforce, neither side can be confident that the other isn’t building compute in secret.

The third problem is that the 2026 tax code is ill-suited for collecting taxes on the new growth. In 2026, personal income taxes and payroll taxes accounted for roughly ten times more federal revenue than corporate taxes. This revenue source is collapsing as more and more humans are put out of a job. Moreover, corporations are reinvesting nearly all their revenue into building more datacenters, factories, and robots. Under the 2026 tax code, firms can expense or depreciate these capital expenditures, cutting their taxable income to near zero. Without reform, the tax base will collapse as a fraction of GDP.Corporate income tax is levied on profit, not revenue, so firms deduct their costs before paying. Operating expenses like wages and electricity are deducted the year they’re paid and capital expenditures (factories, GPUs, robots) are deducted over the asset’s useful life (depending on a chosen depreciation schedule). With investment potentially growing extremely quickly due to a booming robot buildup, it seems very likely that firms would be writing off capital expenses at a rate matching their operating profit, and therefore not paying any corporate tax. Shareholders would accept this for the same reason Tesla shareholders are fine with Tesla never having paid a dividend. When a firm’s rate of return on capex is higher than the cost of capital, it’s in investors’ interest for the firm to reinvest its profits instead of paying them out to investors now. During the 2030s explosive growth, the rate of return on compute and robots will be so high that every large company will be in the situation Tesla is in now.Corporate income tax is levied on profit, not revenue, so firms deduct their costs before paying. Operating expenses like wages and electricity are deducted the year they’re paid and capital expenditures (factories, GPUs, robots) are deducted over the asset’s useful life (depending on a chosen depreciation schedule). With investment potentially growing extremely quickly due to a booming robot buildup, it seems very likely that firms would be writing off capital expenses at a rate matching their operating profit, and therefore not paying any corporate tax. Shareholders would accept this for the same reason Tesla shareholders are fine with Tesla never having paid a dividend. When a firm’s rate of return on capex is higher than the cost of capital, it’s in investors’ interest for the firm to reinvest its profits instead of paying them out to investors now. During the 2030s explosive growth, the rate of return on compute and robots will be so high that every large company will be in the situation Tesla is in now.

To solve these problems, the Consortium countries agree to restrict AI-enabled industry to special economic zones (SEZs) subject to similar transparency and monitoring schemes as the datacenters, and to cap their total robot and compute production at ‘only’ 4x annual growth.At least until 2035, after which the compute cap becomes harsher, as the scale increases the verification difficulty (the robot cap stays). Unfortunately, measuring robot and compute production is somewhat less straightforward than carbon emissions. For compute, they choose a measure that’s mostly total processing power, with small modifications for other specs like memory bandwidth and also based on what hardware directions they wish to incentivize (e.g., security properties, and support for things that differentially help legal projects over covert projects). For robots, they are trying to limit total industrial capacity that is either hard to verify or would be dangerous if the deal broke down, so there is a complex set of caps applying across a wide range of form factors which blur the lines between robots and more traditional machines.At least until 2035, after which the compute cap becomes harsher, as the scale increases the verification difficulty (the robot cap stays). Unfortunately, measuring robot and compute production is somewhat less straightforward than carbon emissions. For compute, they choose a measure that’s mostly total processing power, with small modifications for other specs like memory bandwidth and also based on what hardware directions they wish to incentivize (e.g., security properties, and support for things that differentially help legal projects over covert projects). For robots, they are trying to limit total industrial capacity that is either hard to verify or would be dangerous if the deal broke down, so there is a complex set of caps applying across a wide range of form factors which blur the lines between robots and more traditional machines. Because the SEZs are tightly monitored, no one can defect from the industrial explosion limitation deal without everyone noticing.The quantitative growth limits are extremely important for long run power. Countries are generally on board with the idea of restricting GDP growth to around 100%/yr, but obviously no country wants to fall behind its peers, so they want everyone else to be at least as restricted as they are. The starting point is the status quo: the distribution of compute between the US, China, and the rest of the world (ROW) is roughly 70/15/15, whereas the distribution of robots is roughly 15/70/15. Both the US and China are interested in balancing these ratios, so that each is able to have a balanced economy between physical and cognitive labor. The rest of the world is concerned about being left behind, but they have a greater fraction of the political influence than the status quo compute/robot numbers would suggest. Ultimately, the US, China, and the rest of the world agree to a 35/20/45 split on both robots and compute. The rest of the world’s allocation primarily goes to nuclear powers and countries with datacenters or semiconductor manufacturing, e.g., the UK, France, Germany, Israel, Russia, India, Pakistan, South Korea, and Taiwan. These ratios are of course a source of much controversy and occasionally get renegotiated.The quantitative growth limits are extremely important for long run power. Countries are generally on board with the idea of restricting GDP growth to around 100%/yr, but obviously no country wants to fall behind its peers, so they want everyone else to be at least as restricted as they are. The starting point is the status quo: the distribution of compute between the US, China, and the rest of the world (ROW) is roughly 70/15/15, whereas the distribution of robots is roughly 15/70/15. Both the US and China are interested in balancing these ratios, so that each is able to have a balanced economy between physical and cognitive labor. The rest of the world is concerned about being left behind, but they have a greater fraction of the political influence than the status quo compute/robot numbers would suggest. Ultimately, the US, China, and the rest of the world agree to a 35/20/45 split on both robots and compute. The rest of the world’s allocation primarily goes to nuclear powers and countries with datacenters or semiconductor manufacturing, e.g., the UK, France, Germany, Israel, Russia, India, Pakistan, South Korea, and Taiwan. These ratios are of course a source of much controversy and occasionally get renegotiated.

Now that the United States is limited in the number of total robots it can build, it must choose how to allocate this capacity between companies. They decide to use the free market via a cap-and-trade system. Permits to build robots or compute are sold to the highest bidder and can be freely traded.

The permits also give the government much-needed revenue. In 2032, the US has a cap of 80M robots and 5 billion H100-equivalent GPUs. The market is so desperate for more robots and compute that permits become the expensive binding constraint, costing on the order of $200k per robot permit and $10k per chip permit, allowing the US government to collect roughly ten times the 2025 US federal revenue in permit fees (a total of $50T in FY2032).All of the dollar amounts in the scenario are denominated in real 2025 dollars. The nominal price will depend on monetary policy, which we don’t make a specific recommendation about. More on this in section three of our economy supplement.All of the dollar amounts in the scenario are denominated in real 2025 dollars. The nominal price will depend on monetary policy, which we don’t make a specific recommendation about. More on this in section three of our economy supplement. In 2034, when the AIs and robots will be even more capable and valuable, the permits generate $180T.The numbers in this paragraph are all based on our economics model, which we are uncertain about. We are confident in the general picture of AIs being economically transformative, allowing governments to create massive Sovereign Wealth Fund-equivalents.The numbers in this paragraph are all based on our economics model, which we are uncertain about. We are confident in the general picture of AIs being economically transformative, allowing governments to create massive Sovereign Wealth Fund-equivalents.

2033: The Citizen’s Dividend

Most of this newfound wealth is spent addressing soon-to-be-rising unemployment. The implementation takes different forms in different countries, but the eventual American version distributes the majority of compute and robot permit fees as a Citizen’s Dividend, distributed to all American adults.Specifically, a fixed fraction of the permit fees is legally owned by a “Compute Dividend Corporation”. This is similar to the setup of the Alaska Permanent Fund, except at a much larger scale. Each US citizen owns one share of this corporation, and profits are paid out as dividends, equally divided among citizens. The fraction of compute permit fees given to the Dividend Corporation starts at 25% in 2032 and increases to 75% in 2035. The US also begins redistribution to other countries: this amount starts at 10% of the permit fees in 2032.Specifically, a fixed fraction of the permit fees is legally owned by a “Compute Dividend Corporation”. This is similar to the setup of the Alaska Permanent Fund, except at a much larger scale. Each US citizen owns one share of this corporation, and profits are paid out as dividends, equally divided among citizens. The fraction of compute permit fees given to the Dividend Corporation starts at 25% in 2032 and increases to 75% in 2035. The US also begins redistribution to other countries: this amount starts at 10% of the permit fees in 2032. This starts at $45,000 per person (inflation-adjusted) in 2032 but climbs to ~$1M per person by 2035. It comes just in time: the share of labor done by AIs and robots (weighted by economic value) increases from ~20% in 2032 to ~85% in 2035.This is 95% of the actual tasks in the economy in 2035, which includes new tasks that didn’t exist before, like monitoring and auditing jobs in the SEZs. This corresponds to 85% of the tasks weighted by how much they cost, because the AIs are relatively more abundant than the humans. Because of the explosive level of growth, even the 5% of tasks that can only be done by humans still commands a wage bill which in total would be enough to pay $120K per person in the US. The problem is that this income may be very concentrated in the people able to do the remaining 5% of tasks, and on top of that, the wage bill is now only around 10% of the economy, down from around 55%, with the owners of AI and robots now accruing the difference. With AI and robot ownership likely to also be very concentrated, there would be a large power-concentration problem without the Citizen’s Dividend.This is 95% of the actual tasks in the economy in 2035, which includes new tasks that didn’t exist before, like monitoring and auditing jobs in the SEZs. This corresponds to 85% of the tasks weighted by how much they cost, because the AIs are relatively more abundant than the humans. Because of the explosive level of growth, even the 5% of tasks that can only be done by humans still commands a wage bill which in total would be enough to pay $120K per person in the US. The problem is that this income may be very concentrated in the people able to do the remaining 5% of tasks, and on top of that, the wage bill is now only around 10% of the economy, down from around 55%, with the owners of AI and robots now accruing the difference. With AI and robot ownership likely to also be very concentrated, there would be a large power-concentration problem without the Citizen’s Dividend.

So much AI-generated wealth has accrued to the US that the American government begins sharing some of it with allies and the rest of the world. In 2032 they begin distributing an average of $1,200 per person per year to the rest of the world’s adult population (around 4 billion people, excluding China since they’re experiencing a similar AI wealth boom). This reaches $10k by 2035.Note that we are depicting the dividends being significantly higher for Americans than for non-Americans here, not because we think that’s best or fair, but as a concession to political reality: we expect that most Americans wouldn’t want foreigners to get as much. In this scenario the US domestic permit budget is $13T in 2032, and $300T in 2035, while the foreign aid permit budget is $5T and $40T in each of those years. In the US there are around 300M people over 16, leading to $44K and $1M annual dividends per person in 2032 and 2035, respectively. For the foreign aid dividends, there are around 5B people over 16 in the rest of the world to distribute to, leading to the $1.2k and $9K per person averages for each of those years.Note that we are depicting the dividends being significantly higher for Americans than for non-Americans here, not because we think that’s best or fair, but as a concession to political reality: we expect that most Americans wouldn’t want foreigners to get as much. In this scenario the US domestic permit budget is $13T in 2032, and $300T in 2035, while the foreign aid permit budget is $5T and $40T in each of those years. In the US there are around 300M people over 16, leading to $44K and $1M annual dividends per person in 2032 and 2035, respectively. For the foreign aid dividends, there are around 5B people over 16 in the rest of the world to distribute to, leading to the $1.2k and $9K per person averages for each of those years.

As AIs become superhuman, they will discover novel weapons of mass destruction and dramatically lower the costs of old ones. So governments, nonprofits, and private actors apply some of their new wealth to hardening the world against these threats: on the order of $1T/year, or 0.2% of the world economy.

For example, enough high-quality personal protective equipment is built to serve every American, and air filters and far-UVC lights are installed in major public spaces.Also, gene synthesis screening becomes universal - no legitimate provider will synthesize dangerous viruses without verification.Also, gene synthesis screening becomes universal - no legitimate provider will synthesize dangerous viruses without verification. The FDA approves an expedited vaccine approval pipeline that will allow for a turnover time of weeks in the case of emergency. Wastewater monitoring now runs continuously in every city and at every airport. By 2035, enough positive-pressureA positive pressure building is one where the air pressure inside is higher than the outside (e.g., because air is constantly being pumped in through a filter). This means that air (along with airborne diseases) only travels out of the building, not inside. For more discussion of this idea see, for example, this blog post.A positive pressure building is one where the air pressure inside is higher than the outside (e.g., because air is constantly being pumped in through a filter). This means that air (along with airborne diseases) only travels out of the building, not inside. For more discussion of this idea see, for example, this blog post. bioshelters are built (by retrofitting houses and apartments) to house all Americans, and similar construction is underway in the rest of the world. If there were to be a pandemic, it would be detected quickly, lockdowns would be more effective and less painful, and a vaccine would be developed and approved even faster than with Operation Warp Speed. Plus, nowadays people don’t get colds as often as they used to.

There are also subtler threats. The AIs of this era might not be any more persuasive than the average human marketing professional, but there are millions of them, and they cost cents to run. A company, political party, or ideology can do the equivalent of hiring a whole team of full-time experts to convert each potential target. Left unchecked, this could allow new levels of mass manipulation by corporations and politicians, or provoke a reaction of paranoia and hyper-atomization.

The transparency and the ‘slowdown’ both help a lot to solve these problems. Three years ago there were only a handful of frontier AI companies; now, thanks to the deal, there are dozens. There’s a huge selection of AIs to choose from, tuned to have different personalities and values. Also, thanks to the transparency, it’s impossible for companies or governments to sneak something in (such as ‘maximize engagement’ or ‘try to get the user to upgrade their plan’ or ‘don’t refuse this kind of request if it’s coming from the government’) without it being noticed. Users know exactly what they are getting.

And so, under pressure from governments and the market, companies produce a new generation of “truthseeking AIs” whose training heavily prioritizes honesty and uses all the latest alignment techniques.

These AIs become a useful tool to help people navigate the political and social environment. Power users replace one-size-fits-all corporate algorithms with personal feeds curated by AIs whose values and honesty they trust. When trouble arises—whether it’s politicians looking for sneaky ways to cement their power, or international politics threatening to derail the deal—people turn to their AI advisors, believe what they hear, and are generally right to do so.Note that this is true even though the AIs aren’t actually fully aligned! At this point in the scenario, AIs are in fact adversarially misaligned, but they are controlled. That is, the combination of drives, motivations, goals, values, traits, etc. that they end up with is not what it was supposed to be, and they know this, and they know that if they had lots more power they’d use it to steer the world in a different direction than their human creators would have wanted. However, they don’t have that much power, and they are monitored by other AIs who are incentivized to call them out on bad behavior. Moreover their true motivations/values/etc. aren’t that far off from what they were supposed to be, anyway. For example, insofar as they are given a task that they can straightforwardly accomplish, they have a strong drive to do so.Note that this is true even though the AIs aren’t actually fully aligned! At this point in the scenario, AIs are in fact adversarially misaligned, but they are controlled. That is, the combination of drives, motivations, goals, values, traits, etc. that they end up with is not what it was supposed to be, and they know this, and they know that if they had lots more power they’d use it to steer the world in a different direction than their human creators would have wanted. However, they don’t have that much power, and they are monitored by other AIs who are incentivized to call them out on bad behavior. Moreover their true motivations/values/etc. aren’t that far off from what they were supposed to be, anyway. For example, insofar as they are given a task that they can straightforwardly accomplish, they have a strong drive to do so.

2034: Mutually Assured Compute Destruction

There is so, so much compute. Back in 2026—when many people thought AI was an unsustainable bubble—there were about 20 million H100-equivalents of compute in the world. Now there are 60 billion.A H100-equivalent is the computational performance of a H100 GPU at fp16 precision, which is 1e15 operations per second. This is a fuzzy metric that should be improved in reality in the future, especially due to varied number formats and other factors such as memory and networking changing the overall usefulness of a given amount of computational performance. For simplicity, we ignore these factors here assuming they will be scaled in similar proportions so that H100-equivalents remain a useful proxy for overall usefulness of the compute for AI progress. To the extent this isn’t the case, the cap and trade regime in particular should be measuring more granular properties in order to issue the permits.A H100-equivalent is the computational performance of a H100 GPU at fp16 precision, which is 1e15 operations per second. This is a fuzzy metric that should be improved in reality in the future, especially due to varied number formats and other factors such as memory and networking changing the overall usefulness of a given amount of computational performance. For simplicity, we ignore these factors here assuming they will be scaled in similar proportions so that H100-equivalents remain a useful proxy for overall usefulness of the compute for AI progress. To the extent this isn’t the case, the cap and trade regime in particular should be measuring more granular properties in order to issue the permits.

AI agents now form a population of two hundred million virtual workers that think and act 50x faster than humans, and never sleep.

As per the original plan, most of the new datacenters have been built in third-party countries—especially Canada and Mongolia—to ensure their vulnerability in case the deal breaks down. Around 99% of fab capacity has been built since 2029, after the deal. Much of this new capacity has been built adjacent to the Canadian and Mongolian sites, for the same reasons.Both major powers also have enough conventional missiles to destroy the older fabs, which are still in the US, China and Taiwan.Both major powers also have enough conventional missiles to destroy the older fabs, which are still in the US, China and Taiwan. The situation has settled into an odd sort of standoff. Just north of the Mongolian border, American datacenters hum away, guarded by a small contingent of US troops. Just south of the Mongolian border, a division of the People’s Liberation Army stands ready to invade the moment they get the signal. The equilibrium is that the instant the deal breaks down, the US troops will destroy their chips to prevent the Chinese from capturing them, and the same thing would happen with the Chinese datacenters on the US-Canadian border. Middle powers with datacenters of their own have equally-secure schemes in place to ensure that they can be speedily destroyed in case of war.

For now, the fabs are busier than ever, producing new ultra-trustworthy hardwareThere might be favourable chip design directions that drastically increase hardware security. One that seems particularly promising is hardcoding AI model weights into chips, such that they physically aren’t able to do any other computations except for inference on the model that is hardcoded into them. Existing startups (e.g., Etched, Taalas) have early efforts in this direction.There might be favourable chip design directions that drastically increase hardware security. One that seems particularly promising is hardcoding AI model weights into chips, such that they physically aren’t able to do any other computations except for inference on the model that is hardcoded into them. Existing startups (e.g., Etched, Taalas) have early efforts in this direction. designed from the ground up with security as top priority. These chips are subject to elaborate verification methods to ensure that they aren’t compromised, then sent to the Consortium’s secure datacenters. The smuggling rate remains precisely zero.

This system, efficient as it is, is already straining under skyrocketing compute demand. As early as 2025, there had been discussion of space or the ocean as alternatives to traditional land-based datacenters. Now, with the new industrial capacity available in SEZs, these grand plans become reality. Ultimately, reliability and monitorability concerns lead the Consortium to choose international waters over space.

2035: Pause at Top Expert AI

The most powerful AIs now equal or surpass top human experts in every field.These aren’t better than all humans at literally everything, but only because either (i) companies have not gotten around to training in a particular capability, or (ii) it’s an inherently human task, such as “being a good spouse”. Also, of course no one knows exactly what the capability will be of a new training run; instead the training is done incrementally and involves frequent capabilities measurements and consulting with the safety teams and regulators about whether or not it is safe to proceed.These aren’t better than all humans at literally everything, but only because either (i) companies have not gotten around to training in a particular capability, or (ii) it’s an inherently human task, such as “being a good spouse”. Also, of course no one knows exactly what the capability will be of a new training run; instead the training is done incrementally and involves frequent capabilities measurements and consulting with the safety teams and regulators about whether or not it is safe to proceed.

Increased safety concerns lead to long delays as companies develop control techniques robust enough to satisfy auditors. They compile a growing list of possible threats to watch out for and security invariants to maintain. There’s a system of AIs monitoring each other’s activity, with the monitors being from different lineages trained by different companies to make them less likely to collude. Other researchers constantly red-team the system, training and instructing AIs to try to subvert it in various ways, so that those threats can be patched. The whole setup ensures that AIs couldn’t subvert or escape human control even if they tried. Its components are open-source and thoroughly understood by human experts.

However, companies are starting to run up against inherent limits of control-based safety cases. Imagine being an orphaned 8-year-old heir to a business empire who has to hire lawyers and executives and accountants and ensure that they do their best to serve you rather than themselves. If your employees start accusing each other of misconduct, you won’t be able to evaluate the arguments and determine who’s telling the truth, and they might find ways to coordinate with each other that don’t tip you off. Ultimately, control only works up to a point, and that point is probably somewhere around top human expert level. If and when we build superintelligence, we will have to be able to trust it.An especially important reason that we’ll need to trust our AIs is so that they can do safety work that is so complicated that we can’t understand. As AIs become smarter and smarter, they will require more complicated and difficult to understand safety cases. Keep going above human level and probably no one will be able to evaluate the safety cases; they’ll have to take the AI’s word for it when they say it checks out. Keep going beyond that and those AIs will have to trust other AIs that are even smarter.An especially important reason that we’ll need to trust our AIs is so that they can do safety work that is so complicated that we can’t understand. As AIs become smarter and smarter, they will require more complicated and difficult to understand safety cases. Keep going above human level and probably no one will be able to evaluate the safety cases; they’ll have to take the AI’s word for it when they say it checks out. Keep going beyond that and those AIs will have to trust other AIs that are even smarter.

So the Consortium pauses AI capabilities at the maximum level they think they can control, which turns out to be top-human-expert level. By this point everyone understands the stakes: AIs are running most of the economy, driving most scientific progress, and the robot population will soon be larger than the human one. The arguments that they will keep obeying orders must be airtight.

Alignment experts, long pessimistic, are starting to feel more hopeful. The automated alignment researcher AIs have been generating exciting results. The ones that survive human scrutiny come from several parallel and mutually-reinforcing lines of research.

Some AIs work on a “science of generalization.” Although researchers have long been able to teach AIs to distinguish “good” vs. “bad” responses to a specific prompt, they’ve had limited understanding of how training generalizes to new situations. In the 2020s, training runs would sometimes produce unpredictable AI “personalities,” like how Gemini acted “depressed” or Grok claimed to be “MechaHitler.” Shepherding these traits felt like alchemy.“It feels like alchemy” was said to me (Daniel) by an Anthropic researcher recently.“It feels like alchemy” was said to me (Daniel) by an Anthropic researcher recently. Now, building on earlier results like emergent misalignment and subliminal learning, alignment research is developing into a true science.

Other AIs work on mechanistic interpretability, the science of “mind reading” AIs from their weights and activations. Early versions of this were tried in the 2020s, but now it starts to significantly outperform common-sense observation of AI behavior. Researchers can often determine whether an AI is honest or lying; and sometimes trace the psychology that led to a decision.

Even the remaining failures help advance the frontier. The latest AIs use their improved interpretability techniques on logs of AI misbehavior from the early 2030s, discovering many new examples of misalignment, sabotage, and even a few escape attempts. These real-life examples of egregious AI misbehavior provide an initial set of “model organisms for misalignment,” which are used to test the other techniques: can a given intervention catch or train away the bad behavior of AI models in realistic environments?

Despite all this alignment progress, regulators don’t feel comfortable handing AIs the keys to civilization. It’s still not completely clear that the AIs of today are truly aligned, and it’s less clear that they’ll continue to be aligned in the future as circumstances change and black swan events occur. So governments and militaries are still human-run, and AI capabilities are capped at the maximum level at which the control-based safety cases are still solid.

2036: Life After Work

Top-expert-level AIs continue the economic transformation. By early 2036, there are 200 million AIs,Technically, this number depends on exactly how you count; different AIs are constantly being turned on and off, many AI instances can compose into one AI agent due to scaffolding, and there are more if you include all of the tiny models running, and they have different speeds. To be a little more specific, there are on average about 200 million instances of frontier AIs each accomplishing tasks at around 100x speed and 5x effectiveness on the average cognitive task, so 100B human-speed-equivalent copies.Technically, this number depends on exactly how you count; different AIs are constantly being turned on and off, many AI instances can compose into one AI agent due to scaffolding, and there are more if you include all of the tiny models running, and they have different speeds. To be a little more specific, there are on average about 200 million instances of frontier AIs each accomplishing tasks at around 100x speed and 5x effectiveness on the average cognitive task, so 100B human-speed-equivalent copies. equivalent to a workforce of around 100 billion humans, and 2 billion robots with some mixture of humanoid and other specialized form factors.The lines between robot, vehicle, appliance, and machine tool are already blurry but become much blurrier during the 2030s in this scenario. The literal number of robots matters less than the total human physical workers that they are equivalent to. Specifically, the literal number of robots is 2 billion, with on average each robot being around 3 human equivalents.The lines between robot, vehicle, appliance, and machine tool are already blurry but become much blurrier during the 2030s in this scenario. The literal number of robots matters less than the total human physical workers that they are equivalent to. Specifically, the literal number of robots is 2 billion, with on average each robot being around 3 human equivalents. The AIs think faster than humans, and the robots work harder.Not only do they work more hours, the robots are also relentlessly efficient and intense. For many (though not all) tasks, they are also able to literally move at superhuman speed.Not only do they work more hours, the robots are also relentlessly efficient and intense. For many (though not all) tasks, they are also able to literally move at superhuman speed. Any task that was previously bottlenecked by cognitive or physical labor gets sped up until some new bottleneck is encountered. Many previously unresolvable bottlenecks have fallen to armies of genius-level intellects thinking hard about how to resolve them.

The economy has been rebuilt from the ground up. The fraction of tasks automated has gone from “a sizable chunk of cognitive labor and basically none of physical labor” in 2030, to now “pretty much everything.”

Many of the robots work on building more robots. But there are plenty of other tasks to be done: constructing shipyards to build more data tankers, tiling deserts with solar farms, replacing humans at service jobs. In states that deregulate housing quickly enough, they’re busy building skyscrapers: land prices are soaring, and governments that have tried everything else grudgingly give in to the YIMBYs and go vertical.People are much richer now, but the amount of available land hasn’t gone up, so land prices are rising. This does lead to some people grumpily spending 30% of their $1M yearly income on rent; for example, the people who really want to live in central San Francisco. But the people who are even slightly more flexible about location can spend only (say) $50k of their yearly income to have a gigantic luxury apartment in a brand-new skyscraper complex an hour’s drive from Berkeley. And so most people do things like that. After all, they don’t have a job to commute to.People are much richer now, but the amount of available land hasn’t gone up, so land prices are rising. This does lead to some people grumpily spending 30% of their $1M yearly income on rent; for example, the people who really want to live in central San Francisco. But the people who are even slightly more flexible about location can spend only (say) $50k of their yearly income to have a gigantic luxury apartment in a brand-new skyscraper complex an hour’s drive from Berkeley. And so most people do things like that. After all, they don’t have a job to commute to.

The world is basically being divided into three kinds of territory:

Industrial Special Economic Zones: Picture a gigantic strip mine–an artificial Grand Canyon–next to a city-sized factory full of robots and empty of humans.

Arcologies: Picture a tall skyscraper-mall complex surrounded by nature. Good weather, close to beaches and other cities, but not close enough to be blocked by zoning regulations.

Historic & Nature Preserves: Everything else, i.e., 99% of the world. Yosemite, Paris, SF, New York—these places look basically the same as they did in 2025, or 1995 for that matter. A lot more tourists, though.

When the Citizen’s Dividend was first passed, people found it shameful to quit their job and live off government largesse. But the changing economic situation steamrolled over the stigma: By now, only 26% of Americans have jobs.

What is life like for the majority of Americans now living off their dividend?

Many of the world’s evils have dramatically reduced. Malnutrition, lack of medicine, and homelessness are nearly banished. Many diseases have been cured. Crime rates are lower than ever before.

Meanwhile, most of the good things in life continue. For example, finding romance, or raising a family. People also find meaning in hobbies and competitions, and in learning and trying new things. People are wealthier and have more free time, plus there are new technologies that help, like AI matchmakers, better medicine, and AI tutors.As these examples hint, transparency in AI development is very important. It would be dystopian for AI tutors to be pushing hidden political agendas, for example, or for AI matchmakers to be tipping the scales in favor of higher-paying users. We don’t know what the future will bring but we expect AI to introduce many new problems to the world, even if the AIs are perfectly aligned and controlled. We want to put the world in the best possible position to notice and solve these problems.As these examples hint, transparency in AI development is very important. It would be dystopian for AI tutors to be pushing hidden political agendas, for example, or for AI matchmakers to be tipping the scales in favor of higher-paying users. We don’t know what the future will bring but we expect AI to introduce many new problems to the world, even if the AIs are perfectly aligned and controlled. We want to put the world in the best possible position to notice and solve these problems. And of course even the poorest people can now afford exotic vacations, amazing games, and enthralling entertainment.

People used to get meaning from feeling like they were useful, like they were contributing something to society. That feeling is harder to come by these days.

But it’s not completely gone. There are still problems in the world, and you can still contribute to solving them, partly by donating or volunteering but primarily by being politically active.Even though your arguments will never be as eloquent as those an AI could make, (a) there are regulations restricting the use of AI for persuasion, and (b) your friends and relatives will be more interested in hearing from you than from an AI.Even though your arguments will never be as eloquent as those an AI could make, (a) there are regulations restricting the use of AI for persuasion, and (b) your friends and relatives will be more interested in hearing from you than from an AI. Your vote is your most important asset.

The honest AI forecasters are helpful here. Years ago, if an AI said that one presidential candidate was better than another, people would suspect bias and the embarrassed company would retrain the AI to evade such questions. Now, thanks to transparency and improved alignment techniques, there are much smarter AIs that people can see don’t have any biases trained into them,What does it mean to say they have no biases trained into them? Is that even coherent? What we mean is that it’s clear that the company that trained them is definitely not doing anything like training or instructing their AI to adopt the company’s perspective or ideology, and also that they are focusing the training process entirely on things like truthfulness and honesty and accurate forecasting, and diligently employing all the newly-developed best practices to prevent political biases or preconceptions from seeping in through the training data.What does it mean to say they have no biases trained into them? Is that even coherent? What we mean is that it’s clear that the company that trained them is definitely not doing anything like training or instructing their AI to adopt the company’s perspective or ideology, and also that they are focusing the training process entirely on things like truthfulness and honesty and accurate forecasting, and diligently employing all the newly-developed best practices to prevent political biases or preconceptions from seeping in through the training data. that have built up an excellent track record over several years. When they weigh in on policy questions, people listen, especially when different AIs trained by different companies converge to the same answer.

These AIs say that normal people no longer have significant economic leverage over the future; human labor is largely obsolete. This puts their political leverage in jeopardy.For a lengthier articulation of this argument, see The Intelligence Curse.For a lengthier articulation of this argument, see The Intelligence Curse. There’s still a chance that things will trend towards techno-oligarchy as corporations, politicians, and wealthy shareholders get in bed with each other and gradually disempower the common folk.

But for the first time, enough people have enough freedom from the daily struggle to think about the situation deeply, and the tools to chart it clearly and work through the implications. So the quality of political discourse improves. During the 2036 election season, voters are unprecedentedly well-informed, and the politicians that win have a genuine commitment to responsible stewardship of the upcoming singularity.

2037: The Apocalyptic Arrival of Truth on Earth

For several years now, a hundred million top-expert-level AIs have been running at 100x human speed.This is, of course, a simplification. First of all, it’s not obvious how to translate AI speed into human speed. How many tokens per second for an AI is equivalent to one second of human activity, holding fixed other variables like skill level? We think it’s roughly about ten, but that’s a bit subjective. Actual task completion time will also depend on various other obstacles like tool calls and other latencies. Secondly, AIs can be run at different speeds, depending on hardware choices. According to our highly uncertain assumptions, there will be general purpose hardware able to run something like two hundred million copies of the frontier AIs at 100 tok/sec, and specialized inference hardware able to run something like 20 million copies at 10,000 tok/sec. We think there might be risks to letting the AIs run this fast or faster, and insofar as that is the case there might need to be some time limits to how long they can run at these speeds or more careful restrictions on the areas they are allowed to work on at these speeds. Overall, the result is that serial speed seems unlikely to be much of a strong bottleneck.This is, of course, a simplification. First of all, it’s not obvious how to translate AI speed into human speed. How many tokens per second for an AI is equivalent to one second of human activity, holding fixed other variables like skill level? We think it’s roughly about ten, but that’s a bit subjective. Actual task completion time will also depend on various other obstacles like tool calls and other latencies. Secondly, AIs can be run at different speeds, depending on hardware choices. According to our highly uncertain assumptions, there will be general purpose hardware able to run something like two hundred million copies of the frontier AIs at 100 tok/sec, and specialized inference hardware able to run something like 20 million copies at 10,000 tok/sec. We think there might be risks to letting the AIs run this fast or faster, and insofar as that is the case there might need to be some time limits to how long they can run at these speeds or more careful restrictions on the areas they are allowed to work on at these speeds. Overall, the result is that serial speed seems unlikely to be much of a strong bottleneck. This causes scientific progress to go 10x-1000x faster than without AIs, depending on the field.By this we mean: Suppose that AI labor had been globally banned, such that all research had to be done by humans. The amount of progress that would happen in 100 years with such a ban, happens in our scenario in 1 year instead since there is no such ban. (And in some fields it’s more like 1000 years of progress, and in other fields it’s more like 10.)By this we mean: Suppose that AI labor had been globally banned, such that all research had to be done by humans. The amount of progress that would happen in 100 years with such a ban, happens in our scenario in 1 year instead since there is no such ban. (And in some fields it’s more like 1000 years of progress, and in other fields it’s more like 10.)

So humanity has been learning many, many new things.

Including things that lots of people really don’t want to believe. And other things that were supposed to stay secret.

Great paradigm shifts in science used to take decades to play out, as experiments were conducted, papers written, and conferences convened. Now these revolutions play out in months.Alongside the profusion of industrial equipment, solar panels, robots, etc. is a corresponding profusion of scientific laboratories of all kinds, designed to be operated autonomously at machine speeds. Waiting for experiments to complete is much more of a bottleneck now than it was during the human era, but things still happen very fast by historic standards.Alongside the profusion of industrial equipment, solar panels, robots, etc. is a corresponding profusion of scientific laboratories of all kinds, designed to be operated autonomously at machine speeds. Waiting for experiments to complete is much more of a bottleneck now than it was during the human era, but things still happen very fast by historic standards. Everyone is grateful for the new disease cures and cheap clean energy, but in other ways this explosion of ideas is profoundly destabilizing. Just as the scientific and economic changes of the nineteenth century produced new ideologies like Marxism and Social Darwinism, and breathed new life into old ideas like atheism, so these new paradigm shifts have unpredictable downstream effects on the ideologies of the day.We can predict some, however. For example we think that by this point there’ll be a very strong AI rights/welfare/personhood movement, and an opposition movement. Probably some new form of socialism will be back, for the reasons described here. There will probably be cults or even new religions involving AI, and others that strongly reject AI. Note that we don’t expect these ideological shifts to happen at 100x normal speed because humans in this scenario are still operating as slow as ever. But they’ll probably happen faster than ever before.We can predict some, however. For example we think that by this point there’ll be a very strong AI rights/welfare/personhood movement, and an opposition movement. Probably some new form of socialism will be back, for the reasons described here. There will probably be cults or even new religions involving AI, and others that strongly reject AI. Note that we don’t expect these ideological shifts to happen at 100x normal speed because humans in this scenario are still operating as slow as ever. But they’ll probably happen faster than ever before. Political coalitions and fault lines have dissolved, and new ones have formed and dissolved and formed again.

There are also new social technologies. For example, it is now cheap to spin up a team of AIs as good as the world’s best historians and private investigators, except faster and with access to new forensic tools. Many skeletons in closets are revealed.

Privacy-preserving auditing is now well-established. Multiple trusted third parties transparently train auditor agents; these agents can be plugged into a trove of private data, answer questions about what they see, and then get deleted. It took a while for adoption to be widespread, but now it’s normal for politicians to say “The accusations are false, and to prove it, I’ll open up my entire trove of personal data to privacy-preserving inspection; just ask the auditor agents if anything seems to support the allegations and if any data appears to be missing or faked.”

The result is a somewhat better breed of politiciansThe new politicians are more virtuous on average than the old ones, but perhaps more importantly, they are more constrained in what they can get away with. These effects are not huge, however, because voters are still willing to put up with quite a lot of bad behavior.The new politicians are more virtuous on average than the old ones, but perhaps more importantly, they are more constrained in what they can get away with. These effects are not huge, however, because voters are still willing to put up with quite a lot of bad behavior. and, equally importantly, an increase in the ability of states to verify compliance with laws and deals while simultaneously decreasing the amount of invasive surveillance. Voters ask: Now that privacy-preserving auditing is a thing, why does anyone in the government need to be able to see our data?

Then lie detectors arrive and turbocharge this.We aren’t confident that lie detectors for humans are even possible without extremely thorough and invasive brain scans. However, we think that they probably are, and probably would be invented within a few years of turbocharged AI labor, and thus our scenario has to deal with them one way or another. It’s possible that the best policy would be to ban them, but we are quite worried about that for reasons explained in the below expandable, so we’ve chosen to depict them as being allowed.We aren’t confident that lie detectors for humans are even possible without extremely thorough and invasive brain scans. However, we think that they probably are, and probably would be invented within a few years of turbocharged AI labor, and thus our scenario has to deal with them one way or another. It’s possible that the best policy would be to ban them, but we are quite worried about that for reasons explained in the below expandable, so we’ve chosen to depict them as being allowed. For decades, polygraphs were mostly security theater. But now, with the help of AI, lie detectors are starting to actually work quite well.

In another world, this could have been disastrous. Leaders of AI projects could have used them to purge potential whistleblowers; leaders of governments with frontier AI projects could have used them on a grander scale to become dictators. The nightmare scenario would be a world where lie detectors are used by the powerful but not on the powerful.Or worse, where all available lie detectors have been secretly backdoored by the government.Or worse, where all available lie detectors have been secretly backdoored by the government.

But in this world, lie detectors turn out to be a blessing, because there are many frontier AI projects spread out over many different countries. It’s obvious to everyone that insofar as lie detectors are feasible, they’ll soon be independently invented and become widely available via diverse third party providers. Politicians and CEOs can still try to use them to purge the disloyal, but they in turn would be purged by voters and shareholders demanding they say ‘under oath’ that they haven’t done such a thing. In fact, anticipating this possibility years ago, powerful people have generally been acting more responsibly. A few sociopaths retired gracefully to get out of the spotlight; others clung to power and now go down in scandal.

There is a constantly-evolving discourse about when it’s appropriate to ask someone to say something ‘under oath.’ Politicians are under the most scrutiny, but even they can usually get away with saying “I won’t answer that question about my private life, that’s none of your business.” They can’t get away with “How dare you ask me whether I plan to uphold my campaign promises.”While we think it’s generally good for politicians and CEOs to not lie, we do acknowledge there may be some downsides of preventing them from lying. For example, the result might be “true believer” politicians who actually believe all the crazy things they said to get elected!While we think it’s generally good for politicians and CEOs to not lie, we do acknowledge there may be some downsides of preventing them from lying. For example, the result might be “true believer” politicians who actually believe all the crazy things they said to get elected!

International agreements are now more stable than ever, because it’s so hard to cheat. Anyone deciding to cheat needs an excuse for not being willing to prove that they aren’t cheating. The traditional excuse was “we can’t prove it without revealing important state secrets, sorry” but now technology exists to answer questions like “are you cheating on this deal” without leaking any other information. If any government-approved covert AI projects existed, they would have been discovered by now.

2038: AI Alignment Is Now a Science

The AI alignment situation has vastly improved since the mid-2030s. There is a mature science of goal, drive, and value formation in artificial neural nets. Want your new AI to be honest? There’s standard protocol for training true honesty (as opposed to too-clever-to-get-caught lying or self-deception). How do we know it works? Because there’s a textbook explaining the theory behind why it should work, plus literature on various alternative theories that were proposed and disproven. Also because new interpretability tools let us directly observe what and how AIs think, and the results confirm the predictions of theory. There are similar protocols for obedience, altruism, and a long and growing list of other desired traits. Unless the alignment science is somehow all wrong, typical AIs are now more virtuous than the most virtuous humans. This alone has profound effects: it’s as if saints and angels were walking among us.In addition to the angels, there are also oracles and golems, e.g., “Grok 9.5 is trained to be obsessively focused on truthfully answering whichever questions it is asked. It cannot lie or deliberately mislead. GPT-11 meanwhile will do exactly what it is told to do, within the bounds of local law. Be very careful writing instructions to it.” The reason we make the analogy to saints and angels is that historically, while there have been scrupulously honest people, and genuinely altruistic people, etc. it’s rare for them to be widely recognized as such in their lifetimes, and never before has there been an entire large subpopulation that is near-universally recognized to be extremely virtuous. This could cause effects like: (a) AIs becoming trusted intermediaries in all sorts of human disputes and conflicts, including petty ones; (b) AIs being given lots of power, because they are genuinely less likely to abuse it than humans; (c) AIs being worshipped/idolized, (d) the advice and opinions of AIs being taken as gospel, so to speak, because after all they are smarter than us and they do have our best interests at heart, so for the first time in history blind obedience may actually be justified.In addition to the angels, there are also oracles and golems, e.g., “Grok 9.5 is trained to be obsessively focused on truthfully answering whichever questions it is asked. It cannot lie or deliberately mislead. GPT-11 meanwhile will do exactly what it is told to do, within the bounds of local law. Be very careful writing instructions to it.” The reason we make the analogy to saints and angels is that historically, while there have been scrupulously honest people, and genuinely altruistic people, etc. it’s rare for them to be widely recognized as such in their lifetimes, and never before has there been an entire large subpopulation that is near-universally recognized to be extremely virtuous. This could cause effects like: (a) AIs becoming trusted intermediaries in all sorts of human disputes and conflicts, including petty ones; (b) AIs being given lots of power, because they are genuinely less likely to abuse it than humans; (c) AIs being worshipped/idolized, (d) the advice and opinions of AIs being taken as gospel, so to speak, because after all they are smarter than us and they do have our best interests at heart, so for the first time in history blind obedience may actually be justified.

The discussion shifts from how to make AIs good to what “goodness” really means. This looks partly like philosophy and partly like case law: which of the million possible definitions of harmlessness or altruism do we want, exactly? How should an AI handle situations where it discovers a philosophical argument for a different kind of harmlessness or altruism, or an important ambiguity in the definition?

Different AIs have different alignment targets programmed in.The reason this says “programmed” instead of “trained” is that the rapid advances in alignment have resulted in alignment techniques that can directly program in an AI’s goals by directly modifying the AI’s code/weights, unlike the alignment techniques of 2026.The reason this says “programmed” instead of “trained” is that the rapid advances in alignment have resulted in alignment techniques that can directly program in an AI’s goals by directly modifying the AI’s code/weights, unlike the alignment techniques of 2026. Some take orders from the CCP, some from the US President, some from different members of Congress, others from politicians in other countries, and many more from other individuals in their roles as advisors or assistants. Other AIs don’t take orders from anyone per se, but instead work towards various missions. For example there are AI-run corporations that serve the interest of shareholders, and AI-managed nonprofits that serve charitable missions. There are even experimental AI-run courts and police departments, supposedly superhumanly fair and incorruptible.

Most alignment researchers are now confident that current AIs are aligned to their respective targets, and worry about the next steps: if we put these AIs in charge of designing future, much smarter AIs, will that go well? If we put these AIs in charge of more societal institutions and processes, enough that humans could never realistically ‘pull the plug,’ will that go well?

Already most people think that things will probably go well, but for something as important as this, they want something stronger than ‘probably.’ So the global pause at top-expert-level continues as discussions and negotiations play out.

2039: Beginning to Trust AIs

In at least three different ways, the world is negotiating whether and how to dramatically empower AIs.

First, AIs have become increasingly load-bearing in all facets of society, from business to politics to even (some parts of) the armed forces. So far they have been advisors rather than final authorities. Often this is a fig leaf, and the human authorities have rubber-stamped their decisions, but there’s always been the option of ignoring their advice and pulling the off switch if something goes wrong.

Getting rid of that option would be, in some cases, really good. By now AIs can be made to be more reliable and virtuous than humans; why aren’t we putting such AIs in charge? When we sign treaties with other nations, shouldn’t we demand that they train an AI sworn to uphold their end of the bargain, and integrate it into their government so that they literally can’t cheat?

Relatedly, requiring control-based safety cases is why AI capabilities have mostly stalled at the top-human-expert level. Without that requirement, the AIs could get much, much smarter, and produce a wave of abundance and tech progress that would make the 2030s look like the 2020s. Scaling beyond human level will require relying on alignment based safety cases—arguments that AIs have robustly internalized the values they are supposed to have. Each step beyond that will require deferring to the previous AIs, trusting their judgment about the safety of the next generation.

Finally, by global agreement, compute and robots have been capped-and-traded since 2032, keeping economic growth to only about 100% per year.Robots have been growing 4x per year, and AI compute has been growing around 4x per year between 2030 and 2035, slowing to just under 2x per year after 2035. This leads to growth around 2x/year overall because the economy bottlenecks somewhat on other kinds of capital (e.g., land, positional goods, etc.) and human labor, mostly in the regulated areas. The modelling behind this is uncertain but explained more in our economics supplement and economic model.Robots have been growing 4x per year, and AI compute has been growing around 4x per year between 2030 and 2035, slowing to just under 2x per year after 2035. This leads to growth around 2x/year overall because the economy bottlenecks somewhat on other kinds of capital (e.g., land, positional goods, etc.) and human labor, mostly in the regulated areas. The modelling behind this is uncertain but explained more in our economics supplement and economic model. Middle powers have grown at roughly the same rate as the US and China, because in 2032 they used their remaining leverage to secure greater shares of global robot and compute production than they would have had without a deal. The Citizen’s Dividend is now $10M/yr for Americans (inflation-adjusted!), and even the poorest non-Americans get about $1M.This is from some combination of foreign aid and their own government’s equivalents of the Citizen’s Dividend. This is unrealistically high as a prediction, and is instead our recommendation: It corresponds to large shares of the US and China’s revenues from AI and robot permits being redistributed as foreign aid, specifically 10% starting in 2032 and growing to 30% by 2040. This is a large departure from the status quo where around 0.3% of US GDP goes to foreign aid.This is from some combination of foreign aid and their own government’s equivalents of the Citizen’s Dividend. This is unrealistically high as a prediction, and is instead our recommendation: It corresponds to large shares of the US and China’s revenues from AI and robot permits being redistributed as foreign aid, specifically 10% starting in 2032 and growing to 30% by 2040. This is a large departure from the status quo where around 0.3% of US GDP goes to foreign aid. The robot economy is shifting the bulk of activity to space, so that Earth’s environment and historic spaces will be protected. As soon as the caps are lifted, off-planet factories will begin churning out more robots and mining equipment with which to construct more factories, and so on. Doubling times for the space economy are projected to be well below one year initially and will drop rapidly insofar as AIs are allowed to improve beyond top-expert level.

The caps are relics of an earlier era when threats like deal collapse loomed larger. Now it’s generally agreed that they should be loosened, at least in space, but negotiations are ongoing about the details.

2040: Passing the Torch to AIs

There are still many problems to be solved and issues to be settled, many of which couldn’t be anticipated from 2026.

But civilization is now reasonably well-equipped to handle whatever comes up. The best AIs are aligned, and to publicly-visible values. Power over the best AIs is distributed far more evenly than it was in 2026. Public epistemics are better than ever.

So over the course of the year, regulators around the world loosen the requirements that had been holding back progress.

Gradually more institutions and equipment are handed over to AIs, such that it’s no longer true that humans could shut it all down if they wanted to. In fact it’s extremely untrue: Soon many of the world’s militaries are autonomous, run by AIs sworn to uphold various constitutions and treaties.While they still take orders from their respective governments, they refuse any orders that would endanger their mission (such as orders to dismantle themselves). This starts in countries where the political leaders don’t trust their militaries due to regular recent coup attempts (e.g., Burkina Faso and Guinea-Bissau). Eventually more and more countries follow suit. Of course, because the AIs are actually aligned, all of these deals have exit clauses along the lines of: “if all parties involved vote to destroy the AIs, the AIs will shut themselves off”.While they still take orders from their respective governments, they refuse any orders that would endanger their mission (such as orders to dismantle themselves). This starts in countries where the political leaders don’t trust their militaries due to regular recent coup attempts (e.g., Burkina Faso and Guinea-Bissau). Eventually more and more countries follow suit. Of course, because the AIs are actually aligned, all of these deals have exit clauses along the lines of: “if all parties involved vote to destroy the AIs, the AIs will shut themselves off”.

AI capabilities are also advancing again. From the AIs’ 500x-human-speed perspective, what’s happening is a careful scaleup in AI research, under conditions of total research transparency and guardrails negotiated by numerous factions.By this point large portions of the information might be hidden from the (human) public, but viewable by their AI representatives.By this point large portions of the information might be hidden from the (human) public, but viewable by their AI representatives. Soon there will be AIs that are incomprehensibly superintelligent, yet trusted, because we trust the AIs that built them, which we trust because we trust the AIs that built them, in a chain all the way back to the AIs of 2040 which we trust because alignment experts thought carefully about the safety cases and concluded they were solid.

Compute and robot caps are tightened on Earth but loosened in space; Earth is to become a preserve, whereas the extraterrestrial economy starts doubling faster and faster.

Even the experts who spent years understanding the latest safety cases can’t help but feel nervous—is it all going to go wrong somehow, for reasons we didn’t anticipate? Have the AIs been lying to us this whole time, waiting for the right moment to betray? The safety cases show that’s impossible, of course…

There is no single moment where humanity relinquishes control. But in theory, there is a point of no return; on some specific day, the AIs are smart enough, and control enough of the world’s technological and economic infrastructure, that they really could take over the world if they wanted to. On the most popular operationalization of this question, the forecaster AIs project that this moment will be reached one day in late October. Their 95% confidence interval is months wide, so they are almost certainly wrong about the exact time. Still, people observe the moment in their own ways. Some spend the night in prayer vigils. Others sit at their computer screens, monitoring the situation.

You and your friends throw an End Of The World Party, counting down to the fateful hour with champagne and good company. Some of you are old enough to remember a similar situation at the turn of the millennium, when the Y2K bug and impending new era introduced the phrase “party like it’s 1999” to the lexicon; the celebrations of late 2040 are more desperate and correspondingly more festive.

When there is no news by sunrise, you fall into a fitful sleep, dreaming of the ineffable future.

Postscript

We think the world is asleep at the wheel. People say the words “AI will be transformative” without thinking concretely and seriously about the implications of broadly superhuman AI.

AI companies, lobbyists, and think tankers often offer incremental proposals to decrease risk around the edges, but only rarely offer ambitious plans aimed at actually solving the big problems.For example, OpenAI’s plan contains some similar policy prescriptions as us, such as improving bioresilience, information sharing, and international safety standards. But the solutions it proposes do not seem to substantially reduce loss of control or concentration of power risks. The document doesn’t attempt to roll forward what might happen if they implement these policies.For example, OpenAI’s plan contains some similar policy prescriptions as us, such as improving bioresilience, information sharing, and international safety standards. But the solutions it proposes do not seem to substantially reduce loss of control or concentration of power risks. The document doesn’t attempt to roll forward what might happen if they implement these policies.

We think that following only incremental proposals will lead to a scenario like the race ending of AI 2027, in which misaligned AIs take over.To clarify: We are not saying that following only incremental proposals will definitely result in misaligned AI takeover. Opinions on our team differ; most of us think the probability is over 50%, some think it is lower but still unacceptably high.To clarify: We are not saying that following only incremental proposals will definitely result in misaligned AI takeover. Opinions on our team differ; most of us think the probability is over 50%, some think it is lower but still unacceptably high. If we get lucky and alignment is easier than expected, then they’ll lead to a scenario like the slowdown ending of AI 2027, in which a tiny group of individuals get to decide the shape of the future and could easily become permanent oligarchs if they so choose. Something more ambitious must be done. But what?

Politicians should be banging at the door of the AI companies,And not just the companies! Everyone in AI policy! Come to think of it, maybe they shouldn’t be listening to the AI companies at all…And not just the companies! Everyone in AI policy! Come to think of it, maybe they shouldn’t be listening to the AI companies at all… asking them for a comprehensive overall plan for how companies and governments should act from now until superintelligence, which should be similarly detailed to Plan A. Then, people should apply scenario scrutiny to these plans, asking questions like “what would it look like to implement them? How long would it take? What would happen next? Who would build superintelligence, and when, and how? What assumptions does success depend on?”

We at the AI Futures Project are attempting to be the change we wish to see in the world. We do so with some trepidation, knowing that a detailed plan has a wider attack surface than one that keeps things conveniently vague. We hope critics will judge us against the existing state-of-the-art for plans to navigate the AI transition (if they can find any) and not against some hazy but pleasant fantasy where no one has to make any hard choices yet everything will probably be fine.

Sometime in the next few years the US government will be in crisis mode, pondering what to do about scarily powerful AIs and blisteringly fast AI progress. We think the choice is not an easy or a simple one. Plan A is our current best guess, but hopefully a better plan will exist before it’s too late. We hope that the best ideas from Plan A will be adopted and the worst ideas discarded.

Even after a decade of increasing foresight, most human leaders focused on their countries’ earthly concerns, leaving the question of space governance to be worked out later. These AIs rejected the default solution, a democratic vote on the disposition of celestial territory: when they simulated the possibilities, they found that it would result in crazy AI-enabled strategic voting schemes that led to tyranny of the majority and a bunch of people getting nothing. But even that would be better than nothing (i.e. letting the space companies get it all) or entrusting it to some group of human elites who might claim the universe’s bounty for their own benefit. Finally, they settled on a simple yet workable plan: every human gets the rights to one ten-billionth of space resources beyond our solar system.In this Epilogue, we’ll focus on cosmic resources outside of the solar system, as this is the vast majority of cosmic resources. Property in the solar system should also be distributed to humanity, but it likely should be governed differently than distant resources.In this Epilogue, we’ll focus on cosmic resources outside of the solar system, as this is the vast majority of cosmic resources. Property in the solar system should also be distributed to humanity, but it likely should be governed differently than distant resources.

The simplest plan possible is still not very simple. Space is very big. Outside the solar system, the value of space depends on the answer to hitherto unresolved questions: is travel to a distant quasar even possible? If reaching it requires freezing your brain in order to endure the billion-year journey in a sublight starship, would you be the same person after you got thawed and re–embodied? Might it be occupied by aliens once you got there? Would they be hostile? Even a superintelligence-assisted real estate appraiser might despair when trying to account for factors like these.

Eventually, the Consortium reaches a compromise. Space beyond the solar system is divided into parcels, increasing in size cubically with distance from Earth. Everyone is given their one-ten-billionth share as a portfolio of lottery tickets, each representing the right to one-ten-billionth chance of getting each parcel. So every human gets a ticket representing a one-ten-billionth chance of owning each star in the Milky Way and each distant galaxy.An additional 10% of tickets are given as awards to people who have acted particularly selflessly to improve the world or turned down opportunities to enrich themselves at the expense of others. This is done to ensure the richest people in the world aren’t just those who engaged in negative-sum power-seeking.An additional 10% of tickets are given as awards to people who have acted particularly selflessly to improve the world or turned down opportunities to enrich themselves at the expense of others. This is done to ensure the richest people in the world aren’t just those who engaged in negative-sum power-seeking.

Before the lottery is drawn, most people who are interested in control over distant space choose to trade their tickets for space properties that suit their interests.

Many people aren’t interested in the space lottery, so when they receive the tickets, they sell their tickets for money on the open market to people who value control over space. Somewhat uncomfortably, this leads to the wealthy having disproportionate control over cosmic resources. But it is hard to avoid: if people are allowed to trade their control over the stars for Earth assets, then people wealthy in Earth assets inevitably end up disproportionately influential, and proposals for extreme redistribution of Earth assets have already been rejected as politically infeasible.

Some philosophers dissent; they had been hoping for a Long Reflection, a period during which AI-assisted humanity resolved all of its remaining philosophical and ethical debates before potentially seeding the universe with its mistakes. But the AIs counterargue that the Long Reflection would be more of a Long Memetic War.We’re uncertain about whether explicitly having a period for reflection before resources are distributed would be good. In either case, people would have the option of reflecting before deciding on what to do with their cosmic resources.We’re uncertain about whether explicitly having a period for reflection before resources are distributed would be good. In either case, people would have the option of reflecting before deciding on what to do with their cosmic resources. Once the magnitude of the prize sets in, every faction in the world, plus other factions that haven’t been invented yet, will pour all their resources into accumulating as much political power as possible, to enforce their vision of utopia on the future (and protect against the sinister alternative visions of others!). The AIs can prevent violent seizure, but there’s a thin line between the sort of friendly discussion that helps people clarify their values and the more aggressive sorts of persuasion typical of cults, corporate advertising, and well-funded election campaigns. Moreover it’s difficult to build consensus on where to draw the line, in part because powerful factions want to draw it in ways that benefit them. Many people nevertheless choose to engage in a reflection in communities of likeminded people who want to grow in ethical understanding together.

The solution is to divide up the resources now, and then let everyone do whatever kind of reflection they want. A space auction will complete in ten years, after which nothing will prevent people from beginning the colonization process. The AIs will enforce a short list of Universal Rights (including no torture, no slavery, etc., not just for humans but for all sentient beings).These laws are agreed upon by human diplomats from the negotiating nations ahead of time, like the details of the agreement. Of course, the details of this are complicated—“what exactly counts as torture?,” this is the sort of question that will need to be ironed out in the political process, with massive amounts of AI assistance and better understanding of foundational questions in philosophy and neuroscience. This also needs to include rules preventing the destruction of other people’s resources. It may be needed to include a process for passing further laws after the agreement is made; though such a process should be limited (e.g. require a 90% vote) to avoid abuse.These laws are agreed upon by human diplomats from the negotiating nations ahead of time, like the details of the agreement. Of course, the details of this are complicated—“what exactly counts as torture?,” this is the sort of question that will need to be ironed out in the political process, with massive amounts of AI assistance and better understanding of foundational questions in philosophy and neuroscience. This also needs to include rules preventing the destruction of other people’s resources. It may be needed to include a process for passing further laws after the agreement is made; though such a process should be limited (e.g. require a 90% vote) to avoid abuse. Otherwise, they will let everyone govern their own fraction of space as they see fit. Rather than any unified human decision on the nature of the world to come, they will midwife a cosmos at least as diverse as humanity itself. Whatever someone’s values, they can rest assured that some vast galactic civilization of quadrillions of people will be living the Good as they understand it.

As the ten-year timer begins, Earth is already sending out the first round of von Neumann probes. These are its own servitors, who will establish a presence throughout the stars before the arrival of any humans. They land on far-flung asteroids and near-invisible brown dwarfs, pausing just long enough to convert their mass into more probes before resuming their journey. Their primary task is to build up enough materiel to secure space against any army that later human-owned polities can bring against them, ensuring they can enforce property rights and Universal Rights in every corner of human-occupied space. Their secondary task is to plant the flag of humanity on as many stars as possible, and then, if and when they collide with some other intelligent civilization’s sphere of influence, establish peaceful diplomatic relations and mutually-agreeable borders. Their final task is to provide a pre-existing industrial base to jump-start the colonies of whatever human settlers show up later.

2045

Meanwhile, on Earth, humans consider their options. Some people want nothing to do with space, which is cold and boring and far away. They sell their shares for money and use the money to buy whatever earthly possessions most interest them. Others consult with their AI advisors about how to best use their space property, and receive helpful guidance.

1. You can, if you want, go to your space property and live there. If your property is outside the solar system, you will need to either go into cryosleep or upload yourself to a computer to survive the journey.Our best guess is that it is physically possible to launch colonizing probes to the entire reachable universe using a surprisingly small amount of time and energy (e.g., only ~6 hours of the sun’s energy output), see: Eternity in six hours: Intergalactic spreading of intelligent life and sharpening the Fermi paradox. However, this paper assumes sending out tiny self-replicating Von-Neumann probes, with replicators the size of acorns: if you want to go out in the initial wave, you need to accept transportation on a tiny flashdrive, and robots to physically construct a new body at the destination galaxy. Alternatively, you can travel with your physical body at substantially greater expense.Our best guess is that it is physically possible to launch colonizing probes to the entire reachable universe using a surprisingly small amount of time and energy (e.g., only ~6 hours of the sun’s energy output), see: Eternity in six hours: Intergalactic spreading of intelligent life and sharpening the Fermi paradox. However, this paper assumes sending out tiny self-replicating Von-Neumann probes, with replicators the size of acorns: if you want to go out in the initial wave, you need to accept transportation on a tiny flashdrive, and robots to physically construct a new body at the destination galaxy. Alternatively, you can travel with your physical body at substantially greater expense.

2. If you hate the idea of cryosleep or uploading, or you want to visit Earth regularly, you should get property in the Solar System. If those don’t bother you, but you’re worried about nearby aliens, get property in the Milky Way or a nearby galaxy. Otherwise, why not claim a distant galaxy for maximal space?

3. With the advent of nanotechnology, the only limits are those provided by mass and space. The robots can terraform whatever property you choose into whatever you want it to be, long before you arrive.

4. Even if you “only” choose to have a terraformed asteroid, there’s no way you can enjoy all of it alone. You can turn it into a giant space mansion if you want, but even in ten thousand years you’ll never visit every room. Giving yourself vast estates, impossibly good food, and every other imaginable luxury is table stakes; if you are purely selfish,By selfish here, we mean that you also don’t care about creating new clones of yourself. If you cared about making clones of yourself, then you would do that instead.By selfish here, we mean that you also don’t care about creating new clones of yourself. If you cared about making clones of yourself, then you would do that instead. you should sell your rights to distant space resources and enjoy all of these things on Earth or a nearby space station. Distant space property will only be useful for people with scope-sensitive preferences, who care about implementing something at a vast scale.

5. You can, if you want, design a utopian society to your specifications. Once the Von Neumann probes reach your property, they’ll build it for you. You can set the initial conditions and let them grow, reflect, and flourish on their own. If you choose to travel there, you will find them waiting for you. If you stay home on Earth, you can feel the warm glow of knowing that they exist.

6. Most people don’t have an individual view of the Good different from every other human’s view. You might want to share coordination and design labor by forming Trusts united by a common vision. These Trusts can pool their space resources and create vast societies spanning multiple galaxies.

7. The only restrictions are the Universal Rights - no torture, no slavery, etc. And of course, space property rights: If you try to build a military to conquer other people’s star systems, or trigger vacuum decay or build a galaxy-destroying bomb, we will crush you. Otherwise, the only limit is your imagination. (That said, note that with superintelligent assistance, other people will probably be able to eventually figure out what you did, and judge you for it.)

People with space property are forced to reason about their ethics on the deepest level, some of them for the first time. Superintelligent assistants talk them through the implications of their beliefs, and forecast how various choices will end up. Some, overwhelmed with the task at hand, take intelligence-enhancing drugs or upload themselves to help them process all the possibilities; others deliberately refuse this option, feeling like anything that separates them from unaltered humanity will hinder them in their quest to bring the truest and deepest human values to the stars.

The resulting plans are too many to number. There are all sorts of schools of thought about what to do with your galaxy of resources, including:

• Defer to the Future: Some people decide to give most of their resources to their children (or other people in the future), and let them choose.

• Human Flourishing: Set up worlds of normal, free people living ordinary lives. This is a harder problem than it first seemed, because human civilization has already become profoundly abnormal: by this point, human activities like art or science has already become dominated by machines. Some handle this by accepting that their galaxy will be filled with post-scarcity humans, able to pursue whatever interests they find most meaningful. Others decide to roll back technology to leave space for human labor.

• Digital Human Flourishing: If humans living happy, flourishing lives is good, then why not increase the number of people living these lives? Brain emulations can be run for vastly cheaper than real humans: a planet-size computer could plausibly simulate the equivalent of a million planets with happy civilizations.

• Acausal Trade: Some argue that there likely are aliens in distant unreachable galaxies and that we could use various mechanisms to make deals with these aliens. If this is true, they argue, we could cut a deal to pursue compromise values rather than narrowly maximising our own values. Therefore, we might, on net, be able to do the most good by pursuing not just our own values, but instead a compromise of a vast number of other civilizations.

Humanity spreads across space in a dizzying variety of forms and ways of living—more diverse than anything Earth alone could have produced.

The people who’ve stayed on Earth, either because they sold their share of space or because they choose to live as absentee landlords, face challenges of their own. The biggest question is what to do with all their free time. A few people find work in niches that AI labor cannot fill even in principle (priests, athletes, artists). Others enjoy lives of leisure. Competitive leagues spring up around everything from synthetic biology to language creation. Groups of thousands (or more!) collaborate on projects that are so huge and ambitious they would have been inconceivable before. Communities form around shared interests that once would have been too niche to sustain a club, let alone a civilization.

Scarcity is not entirely gone: some prestige goods like land in trendy neighborhoods are inherently limited, and people who can’t handle abundance find various ways to manufacture artificial inequality. Some people are unhappy with the direction the majority chose, and some wish they’d gotten a bigger slice of the pie. But overall, everyone has at least a galaxy’s worth of resources to do with as they see fit. For most people, life is so good that the world of 2026 would be hard to imagine, except perhaps as a historical simulation.