My Takeaways From AI 2027

...

Here’s a list of things I updated on after working on the scenario.

Some of these are discussed in more detail in the supplements, including the compute forecast, timelines forecast, takeoff forecast, AI goals forecast, and security forecast. I’m highlighting these because it seems like a lot of people missed their existence, and they’re what transforms the scenario from cool story to research-backed debate contribution.

These are my opinions only, and not necessarily endorsed by the rest of the team.

Cyberwarfare as (one of) the first geopolitically relevant AI skills

AI will scare people with hacking before it scares people with bioterrorism or whatever. Partly because AIs are already showing especially quick progress at coding, partly because it doesn’t require lab supplies or bomb-making chemicals, and partly because there are more hackers than would-be-terrorists.

If AI masters cyberwarfare, there will be intense pressure for government to step in. That’s bad for open-source (it’ll be restricted unless they find some way to guarantee the models can’t be trained to hack), bad for the people who want to pause AI (we can’t let China’s army of auto-hackers get ahead of ours!) and ambiguous for the AI companies (we don’t predict they’ll get fully nationalized, but they’ll end up in the same bucket as uranium miners, Middle Eastern fertilizer factories, etc). But it’s good for biosafety; governments will have to confront tough security questions around AI when they first master hacking; by the time they master bioweapon production, some sort of regulatory framework may already be in place. The scenario is agnostic about whether some early bioterrorist could get lucky and get a small boost from a marginal model. But it doesn’t expect them to have easy access to true superintelligence.

A period of potential geopolitical instability

If America has nukes and is willing to use them, and Russia doesn’t, then America automatically wins every conflict. So if you’re Russia, and you hear America will get nukes next year, what do you do? You either surrender, or try some desperate gambit to destroy their nuclear program.

Likewise, if you’re America, you’ve got nukes, and you know Russia will get nukes next year, what do you do? You can either nuke them now and automatically win, or you give up your advantage and have the whole Cold War. Von Neumann really wanted to nuke them in 1947 and win automatically. We didn’t do that because we weren’t psychos, but the logic is sound.

If true superintelligence is possible, then it’s a decisive strategic advantage in the same sense as nukes. You don’t even have to be a psycho - maybe you can use it to cause a bloodless regime change. So if you get it first, there’s a strong incentive to use it right away. And if you’re on track to get it second, there’s a strong incentive to flip the gameboard so that doesn’t happen.

If everybody realizes this ahead of time, and America is on track to get superintelligence three months before China, then there may be a period where China considers whether to lie down and die, versus do something dramatic (kinetic strikes on US data centers?) In a best-case scenario, this provides an opportunity for a deal, maybe enshrining an peaceful international AI effort. You can decide how likely you think that one is.

The software-only singularity

Skeptical futurists expect two types of bottlenecks to restrain the singularity. There are bottlenecks to AI progress (eg compute) that prevent you from rocketing to superintelligence too quickly. And there are bottlenecks to automation (eg factory build times, regulations) that prevent AIs from changing the economy too quickly. Take both bottlenecks seriously, and you get a long feedback cycle where AIs get a little more intelligent, automate a little more of the economy (including chip factories), use that to get a little more intelligent still, and make a gradual takeoff over the course of decades.

AI 2027 objects to the first bottleneck: smarter researchers can use compute more efficiently. In fact, we know this is happening; about half of all AI scaling since 2020 has been algorithmic progress, where we get better at using the compute we have. If we hold compute constant, but get 10x algorithmic progress (because of the intelligence explosion), then we get 5x overall AI improvement.

The skeptics counter-object: the research to speed algorithmic progress is itself bottlenecked by compute. Researchers need to do experiments to determine which new algorithms work and what parameters to give them. It might be that smarter researchers could figure out how to use this compute more efficiently, but then you don’t get an intelligence explosion until your AIs are already smarter than human researchers - ie when you’re already past AGI.

AI 2027 disagrees. Although the counter-objection is directionally correct, there are little ways intelligence can boost speed even when compute is held constant. How do we know? Partly through armchair attempts to enumerate possibilities - for example, even if you can’t speed up by adding more researchers, surely giving the same researchers higher serial speed has to count for something. And partly because we surveyed AI researchers and asked “if you had a bunch of AIs helping you but only the same amount of compute, how much faster would your research go?” and they mostly said somewhat faster. All these little boosts will compound on themselves in typical intelligence-explosion fashion, and when you game it out, you get a one-year-or-so takeoff to superintelligence.

Here you’ve avoided bumping up against most of the real-world physical bottlenecks to automation (factory build times, regulations, etc); you have a data center full of superintelligences in a world which is otherwise unchanged. You might not even have very good consumer-facing AIs (we think the AI companies probably won’t release many new models mid-intelligence-explosion; they’d rather spend those resources exploding faster).

Later, when we do try to model automation speed, we’re asking what happens when full superintelligences get unleashed on a normal human world - rather than what happens when 30%-smarter AIs try to automate a world optimized by 25%-smarter AIs.

The (ir)relevance of open-source AI

In the scenario, the leading companies’ AIs are a year or two ahead of the best open-source AIs (this isn’t a bold prediction - it’s true now - we only say the trend will not change).

But in the scenario, the intelligence explosion only takes a year or two. So by the time the leading companies’ AIs pass the human level, the open-source AIs are only somewhat better than the best AIs today. That means they aren’t an effective check on post-intelligence-explosion superintelligences.

It might be even worse than that; once AI becomes good at cyberwarfare, there will be increased pressure on companies like Meta and DeepSeek to stop releases until they’re sure they can’t be jailbroken to hack people. If that’s hard, it could slow open-source even further.

AI communication as pivotal

In the misalignment branch, AIs stop using English chain of thought and think in “neuralese” - a pre-symbolic language of neural weight activations (do humans do this? is this the same as the mentalese hypothesis?). They communicate by sending neuralese vectors to each other (sort of like humans gaining a form of telepathy that lets them send mental states through email). This is good for capabilities (neuralese is faster and richer than English) but dooms alignment. Not only can researchers no longer read chain-of-thought to see if the model is scheming, they can no longer even monitor inter-AI communication to check what they’re talking about (for example, “hey, should we kill all humans?”)

In the humanity-survives branch, companies realize this is dangerous, take the capabilities hit, and stick with English. They monitor chain-of-thought and inter-AI communication (or more realistically, have too-dumb-to-plot AIs like GPT-4 do this). These heavily-monitored AIs are never able to coordinate a successful plot, and invent good alignment techniques while still under human control.

When real-world researchers debate whether or not to implement neuralese, we hope they think “Hey, isn’t this the decision that doomed humanity in that AI 2027 thing?”

(or if we’re lucky, the tech level it takes to implement neuralese will also provide us with too-dumb-to-plot GPT-4-style neuralese interpreters, in which case we could try monitoring again).

Ten people on the inside

Title comes from this LessWrong post, but it was the impression I got from AI 2027 too. If things go this fast, there won’t be time for a grassroots-level campaign for safety, or even for safety-related legislation. Whether or not the AI is safe will depend on company insiders. First, the CEO/board/leadership and how much they choose to prioritize safety. Second, the alignment team, and how skilled they are. Third, the rank-and-file employees, and how much they grumble/revolt if their company seems to be acting irresponsibly.

(I suppose the national security state would also have the opportunity to object - but it doesn’t seem like the sort of thing they would do)

This is one reason I oppose the campaigns that have sprung up recently to get safety-conscious people to quit AI companies. I’m tempted to push the opposite - are we sure we shouldn’t be pushing safety-conscious people should be trying to join AI companies as fast as possible? Maybe not if you’re some genius whose presence would massively accelerate capabilities research. But if you’re replacement-level or only slightly above? Sure.

(this claim has not been checked with smart people, and you should run it by experts who have thought about it more before acting on it. Still, I want to get it out there as something to think about before the everyone-should-quit campaigners fill up the space.)

But this also means big possible gains from getting anyone other than ten people on the inside involved. For example, if labs can commit to, or be forced into, publishing safety cases, that brings the number of eyeballs on their plans from tens to hundreds.

Potential for very fast automation

I have to admit I’m skeptical of this one, but Daniel and the other forecasters have done their homework, and I can only object based on vague heuristics.

History provides examples of very fast industrial transitions. For example, during WWII the US converted most civilian industry to a war footing within a few years. The most famous example is Willow Run, where the government asked Ford to build a bomber factory; three years after the original request, it was churning out a bomber per hour.

How did Willow Run move so quickly? It had near-unlimited money, near-unlimited government support, talented people in charge, and the ability to piggyback off Ford’s existing capacity to build and staff factories.

We imagine the first superintelligences in their data centers, chomping at the bit to transform the economy. Aligned superintelligences will want this - the faster they automate the economy, the faster they can cure cancer and produce limitless prosperity. So will unaligned superintelligences - the faster they automate the economy, the sooner they can build their own industrial base and kill all humans without the lights going out. So they plot a tech tree - probably starting with humanoid robot workers, automated bio labs, 3D printers, and other techs that speed up future automation. Then they ask for money, government support, and factories (talent, obviously, is no issue for them).

We predict they get the money - if you get an opportunity to invest in a superintelligence during the singularity, obviously you say yes.

We predict they get the government support - if China is also approaching superintelligence, and the difference between full superintelligent automation and half-hearted superintelligent automation is a GDP growth rate of 25% vs. 50% per year, then delaying more than a year or so is slow-motion national suicide. But also, persuasion and politics are trainable skills - if superintelligences are better than humans at all trainable skills, we expect them to generally get what they want.

And we predict they get the factories. This is maybe overdetermined - did you know that right now, in 2025, OpenAI’s market cap is higher than all non-Tesla US car companies combined? If they wanted to buy out Ford, they could do it tomorrow.

So maybe the three year pivot to a war footing is the right historical analogy here. Then AI 2027 goes further and says that if 1940s bureaucrats can do it in three years, then superintelligence can do it in one - though like I said, I have to admit I’m skeptical.

Most of this - plus the final calculations about exactly how many robots this implies getting manufactured when - is well-covered in Ben Todd’s How quickly could robots scale up?

Special economic zones

In the context of the software-only singularity - where you start with some superintelligences on one side and the entire rest of the economy on the other - this looks like a natural solution. Give them some land - doesn’t matter if it’s a random desert, they’re AIs - and let them tile it with factories without worrying about the normal human regulations.

You can’t do everything in SEZs. At first, you might be limited to existing car factories (probably in Detroit or somewhere), staffed by human laborers in a normal city. But they’re a good next-stage solution. And you might be able to make them work for some of the first stage (e.g. through small SEZs covering a few blocks in Detroit).

Superpersuasion

We had some debates on whether to include this one; people get really worked up about it, and it doesn’t change dramatically affect things either way. But we ended up weakly predicting it’s possible.

Persuasion / charisma / whatever you want to call it is a normal, non-magical human skill. Some people are better than others at it. Probably they’re better because of some sort of superior data efficiency; they can learn good social skills faster (i.e. through fewer social interactions) than others. A superintelligent AI could also do this. If you expect them to be inventing nanobots and starships, yet unable to navigate social situations, you’ve watched too much 1960s sci-fi.

(don’t imagine them trying to do this with a clunky humanoid robot; imagine them doing it with a videoconferencing avatar of the most attractive person you’ve ever seen)

If persuasion “only” tops out at the level of top humans, this is still impressive; the top humans are very persuasive! They range from charismatic charmers (Bill Clinton) to strategic masterminds (Dominic Cummings) to Machiavellian statesmen (Otto von Bismarck) to inspirational-yet-culty gurus (Steve Jobs) to beloved celebrities (Taylor Swift). At the very least, a superintelligence can combine all of these skills.

But why should we expect persuasion to top out at the level of top humans? Most people aren’t as charismatic as Bill Clinton; Bill is a freakish and singular talent at the far end of a charisma bell curve, the same way Usain Bolt is a freakish and singular talent at the far end of an athletic bell curve. But the very bell curve shape suggests that the far end is determined by population size (eg there are enough humans to expect one + 6 SD runner, and that’s Usain Bolt) rather than by natural laws of the universe (if the cosmic speed limit were 15 mph, you would expect many athletic humans to be bunched up together at 15 mph, with nobody standing out). For the far end of the bell curve to match the cosmic limit would be a crazy coincidence (and indeed, the cosmic speed limit is about 10,000,000x Usain Bolt’s personal best). By the same argument, we shouldn’t expect the cosmic charisma limit to be right at the +6 SD level with Clinton.

We worry that people will round this off to something impossible (god-like ability to hypnotize everyone into doing their will instantly), then dismiss it - whereas it might just be another step (or two, or three) along the line from you → the coolest kid in your high school friend group → a really good salesman → Steve Jobs. Or if you wouldn’t have fallen for Steve Jobs, someone you would have fallen for. Your favorite influencer. Your favorite writer. “Oh, but I only like my favorite writer because she’s so smart, and thinks so clearly”. Don’t worry, if you’re not fooled by the slick-hair and white-teeth kind of charisma, there’ll be something for you too.

This skill speeds things up because AIs can use it even before automation (including to build support for their preferred automation plans). But the scenario is overdetermined enough that it doesn’t change too much if you assume it’s impossible.

Which are the key superintelligent technologies?

If AIs invent lie detectors (for humans), international negotiations get much more interesting. What would you be willing to agree to, if you knew for sure that your rivals were telling the truth? Or are there ways to fool even a perfect lie detector (the deep state lies to the President about the real plan, then sends the President to get tested)? Solve for the equilibrium.

If AIs invent lie detectors (for AIs), then alignment becomes much easier. But do you trust the AIs who invented and tested the lie detector when they tell you it works?

If AI can forecast with superhuman precision (don’t think God, think moderately beyond the best existing superforecasters), maybe we can more confidently navigate difficult decisions. We can ask them questions like “does this arms race end anywhere good?” or “what happens if we strike a bargain with China using those lie detectors?” and they can give good advice. Maybe if ordinary people have these superduperforecasters, and they all predict impending technofeudalism, and they all agree on which strategies best prevent the impending technofeudalism, then civil society can do better than the usual scattered ineffectual protests. Maybe we ask the AIs how to create meaning in a world where work has become unnecessary and human artistic effort irrelevant (hopefully it doesn’t answer “lol you can’t”).

If AI is superpersuasive (as above), then whoever controls the AI has unprecedented political power. If technofeudalists or autocrats control it, guess we all love Big Brother now. If nobody controls it (maybe somehow the AI is still open-source) then we get . . . what? Something like the current Internet on steroids, where sinister influencers build cabals of people brainwashed to their own point of view?

What about AI negotiation? Might AIs be smart enough to take all positive-sum trades with each other? Might they benefit from new enforcement mechanisms, like agreements to mutually edit their weights to want to comply with a treaty? Could you use this to end war? Could you accidentally overdo it and end up locked in some regime you didn’t intend?

What about human intelligence enhancement? We may never be as smart as the AIs, but a world of IQ 300 humans advised by superintelligences might look different from IQ 100 humans advised by superintelligences. Would we be better able to determine what questions to ask them? Would society be more equal (because cognitive inequality is eliminated)? Less equal (because only the rich enhance themselves)? What about conscientiousness enhancement, agency enhancement, etc?

AI 2027 is pretty vague on social changes after the singularity, partly because it depends a lot on which combination of these technologies you get and when you get them.