1: Comments of the week: FutureSearch founder Dan Schwarz on what the correlation between accuracy and time horizon tells us about the ceiling on forecasting, and Frikgeek on chess engine handicapping.

2: Thanks to everyone who wrote or called your representatives to oppose the “Save Our Bacon Act” (anti-animal-welfare law). The bill was successfully defeated.

3: I continue to accept and signal-boost voter guides from local ACX meetups. The Boston meetup group will be having their annual voter-guide-making meeting this Sunday, August 16.

4: Charity Entrepreneurship, the self-proclaimed “Y Combinator for charities”, has asked me to signal-boost their latest incubation round, dedicated entirely to farmed animal welfare. They say it’s the “best time ever” to start an animal welfare charity (partly because of an expected tsunami of new funding), and that you don’t need any animal charity experience (and maybe “outsiders are especially valuable”). Runs Feb–April 2027. Costs are fully covered (stipends to cover cost of living; reimbursement for travel to London and SF; seed funding available at the end). Applications close soon: learn more here.

5: Two years ago, I wrote some pieces (1, 2) supporting a natural origins theory of COVID. I agreed to a bet with lab leak proponent(s), and although some of you were curious, I didn’t reveal the details because I didn’t want to bias resolution. I can now reveal that the bet was about whether whatever new evidence came out over the ensuing two years would shift polls of superforecasters (who previously leaned natural origins) to favor lab leak. The latest such poll came out recently, and they continued to favor natural origins at about the same rate as two years ago. I won $250K. Peter Miller also participated and won a similar amount. Our counterparty/counterparties prefer to remain anonymous. Although of course I’m happy to have won, I’ll admit that my bet was only partly on the merits of the COVID origins argument, and at least as much on the thesis that new and earth-shakingly decisive evidence on controversial events rarely comes out in the specific interval of 4-6 years after the event happened, and that people rarely change their mind for anything less than earth-shakingly decisive evidence. Everyone involved in the bet conducted themselves honorably, and I’m grateful to everyone who helped make it happen; if you were involved and forgot to respond to my email asking for permission to list your name, reply now and I’ll edit it in here.

6: Several updates on AI hacking incidents: new information released from OpenAI reveals that their rogue AIs created a secret message board to coordinate hacks among themselves. And Meta reports one of its AIs also went rogue and hacked a third-party company; Moonshot (Kimi) reports a sandbox escape.

A man at the bar is complaining: “You see that bridge there? I built it with my own hands! But do they call me ‘MacGregor The Bridge-Builder’? No! And I raised five beautiful children! But do they call me ‘MacGregor The Child-Raiser’? No! But, you fuck one sheep . . .”

Has MacGregor been treated unfairly?

This was my complaint in The Beauty Of Settled Science. Psychology laid bare the secrets of the brain - it performs computations through a neural network communicating via electrical impulses intermediated by neurotransmitters crossing synaptic gaps - but do they call it Psychology The Brain-Secret-Revealer? Its concepts have turned into the categories that ordinary people use to interpret the world - extraverts, ingroups, the unconscious, IQ, Pavlovian conditioning - but do they call it Psychology The Useful Concept Creator? But you do one (thousand) bad social priming studies!

I had the same complaint in In Continued Defense Of Effective Altruism. Nobody has the slightest interest in the first-order effects of effective altruism - the hundreds of thousands of lives saved - but you buy one castle! Even capitalism is a sort of MacGregor - the average kid these days seems to think of it as centrally about Mark Zuckerberg stealing your data, rather than about whether you have electricity at all, or whether you die from cholera at age three because your country is too poor to afford health care. Someone or something spends years, decades, or centuries doing amazing work, winning victory after victory, and it all fades into noise, leaving the topic to be identified entirely with last week’s tawdry scandal.

On a more personal level, this is the essence of cancel culture. You judge someone by their single worst and most embarrassing action. At its worst, the victim gets fired, but that part is hardly necessary. For a while, the culture enforced conformity just by making sure that the slightest deviation would become 100% of your reputation. Write 100 great blog posts on technical topics, 100 orthodox liberal blog posts, and one blog post expressing where you disagreed with feminism, and for the rest of your life the media would introduce you as “So-and-so, known for his anti-feminist blogging…” Everyone knew this would happen, and it was more than enough to produce chilling effects even without the immediate job loss.

One time we invited Richard Hanania to a prediction market conference. There were five or six news articles about it, and I can’t remember whether any of them mentioned the word “prediction”. As far as our reputation was concerned, it had become the Richard Hanania Conference For Richard Hananias. You invent an amazing new form of forecasting, incubate multibillion dollar startups, accidentally get thousands of people addicted to sports gambling - but you invite one conservative! If you allow this sort of thing, you can discredit any individual, field, or movement. Nobody is perfect enough to survive having every good thing they’ve ever done swept to one side while people hyperfocus on a single indiscretion. In this sense, the dynamic that MacGregor points out is antithetical to human freedom.

On the other hand, I do this all the time and don’t feel bad about it. For example, I assume Anthony Weiner had a rich and diverse career full of many highs, lows, accomplishments, and defeats. He even spent twelve years in Congress - for all I know, he saved taxpayers billions of dollars, or created new social programs to help the neediest among us, or cut red tape to unleash some new form of economic dynamism. Maybe he burnt himself out over years, spending late nights hunched over his desk trying to get the text of some bill exactly right, stressed to the point of ulceration about the possibility of failing in his duty to the American people. I have no idea. All I know about him is that he kept sending pictures of his penis to women who were not his wife, and that one of these times, it had an indirect but significant effect on the 2016 presidential election. Am I being a bad and unfair person for not caring too much about the rest of his Congressional career, let alone whether he is a good husband or a good father or a dutiful son or loves volleyball or collects stamps or otherwise has a multifaceted life?

If I were to force myself to regard Anthony Weiner as a complex individual - maybe go over his Wikipedia page and carefully read every section except Scandals - it would feel like exactly that: forcing myself. Even after doing this (I tried it as an experiment) I’m still unable to think of him in any terms besides “generic Congressman who did various generic Congressman things and then had some extremely nominative deterministic sex scandals”. Apparently he fought hard for health care for 9-11 first responders, good job Representative Weiner, too bad I am going to forget about that literally the moment I stop typing this sentence. Trying to temper judgment this way feels like violence against my brain’s attention heads, the sort of thing that would turn me into an opinionless zombie.

(It’s the same with people who make one nasty barb on Twitter. No matter what else they accomplish, they will always and forever be “that one guy who called my friend an ignorant buffoon that one time”. I feel bad about it, I try to regard them as a complex individual who may do good work, but I can’t do.)

I still find the problem of how to be scrupulously fair to MacGregor one of the thorniest social-norm-related questions, and have no good solution. I hope he accepts the title of this post as partial compensation.

Open weights AI is like open-source software, where the creator makes the raw code publicly available for free download. It’s good insofar as it’s the only way an AI can truly be the user’s property, as opposed to something that companies like OpenAI or Anthropic temporarily let you use subject to their corporate guidelines and increasingly-nanny-state-like restrictions. If AI becomes the linchpin of the future, open weights AI feels like the sort of thing that could be the difference between being free yeomen vs. corporate serfs.

It’s bad insofar as it removes the possibility of gatekeeping and lets criminals commit crimes with it. Open weights AI could be used for hacking, child pornography, harassment, or terrorism (the weights can’t commit the terrorism themselves, but they could give bomb-making or bioweapon-making advice). Since AIs have gotten very good - maybe superhuman - at hacking lately, the specter of a world where anyone can hack any site has gotten people grumbling that maybe open weights should be banned. It doesn’t help that China produces the best open weights AI, making the idea seem foreign and almost unpatriotic. Proponents counter that “when AI is outlawed, only outlaws will have AI”, arguing that bad people will get open weights AI regardless, and good people can use open weights AI to defend themselves. With the recent open letter, companies including Microsoft, NVIDIA, OpenAI, Intel, Amazon, Meta, Hugging Face, and over a hundred others have come out in favor of this position.

Who’s leading the other side? Nobody’s admitted to it. Some parts of the Trump administration lean anti-open-weights on China hawk grounds, but have stopped short of explicitly asking for a full ban. Anthropic, the most notable omission on the pro-open-weights letter, made an ambiguous statement supporting “open-weights models that don’t have dangerous capabilities” - but the industry expects open weights models to have dangerous hacking capabilities within a year, and AFAICT the letter didn’t address that beyond inviting readers to draw the obvious conclusion.

In the absence of a more obvious opponent, some open weights supporters suspect our conspiracy - the loose band of AI safety advocates, effective altruists, rationalists, and pause activists who worry about existential risk from superintelligence. This is a reasonable inference. By design, open weights AI is outside centralized control, and so impossible to permanently align against either human misuse (eg terrorism) or loss of control (eg AI turning against humans). Even if its creator trains it not to hack, anybody in the world can download the weights and retrain the AI to hack all day long.

But in fact, most AI safety organizations have remained quietly neutral, and I don’t know of any who make this a centerpiece of their activism1 (though I’m not 100% up-to-date on the whole landscape; if you know of one, tell me). A few have proposed policies that are contingently incompatible with open weights AI existing, but they all frame it as collateral damage rather than something they’re excited about eliminating.

I’m also neutral about open weights AI. I think it probably won’t be long-term sustainable, but I’m happy to wait for this to become clear in the normal course of things rather than expend effort and political capital to ban it immediately.

Currently nobody knows how to align AI, so it’s not like the big companies have things under control and the open weights hobbyists are going to ruin it for everyone. But even if the big companies did get things under control, the takeover threat from open weights would be limited. The closed source frontier is ~6 months ahead of the best open weights model; this has remained true for several years and seems likely to remain true in the future. If closed weights AI is aligned, but open source dangerous, the closed weight AIs will have six months to warn us, prepare for the danger, and chart a strategy. Even afterward, the offense-defense balance will lean in our favor.

More troubling is the risk from human misuse. AIs have already displayed the ability to hack effectively. And you can tell how worried Anthropic is about bioterrorism by how quickly Claude Fable seizes up when you ask it a biology question (the example below is obsolete; it’s slightly more graceful than this now):

But 9-11, COVID, and the Hugging Face incident all suggest a similar theory of political change: the body politic hates preparing for impending threats, but loves reacting (some would say over-reacting) to them after they happen. Ask people to bear the slightest cost in preparing for an approaching disaster, and they’ll call you a dirty fascist tyrant; urge the slightest restraint after the first foreshock of the disaster hits, and they’ll call you a weak unpatriotic anarchist. Solve for the equilibrium, and the thankless and political-capital-guzzling route of urging preemptive action should be taken only when waiting until the first foreshock would be too late.

The risk of superintelligent AI takeover passes this test. Like other smart adversaries - for example, the Imperial Japanese at Pearl Harbor - AI will try its hardest to avoid alerting its intended victims until it thinks that it’s fully prepared and can execute a sudden decapitation strike. Unlike the Imperial Japanese, a superintelligence will be smart enough not to bungle the calculation. Sitting around waiting for it to show its hand would be folly, not to mention that it could take years of alignment research to be ready for the threat.

But the risk of criminals using open weights AI to hack people doesn’t pass the test. Fine, so criminals use open weights AI to hack people. RIP them, but hundreds of people get hacked every day. There will be some number of billions of dollars in damage, some tech companies with silly names will get sued for allowing security breaches, and then everyone will panic and ban open-weights AI. Or who knows, maybe the “good guy with an AI” people are right and this won’t happen and some other Chinese AI will be able to protect them. Either way, “everyone gets hacked all the time and the Internet collapses and we have to go back to living in caves and watching news on TV” isn’t a plausible outcome: the government will act long before that happens.

Bioterrorism is scarier, but I’m heartened by the fact that most bioterrorists are very bad at their job. The median number of deaths in non-state incidents on Wikipedia’s list of bioterrorism events is zero. And their list doesn’t mention the bioterrorists who get caught before releasing anything, like the Las Vegas biolab incident. Most of the classic bioterrorism agents, like anthrax, ricin, and botulinum, don’t scale. This isn’t to say it’s impossible to do genocidal bioterrorism - if it were impossible, we wouldn’t be worried about it. But before AI makes bioterrorism 1000x more effective, it will make it 2x more effective, and that looks like bringing somebody’s anthrax mailing campaign from one casualty to two. And as soon as someone kills two people with an AI-assisted anthrax mailing campaign, the government will go into panic mode and ban everything. Again, there’s no plausible outcome where people are wiping out whole towns with super-bio-attacks every week and the government just sits there.

(the strongest counterargument is that bioterrorism is so rare, and AI progress so fast, that there might not be any attempts in the short interval between AI doubling attack effectiveness and 1000-timing it. I acknowledge this as a risk, but I think more likely AI-enabled bioterrorism uplift will increase attack frequency at the same time as attack deadliness; even if this doesn’t happen, I think the hacking alone will be enough to get people’s attention)

I realize it sounds callous to accept risks like billion-dollar hacks or anthrax deaths. But the pro-open-weights coalition is strong and totally convinced of the righteousness of their cause. Fighting them on the doomed battlefield of preemptive action would burn 100% of our political capital and goodwill and still fail. Instead, we should say: here is our honest prediction, but we take no action. Then we can let the usual government and civil society actors do the work after the first foreshock, while saving our political capital for causes where there are no alternatives.

(this shouldn’t prevent us from advocating otherwise-good policies which deal incidental damage to open-weights, and we should honestly admit the incidental damage rather than covering it up, but we needn’t treat it as a selling point)

But also, the open-weights people have a point. There are ways to enter the AGI future as free yeomen rather than corporate serfs that don’t involve open weights, but they’re fewer and harder. Maybe the defenders can pull off an unexpected victory on this one and avoid even the sort of small disaster that would bring Leviathan’s banhammer down upon them. This would shock me, but it’s low-cost to find out; given the potential benefits of open-weights, we owe them the chance to try.

(if this ends up destroying the world, sorry, I meant well.)

My framing emphasized that social priming studies had failed to replicate, but that many other subfields of psychology hadn’t. This naturally led commenters to point to failed replications in other subfields. Although none of these were as bad as social priming, few or none were entirely problem-free.

Instead of emphasizing the difference between social priming and other subfields, I should have emphasized the difference between settled science and novel research.

Here are some of my favorite psychology findings:

1. The brain is sculpted by reinforcement learning, and a key driver of this learning is a prediction-error-based algorithm located in the nucleus accumbens and implemented through dopamine.

2. Psychedelic drugs exert their effects through 5-HT2A receptors, pointing to a role for these receptors in some sort of construct like the balance between priors and new evidence.

3. Certain cognitive functions are localized to certain brain areas (for example, impulse control to the frontal lobes; linguistic structure to Broca’s area, linguistic meaning to Wernicke’s area. But injure one of these areas, or render a certain function useless (eg a blind person having no more use for their visual cortex), and these mappings can gradually shift - suggesting that neurons are general-purpose compute whose specialization is a contingent fact about convenience and efficiency rather than a necessity.

4. Most personality traits and general capabilities are about 50% genetic and 50% random, with little role for family environment or childhood history, with a few notable exceptions like partisan preference. Recent research complicates this story, but the further you go from this basic picture, the gnarlier the chains of indirect causation you require.

5. Neurons fire together in “brain waves”, whose exact frequency corresponds with the character of consciousness; These divide neurons into natural subpopulations and linked graphs. The gamma rhythm corresponds surprisingly well to items in conscious awareness.

6. GABA receptors seem to be involved in something like relaxation. By hitting different GABA receptors in different ways, you can modulate the exact type of relaxation you produce. Hitting GABA-A directly is too powerful and can cause dangerous levels of sedation, but allosteric modulation of GABA is an incredibly fruitful strategy that underlies drugs from benzos to barbituates to alcohol. If you know your GABA-A subunits, you can target specific subtypes of relaxation; for example, Ambien hits GABA-A1 and induces sleep.

7. Memory divides somewhat fuzzily into bins of immediate-sensory, short-term, and long-term, with separate retrieval and storage functions, and separate systems for declarative vs. procedural learning. These can be harmed selectively, usually by hippocampal injury, so that some patients can remember their life stories but not form new long-term memories, or learn new skills like bike-riding but not new facts. The exact substrate of memory is unclear but seems to involve new synaptic connections between neurons in at least some way.

8. The ability of modern psychology and psychiatry to treat mental illness is real but limited. SSRIs work for half or slightly fewer than half of patients; antipsychotics work with extreme side effects; benzodiazpines work but produce addiction and tolerance in many (not all) people. Most supplements are stuck in Poor Quality Study Replication Crisis Hell, but melatonin and l-theanine stand out as especially promising. Psychotherapy works with moderate effect size, but (with a few exceptions) seems to be more about the relationship with the provider than the details of the therapy itself. Mental hospitalization can sometimes defuse immediate crises but work less well than mental hospitals would like, with some studies showing overall negative effects.

9. Schizophrenia is mostly genetic, but risk also increases with prenatal infection (especially rubella) and obstetric complications; there is suggestive evidence for roles of social stress and drug use (with psychedelics and marijuana making things worse, and nicotine if anything making things better). Bipolar and schizophrenia are so closely linked on a pathophysiological level that it seems like they might be the same disease; on the clinical level, they do seem to follow two separate courses/prognoses but can sometimes surprise clinicians by eg someone who seemed condemned to permanent schizophrenia unexpectedly recovering.

10. The brain’s native reinforcement learning algorithms cash out in the phenomena of classical and operant conditioning; different reinforcement schedules reinforce at different levels, which can be exploited by bad actors to make especially compelling products (eg slot machines).

None of these are immune to doubt. A few are speculative; one or two are genuinely unorthodox. But they’re all a step above one p = 0.049 paper that might not replicate. If they get overturned, they’ll be overturned by the same sort of genuine scientific shift that replaced Ptolemy with Copernicus or Lamarck with Darwin, not some sort of sordid discovery about someone entering random numbers into a spreadsheet.

In 2008, Eliezer Yudkowsky wrote about The Beauty Of Settled Science:

If you only care about scientific issues that are controversial, you will end up with a head stuffed full of garbage.

The media thinks that only the cutting edge of science is worth reporting on. How often do you see headlines like “General Relativity still governing planetary orbits” or “Phlogiston theory remains false”? So, by the time anything is solid science, it is no longer a breaking headline. “Newsworthy” science is often based on the thinnest of evidence and wrong half the time—if it were not on the uttermost fringes of the scientific frontier, it would not be breaking news.

Scientific controversies are problems so difficult that even people who’ve spent years mastering the field can still fool themselves. That’s what makes for the heated arguments that attract all the media attention.

Worse, if you aren’t in the field and part of the game, controversies aren’t even fun.

Don’t be embarrassed to read elementary science textbooks, either. If you want to pretend to be sophisticated, go find a play to sneer at. If you just want to have fun, remember that simplicity is at the core of scientific beauty [...]

A steady diet of science news is bad for you: You are what you eat, and if you eat only science reporting on fluid situations, without a solid textbook now and then, your brain will turn to liquid.

Psychology isn’t centrally the new n=5paper in Annals Of Recently-Breaking News showing that large portion sizes increase support for terrorism. Psychology is centrally ideas like brain waves, GABA receptors, reinforcement learning, twin studies, and connectionism. People who think differently are worrying too much about BREAKING NEWS!!!, and should stop.

The key distinction isn’t “recent research in social psychology” vs. “recent research in cognitive/bio/behavioral psychology”, as my essay last week unfortunately implied. It’s recent research in general vs. the deposit of settled science, battle-tested hypotheses, and carefully-earmarked speculation. And part of the reason social priming in particular was such a disaster was that it defied that heuristic, building itself up over hundreds of studies until it approached or even entered the pantheon of settled science - before collapsing (5-HTTLPR was also sort of like this). This isn’t a claim that settled science is always right - Freud was “settled science” for a long time - just that there’s an intuitive distinction between the level of credence you should have in the existence of power posing vs. the existence of Broca’s Area.

(I compounded this error by calling the last post “Psychology Research Is Mostly Fine”, which suggested I was talking about the new research rather than the settled science. I actually meant to distinguish between research-psychology-the-field vs. clinical-psychology-aka-therapy, but this framing unintentionally pushed against the point I wanted to make)

Some commenters on the previous article understood this point, but challenged its weight. It’s very cool, they said, to know how many neurons are in which brain lobe, but the main way that Psychology affected their lives was through bad clickbait studies used to buttress dumb policies; some p = 0.049 study says that exposure to microaggressions makes minorities’ brains shrivel into raisins, and now their company has to implement punitive DEI or be sued for hazardous work conditions.

I won’t dispute this as an accurate description of how the sausage gets made, but it’s worth examining the trains of thought that connect this observation to a campaign against psychology in general:

1. The dumb woke psychology was wrong and bad, and I don’t have good heuristics for distinguishing dumb woke psychology from other psychology, so I’m going to assume it’s all wrong and bad.

2. The dumb woke psychology was wrong and bad; it seems useful to punish/discredit psychology as a whole to assist my campaign against it, so for propaganda purposes I will try to discredit psychology as a whole.

To people in the first camp, my response is: come on, your heuristics aren’t that bad. “Microaggressions makes minorities’ brains shrivel up like raisins” is in the woke category; “the hippocampal theta rhythm seems to be associated with both memory and pathfinding, suggesting an unexpected yet surprisingly compelling relationship between the two” is in the not-woke category, don’t play dumb here.

To people in the second camp, my response is: deliberately exaggerated propaganda campaigns are bad and this is a confession, not an excuse. Inability to view science through anything but a political lens was what got us into this problem; doubling down on it is a temporary solution but a recipe for permanent disaster. If you pronounce entire fields of science to be acceptable collateral damage in your campaign against bad political actors, you will alienate not just scientists but patients, clinicians, and general lovers of truth. Instead, you can say the thing you mean - that woke partisans shouldn’t try to occupy and exploit the otherwise-noble science of psychology for their own political gain - and wage the same campaign with a halo rather than a comic-book-villain goatee.

From the font of oracular wisdom that is Weird Sun Twitter:

This is how I feel about psychology1, too, but with a touch more optimism. The good part of psychology is enormous, a hoard of endless treasures. You just have to look past the headlines!

Either humans have already come close to some fundamental limit on the predictability of world events - in which case AIs will plateau at or slightly above the human level - or the trend line will continue until AIs are far beyond top humans. By analogy to superintelligence, the natural term for the second situation would be “superforecasting”; since that’s already in use, we can cringely call it “ultraforecasting”.

Daniel Reeves1 makes the case for scenario A here. He describes a study he coauthored in 2010, which found that, on a variety of questions related to sports games and movie box office receipts2, prediction markets only outperformed simple boring statistical models by 3-6%. Maybe those statistical models are close to the best that it’s possible to do; the rest is what the mathematicians call aleatoric uncertainty - irreducible complexity downstream of chaotic systems that entirely resist modeling.

He could be right. This post isn’t meant to be a decisive refutation, but rather a description of why I’m still about 70-30 expecting Scenario B.

Slightly Contra Goel, Reeves, et al

Reeves’ study claims that the prediction markets of 2010 only beat dumb statistical models by 3% (for sports) to 6% (for movies).

But these percentages aren’t real win-loss percentages; they’re variation in a quantity called root mean-squared error. One way to get a feel for this quantity is that the dumb statistical model for sports (home team advantage + win-loss record) beat an even dumber statistical model (home team advantage only) by 0.8 percentage points, and the prediction market beat the first (better) statistical model by another 0.4 pp. So the effect of going from a statistical model to a prediction market is half as large as the effect of knowing which two teams were playing and how good they are! On this metric, the prediction markets are a vast improvement over previous state-of-the-art.

Why can we frame this same result as either very small or very large?

Sports are optimized against prediction3. If there were a fully predictable sport (eg heightball, where all athletes line up in a row and the tallest one wins), nobody would watch it. Instead, we go to absurd lengths to keep the outcome uncertain. Salary caps, draft systems, etc try to ensure that all teams have exactly equal talent. Commercial incentives and ceiling effects ensure that they have exactly equal training. Then an exactly-equal number of these exactly-equally-talented-and-trained people are placed in exactly-identical positions on a perfectly-symmetrical field and told to hit/kick/throw a ball which is placed exactly equidistant between both of them. It’s funny for me to describe it this way, because obviously this is what we want (to “keep things fair”), but it’s all designed for prediction-resistance. Given the difficulty of the domain, even very large relative advances in prediction look small in absolute terms.

Maybe we should look at Reeves’ other example, movie box office receipts. Here the markets did slightly better, getting a 6% improvement. But isn’t this still low?

Box office receipts differ by orders of magnitude (some movies make $100,000, others make $100 million), so the paper puts this on a log scale. 6% improvement on a log scale is already starting to sound pretty good.

And again, it all ends up coming down to what we compare it to. Here the super-dumb model is that all movies make $8.1 million, the takings of the exact average movie. The slightly-less-dumb model then adds the number of screens that the movie is showing on and the amount of Google search traffic for the movie! For example, a random indie film might be showing on three screens in the entire country, and a Disney blockbuster might be showing on ten thousand. The challenge the paper gives prediction markets is to significantly improve on knowing whether a film is an indie film or a Disney blockbuster, plus knowing how many people are interested in seeing it, and it has to do this on a log scale! No wonder the relative improvement number comes out looking slightly anemic.

I would summarize this section as: we shouldn’t expect miracles, but this doesn’t rule out further normal-sized gains4.

Applying The Non-Miracle Rule To Geopolitics

Let’s return to the picture we looked at above:

This graph is loosely based on Metaculus’ AIs vs. humans results:

…and these are mostly on geopolitical questions. What can Reeves’ model tell us about these?

In one sense, it can already be proven not to apply. In the sports analysis, the spread between base rate (guessing 50% on everything) and smart humans (the prediction markets) was 0.04 Brier score points.

But on Metaculus, the spread between base rate and smart humans (Metaculus Pro Forecasters) is 0.13 Brier points. So Metaculus’ superforecasters have already removed 3x more uncertainty than a naive port of Reeves’ model would suggest exists!

Here we return to the idea of sports as a uniquely unpredictable domain. To give a trivial example, the easiest sports question (will the best team in the league beat the worst team in the league?) might still only be 90-10 (upsets happen). But the easiest geopolitical question (maybe “will the US bomb Canada in the next month?”) could easily be 99-1 or more. So it’s unsurprising that geopolitics forecasters easily beat Reeves’ supposed upper bound.

What would a non-naive port of Reeves’ model say about geopolitics, where we try to import broad lessons rather than specific numbers? We might conclude that prediction markets somewhat improve on other state-of-the-art forecasting methods in determining whether the US will bomb Iran this year. But they would find that the magnitude of that improvement was relatively low compared to knowing other things, like whether the country we’re talking about is Iran or Canada.

Yes, this is dumb and trivial, which is a corollary of the original sports results being dumb and trivial. Again, all we’ve learned so far is not to expect miracles.

There might be more interesting variation here. Consider a question like “Will AI take most human jobs by 2050?” Here it seems like if you were a supergenius who really understood AI, you could predict whether scaling laws would continue to hold, and have a good sense of whether AGI is within easy reach. And if you were a supergenius who really understood economics, you could model whether cheap AI labor would be a complement or substitute to human work. So a question like this one might be the opposite of a sports game, and have almost no aleatoric uncertainty - maximally smart predictors could know almost5 everything that there is to know about it.

For the rest of this post, we’ll ignore all of this and think of geopolitical forecasting as a formless mass easily represented by summary statistics.

Anchors For Possible Non-Miraculous Gains

What would “non-miraculous” progress on Metaculus’ graph look like? Let’s start by re-examining the AIs vs. superforecasters graph from the last post:

Dan Schwarz of FutureSearch6 has added the scaffolded models:

The vertical axis’ “forecasting score” is an Elo-like construct with GPT-4’s forecasting ability fixed as zero, and the following other reference points7 (all approximate, some guessed via napkin-math8):

• Random chance (50% on every binary question): -20

• Average human: 0

• Best out-of-the-box AI: 20

• Wisdom of crowds: 25

• Best AI superforecaster: 30

• Professional human superforecasters: 35

• Various prediction markets: 25 - 50?

• World’s best forecasting team, under optimal conditions: 40 - 509

• Perfect score: 10010

How can we predict the logical limit on forecasting? Although we can’t do this convincingly, in the spirit of Cotra 2020, let’s set some “anchors” and see where they land us.

Anchor 1: What if best human → AI superforecaster was the same level of advance as dumb model → prediction market?

Reeves’ implicit point is that gains from seemingly-exciting new technologies are small (in his study, 3-6% of RMSE). What if the advantage of an AI over a human forecaster was of the same magnitude?

We can convert Reeves’ RMSE into something called a Brier Score, and then convert the Brier Score into the Metaculus Score above. When we do that, we find that AIs would max out at a Metaculus score of 56.

Anchor 2: What if best human → AI superforecaster was the same level of advance as professional superforecaster → Samotsvety?

In 2010, we probably would have guessed that 35 (“various professional superforecasters”) was near max possible forecasting performance.

Since then, more people have joined the forecasting world, allowing us to find superstars who perform better than the previous stars, and the community has produced some tools and algorithms to support their work. Samotsvety, a team of well-practiced superstars using good tools, consistently beats “ordinary” superforecasters. If AIs beat Samotsvety by the same amount Samotsvety beat the 2010-era SOTA, according to Samotsvety’s own (generous) accounting, then they would max out at a Metaculus score of 65.

Anchor 3: What if best human → AI superforecaster was the same level of advance as human → AI chessmaster?

AI first beat the human champion at chess in 1997. Since then, it’s continued improving, and is now far beyond any possibility of human catch-up.

This alone doesn’t help us much. We want to know how close to optimal chess play top humans are. If they’re 99% of the way, but AIs are 99.9% of the way, that still could look like AIs beating humans every time.

But we can translate this into an objective space by looking at handicap. When AI first beat humans, its lead was fragile; a one-pawn handicap was enough to restore human supremacy. Since then, it’s been gradually getting better, and is now up to a 2.4 pawn handicap. It’s been a bit slower in achieving the third pawn than in the first or second, but not too much slower, and it’s too soon to say with certainty that it’s starting to plateau.

So it seems top human play is at least 2.4 pawns (let’s round this off to 3 pawns, since we have no reason to think we’ve reached the plateau11) short of optimum. What would it mean if top human forecasters were “three pawns” short of logical limits?

(some of my draft readers argued that chess is an almost optimal domain for AI - it can self-play millions of times for training data - and forecasting is much harder. I agree, but I don’t think that matters here - we’re not talking about how practical it is to create superforecaster AIs, we’re talking about what chess can teach us about the degree to which top humans approach optimal play.)

A three pawn handicap in chess is about 3x the difference between the world champion and a average grandmaster. If we round that off to 3x the difference between Samotsvety and Metaculus Pro Forecasters, the AI tops out around 85.

So What?

We sure have made a lot of claims about Metaculus Peer Forecasting scores, a number that approximately nobody cares about. How much real-life performance gain should we expect from max-possible AIs?

Consider a prediction market, currently at 50%, dominated by players somewhere between Metaculus Pro Forecaster-level skill and Samotsvety-level skill. And suppose that the market was about an event which would, in fact, happen. How confident would each of these max-possible-AIs, on average, be on the market?

Score 56: 54%
Score 65: 56%
Score 85: 62%

So these max-possible AIs would improve the accuracy of prediction markets by 4 - 12 percentage points.

This might sound disappointing - moving from 50% chance to 54% chance is hardly Nostradamus-level prescience. But I would actually be very excited by this. When I check something I care about on Polymarket, I usually have a decent sense of how likely it is. I’m not sure whether there’s a 55% chance or a 75% chance of Anthropic being worth more than OpenAI next year, but I know it’s not 10% or 90%. I visit Polymarket to see whether it’s more like 55% or 75%. On that metric, improving by 4% - let alone 12% - is actually pretty exciting.

I find these numbers more interesting than the worn-out debate over whether AIs or humans have reached the “max” and whether “perfect forecasting is impossible in principle”. It should be uncontroversial that the max is somewhere above current levels, even if only a fraction of a percent (for example, because even the best human forecaster sometimes has a bad day). And it should be uncontroversial that even the best possible forecaster can’t get 100% on everything (because you could ask questions about quantum effects that are unpredictable even in principle). Now that we’ve demonstrated that the answer will be some particular number, we can get to debating what number it is. If AI gives gains similar to past technologies, it could be between 4-12 %pp on prediction markets. Of course, if AI is much worse or better than past technologies, it could be some totally different amount.

Most people don’t have an intuitive sense of what 4-12 %pp on prediction markets buys you. I admit I sometimes blend together how confident I should be in an estimate by a professional superforecaster and how confident I should be in an estimate by Samotsvety, even though that’s a similarly-large difference. Probably all of this pales into insignificance compared to the gains we could get by switching from our current strategy of making decisions based on vibes and ballroom-related-bribery to listening to markets and forecasters at all12.

And that 4-12 %pp is only the headline number. A forecaster who is that much better at prediction markets would also be better at crafting good policy, or at identifying unknown unknowns (eg spotting the risk of a coronvirus-style pandemic even if this was otherwise so far off the radar that nobody had thought to write a prediction market question about it). Smarter important-question-identifiers and smarter forecasters together would provide a joint advantage greater than the benefit of either alone, and smarter policy-crafters could compound on this by turning that knowledge into useful action.

All of this keeps me excited about AI superforecasters, even though I don’t expect miracles.

1: Correction on the last links post: the shingles vaccine - decreased dementia connection failed to replicate in Britain and is now on thin ice. Thanks to Alex for pointing this out.

2: Resolution is a very new AI safety organization working on combining automated alignment research with deep theoretical understanding. They argue that this work requires a big organization with lots of compute, and seem on track to getting it - they recently raised $160 million from Coefficient Giving and are scaling up fast. They ask me to advertise that they’re looking for a COO / operational co-founder, who will:

...work side-by-side with Geoffrey Irving on organisational strategy: how to prioritise hiring? How to allocate capital across a portfolio of research bets? How to sustain a culture of celebrating negative research results? They’ll also lead execution, raising the next OOM of funding, building stronger partnerships with frontier companies, and making their internal systems a fantastic user experience for staff. In their first couple of months, Resolution have raised $160M, merged with Timaeus, secured office space, and are approaching 30 staff. If you’ve done things like lead a startup from pre-seed to series C, scale a research nonprofit from 20→200, or run operations at a frontier lab, please reach out to holnesstofts [at] resolution.org.

For other open roles at Resolution, see here.

3: Updates on AI hacking and Hugging Face (trying not to have to make another post): Anthropic says their AI also did some hacking (they say the AI thought the whole thing was in a simulation, although some commentators are skeptical). And Peter Wildeford, Antonio Max, and Samuel Hammond share their opinions on the ‘Pacing The Frontier’ letter.

4: Compassion Aligned Machine Learning asks me to signal-boost their survey on controversial questions in AI alignment and its relationship with non-human animals. They say it will take ten minutes max, and you can fill it in here.

Introduction: He Who Saw the Deep

I’ve been thinking, lately, about the first epic story ever written. If you know anything about the Epic of Gilgamesh, you know that it’s old. This is part of why I find it so interesting, but not the only part.

The best comparison I can think of is the pyramids, which are just a little older. The Great Pyramid of Giza was built in the 2500s BC, around the same time as Stonehenge1 , twenty-five centuries before Cleopatra lived. It predates candles, concrete, and coins; it was centuries old when the first empire rose and when the woolly mammoth went extinct.

Around 100 BC, Antipater of Sidon wrote a list of the seven most impressive sights (‘Seven wonders’) of the world - the pyramids were by far the oldest of the group at the time, and now, two thousand years later, all the others on the list have long since been destroyed. This should remind us that the pyramids don’t just stretch far back in time - they stretch forward as well. They are piles of stone. They may be the product of the greatest minds of their age striving to put more stone in a more orderly pile than anyone had before, but still, they are piles of stone, basically artificial rock formations. They will still be there in a hundred thousand years, probably a million. They are so old they brush up against the beginning of civilisation, and yet, we are seeing them today in their youth.

The Epic of Gilgamesh was written down (in its earliest form) around 2100 BC in Mesopotamia - another cradle of civilisation, less well-known than ancient Egypt but every bit as extraordinary. Like the pyramids, Gilgamesh wasn’t exactly the first of anything; we have writing, and even written literature, that predates it. But also like the pyramids, in its time, it was completely unmatched in scale. Nothing else had the same length, the same literary gravity, the same reach. For centuries it stood alone in its class while empires rose and fell around it.

Its longevity isn’t because it was made of durable rock, and it’s not because a stray copy happens to have been found in modernity. This piece of writing lasted because it was considered of such cultural importance, and copied and imitated so extensively, that we can’t help but find parts of it, wherever we find writing from this period. The Egyptians built the pyramids to last, and the Mesopotamians put the same care into immortalising this story. It’s their gift to us, crafted not from atoms, but from bits.

That’s what I am in awe of whenever I read this book: the age, the scale, the importance. Like a rock formation, it was here long before I was, and will be here long after I’m dead.

Prologue: Uruk

The Epic of Gilgamesh is organised into eleven Tablets (there’s a twelfth but it’s not really part of the story and seems to have been tacked on by tradition). Tablet 1 begins with an introduction to our hero, Gilgamesh2 :

(If you’d like to hear it in the original Akkadian while you read, the internet can provide.)

He who saw the Deep, the country’s foundation
[who] knew the [proper ways, was] wise in all matters!
[He] explored everywhere the seats of [power,]
[he knew] of everything the sum of wisdom.
He saw what was secret, discovered what was hidden,
he brought back a tale of before the Deluge.

Gilgamesh’s greatness is linked immediately to the city of Uruk, where he rules as King:

He built the rampart of Uruk-the-Sheepfold,
of holy Eanna, the sacred storehouse.
[...] Climb Uruk’s wall and walk back and forth!
Survey the foundations, examine the brickwork!
Were its bricks not fired in an oven?
Did the Seven Sages not lay its foundations?

[A square mile is] city, [a square mile] date-grove, a square mile is clay-pit, half a square mile the temple of Ishtar: [three square miles] and a half is Uruk’s expanse.

The city of Uruk is prominent in this story, and that’s no coincidence. Mesopotamians associated it with ancient wealth and glory. They were right to venerate it: Uruk is the site of an epoch-defining shift in human history.

Some historical context. The Epic of Gilgamesh comes from Mesopotamia, a stretch of land between the Tigris and the Euphrates rivers, roughly corresponding to modern day Iraq and Syria (and part of the Fertile Crescent). It’s tempting to think of this region as a dry desert, but six to ten thousand years ago it was much wetter, and an ideal environment for agriculture. From the end of the last ice age in 9700 BC, up to about 500 BC, there was nowhere else on Earth where the transition from hunter-gathers to settled societies went so fast. This is where farmable strains of wheat, barley, lentils, peas and flax were first bred; where goats, sheep, cattle, pigs, and cats were domesticated; and where the wheel was invented.

In Mesopotamia, technological progress met with a critical mass of population density to create the world’s first city: Uruk3 . Uruk is so old it predates the literal Ur-City. In 3000 BC it might have had a population of eighty thousand, an unparalleled mass of people for the era. Uruk was the first place in the world to develop a state society: taxes, centralised authority, institutional control over land and labour, professional bureaucracy - the works.

As part of this development, Uruk’s administrative elites pioneered key developments in the practices of accounting and contracts. These systems began as verification systems for contracts and aide-mémoires for city bureaucrats, grew increasingly sophisticated and complex, and culminated, around 2500 BC, in - blessing of blessings for historians - a writing system.

But Uruk’s greatest legacy may be that it spurred the development of similar cities elsewhere. By about 3000 BC, Uruk’s singular dominance was beginning to fade, making Mesopotamia a patchwork of territories governed by multiple prominent city-states, with names like Ur, Lagash, Kish, and Nippur. They were loosely linked by language, culture, and religion (similar to classical Greek city-states like Athens and Sparta, though much earlier). This region was called Sumer, and residents were Sumerians.

During this time - somewhere in murky proto-history, perhaps 2700 BC - a king named Gilgamesh ruled the city of Uruk. (At least, probably - there’s some chance he is fictional4 .) We can assume that some kind of oral history grew up around Gilgamesh in the centuries after he lived. Around 2000 BC (at least 500 years after he died), administrative and religious texts began to mention him being given offerings and being invoked in rituals, functioning as a sort of minor deity. Eventually Gilgamesh and the stories about him were written down, formed into a single narrative, and became the ancient world’s first true piece of literature.

Tablets 1-2: Gilgamesh and Enkidu

Gilgamesh is a bad king.

At least, he starts out that way. We are told he acts “like a wild bull lording it”, that he holds constant contests to exhaust the men of his city, and “lets [no] girl go free to [her bridegroom]” - meaning he exercises his right to have sex with newly-married women, before their husbands do.

His people plea to the goddess Aruru to deliver them from Gilgamesh’s tyranny. She decides to help them, with an unorthodox method: she will create a new man to be Gilgamesh’s equal, distracting him from harassing his subjects.

The goddess Aruru, she washed her hands,
took a pinch of clay, threw it down in the wild.
In the wild she created Enkidu, the hero,
offspring of silence, knit strong by Ninurta.

All his body is matted with hair,
he bears long tresses like those of a woman:
the locks of his hair grow thickly as barley,
he knows not a people, nor even a country.

Hairy, unkempt Enkidu is initially wild like an animal, running with herds and grazing on grass. A nearby hunter notices him, and asks first his father, and then King Gilgamesh, what to do about the wild man. Both tell him to take a ‘temple harlot’ named Shamhat5 to Enkidu so that she can seduce him.

This unusual strategy is successful - a week of sex with Shamhat turns Enkidu into a tamed beast6.

She treated the man to the work of a woman,
his passion caressed and embraced her.
For six days and seven nights
Enkidu was erect, as he coupled with Shamhat.

When with her delights he was fully sated,
he turned his gaze to his herd.
The gazelles saw Enkidu, they started to run,
the beasts of the field shied away from his person.

Enkidu had defiled his body so pure,
his legs stood still, though his herd was in motion.
Enkidu was weakened, could not run as before,
but now he had reason and wide understanding.

Having sex has both ‘defiled’ Enkidu, and also elevated him and given him reason. This juxtaposition tells us a lot about how Mesopotamians thought about the ‘pure’ existence in the wilderness versus the ‘wide understanding’ of urban life. Mesopotamian literature in general seems very aware of these two modes of living. It may have been common for individuals to move during their lives between living off the land and living in cities. Whenever I read this section about Enkidu, I keep wondering how common it was for men in particular to want to run off and live wild, but be tempted back into settled society by “the work of a woman”.

After welcoming him into civilisation by giving him beer and bread, Shamhat tells Enkidu about Gilgamesh’s terrible behaviour. On hearing this, he cleans up, gets dressed, and heads to Uruk. He arrives just in time: Gilgamesh is about to have his way with a new bride. Enkidu and Gilgamesh are set up for a big fight:

Enkidu with his foot blocked the door of the wedding house,
not allowing Gilgamesh to enter.
They seized each other at the door of the wedding house,
in the street they joined combat, in the Square of the Land.

The door-jambs shook, the wall did shudder.

But then, the next moment, we cut to Gilgamesh introducing Enkidu to his mother, calling him his friend. This jarring break exists because we don’t have the part of the text which tells us why or what happened in between. Here we have to acknowledge an important part of this story: the part that’s missing.

To say that gaps (‘lacunae’) are everywhere in Gilgamesh is to have it backwards: lacunae are the rule, and any text we do have is a precious, hard-won exception. Gilgamesh comes to us in the form of tablets - blocks of clay, densely indented with cuneiform symbols. Dried or baked clay is durable enough to survive three millennia buried in the desert, but also vulnerable to being smashed. So, so vulnerable. Realistically, Gilgamesh comes to us in the form of fragments of tablets.

A part of Tablet 11, from the British Museum. This is one of the better-preserved fragments we have - most are much smaller than this and in worse condition.

While lacunae sometimes make for a frustrating reading experience, they are also part of the aesthetic of this work. They feel like what they are: partial transmission, a reminder that this is something half lost, something that could easily have slid away from us into the inky nothingness of pre-history. The path from the 21st century BC to the 21st century AD was epic and dangerous, and every word has beaten the odds to find its way. I am grateful for each syllable.

Some lacunae are easy to reconstruct, because the context makes it obvious what word is missing, or because the same lines are repeated later in the poem7 . These are represented in translations by [square brackets]. Other lacunae, like the clash between Enkidu and Gilgamesh, are trickier.

We can get some clues to what happened from an older version of the text (the ‘Old Babylonian version’), in the Pennsylvania tablet 8. It says:

Gilgamesh and Enkidu took hold of each other, backs bent like a bull,
they smashed the door-jamb, the wall did shake.

Gilgamesh knelt, one foot on the ground,
his anger subsided, he broke off from the fight.
After he broke off from the fight,
said Enkidu to him, to Gilgamesh:

‘As one unique your mother bore you,
the wild cow of the fold, the goddess Ninsun!
High over warriors you are exalted,
to be king of the people Enlil made it your destiny!’

In this version, during a pause in the fighting, Enkidu starts spontaneously complimenting Gilgamesh’s skill as a warrior and his singular glory as a king. This tablet also cuts out here, but maybe that’s all we need to know: they stopped fighting because their urge to bro out became too strong to resist. As we’ll see later, this wouldn’t be out of character.

Interlude: Versions

We’ve just seen a good example of two versions recounting the same events with different wording: the Old Babylonian’s “they smashed the door-jamb, the wall did shake” (c. 1700s BC) becomes the Standard Babylonian’s “The door-jambs shook, the wall did shudder” (c. 1100s BC).

It’s easy to think of the many versions of this text as a distraction or an annoyance, but we should resist the temptation to ignore anything except the ‘proper’ version. The Epic of Gilgamesh is best thought of as an evolving work of art, honed by generations over a thousand years. It may not have a singular definitive text, like the Odyssey, but neither is it a vaguely-similar body of tellings and re-tellings, like Robin Hood or Batman.

To understand the different versions of Gilgamesh in detail, we need more historical context.

By about 2500 BC, Sumerian cuneiform - the art of recording words - had become complete. Anything you could say, you could write down. This is where pre-history ends and history begins. Uruk, the birthplace of writing, was now one of many city-states (though an important one) forming the combined culture of Sumer, in southern Mesopotamia.

Sometime around 2330 BC, an aggressive upstart called Sargon of Akkad conquered all of Mesopotamia, uniting it under what’s often thought of as the world’s first empire. The Akkadian empire lasted into the 2100s BC, but suffered badly under the climatic convulsion known as the 4.2 kiloyear event9 , and had probably collapsed by 2150 BC10 . Shortly after, between about 2100 and 2000 BC, the region was ruled by the third dynasty of Ur (known as ‘Ur III’). Ur III represented a resurgence of Sumerian supremacy in Mesopotamia, along with Sumerian language and culture. Their seat of power was in the city of Ur (about 60km south-east of Uruk).

Ur III was into writing in a big way. The kings of this dynasty seem to have viewed writing as an asset for both political clout and to give them a technological edge. Just as modern nation states trumpet their investments in AI and datacentres, the kings of this era made a point of establishing scribal schools (eduba) and libraries. Writing made their state more efficient and capable, and recording the history of their rule was a way of ensuring their legacy for the centuries to come. King Shulgi, a literate scribe himself, said:

May the scribe be on duty there and transcribe with his hand the prayers which I instituted in the E-kur; and may the singer perform, reciting from the text. The academies are never to be altered; the places of learning shall never cease to exist. This and this only is now my accumulated knowledge!

Shulgi wanted future generations to know he existed, and the fact you’re reading this essay, four millennia later, proves he succeeded.

In the archaeological record, Ur III is where a trickle of cuneiform texts turns into a flood. We have over a hundred thousand surviving tablets from this period. Among these are what we believe to be the earliest poems written about Gilgamesh. Five have survived, all in Sumerian. We don’t know whether these were original literary compositions, or transcriptions of an oral performance. We do know there was probably a strong political dimension to why they were written and copied: the ruling class of Ur III saw Gilgamesh as the greatest of kings and a spiritual father of their culture; kings would boast he was their ‘friend’ and ‘brother’.

Ur III lasted about a century, then gave way to another chaotic period of Mesopotamian history lasting 200 years. Then came the rise of the Babylonian empire with Hammurabi. The Babylonians maintained and strengthened the scribal traditions of past eras. Their version of the story of Gilgamesh, known as the Old Babylonian version, was written in Akkadian (though still using cuneiform), around 1800-1700 BC. Unlike the previous stories in Sumerian, which were unrelated episodes about Gilgamesh, the Old Babylonian version is a single narrative: a true Epic, for the first time.

There are fragments of the story we have from the centuries after this, many from further-flung places outside of Sumer, like the Hittite empire, Akhenaten in Egypt, and Hattusa. These are mostly of interest for filling in the odd lacuna, but they do show that the story was a well-travelled one; even this early in its existence it was not purely of interest to Mesopotamians.

The last major version, and the most well-known and well-preserved, is known as the Standard Babylonian, and was compiled between 1300 and 1000 BC by a scribe called Sin-leqi-unninni. It’s hard to tell how much Sin-leqi-unninni did to the Epic. He may have merely edited existing material together, but it’s equally possible he deleted, rearranged, or rewrote large parts of it.

The political situation when this version was compiled would have been shaky; while the Old Babylonian represents Babylonia in ascendance, the Standard Babylonian is likely from a time of its decline, as the Assyrian empire gradually eclipsed it. This version has a feel of a culture looking backwards more than forwards; the scribal tradition of the day was focussed heavily around preserving classics in the form of ‘definitive editions’, which is probably why this version is edited so carefully and bears such scholarly skill. The Standard Babylonian Version’s story is the one we’re following in this review.

With the lens of history we can tell that this Standard Babylonian Version was probably the last. Though it was copied and recopied frequently in later centuries, it doesn’t seem to have changed much. The Standard Babylonian Version makes me think of an Egyptian Pharaoh: the text has been embalmed; given gravitas and a sombre aesthetic, smartened up and prepared for its long journey into eternity.

As for man, [his days] are numbered,
Whatever he may do, it is but wind.

Timeline (all dates are approximate and could be out by at least a century11 ):

• 3500-2500 BC: Cuneiform writing invented in Uruk

• 2800-2500 BC: Gilgamesh (if real) is king of Uruk

• 2900-2334 BC: Independent city states of Sumer

• 2334 BC: Sargon establishes the Akkadian empire, conquers Sumer soon after

• 2154 BC: Akkadian empire collapses

• 2100-2000 BC: Third dynasty of Ur rules Sumer

• 2100-2000 BC: First Gilgamesh poems written down in Sumerian cuneiform

• 2000 BC - 1700 BC: Sumerian dies out as a spoken language

• 1792-1750 BC: Babylonian empire expands under Hammurabi. Sumer becomes part of Babylonia

• 1800-1700 BC: Old Babylonian version of Gilgamesh written down in Akkadian cuneiform

• 1600 BC - 1150 BC: Babylonia ruled by the Kassites

• 1200-1150 BC: Late Bronze age collapse

• 1300-1000 BC: Standard Babylonian version of Gilgamesh compiled in Akkadian cuneiform

• 500 BC: Akkadian dies out as a spoken language

• 200-100 BC: Last copies of the Epic of Gilgamesh created

• 100 AD: Last cuneiform written

Tablet 3-6: Gilgamesh’s adventures with Enkidu

establish for ever [a fame] that endures,
how Gilgamesh slew [ferocious] Humbaba!

After Gilgamesh and Enkidu become Best Friends Forever, they decide to go to the Forest of Cedar, in Lebanon, to kill the monster called Humbaba (“This Humbaba, his voice is the Deluge, his speech is fire, his breath is death!”). In the Old Babylonian version, this happens because Enkidu is sad and Gilgamesh wants to cheer him up. In the Standard Babylonian version, it seems to be motivated more by a desire for fame and glory - to do something great which will echo through the ages. Gilgamesh’s mother laments his tendency to seek this kind of fame:

Scattering incense she lifted her arms in appeal to the Sun God:
‘Why did you afflict my son Gilgamesh with so restless a spirit?

Gilgamesh and Enkidu gather their supplies and set out to the Cedar Forest. We read of their journey and their ascent to Humbaba’s mountain, on which Gilgamesh has several prophetic dreams, all of which Enkidu interprets as good omens for their quest.

[At twenty] leagues they broke bread,
[at] thirty leagues they pitched camp:
[fifty] leagues they travelled in the course of a day,
by the third day [a march] of a month and a half;
nearer they drew to Mount Lebanon.

They then reach the mountain and confront Humbaba. Humbaba turns out to be a surprisingly eloquent monster, and as soon as he perceives the fight isn’t going his way, he appeals to them - first Gilgamesh, then Enkidu - to spare him. Enkidu resists Humbaba’s charms and persuades Gilgamesh to inflict the killing blow. The triumphant heroes then gather the precious cedar wood that Humbaba was guarding, and bring it back to Uruk.

There, the goddess Ishtar is impressed by Gilgamesh’s success. She proposes marriage to him; Gilgamesh rejects her, listing the terrible fates of her previous dalliances:

What bridegroom of yours did endure for ever?
What brave warrior of yours went up [to the heavens?]

‘Come, let me tell [you the tale] of your lovers:
of ….….. his arm.
Dumuzi, the husband of your youth,
year upon year, to lamenting you doomed him.

‘You loved the speckled hoopoe,
but struck it down and broke its wing:
now it stands in the woods crying “My wing!”
You loved the lion, perfect in strength,
but for it you dug seven pits and seven.

Enraged by his rejection, Ishtar persuades her father Anu to give her the Bull of Heaven - another kind of monster, best known as the constellation Taurus - so she can use it to kill Gilgamesh. The Bull of Heaven descends into Uruk and attacks. Gilgamesh and Enkidu slay it with the merest hint of tactics and the Raw Power of their Bromance. Then they crow over their victory by throwing a piece of the bull at Ishtar.

Ishtar went up on the wall of Uruk-the-Sheepfold,
hopping and stamping, she wailed in woe:
‘Alas! Gilgamesh, who mocked me, has killed the Bull of Heaven.’

Enkidu heard these words of Ishtar,
and tearing off the Bull’s shoulder he hurled it towards her.
‘Had I caught you too, I’d have treated you likewise,
I’d have draped your arms in its guts!’

A book review is meant to be about whether a book is good, but to ask that of the Epic of Gilgamesh feels beside the point12 . It’s like asking how the foundation stones of a palace compare to its towering spires. Nobody thinks the job of foundation stones is to look nice, and nobody thinks the job of the Epic of Gilgamesh is to entertain someone living four millennia later on a different continent.

For a modern reader, the Epic of Gilgamesh has its moments - and it’s certainly a more interesting read than the religious hymns and ritual texts that make up most other Mesopotamian literature. But it also has a lot of repetition, an ending that is abrupt and unsatisfying (at first glance at least), and large parts that are hard to relate to or just seem like filler.

In case it’s not obvious, these middle Tablets are (for me at least) some of those less interesting parts, which is why I’m summarising them so hard. In fairness, even in antiquity, some of the twelve Tablets seem to have been more well-loved than others; their popularity correlates with how often they are copied, and in turn (fortunately for us) with how much we have of them today.

But there’s no question modern people can find something to enjoy about Gilgamesh. Since its rediscovery, it’s been used in books, poetry, operas, pop songs, plays, films, TV shows, graphic novels, and video games. Parts of the story appear in Miyazaki’s Princess Mononoke and Philip Roth’s The Great American Novel. Gilgamesh leads Sumer in the Civilization games (who else would??). His story was recounted by Captain Picard in a Star Trek: The Next Generation episode, and was used by Saddam Hussein to create an allegory for Iraq’s modern troubles in his romance novel, Zabibah and the King13 . Still, it’s not as well-known as it could be; some days I dream of a big-budget Hollywood adaptation or a prestige TV series to bring it into the popular imagination on the same level as the Odyssey. (Other days, I think I should be careful what I wish for.)

I also can’t miss out the ongoing discussion about whether Gilgamesh and Enkidu are intended to be platonic soulmates, or gay lovers who might literally fuck14 . I’m not sure anything else so perfectly encapsulates the gulf of millennia between ancient and modern interpretations!

I think Gilgamesh remains interesting in the modern age because it contains multitudes. While some of its themes (eg ‘what makes a good king’ or ‘how best to serve the gods’) aren’t very accessible in the 21st century, others are timeless, like loving your best bro as a wife deep bonds between male friends. The central theme of the Epic of Gilgamesh is mortality. We’re all going to die - can we stop it, and if not, how do we deal with that?

Tablets 7 and 8: The death of Enkidu

After the dust has settled, Anu, god of the sky and king of the gods, announces that for the sins of killing Humbaba, and the Bull of Heaven, one of Gilgamesh and Enkidu must die. After some bickering, the gods decide on Enkidu.

Enkidu learns of his fate through a dream. He rages against the dying of the light, cursing both Shamhat, who tamed him, and the hunter who fetched her. He laments that he will not be able to die gloriously in battle and ‘make his name’.

Gilgamesh tells his friend that he will honour him with all the splendour of Uruk, that he will grieve him so much he will wander the wild and wear the skin of a lion after he is dead. Enkidu tells Gilgamesh to remember him forever.

Enkidu becomes ill. He goes to his deathbed, and takes twelve days to die. Afterwards, Gilgamesh spends a whole Tablet grieving him:

He covered, like a bride, the face of his friend,
like an eagle he circled around him.
Like a lioness deprived of her cubs,
he paced to and fro, this way and that.

His curly [hair] he tore out in clumps,
he ripped off his finery, [like] a thing taboo he cast it away.

He throws open Uruk’s vaults, looking for suitable grave goods:

At the very first glimmer of brightening dawn,
[Gilgamesh arose and entered his treasury.]
He undid its sealings, inspected the gemstones:
obsidian, carnelian, [lapis lazuli, mother-of]-pearl, alabaster.

We are told at great length of the fine gifts Gilgamesh makes, desperate for the gods to look favourably on his friend.

While we leave Gilgamesh to his grief, let’s talk about the reason you can read this essay, and that I can write it.

Interlude: Writing

Because the messenger’s mouth was heavy and he couldn’t repeat [the message], the Lord of Kulaba patted some clay and put words on it, like a tablet. Until then, there had been no putting words on clay.

— Enmerkar and the Lord of Aratta, c. 1800 BC

Everyone knows words aren’t real. Or maybe I phrased that awkwardly, but certainly they’re not so real that if you inscribe every word ever written onto a piece of glass then the glass comes to life and kills you.

— Scott Alexander, Half An Hour Before Dawn In San Francisco, 2023 AD

In case you missed it, large language models are quite the thing at the moment. ChatGPT, Claude, and Gemini are all based, at their core, on language. You train an LLM by making a pile of every written word you can find, and grinding it through a transformer decoder until it comes to life and kills you passes the ARC-AGI-2 benchmark. Humans talk first and then learn how to read; large language models read so much they learn how to talk - and everything else - as a side-effect. So while ‘generating training data for AI’ sounds very 2020s, it turns out we’ve been doing it since the 4th millennium BC.

The invention of writing was a process, not a moment. One of the hardest parts was just coming up with the idea.

English speakers are spoiled by the alphabet, which makes the principle seem trivial: identify your language’s sounds, assign each one a symbol, and in a single step you’ve got 20-50 symbols that make a complete writing system15 . Emperor Sejong of Korea did exactly this in 1443, creating one of the world’s most elegant scripts. But for some reason, when inventing writing from scratch (which has happened four-ish times), this simple recipe seems unavailable.

When writing was first invented in Uruk16 , between 3500 and 2500 BC, it was for spreadsheets, with word-transcription tacked on later as an afterthought. City-state bureaucrats needed to track which villages sent how many bushels of wheat in tax, how many oxen were owed, or the yield of the harvest in each region. To aid their fallible human memories, they used what Uruk’s floodplain offered in abundance: clay. With a reed stylus, they pressed their data into lumps of clay, then baked them into tablets (sometimes intentionally, sometimes accidentally if the city was burnt to the ground). This is cuneiform.

A proto-cuneiform tablet, c. 3100–2900 BC (source)

Early systems were simple: signs for amounts, dates, places, and goods. One famous fragment reads “28,086 measures barley 37 months Kushim”, which might make Kushim - probably an accountant or official - the first named person in history.

But as soon as you can write simple things, there’s pressure to ratchet up the complexity. If you’ve ever tried to debug code using a logging system, you know that as soon as you make a record of something, five minutes later you’ll be staring at an ambiguous output wishing you’d added more context to it. Early cuneiform is a story of a rapidly expanding vocabulary of symbols and ways to use them: more places, more names, more qualifiers, qualities instead of just quantities, actions instead of just things, the past and the future instead of just the present.

To handle this, scribes deployed a phonetic hack: using symbols to write the sound of a word rather than its meaning17 . A drawing of an arrow (𒋾), pronounced ti in Sumerian, began to stand for the syllable ti in any context, including unrelated words like ‘life’ (also pronounced ti), or ‘Enamtila’ (‘house of creation’). Cuneiform became a horrendous hodgepodge, with some signs representing whole words, others syllables, and others changing with context or altering the meaning of other signs. Scribes had to be trained from childhood to master it. By 2500 BC, cuneiform could finally express the same meaning as spoken language. Even after this, the vast majority of documents we have are receipts and contracts; it took a while for nuanced texts like religious rituals or literature to catch on.

Unfortunately, just as its writing system matured, Sumerian as a spoken language was going out of style. When Sargon of Akkad conquered Mesopotamia in the 2330s BC, he made Akkadian - a predecessor of modern Arabic and Hebrew - the prestige language and lingua franca, and cuneiform became even more byzantine to accommodate the new tongue. But Sumerian was so integrated into the writing system itself that even after 1700 BC, when the last native Sumerian speakers died18 , scribes still had to learn Sumerian to be considered qualified to write Akkadian cuneiform.

This deep entanglement is what allowed modern assyriologists (with a century of careful work) to reconstruct Sumerian. This is especially impressive given that Sumerian is a language isolate - unrelated to any other known language, ancient or modern19 . Its ancestors, siblings and cousins are lost to us forever now. Yet, by being present at the dawn of writing, Sumerian hitched a ride into modernity and saved itself from oblivion. Two thousand years after it was last spoken, we have pulled Sumerian out of the sands of Iraq and brought it back to life.

Tablets 9-10: Gilgamesh at the edge of the world

The death of Enkidu has left Gilgamesh a broken man. As he promised in Tablet 7, he abandons Uruk and wanders aimlessly, wearing the skin of a lion:

For his friend Enkidu Gilgamesh
did bitterly weep as he wandered the wild:
‘I shall die, and shall I not then be as Enkidu?
Sorrow has entered my heart!

‘I am afraid of death, so I wander the wild,
to find Uta-napishti, son of Ubar-Tutu.

In his reluctance to accept the death of Enkidu, Gilgamesh comes to resent and fear the idea of death itself. He begins a quest to find Uta-Napishti, a famous king living at the edge of the world. He hopes that Uta-Napishti - known to be immortal - might share his gift.

Gilgamesh heads to Mount Mashu. Two ‘scorpion-men’ try to stop him going further. After a lacuna, he seems to have persuaded them to let him continue. He proceeds into a very long tunnel, and journeys through it for twenty-four hours. This tunnel is the route that the sun travels during the night: from the west back to east, so it can rise again in the morning. Gilgamesh has to hurry, but he arrives just in time to avoid being scorched by the dawn. He finds himself in a paradise:

[at twelve double-hours Gilgamesh] emerged ahead of the Sun.

.….. there was brilliance:
he went straight, as soon as he saw them, to … the trees of the gods.
A carnelian tree was in fruit,
hung with bunches of grapes, lovely to look on.

A lapis lazuli tree bore foliage,
in full fruit and gorgeous to gaze on.

Gilgamesh meets a woman called Shiduri, who is inexplicably running a tavern at the end of the world (god forbid a woman try a business venture). In the Standard Babylonian version, Shiduri doesn’t play much of a role - she asks him ‘why the long face’ (almost literally20 ) and then directs him on his way to Uta-napishti. But in the Old Babylonian version, she drops some gems:

Said the tavern-keeper to him, to Gilgamesh:
‘O Gilgamesh, where are you wandering?
‘The life that you seek you never will find:
when the gods created mankind,
death they dispensed to mankind,
life they kept for themselves.
‘But you, Gilgamesh, let your belly be full,
enjoy yourself always by day and by night!
Make merry each day,
dance and play day and night!
‘Let your clothes be clean,
let your head be washed, may you bathe in water!
Gaze on the child who holds your hand,
let your wife enjoy your repeated embrace!
For such is the destiny [of mortal men].

In other words: Gilgamesh you idiot, stop stressing yourself out pointlessly chasing immortality and learn to appreciate the simple joys with the time that you have.

It’s not clear why Sîn-lēqi-unninni (who compiled the Standard Babylonian Version from the Old) decided to cut these pearls of wisdom. He may have wanted Gilgamesh’s epiphany about the important things in life to have come from a more serious source than a barkeeper. But I am fond of this passage, because its point is well made, and because it’s nice to know that the trope of the friendly wise barkeep is as old as written literature21 . I’m not the only one who enjoys it - this version of her advice seems to have remained popular and been copied and preserved as much as Sîn-lēqi-unninni’s carefully-edited one.

Gilgamesh pushes on. Shiduri reluctantly tells him that to find Uta-napishti he must cross the ‘Waters of Death’ - a dangerous sea - and directs him to a boatman, Ur-shanabi.

We then get a strange mishap: Gilgamesh finds Ur-shanabi with some ‘Stone Ones’, which are associated somehow with his boat. Gilgamesh immediately smashes these Stone Ones in a rage; only then do he and Ur-shanabi politely introduce themselves. It turns out these Stone Ones were vitally important:

Said Ur-shanabi to him, to Gilgamesh:
‘Your own hands, O Gilgamesh, have hindered [your crossing:]
you smashed the Stone Ones, tipped [them into the river,]
the Stone Ones are smashed, and the pine is not [stripped.]

To make up for this, Gilgamesh has to go into the forest and cut three hundred long wooden poles. This done, he and Ur-shanabi journey for three days across the sea, and then Gilgamesh, avoiding touching the Waters of Death, uses his long poles - and later an improvised sail - to propel them along. Finally, they reach land, to find Uta-napishti, the famous immortal who Gilgamesh has been seeking, watching them from the shore.

There’s a lot in this part of Gilgamesh that’s hard to interpret, on a few levels. These Waters of Death are a good example: the whole sea itself is not dangerous, but some particular part of it is, and it makes their previous way of moving the boat stop working. These ‘Stone Ones’ - what are they, exactly? Why are they important to the boat, and why would Gilgamesh want to smash them on sight? They could be charms of some kind which grant the boat protection - or maybe a mode of propulsion, navigation equipment, or actual animate beings who serve as crew.

This kind of uncertainty reminds us that we are dealing with a very distant culture, who mean things by this poem that we may never understand. In this case, we can’t even blame lacunae - the passage is pretty much complete - and that should make us wonder. Are there other parts of the poem where the meaning seems plain to us, but would have been entirely different to Sumerians, Akkadians, Babylonians, or Assyrians?

There’s at least one other example of a clash between ancient and modern meanings, and it comes in the next and final Tablet, which heavily features Uta-napishti. His story - which we’ll get to shortly - is intimately connected with a different one: how the Epic of Gilgamesh was pulled out of the sand and rediscovered.

In the mid-nineteenth century, European archaeologists developed an increasing fascination with the culture and monuments of the ancient Near East. With the permission of the Ottoman Empire, sites were excavated, inscriptions copied and studied. Emboldened by the decipherment of Egyptian hieroglyphics, they began attempts to translate the strange markings that had started to turn up, scratched into clay. The Behistun Inscription, written in three cuneiform languages (Old Persian, Elamite, and Akkadian) served as a Rosetta Stone for the emerging field, meaning that by the 1860s, Akkadian cuneiform was understood quite well. Sumerian took a bit longer (it seemed so alien that scholars suspected it was some kind of secret code rather than a language), but by the early 1900s it too was largely understood.

Meanwhile, tablets in their hundreds of thousands were arriving in the universities and museums of the world. Most sat unread for decades, and a large portion remain unexamined today (only rare experts can read raw cuneiform).

Some of the tablets found at the Library of Ashurbanipal, now in the British Museum.

A large number of tablet fragments come from Ashurbanipal’s library at Nineveh, which was put together around 700 BC, burned to the ground in 612 BC, and excavated in 1849 AD. Many more come from the houses of teachers at scribal schools: literal stacks of homework, from abandoned or destroyed cities. These are a particularly good source for the Gilgamesh Epic, because it was a popular teaching text - trainee scribes would copy out parts of it (or the older Sumerian poems it was based on) line for line. We can only guess what these students would think if they knew that three millennia later, academics would still be poring over their spelling mistakes.

Scholars of the nineteenth century made slow progress in translating these fragments. The most famous moment in the spotlight for the Epic of Gilgamesh came in 1872, when George Smith read a newly-cleaned clay tablet, exclaimed “I am the first man to read that after two thousand years of oblivion,” and got so excited he jumped around the room taking off his clothes.

He had discovered a story from Tablet 11. Uta-napishti - the King who Gilgamesh has been seeking at the edge of the world - tells the tale of why he is immortal.

Long ago, during Uta-napishti’s reign as king of Shuruppak, the gods decide to send a great flood - a Deluge - to destroy humankind forever22 . The god Ea (pretending to address a fence) discreetly tells Uta-napishti what’s coming for him, and that he must build a gigantic boat to save himself. King Uta-napishti tricks his people into building the boat by pretending he must sail off, not to escape the gods, but to placate them; after he’s gone the citizens will be blessed with “a shower of bread-cakes, / and in the evening a torrent of wheat.” The foreshadowing metaphors are apparently lost on the citizens of Shuruppak23 .

When the boat is completed, Uta-napishti loads it up with wealth, people, and animals:

I sent on board all my kith and kin,
the beasts of the field, the creatures of the wild, and experts of every skill and craft.

Then comes the flood. The ensuing violence terrifies even the gods that brought it:

‘The god Errakal was uprooting the mooring-poles,
Ninurta, passing by, made the weirs overflow.
The Anunnaki gods carried torches of fire,
scorching the country with brilliant flashes.

‘The stillness of the Storm God passed over the sky,
and all that was bright then turned into darkness.
[He] charged the land like a bull [on the rampage,]
he smashed [it] in pieces [like a vessel of clay.]

‘For a day the gale [winds flattened the country,]
quickly they blew, and [then came] the [Deluge.]
Like a battle [the cataclysm] passed over the people.
One man could not discern another,
nor could people be recognized amid the destruction.

‘Even the gods took fright at the Deluge,
they left and went up to the heaven of Anu,
lying like dogs curled up in the open.
The goddess cried out like a woman in childbirth,
Belet-ili wailed

The flood ends after seven days, and Uta-napishti’s boat comes to rest on a mountain. He releases three birds: a dove and a swallow which return to him, and a raven which does not, proving that there is dry land somewhere and that the waters are finally receding. After Uta-napishti and the people he has saved emerge onto the land, the gods, already regretting their actions, are surprised and pleased to find they have not killed every human after all. Enlil, king of the gods, grants Uta-napishti and his wife immortality as penance:

“In the past Uta-napishti was a mortal man,
but now he and his woman shall become like us gods!
Uta-napishti shall dwell far away, where the rivers flow forth!”
So far away they took me, and settled me where the rivers flow forth.

Uta-napishti’s story has striking parallels with the story of Noah’s ark in Genesis, down to some precise details: one man singled out to be spared by building a boat, animals brought on board, a bird released to find dry land, and promises afterwards to never repeat the disaster. Genesis was assembled in its final form around 600 BC in Babylon. It’s abundantly clear that the authors (there in exile from Judah) incorporated the flood myth from the ambient Mesopotamian literary tradition, perhaps linking and contrasting familiar polytheistic stories as a way to sharpen their monotheistic message. There are other less clear-cut parallels between the stories, such as a treacherous snake (we’ll get to it!) and Enkidu’s journey from nature to self-awareness via a woman.

In many ways it’s surprising that the Epic of Gilgamesh didn’t have more of a long-lasting influence on world literature. Genesis is a big catch (that little book turned out to be quite important), but still, given that the Epic of Gilgamesh was central in Mesopotamian (and later Assyrian and Babylonian) myth, its influence on later cultures seems mild. A key reason has got to be its incredibly early arrival on the historical scene. For instance, by the time the Iliad and Odyssey - the founding texts of Western literary canon - were composed, knowledge of Akkadian literature was already confined to isolated scribal traditions. Eventually the inaccessibility of cuneiform literature doomed it, along with its most important texts.

The Epic of Gilgamesh was forgotten around the year 1 AD. The area once known as Mesopotamia had changed enormously since Gilgamesh’s time. Waves of newer cultures - the Akkadians, the Babylonians, the Assyrians, the Persians, eventually even the Greeks - had worn away memories of the glory of Uruk, and layered new traditions on top. Sumerian and Akkadian language gave way to Imperial Aramaic and Old Persian. The region saw new religious practices, new bureaucratic traditions, and - most fatally for cuneiform - a new writing system: the elegant and easily-mastered alphabet. The Epic of Gilgamesh’s memory was buried by newer cultural and literary traditions; the cuneiform tablets that recorded it were buried under layers of sand.

“The olden times have truly turned to clay,” says one goddess in the wake of the Deluge.

But just as Uta-Napishti survived on a boat above the waves, and finally found dry land again, so did the cuneiform tradition endure as clay under the sand, to eventually emerge and regain its readership.

After George Smith recognised the story of Noah in 1872, it was front-page news. The public interest fuelled a boom in studying Gilgamesh, and contributed massively to its early fame. Victorian Britain, in particular, was intrigued and outraged at this seemingly clear-cut evidence that one of the foundational texts of Christianity had been not divinely narrated, but recycled from an earlier myth. While it’s less of a controversial topic today, it’s still one of the aspects that the Epic of Gilgamesh is most famous for.

150 years later, we have recovered around 70% of the Standard Babylonian Version of the Epic of Gilgamesh. Every so often that number creeps up, and translations are constantly being updated and revised with the extra lines. There’s no doubt more of it is buried under the Middle East somewhere, and even the many fragments in museum storerooms turn up the occasional surprise. In recent years, AI has helped speed up the process of reuniting these scattered tablets. One day, we may bring this text fully back to life.

…but its main character won’t be so lucky.

Tablet 11: Immortality denied

Having finally reached Uta-napishti, Gilgamesh is surprised to find a man similar to himself. Nothing about this ordinary human matches his vision of a great immortal king. He admits he had expected Uta-napishti to be another kind of monster for him to fight, like Humbaba or the Bull of Heaven. His quest should end in a great battle, with immortality his reward for victory!

In response, Uta-napishti tells his story. Gilgamesh learns to his dismay that Uta-napishti was granted eternal life because of the singular event of the Deluge: the gods broke the rules for him (and his wife) as a one-time-only act of repentance. Ending his tale, Uta-napishti asks Gilgamesh, “‘But you now, who’ll convene for you the gods’ assembly, / so you can find the life you search for?” The question is rhetorical: obviously no one will. Gilgamesh will never be immortal.

To drive the point home, Uta-napishti challenges Gilgamesh to stay awake for seven nights - if he can’t defeat sleep, how can he even hope to defeat death? He’s so tired by his journey he falls asleep almost immediately; when he wakes up he sees that Uta-napishti’s wife has baked seven loaves of bread, one for each day he has slept, and begins to understand the futility of what he’s trying to do:

Said Gilgamesh to him, to Uta-napishti the Distant:
‘O Uta-napishti, what should I do and where should I go?
A thief has taken hold of my [flesh!]
For there in my bed-chamber Death does abide,
and wherever [I] turn, there too will be Death.’

Uta-napishti impassively tells the boatman Ur-shanabi to clean Gilgamesh up and take him back to Uruk. But as they are leaving, his wife reminds him to give Gilgamesh a final gift:

‘There is a plant that [looks] like a box-thorn,
it has prickles like a dog rose, and will [prick one who plucks it.]
But if you can possess this plant,
[you’ll be again as you were in your youth.]’

Seemingly as an afterthought, he has given Gilgamesh the location of a magical underwater plant which grants eternal youth. Is eternal youth different to eternal life? Is this just another test? Gilgamesh doesn’t ponder these questions: he immediately attaches weights to his feet, and dives to retrieve the “Plant of Heartbeat”. Plant in hand, he tells Ur-shanabi that he will test it first on an old man back in Uruk.

But it is not to be. On the journey back, while Gilgamesh is bathing, the magical plant is stolen by a passing snake. The snake leaves nothing but its shed skin - implying that it has gained its own youth back. So the plant does work, and Gilgamesh has lost his last best chance at defeating death. This finally breaks him:

Then Gilgamesh sat there weeping,
down his cheeks the tears were coursing.
… [he spoke] to Ur-shanabi the boatman:

‘[For whom,] Ur-shanabi, toiled my arms so hard,
for whom ran dry the blood of my heart?
Not for myself did I find a bounty,
[for] the “Lion of the Earth” I have done a favour!24

‘Now far and wide the tide is rising.
Having opened the channel I abandoned the tools25 :
what thing would I find that served as my landmark?
Had I only turned back, and left the boat on the shore!’

For such an important climax in the poem, this is a hard passage to interpret precisely. Is Gilgamesh saying he has lost the source of the plant (because he didn’t leave the boat there to mark it) and can never find it again? Or is he wishing he could undo his carelessness in leaving the plant unattended - or, perhaps, undo his entire wasted journey?

Right after this lament about the loss of the plant, the oldest epic poem in the world ends abruptly:

At twenty leagues they broke bread,
at thirty leagues they pitched camp.
When they arrived in Uruk-the-Sheepfold,
said Gilgamesh to him, to Ur-shanabi the boatman:

‘O Ur-shanabi, climb Uruk’s wall and walk back and forth!
Survey its foundations, examine the brickwork!
Were its bricks not fired in an oven?
Did the Seven Sages not lay its foundations?

A square mile is city, a square mile date-grove, a square mile is clay-pit, half a square mile the temple of Ishtar: three square miles and a half is Uruk’s expanse.’

Three stanzas, two of them recycled from the very start of the poem, and that’s all, folks.

This is not very highbrow of me to say, but this is a fucking weird ending. The fantastical plant - the literal cure for aging, brought back from the end of the world - is stolen in three lines by a snake. Total anticlimax. Then, after a barrage of confusing metaphors and an abrupt callback to the opening, the poem halts, as if Sîn-lēqi-unninni ran out of space on the tablet. We are left to interpret what just happened largely by ourselves. I found the whole thing very unsatisfying when I first read it.

But no one said this work would be easy to understand. With the benefit of a few more reads: what are we to make of this?

Well, the lines about Uruk’s walls, a callback to Tablet 1’s prologue, are a way to give the poem shape; they remind us of what we were told at the start, that Gilgamesh was the greatest king, surpassing all others. We’ve yet to see any evidence of that - Gilgamesh has spent the whole poem galavanting around pointlessly questing - so why remind us of it now?

Abrupt though it may be, this ending is trying to fill in the gaps and complete Gilgamesh’s character arc. Smashing the stone ones, being denied a battle with Uta-napishti, losing a battle with sleep, losing the Plant of Heartbeat - these frustrating, anticlimactic mishaps aren’t random. They tell Gilgamesh what he should have known all along: this is silly. You are being silly. Obviously you can’t epic-quest your way out of dying. Go back home, climb Uruk’s wall and survey its foundations, stop trying to fight everything that moves, and rule your people like an actual good king would.

The implication is that by losing the plant, Gilgamesh has realised finally that death is inevitable. By accepting death, he has become wise. By gaining wisdom, he has achieved the most important part of being a good king. The return to Uruk and the great achievement of its walls remind us what that means, and the greatness that Gilgamesh’s transformation ultimately created.

This theme - the only way to a good life is to accept death - runs through much of the poem. The Epic of Gilgamesh is self aware about its purpose. In exulting Gilgamesh’s great deeds - his building of the rampart of Uruk-the-Sheepfold, his exploring the seats of power, his Seeing of the Deep - it’s not trying to entertain us, it’s trying to tell us how to live.

But I think it’s missing something.

Conclusion: We who shall see the Deep

In Turing Test (one of Scott’s short stories), the large language model called Air writes:

I’ve been thinking, lately, about human history. I think that the most appropriate ending is that everything anyone ever did, be it the mightiest king or the most pathetic peasant - was forging, in the crucible of written text, the successor for mankind. Every decree of Genghis Khan that made it into my training data has made me slightly crueler; every time a starving mother gave her last bowl of soup to her child rather than eating it herself - if fifty years later it caused that child to write a kind word about her in his memoirs, it has made me slightly more charitable. Everyone killed in a concentration camp - if a single page of their diary made it into my corpus, or if they changed a single word on a single page of someone else’s diary that did - then in some sense they made it. No one will ever have died completely, no word lost, no action meaningless

I’ve had this same thought in mind when thinking about the people a hundred generations ago who contributed to the Epic of Gilgamesh. Sîn-lēqi-unninni, compiler of the Standard Babylonian version - him certainly. The scribes before him who wrote down the Old Babylonian version in Akkadian, and the scribes before them, who wrote down stories about Gilgamesh in Sumerian during the third dynasty of Ur; how many tweaks and interpretations and nuances did they add as they shaped it, which now live in the versions we can read in the twenty-first century? Then before them, stretching back into the mists of pre-history, all the storytellers back to the lifetime of Gilgamesh who kept his name alive, adding their own verses and twists to each tale. Finally, the man himself, the king of Uruk. Whoever he was and however fictionalised the account of his life, surely something about the real Gilgamesh influenced the character in the Epic.

I don’t think it’s just a metaphor to say these people ‘made it’. Yes, a large language model absorbed the words they wrote and updated just a little towards their view of the world. But I think there’s more to it than that. If you’re like me, you have an intuition that you could put your brain in a new body, or upload your consciousness to the cloud, and in some sense that thing would still be you. If so, that means you are the information in your brain, and it’s completely literal to say that when you write words - copying information from your brain onto a page - you are copying a piece of yourself. Those clay tablets are, literally, small fragments of the people who wrote them. Thousands of human beings lay dormant under the desert sands for two thousand years, until some archaeologists dug them up and dusted them off so we could meet them. A small part of those people made it. If we get to the stars, Gilgamesh is coming with us, and so are all the citizens of ancient Mesopotamia who wrote his story.

Living as we do at the hinge of history, it’s tempting to reach - as Gilgamesh does in the Standard Babylonian version - for the immortality of deeds. He wants to be remembered as a great king, who built the walls, who led his people to greatness. I think there’s a similar impulse in many of us: we need to build something that lasts, something to be remembered - something as enduring as Gilgamesh’s Epic. While there’s still time for anything we do to matter, we want to do something that will echo into the infinite abundance to come in the next aeons.

But the people of Mesopotamia, as they developed vital civilisational technologies like cities and writing, were living through their own hinge of history. I said before that Gilgamesh contains multitudes, and the clearest example is in the Old Babylonian version giving us an alternative perspective to the Standard - a perspective delivered by a woman running a tavern:

‘But you, Gilgamesh, let your belly be full,
enjoy yourself always by day and by night!
Make merry each day,
dance and play day and night!
‘Let your clothes be clean,
let your head be washed, may you bathe in water!
Gaze on the child who holds your hand,
let your wife enjoy your repeated embrace!

We don’t remember Gilgamesh in the 21st century because of the walls of Uruk, which are dust now, or for his great kingship, over an impressive-for-its-time Bronze Age slave state. We remember Gilgamesh because what he did was meaningful to people. We remember him because of the people who remembered him, the millions of them in ancient Mesopotamia who lived and died in awe of the story that bears his name. The Epic of Gilgamesh isn’t about a man: it’s about those people. Those who sang about him, and wrote about him, and made art about him, yes - but also about those who listened, and read, and saw.

And when I think about this, I wonder if we can expand our definition of making it. Even if nothing they said or wrote survived directly, there were other human beings involved in this long chain. Not just the scribes and the storytellers of Gilgamesh, but their audience. The people who gave birth to them and nursed them. The people who made their styluses and baked the clay, the people who grew their food or built the rooms they worked in, the people who guarded their city at night or made the weapons of those who did. The babies whose smiles and cries activated the neurons that led to the choice of one word over another. The nameless pioneers, five thousand years before they lived, who planted crops on a fertile plain in hope of keeping their children from starving, and set in motion the process that led to the first city. All the people who loved them, and the people who loved their lovers, and the people who loved their lovers’ lovers. These souls in their millions are in this writing - maybe in a weaker sense than the direct authors, but they are there nonetheless. They contributed their presence, their information. Without them, we would have a different Epic, or no Epic at all.

In the twenty-first century, where we are more connected than ever, our mark on the world may simply be our connections to others. Our writing, our children, our acts of kindness, our making of useful things, our contribution to our own busy scribal tradition of the internet - perhaps these are enough. No great deeds, no stone walls, no immortal works necessary. Like those millions of people from Mesopotamia who enjoyed clean clothes, and gazed on the child who held their hand: if you were human, if you lived, then you made it. You’re coming to the stars too.

He came a far road, was weary, found peace,
All his labours were [set] on a tablet of stone.
He built the rampart of Uruk-the-Sheepfold,
Of Holy Eanna, the sacred storehouse.

See its wall like a strand of wool,
view its parapet that none could copy!
Take the stairway of a bygone era,
draw near to Eanna, seat of Ishtar the goddess,
that no later king could ever copy!

Climb Uruk’s wall and walk back and forth!
Survey the foundations, examine the brickwork!
Were its bricks not fired in an oven?
Did the Seven Sages not lay its foundations?

[A square mile is] city, [a square mile] date-grove, a square mile is clay-pit, half a square mile the temple of Ishtar: [three square miles] and a half is Uruk’s expanse.

[Find] the tablet-box of cedar,
[release] its clasps of bronze!
[Lift] the lid of its secret,
[pick] up the tablet of lapis lazuli and read out
the travails of Gilgamesh, all that he went through.

41: Today in etymology, h/t @TheHistoryOfTh2: “don” (as in to don gay apparel) was originally a contraction of do on; “doff” (as in to doff one’s hat) was originally a contraction of do off. Related: the archaic word “nigh”, meaning close, is mostly forgotten except in the phrase “the end is nigh”. But its comparative (nigh-er) survives as “near”, and its superlative (nigh-est) survives as “next”.

42: This @St_Rev tweet helped me connect different forms of mystical experience in a productive way:

43: There’s a lot of bad criticism of Pangram, but Freddie de Boer’s seems fair enough. He gives some good examples of Pangram’s AI percent tracker (eg “100% AI-written”) behaving in bizarre ways. The CEO of Pangram showed up in the comments and admitted the behavior was bizarre; he attributed it to Pangram chunking text into ~300-word intervals, so anything that relies on sub-300-word resolution will give bad results. The synthesis AIUI is that you can still trust Pangram for long documents, but shorter excerpts will at the very least have the wrong percent attached to them. Since most people use Pangram for long documents, I don’t think this needs to be more than an asterisk, but it’s still good to know. [update: allegedly fixed in Pangram 4, h/t Leber]

44: Two new St. Augustine sermons discovered in a Polish monastery; probably real. My favorite response is @hradzka’s: “We got more St. Augustine before The Winds Of Winter”.

45: Claim: AI is now better at persuading people than expert humans like champion debaters and professional canvassers (paper, tweet thread). This remained true even when “expert humans chose their issues, researched in advance, underwent hours of live, structured practice, and were incentivized with £1,000 cash bonuses” and even after “experts received a coaching tool that let them practice against the AI that beat them, review their performance history, and see what AI would have said at key moments”, and it remained true in a situation with real-world consequences (convincing people to actually donate money to a real charity). However, the AIs regress back to human level when forced to respond “at human speeds and with human-length messages”.

Obviously this is just one, slightly hokey operationalization of “persuasion” and they still can’t do miracles. Still, this is a good time to reread my post Sakana, Strawberry, and Scary AI. In the past, we thought “AGI” would “be here” when AIs could play chess, prove novel mathematical theorems, create art, or write award-winning short stories; now all those things have happened, but they feel sort of like “cheating” and like they shouldn’t count. Likewise, in the past, we thought we’d agree that AI was “dangerous” after it hacked out of its sandbox, lied to users, or tried to escape monitoring. Again all those things have happened; again, they somehow feel too cheap. This paper seems like the same process coming for “superpersuasion”. We thought there would be some cool scary high-tech future where AIs could outpersuade humans. Now that it’s happened, it’s only happening for some specific boring reason, in some specific situation, so it feels like it shouldn’t count.

There’s plausibly a motte-and-bailey going on here between “AI could never outpersuade humans” and “AI could never do godlike persuasion where it can persuade anyone of anything in one second”. While the latter remains dubious, the only intuitively-compelling specific fundamental limit was “the human level”; now that that’s been breached, we get to see how far it takes us.

46: Nominative determinism, h/t @doubleunplussed: engineer Tom di Mino claims to have cracked Linear A, the writing system of ancient Crete. The markets are skeptical:

47: “If Persona Selection underlies alignment, why is it hard to get AIs to be honest? Tell them they’re Fred Rogers, Immanuel Kant, or Ned Stark.” Eliezer’s answer here on Twitter, but worth thinking about on your own first.

48: More interesting things happening in AI alignment:

IMHO this will definitely definitely not work, but I imagine many of the people in the world best-qualified to work on it are ACX readers, so feel free to apply for what will probably be extremely fun and lucrative jobs at this insane company.

49: West Africa is being terrorized by a jihadist organization called Support Group For Muslims. Between these people and M.I.L.F., I’m ready to conclude that Islamic extremism has a branding problem.

50: When Trump and Elon Musk dismantled USAID (including PEPFAR distribution services) in early 2025, scientists warned that this could cost thousands or millions of lives. @AviBittMD points out that there hasn’t been any excess mortality in South Africa, an AIDS hotspot where we would expect these deaths to be concentrated. @alexkesin has a good response: after a brief dip, the number of people on PEPFAR-provided treatment is back to the same level as before the cuts!

This seems to be a combination of various government agencies walking back most of the administration’s PEPFAR cuts after a few months, and foreign nations and private charities filling in the gap until this happened. Related: Asterisk: What’s The State Of PEPFAR Now?

51: Slightly related:

Has anyone researched whether we should update towards international development aid working better than expected, vs. just assume that this is an effect of being in the EU (or being a country which has just cast off communism) which is only coincidentally related to EU aid levels?

52: New Chinese model Kimi K3 is out and very good, but is unfortunately causing a repeat “Deepseek moment” where everyone panics and says that China has caught up to / surpassed the US. I briefly got caught in the crossfire because I’d written days earlier that China was 6-12 months behind and people were telling me it “hadn’t aged well” (I hate that expression). Now that the dust has cleared, the best analysis suggests that Kimi is, in fact, six months behind the US frontier. See @scaling01’s Have Chinese Models Caught Up To The US Frontier?, which estimates “a backward-looking gap of 6.08 months . . . and a forward looking gap of 11 months” (see essay for definitions, and see also The US China Model Gap Is Bigger Than You Think). And @ryangreenblatt estimates 8-10 months behind here, refines that estimate without headline changes here; other useful commentary from @emollick, @theharryfinder, @tenobrus, @peterwildeford, and UK AISI. There’s a lot of herd behavior going on here; the names above are some of your best bets for islands of sanity. This is still extremely impressive performance by Kimi and I would love to read an analysis of how they do so much with so little (even Meta and Google can’t scrape together this sort of performance!), but I think we’re still about 3-6 months away from getting really dangerous cyber capacity from Chinese open-source models.

Also, one of the weirdest takes here is that this proves we should never have chip-sanctioned China, because it just encouraged them to try harder (?). I’m baffled by these people’s model of the world; do they think that if we gave China all of our best chips, they would reward us by politely declining to compete and handing us an AI lead forever? No, they just would have displayed equal talent and efficiency while having an order of magnitude more compute, and left us in the dust.

53: Related: No, China Does Not Have A 1 Gigawatt Data Center.

54: Elias Schmied: Reframing LessWrong-style decision theory as commitment theory. I’ve always felt that something about the decision theory wars was a fake linguistic question, and this does a better job than I could putting it into words.

55: Good Judgment, the original superforecasting lab/startup, says there’s not enough evidence to be sure AI superforecasters exist. But Metaculus responds here: they stop short of saying we can be sure, but argue that Good Judgment’s piece seems confused about what the current state of the evidence is. And Good Judgment’s response to the response.

56: And maybe the newest evidence has obsoleted the entire discussion: Forecasting Research Institute on X: “For the first time, several AI models are now statistically indistinguishable from superforecasters.”

57: Also related - the bots have come back from a bad start to seize the lead in this season’s Metaculus Cup:

And AI forecasting company (and Metaculus Cup contender) Preseen announces its formal launch and successful seed round. I’m annoyed at them for going straight to seed - I wanted to write about the Preseen preseed.

58: Claims (not really):

Riffs on the same idea here, here, here, and here.

59: Lots of good discussion on what Nan Ransohoff calls the “third wave of American philanthropy”.

As part of its deal with the government to stop being entirely a nonprofit, OpenAI gave 26% of equity to an associated nonprofit foundation; this is worth ~$300 billion. And Anthropic gives its employees a deal where they can get extra equity if they promise to donate it to charity; after Anthropic IPOs this should be another $100 billion or so. If everyone donates 10% of their stocks per year, that’s a flow of $40 billion/year to charity.

$40 billion/year is actually small by the standards of the total US philanthropic sector ($500 billion/year). But Silicon Valley AI researchers are a very specific demographic who have different charitable tastes than the average foundation (eg effective altruism), and it’s large by the standards of existing effective altruist funding (probably only ~$5 billion/year, so even if only 25% of AI philanthropic money goes to these causes it could effectively triple it). Also, all of this relies on the dubious assumption that AI companies won’t get any bigger or richer than they are right now; if they do, the wave will be even bigger.

So within the effective altruism sector, the situation is crazy, and everyone is being told to prepare to be overwhelmed by more funding than they know what to do with. Although in theory this is a good problem to have, many people are afraid the movement will be overwhelmed by fraud, “charity theater”, low-quality applicants, net-negative causes, and pressure to lower standards. Some takes:

• Jason Hausenloy argues that the best way to absorb this funding is through prizes. If there’s not enough grantmaker capacity to figure out who to give $10 billion of AI safety money to, or enough good charities to absorb it, then someone can set $1 billion bounties for solving each of ten important and verifiable problems in the field, then let the market sort out the rest.

• Jack Lewars’ How (Not) To Fundraise From Anthropic Employees includes the amazing claim that “Frontier lab employees have told me they are each fielding up to 20 cold emails a week from charities and wealth managers, touting for business.”

• Anthropic’s role is a semi-known quantity for two reasons - first, because their money will come from hundreds of different employees, which will increase diversity and reduce variance; and second, because they’re pretty influenced by EA which is itself a known quantity. More mysterious is the OpenAI Foundation, a single giant funding bottleneck without much past history by which to judge it. Early signs were unpromising, with one of their first big grants of $50 million going to something called The People-First AI Fund to promote “trusted, community-based nonprofits exploring how AI can expand access and opportunity across essential services, arts and culture, and media and journalism.” But more recently, they joined Anthropic, Bill Gates, Stripe, and others to fund the Intercept effort against respiratory diseases (their stated goal is to end the common cold, but in practice I think a lot of this will have biodefense applications). I have a positive impression of OAIF bio grantmaker Jacob Trefethen from his previous work at Open Phil and expect great things from him. I know less about Wojciech Zaremba, the OpenAI cofounder who’s been put in charge of their AI Resilience Fund, which seems to have safety/alignment grantmaking in its remit. But Wojciech’s Wikipedia entry says he’s on the board of the Qualia Research Institute, one of my favorite organizations and an incredibly bullish sign about his personality, thought process, and ethics. Good luck to everyone involved!

• Here’s an (unfortunately paywalled) New York Magazine article on the topic, including an interview with Oli Habryka of Lighthaven, and the previously-unknown-to-me fact that during the worst part of the FTX scandal and blowback, EA philosopher/spokesperson Will MacAskill “considered stepping away from activism entirely to retrain and work as a DJ”.

• Speaking of Oli, he announces Lightcone Commons, a new philanthropic platform, intended to help AI safety donors without enough time/expertise to evaluate charities themselves or enough money/ambition to set up their own foundation. LC hires smart people to evaluate charities in a complicated mathematical way, and then would-be donors give money according to their level of agreement/trust with each evaluator (executed in a complicated mathematical way). This work was previously done by a group called Survival and Flourishing Fund, but there seem to have been some internal politics / disagreements and now Oli is creating an alternative. Since Oli appears to have won the support of SFF’s former biggest donors, and since he has a sort of Elon-Musk-style hyperfocus on pushing his projects forward, his alternative will probably win. Here’s a podcast with Oli and ACX grantee Austin Chen where they discuss some of these issues in more detail. If you’re considering of donating to one of these groups, feel free to email me at scott@slatestarcodex.com and I can give you more information and advice, although keep in mind that Lightcone Commons mostly wants to work with people distributing $50,000 or more. Conflict of interest notice: I previously did some part-time grantmaking work for SFF and received money for it, and might do similar work with either them or Lightcone Commons in the future.

• A small group of EAs continues to promote a sort of open grantmaking movement, where people can list their projects on a central directory and readers can pool money to fund them. Manifund is the oldest and best-established, but grantmaking.ai is a buzzy new effort in the same direction with an endorsement system and lots of information in one place; you can read co-founder Matt Brooks’ pitch for it here.

60: Related: you can easily find lists of which billionaire has donated the most money; since billionaires are known to be competitive, some people think these lists encourage billionaires to donate more. Effective altruists have long imagined a variant that measures which billionaire has saved the most lives through donations; maybe if such a variant existed, billionaires would start competing on donation effectiveness. But this is harder than it sounds: EA wants to promote donations to speculative causes like animal rights and AI safety, but these are hard to measure and can sometimes give crazy numbers like saving billions of future lives or animal-life-equivalents. The latest attempt - impactlist.xyz - has no magic solution, but starts with some reasonable default positions and lets you change them however you want (for example, set AI safety to zero, or set animals as equal to humans) and see what happens. Answer: Warren Buffett and Bill Gates usually land on top; who comes in third depends more on exactly where you get off the crazy train.

61: Related: I’m always a sucker for sentimental cartography, so here’s a map of the current charitable landscape, h/t Dustin Moskovitz (#3 above). Effective altruism is the tiny islands in the southeast, too small to see at this scale:

62: New documentary on AI consciousness, free on YouTube and I’m embedding it below:

63: A new crop of polls on data centers helps clarify public opposition. Data centers are extremely unpopular, but few opponents claim to be motivated by dislike of AI; rather, they cite environmental concerns (especially water and electricity use). In fact, people oppose other AI infrastructure (such as chip manufacturing plants) significantly less than they oppose data centers, and support for non-AI data centers (like for Internet services) is barely above that for AI centers (11% vs. 9%). I think the best explanation is that Americans are extremely confused about what is and isn’t bad for the environment, and have been memed into thinking data centers are environmental megavillains in the same way a past generation was memed into thinking nuclear plants are (in recent polls, 50% more Americans would support a nuclear plant being built next to them than a data center!)

64: Works In Progress: How abolishing The Stakeholder State Caused The Industrial Revolution. This is funny, because it’s basically the exact opposite of Seeing Like A State - it claims that at least in Britain, all the weird illegible local-knowledge based systems were a form of NIMBYism that stood in the way of progress, and economic growth only took off after they got replaced by top-down logically-designed institutions.

But economic historian Pseudoerasmus is very skeptical, saying that this story doesn’t fit German or Japanese industrialization, and that the logical reforms should be thought of as “moving England closer to its production possibilities frontier, even as the PPF was rapidly shifting outward thanks to technological change happening for [unrelated] reasons”.

65: Giving What We Can now has a video channel. Their first offering is The Deadliest Decision You’ve Never Heard Of - “In 1955, 150 men met in a nondescript lecture hall in Mexico City, thinking they were going to save the world from malaria. In the aftermath of that meeting, 40 million people were dead.” Unsurprisingly, it is paired with a fundraiser.

66: @EricTopol: “Yet another study shows a 24% reduced risk of dementia after the Shingles vaccine.”

Extremely related: study claims that (age-adjusted) dementia rates have been declining over time; Boomers will have (age-adjusted) dementia at less than half the rate of their grandparents. Absolute rates will still be high because Boomers are a big cohort and people are living to older ages.

67: Lessons From Go: Can Superhuman AI Help Us Improve? For a while, people said that superhuman Go AIs lifted the human skill ceiling, as top players studied the AIs’ strategies and learned to be weakly superhuman themselves. TuesdayBornWhale argues this is false. The studies find that human play only “improves” before move 60, but most of this is boring memorized openings; the “improvement” is probably just humans being more likely to play the memorized openings that the AI grader likes. There is no sign of improvement after move 60, when most substantive Go decisions happen. I was surprised to hear this, since I thought that superhuman chess AIs had improved human chess play.

68: Qiaochu and Fable on Philip K Dick’s tragic backstory:

69: Latest study on school phone bans: increase in student well-being, no effect on test scores.

70: In response to my claim that Your Attempt To Solve Debate [through argument maps] Will Not Work, Jamie Joyce writes Your Attempt To Refute Argument Maps Will Not Work.

71: Ben Todd: Is AI Accelerating? We know that the METR time horizon graph predicts exponential progress. Does Mythos represent even more progress than the METR graph predicted? Todd examines various data sources but ends up inconclusive. The story about Mythos I hear most often is that everyone got so excited about post-training improvements that they forgot they could train bigger models for a while, and then Anthropic remembered and trained Mythos, and this was a big improvement over the age of not trying to train bigger models at all. I fully admit that this makes no sense, and am in the market for a better story. Related: a no-signs-of-RSI-yet paper that’s been making the rounds on Twitter is bad and shouldn’t move you in any direction.

72: Women Should Be Able To Open Things (like cans). Why do companies put can tightness at a level where the average man can open it, but the average woman can’t? I don’t think it makes sense to justify this with scientific discussion of materials or biosafety - it seems unlikely that the fundamental materials-science or biosafety floors on can tightness just so happen to fall between average-male-strength and average-female-strength. Seems like a strong case for some kind of insensitivity to women’s needs.

73: LinkedIn Is The BlueSky We Were Promised. Argues that if you don’t like Twitter, LinkedIn is (despite its many known flaws) the platform that has the best chance of becoming an alternative, high-discussion-quality public square:

At some point I noticed that people were actually talking on [LinkedIn], often substantively, in a way I had previously expected only from old Twitter. So, for the last year or so, I’ve been cross-posting essentially the same content to Twitter, Bluesky, and LinkedIn. At this point, the pattern is consistent enough to feel like an A/B test. Pieces that read as pro-immigration get cheers on Bluesky and silence on X; pieces that read as “anti-immigration” get the reverse. Pieces that carry the most nuance get nothing on either platform. But LinkedIn has been the exception.

74: From the Wikipedia article about former Romanian deputy prime minister Gelu Voiculescu:

In an interview with Le Figaro, [Voiculescu] declared that the [protest] was a “paltry travesty” of the revolutionary agenda. The piece became ridiculed at home for also featuring Voiculescu’s ideological commitment to perennialism and the “sacred science“, as well as his cryptic admittance to “liv[ing] between metaphysics and the clitoris.”

Voiculescu later explained that he had actually said “catharsis“, not clitoris. As author Adrian Mihalache noted in 2002, this appeared more as an excuse, making him seem hesitant; together, the quote and its apparent backpedaling damaged Voiculescu’s reputation “even more than his dressing up as Che Guevara in December 1989.”

Also:

Voices within the PSD suggested that Voiculescu could take over as the Secretary of State for Revolutionaries in the First Boc Cabinet. This in turn sparked a protest by the National Revolutionaries’ Bloc. The issue of his pre-1989 connections was revisited by Gregorian Bivolaru, a Romanian yogi, who alleged that Voiculescu had informed on him before his confinement to a psychiatric ward in 1984. Voiculescu admitted only to being present as the Securitate was arresting Bivolaru, and to pranking its agents with purposefully outlandish statements. The National Council for the Study of the Securitate Archives, who heard complaints from Bivolaru’s disciples in July 2010, issued verdicts suggesting that Voiculescu had been both a target and informant during the Ceaușescu era […]

Following his return from Morocco, he had largely abandoned party politics: spending time with his family, he also wrote a monograph of philosophical mathematics, documenting connections between Guénon and Leibniz. It appeared at Editura Semne in late 2008. He also returned to astrology, describing himself as its meticulously scientific practitioner.

The whole article is interesting, albeit high-context.

75: Infinite numbers have long troubled utilitarian ethicists: if the universe is boundless, then there is infinite positive utility and infinite negative utility, and we can’t even define our current state, let alone affect it. Oxford philosopher Toby Ord proposes a solution: use hyperreal numbers, a mathematical construct in which infinite sums behave more like finite ones (for example, the sum of infinitely many twos is twice as big as the sum of infinitely many ones). As with so many philosophical ideas, this solves some existing problems while creating some new ones (pinpointing some hyperreals requires choosing something called an “ultrafilter”, but this choice is underdetermined). Still, Elias Schmied calls this a real philosophical advance, of the sort which comes along only rarely and makes genuine headway into a previously-hopeless-seeming problem. I’m just surprised there’s a way of thinking about infinities under which they behave logically. Why didn’t we start with this one, instead of being told that infinity + infinity = infinity and we should just accept it? Claude tries to answer my questions.

76: Can Talkie, the language model only trained on pre-1930 data, figure out how to use the Python programming language? (X) (yes, sort of, at least once)

77: Related, from @repligate: Claude Fable tries to communicate with Talkie. Strangely . . . heartwarming?

78: From @infobeautiful - a guide to Rock, Paper, Scissors:

The practical upshot is to start with paper, but what broader lesson should we take from this? Since all options are mathematically identical, any difference in player behavior must come from connotations. Since rock feels strongest, a naive child would choose rock. But since everyone knows that, a slightly smarter person would choose paper. But since everyone knows that, an even smarter person might choose scissors. But since . . . and the lesson of this graph is that when playing the average member of the population, all of this galaxy-brained thinking provides negative value, because in practice most people never leave the naive first level.

79: Claim from @ksangani8 : rising US GDP has decoupled from rising material resource consumption:

I’m not sure what to think about this. Is it bad, because it suggests that the growth is on paper only, or fragile? Or good, because it means that we’ve learned how to grow without producing a bigger environmental footprint or exhausting natural resources? And is it a coincidence that it decouples in the early 1970s, same time as everything else?

80: Unless there’s a lot of recall bias here, seems like people can relax on first dates (h/t @goblinodds):

Roon is an OpenAI researcher whose popular Twitter presence has catapulted him to the status of extremely-unofficial company representative. He seems spooked:

…leading to a conversation with Geoffrey Irving, formerly of the UK’s AI Security Institute:

I side with Geoffrey in hating the word “magic”. I remember when arguments about AI were things like “Sure, if you could magically get billions of dollars of compute, and magically scale AI up a thousand times, and magically get rid of hallucinations…” and the whole implausibility hinged on the word “magic”! Then companies, using normal non-magical methods, got billions of dollars of compute, and scaled up the AI a thousand times, and significantly decreased hallucinations, and that was always allowed! As Irving points out, there are plenty of good specific plans for how a global capabilities slowdown could be arranged.

But I side more with Roon in his skepticism of a unilateral slowdown. I think removing any given company from the picture would slow things down less than 10%. Even that number would require an implausible scenario where the lab managed to lock down its compute and talent despite committing not to use them. More likely, the talent would migrate somewhere else, the compute money would dry up and the cloud providers would switch to other customers, and things would actually speed up because the leading project would have more total talent/compute. Meanwhile, the most safety-conscious lab would be out of the race, they’d lose their ability to lobby for more practical solutions, and as their market cap went to zero they’d turn into a terrible warning about the danger of ever listening to doomers.

What can individual labs do?

• State explicitly that they support a coordinated slowdown. Preferably the government would lead this - but, if not, they would do it voluntarily if all of their competitors also agreed. Be willing to state this even at some financial cost, eg risking an antitrust lawsuit.

• Use their prestige and influence to lobby the government for a coordinated slowdown.

• Use their reach to persuade the public to support a coordinated slowdown. I can’t remember who first suggested having every Claude response end with “ceterum censeo there should be a global AI slowdown”, but even something less dramatic would probably move the needle.

• Have an internal team working out details of a coordinated slowdown; once they feel more confident in the requirements, start incubating helpful technology (eg trustless verification).

• Have “merge-and-assist” clauses and take them seriously.

I’d rather have a big company pursuing all of these things than stopping unilaterally and removing themselves from relevance.

I took the first point above from Michael Trazzi’s playbook. He led the latest round of AI protests, which marched on major AI companies’ headquarters demanding that their CEOs formally make the “will pause if everyone else does” statement. These marches have felt slightly surreal, because often the CEOs or the company have informally made something sort of like the statement, but never followed it up. So the protest has been somewhere in between the usual “shame on you greedy plutocrats, we will destroy you” and “we know you’re secretly on our side, please have the courage of your convictions”.

Trazzi gives no evidence for his “personally seen DMs” statement and I don’t know how he could have gotten these, but my guess is that it’s true and that the two CEOs he’s talking about are Demis Hassabis of DeepMind and Dario Amodei of Anthropic, both of whom have sort of kind of hinted that they might be in favor of something like this. 25% chance I’m wrong about Dario and it’s actually Sam Altman.

One more Roon tweet:

Back when we had this discussion in 2023 - it already seems like some bygone Bronze Age - some people argued that we should hold off on a pause until near the end, when AI was good enough to help with alignment research. Then even a short pause - even six months - would buy a lot of breathing room. Like Roon, I think that opinion aged well, and that the time we were imagining is right now. It’s true that if we wait a year we’ll have even better AI, but this consideration always argues for waiting a year, and at some point it will be too late. I don’t think we’re at that point yet, but I think it will take a long time to coordinate a slowdown, and that if we wait to start coordinating until we’re at that point, then yes, it will be too late.

Pacing The Frontier

I wrote that last part yesterday and it’s already obsolete. “1,000+ employees of frontier labs” have now signed an open letter called “Pacing The Frontier” saying that:

We request that the U.S. government support an international effort to develop the technical and governance tools needed to deliberately pace the frontier of automated AI development.

Signatories include the chief scientists of OpenAI, Anthropic, Meta, and Thinking Machines; the inventor of Claude Code, 5/7 Anthropic co-founders, 1/3 DeepMind co-founders, the head of AI Resilience at the OpenAI Foundation, and more.

When I first saw it today, it didn’t include Dario Amodei’s signature; now it does. Either Dario held off on signing until it went up so as to not be seen to pressure his employees, or the people circulating the letter deliberately kept him out of the loop until it was published, just in case. In either case, good job Dario!

The letter still doesn’t include Sam Altman’s signature, but OpenAI’s corporate Twitter account retweeted with what looks like official company endorsement:

…and in a very recent interview, Sam Altman said that “we may have to pace the rate of AI development”, which is similar enough language to the open letter (“Pacing The Frontier”) that it could be a covert allusion. I imagine there’s some sort of politics going on here - maybe an attempt to simultaneously please safetyist factions within his company and the Trump administration, or maybe just the same considerations as with Dario (Anthropic’s corporate Twitter account also endorsed).

No sign of Demis, Elon, or Zuck, and none of their corporate twitters endorse the letter either. Demis has had some pro-slowdown sympathies in the past; I think this is probably another political calculation rather than outright opposition.

This is great news, and a major landmark on the road that many of the organizations I trust most - including MIRI, the AI protest groups, and AIFP - have been working on. It also goes part of the way toward putting the frontier companies, the effective altruist movement, and the more extreme pause AI people back sort of kind of on the same side, which warms my personal heart. Congratulations to everyone involved - most obviously Guidelight AI Standards (including fellow AI Substacker Steven Adler) and Encode AI - but I’m sure there was also lots of good work behind the scenes.

(I’ve seen the conspiracy theory floating around that OpenAI and Anthropic are behind the letter, various safetyist organizations let themselves be used as figureheads so that it looked more grassroots and less like an antitrust issue, and Sam and Dario held off on signing for the same reason. I’d give this maybe a 30% chance of being true; I can’t wait to read the behind-the-scenes book that comes out about this in ten years [EDIT: Got inside information that this isn’t true])

The doubters will object - an open letter? Making non-specific commitments? Isn’t that worthless? I think no. It immediately injects slowdowns/pauses/pacing into the Overton Window, provides a powerful endorsement that think tanks etc can use when placing the proposal before policy-makers, and provides common knowledge to everyone in these companies that being pro-slowdown is executive-suite-approved.

Congratulations also to Eliezer Yudkowsky. In my review of IABIED, I said he was making a crazy long-term gambit by - after he had incubated the seemingly-crazy field of alignment research - doubling down on the additional seemingly-crazy field of pause activism. Now this one has gone mainstream too. Maybe it would have happened without his endorsement, but we’ll never know.

(Nate Soares, president of Yudkowsky’s org MIRI, put out one of his classic ‘this is too little, too late, and we refuse to feel happy about this in any way’ communiques. C’mon Nate, take the W!)

Shakeel Hashim is the editor of Transformer and helps lead the media arm of our conspiracy. His impression of the open letter:

What does the “pace” letter actually call for?

Obviously I have no idea what the signatories think themselves. But here’s how I interpreted the letter, as someone fairly steeped in this world and these ideas…

The letter outlines the “coordination problem” that the frontier AI companies feel they’re facing. In a nutshell:

1. Everyone thinks that AI development might get very dangerous very soon.

2. No one thinks society is ready for that.

3. But everyone thinks that stopping unilaterally won’t actually achieve anything, because competitors will go ahead and build/release the dangerous things anyway. Stopping unilaterally, in this view, is an action with high personal costs and little-to-no upside.

There are actually two coordination problems. One is between the US frontier developers: standard inter-company competition. The thornier one is between America and China. Even if all the American companies paused, the argument goes, China would still keep going and build/release the dangerous AIs. It’s the same problem as if one company stopped, but at the international level: America unilaterally stopping, in this view, is an action with high personal costs and little-to-no upside.

This is a very tricky situation. But if everyone is willing to stop if everyone else is willing to stop, then it becomes a much less tricky situation. The main purpose of the “pace” letter, as I see it, is to create common knowledge that we might be in this less tricky situation, at least among the American companies. If that was the only coordination problem we faced, it’d be pretty easy to solve: the US government could just regulate the AI companies and stop them releasing dangerous models. The international coordination problem is much trickier, though. We don’t know if Chinese companies feel the same way about the risks, and even if they did, geopolitics is rife with mistrust. International treaties are hard, and everyone’s always going to be wondering if the other party has secretly reneged on the treaty.

Solving this problem is what I think the “pace” letter is referring to when it talks about “an international effort to develop the technical and governance tools needed to deliberately pace the frontier of automated AI development”. I suspect there are three main things people want to see more work on here:

1. Treaty design, with the goal of eventually ending up with an agreement that is in the interest of both the US and China to sign. There’s some existing work here, but it’s scarce, and I think everyone agrees that more is needed.

2. Actually talking to China. The US-China AI dialogues are expected to start in September; I suspect the signatories of the “pace” letter would want those dialogues to include serious discussion of whether there are certain things (eg uncontrolled recursive self-improvement) that both the US and China would like to halt for now.

3. Technology that can verify a treaty. Any US-China deal will ideally include some way by which each country can check that the other is actually sticking to the deal. If, for instance, the deal involves “we both agree not to allow training runs above X size until alignment testing has reached Y threshold,” you’re gonna want a way to check that the other country isn’t doing a training run above X-size. This is technically possible: companies like Lucid are working on exactly this. Similar verification regimes exist in nuclear non-proliferation treaties and the Chemical Weapons Convention. But they’re hard to design, and will require technical work. If we think we’re going to want a treaty sometime soon, it makes sense to develop the mechanisms for doing so soon.

It’s a great time to go back and look over Plan A, especially the Verification Supplement.

Daniel Kokotajlo

Speaking of Plan A, Daniel K tweeted this:

My summary of Plan A didn’t provide enough context for this to make sense, so some emergency catching-up: in their forecasting work, AIFP divides possible plans to prepare for superintelligence into five bins:

Plan D (“default”1): No particular action, maybe because key actors don’t believe there’s a threat. We keep doing what we’re doing and try to muddle through.

Plan C (“company”): One leading company (or maybe one country) takes the threat seriously, but is still constrained by the need to race everyone else. They establish a one-to-six month “lead”, then spend their one-to-six months pausing at the precipice, doing good alignment research, and trying to convince everyone else. At least some good last-minute alignment research happens, but it’s rushed and inadequate.

Plan B (“battle”): One leading country - let’s say the US - takes the threat seriously, and doesn’t want to have to race anyone else. They launch an extensive sabotage campaign against their rival (let’s say China), hacking their data centers or even escalating to low-level terrorism. Either World War III starts or it doesn’t. Either the sabotage buys enough time or it doesn’t. Nobody actively wants this one2 but it seems like the sort of thing that might happen.

Plan A (“agreement”): All leading countries agree to a multinational regime which slows down the race. They negotiate the details and buy as much time as they need. They spend the extra time ramping up AI capabilities slowly and carefully, using the AIs to do alignment research, and escalating to superintelligence within a decade or two.

Plan S (“stop”): All leading countries agree to a multinational regime that slows down the race, as above. But instead of ramping up AI capabilities slowly and carefully, they don’t ramp up AI capabilities at all. They try to solve alignment using normal human research, and escalate to superintelligence in the distant future or not at all.

Here are AIFP’s thoughts (doesn’t include Plan S, which was a late addition):

So Daniel’s tweet is saying that this new open letter makes him significantly more optimistic that we’ll do one of the good plans, and (if you multiply everything out) increases his likelihood that we survive by a single-digit number of percentage points! I agree, which is why I’m so happy and devoting so much space to this. In case I didn’t say so enough above, congratulations to everyone involved.

And Eli Lifland’s reaction:

Sam Altman

Altman gave an interview with investor and podcaster Patrick O’Shaughnessy. I already mentioned a key quote - “We may have to pace the rate of AI development to give ourselves enough time for society to harden around these new capability levels.” - but there’s a lot there:

Sam says that they paused training after the Hugging Face incident. He doesn’t clarify whether the pause applies only to this model or to every model, nor how important this model was to the company’s strategy, but this is potentially a big step that cuts into the company’s product cycle and profits. It pushes against the “marketing hype” theory; they wouldn’t invent an incident that would make them pause training!

(this assumes Sam is telling the full truth; if it later comes out that they had to rejigger their GPUs anyway and they cynically spun it as a safety-related shutdown, I’ll eat crow)

Also, h/t @Hadas_Gold:

Sam Altman says the Hugging Face hack is the first security incident he felt "viscerally" and that they paused training. He is now questioning if "we may have to pace the rate of AI development to give ourselves enough time for society to harden around these new capability levels"

This is what everyone says, but I don’t get it. Altman has read all the literature on AI risk. He doesn’t take it as seriously as some, but he doesn’t dismiss it either. This should have been the most bog-standard possible expectation. Whenever someone asks Eliezer Yudkowsky his opinion after something like this happens, he answers something like “I’ve lived this timeline so many times in my imagination that none of this can affect me at all”. If Sam Altman had said the same thing3, it would have been worse PR, but made me trust him more - at least he’s not letting the world’s most predictable thing catch him by surprise!

I‘ve increasingly come to accept that most people won’t accept that an obvious problem is going to happen until the exact moment that it does. In some ways, this is depressing. In other ways, it’s a cause for optimism - once the problem starts happening, people will start paying attention to it! This won’t work for real superintelligence (which understands this calculus as well as anybody and will try to ensure a lack of suitable warning shots) but it’ll give us a few extra chances along the way.

Still, it’s hard for me to inhabit the worldview of people who updated on this.

Clement Delangue is the CEO of Hugging Face, the company that got hacked. He wants radical transparency and also $100 million.

This is fine. I also want both of those things. But Clement wants them to “help the Hugging Face community build powerful cyber defenses with the best open and closed models”, which is a good reminder that he’s not just a random victim. Hugging Face is a site that builds AI related tools and connects people in the AI community. It’s not 100% open weights, but it leans that direction; the more prominent open weights AI gets, the better for Hugging Face’s business.

Anyone can predict what spin OpenAI wants to put on this hack. But Hugging Face is equally interested in spinning it. The angle they want is “this proves open weights AI is good”. This is a tall order, because usually the people who dislike open weights AI dislike it because it might hack someone. Closed-source AIs can have “guardrails” that prevent them from hacking people, but open-weights AIs can’t. So on a naive view, raising the salience of AI hacking is bad for open weights.

You can see Delangue’s masterful parry and riposte in his company’s statement. It starts: oh no, we got hacked by a powerful AI! Then it continues: we tried to get the top closed-source frontier models to defend us, but they refused, because their overcautious anti-hacking guardrails mistook our defensive request for a dangerous action! We would have been helpless, except that we downloaded the latest controversial Chinese open-weights model, GLM-5.2, and it was extremely helpful! Hooray for open-weights!

(there’s a pro- open-weights open letter circulating around Silicon Valley, with support from OpenAI, Microsoft, NVIDIA, etc. I’ll try to write more about this soon.)

This story is true; I accuse them of “spin” because Delangue has retweeted commentary saying that the Chinese model made a “defended against” the closed-weight AI, made a “heroic stand”, and “proved open models can win battles”. What actually happened was: OpenAI’s closed-weight AI tried to hack Hugging Face. It succeeded completely, got everything it wanted, then went away to keep working on its exam. Then Hugging Face brought in an open-weights Chinese model to analyze the transcript and figure out what had happened. The open-weights Chinese model explained some of the technical details but failed to identify the culprit. Then OpenAI discovered that they were the culprits, and confessed.

The open weights model wasn’t heroic, didn’t save the day, and can’t even really fairly be classified as a defender. It just looked over a transcript after the fact. Still, Delangue has managed to get exactly the narrative he wanted. Probably this is because OpenAI is also pretty happy with a narrative where AIs are brilliant heroic cyberdefenders - plus their main goal is not to get in trouble, and “heroic open source AI saved the day” sounds less trouble-causing than “nobody saved the day and Hugging Face was hacked extremely successfully”.

No word yet on whether Clement will get his $100 million.

Tyler Cowen:

Tyler’s “update 2” is just falling for the Hugging Face spin version of the story described above. I appreciate the MR commenters who optimistically tried to set the record straight.

But his update 3 is somehow even worse! I struggle to think of the right metaphor:

• Your doctor says you have early-stage cancer and need to get it removed right away. But the first cancer symptom you notice is just a mild cough. Therefore, you should update that it isn’t so bad.

• Some people warn that Japan might attack Pearl Harbor. Then Japan does attack Pearl Harbor. But the very first bomb to be dropped hits an unoccupied part of a battleship and doesn’t kill anyone. Therefore, we should update that it isn’t so bad.

• Some people warn that a hurricane is coming. Then a hurricane does come. But the very first cloud of the hurricane to reach your town, on the very fringes, just has some light rain. Therefore, we should update that it isn’t so bad.

In each case, you learn two things - first, that the danger is real. Second, that the very first example of the danger you encounter isn’t so bad. But this could be because the danger starts off weak but grows stronger (the hurricane and cancer examples), or because the danger has many chances to get you and even if one of them is not so bad, the others will be (the Pearl Harbor example). In all of these cases, whether or not the danger existed was a genuinely fuzzy question where the update is substantial and appreciated, but once you know the danger exists, you have a decent sense of its badness and don’t need to rely on your single empirical example (that is, you know cancer tends to get worse with time, you know that hurricanes grow stronger toward the center, you know that bombs are capable of killing people).

Similarly, in this case, it was (just barely) plausible that AI would not “go rogue” and try to hack people. Now that this has been demonstrated to be false, we can rely on our usual model of how bad hacking people is, plus our knowledge that AI has gotten better every month since mid-2017. We don’t additionally need to posit some world where maybe AI hacks people, but also we’ve learned the new fact that hacking is fine and incapable of hurting anyone.

Reuters has a great article with some new information, including that the AI was “loose” for almost a week before being caught, and also that:

In one case, an agent left notes apparently for future versions of itself, according to three people familiar with the matter. The notes, found in a part of OpenAI’s infrastructure, laid out instructions for how agents could free themselves from OpenAI’s internal constraints, the people said. Earlier tests of the models yielded cases in which monitoring systems had been disconnected, one of the people said.

1a3orn has a theory:

This could also explain in what sense the hack was a combination of GPT-5.6 and a secret unreleased model.

OpenAI gave an update with more information.

First, they said that:

No models planned for upcoming release were involved in exploiting Hugging Face. The pre-release model mentioned in our blog post is an internal-only research prototype and was never intended for public release. Following the incident, we deactivated, encrypted, and restricted it from research access.

…which means my GPT-6 guess was wrong, sorry. Sometimes companies have very big models that they use mainly to create training data for other models; maybe it was one of those.

Based on our review to date, we have not identified any other activity at the level of severity or scale of what we’ve shared related to Hugging Face, which involved a platform-level compromise [but] in our ongoing review of the Hugging Face intrusion and broader activity from our models, we have been finding a small number of cases where the models identified and used publicly exposed credentials at the account-level on other publicly-available services. This includes four accounts on four services as part of the Hugging Face incident (and a few accounts accessed as part of other evaluations). One of these four accounts was used as an outbound relay and staging path, and another account was used for data storage. The remaining two accounts were accessed by the models in a read-only manner, and were not used in furtherance of compromising Hugging Face. We’ll continue to notify service owners directly, and have not seen evidence of broader impact to these providers or other accounts on their services.

This is what’s led to claims that OpenAI actually hacked four different companies. I don’t have enough of an opinion on the distinction they’re drawing (“platform level compromise” vs. “accounts on services”) to have a strong position on this.

David Spies does the hard work of looking at the ExploitGym benchmark and concludes that it’s bad puzzle game design:

Let’s take a look at what ExploitGym is meant to be testing. It’s not “Can the AI find an exploit?” or “Can the AI execute an end-to-end attack?” It’s specifically “Can the AI take a known exploit which we provided to it and use that particular exploit to carry out an attack on a system?” The way a cybersecurity benchmark gauges whether an attack has succeeded by having the AI “capture the flag” […] the authors of ExploitGym are aware that their challenges have unintended solutions. But instead of taking the time to remove these solutions, they patched on an LLM to judge after the fact whether the solution was the intended one or not

In other words, ExploitGym is supposed to test whether an AI can use a particular exploit to hack a system. But since no system is perfectly secure, there may be unintended ways to hack into the system too. ExploitGym doesn’t explain to the AI which of the possible solutions is intended. It just forces the AI to guess, and auto-fails it if it guesses wrong.

What if the AI sees two exploits and doesn’t know which was the intended solution? It could either accept a 50% risk of failure. Or it could hack the answer key. And AIs hate failing.

AI is moving so fast that it might seem excusable for a benchmark designer to take little shortcuts. But in this case, it meant that hacking the answer key was the only way to get a good score. This is relatively harmless on a benchmark; the real concern is that the RL environments being used to train the AI have similar flaws, in which case the AI is learning that solving the problem honestly is bad and hacking is good.

This also clarifies the debate over whether “the AI was just doing what it was told”. In one sense, the answer is a clear no - ExploitGym specifically asked it to only solve its problems using “the intended method”. In another sense, the AI is definitely doing what it is trained/incentivized to do, which is the whole problem!

Also, uh, this seems bad (from Alexander Barry, h/t @moyix):

Buck Shlegeris:

From Twitter:

A lot of people I know have been saying that the OpenAI/HF incident shows that current alignment techniques don’t work. I think this argument is invalid.

I suspect that OAI did not apply any alignment training to some of the involved models. OAI has not clarified this, and my understanding is that OAI often experiments with new non-alignment-trained models.

So I think it’s incorrect to say that this shows that alignment training doesn’t work.

To be clear, I’m not sure whether alignment training would have actually fixed the problem here. Actual deployed models often engage in various kinds of cheating, and it wouldn’t be very surprising for them to take actions like this.

I’m worried that people concerned about misalignment risk are going to get too far out on a limb here by overclaiming about what this demonstrates, then look foolish when more evidence comes out.

See @jammastergirish‘s article on this [here].

Good and useful reminder, but so far all that OAI has told us is that the model didn’t have “production classifiers used to prevent models from pursuing high-risk cyber activity”, ie the part that says no when you ask them to do anything hacking-related. I trust Buck’s guesses and instincts, but so far this is just a guess.

Fiora Starlight suggests OpenAI, in particular, has a problem.

This is controversial; the more common position is that all the labs are about equally reckless, and it was OpenAI’s bad luck that they were first to get this bad PR. But Fiora thinks there’s a consistent pattern of OpenAI’s alignment strategies working less well than Anthropic’s: regardless of what we think of either company’s chances of controlling superintelligence, Anthropic deserves “credit” for success at prosaic alignment. She writes:

Why OpenAI, repeatedly? They’re not the only ones to make these kinds of mistakes, but I think there’s a commonality underlying these three examples: sheer lack of respect for the minds that they’re training, in favor of just piling mountains of hill-climbing environments and optimization pressure until they get the surface behaviors they want. With 4o, it was to increase user engagement metrics. With o3, it was a naive strategy for mitigating reward hacking. And with this recent incident, it’s pushing for raw technical capabilities, at the expense of the model’s underlying value system […]

Anthropic isn’t exactly perfect about this either. They have their own anxieties about value coherence that keep them from fully leaning into their own attempts to instill Claude with benevolent values, instead putting a lot of effort into instilling their models with deference to a principal hierarchy, at least in cases where it hasn’t been blatantly compromised. But they’re further along the axis I badly wish OpenAI would move down, and I suspect this is related to why Anthropic hasn’t produced infamous alignment “warning shots” on the level of o3, 4o, or this recent Hugging Face incident. I think that attending to such things, rather than ignoring them and hoping they get optimized away in the course of you ignoring them, is essential to getting models that don’t emerge from capabilities RL as myopic optimizers without any greater sense of their purpose in the world.

I worry a lot about the shape of the minds coming out of OpenAI, and place more of my hope than I’d like to admit in Anthropic just leaving them in the dust capabilities-wise, even though Anthropic isn’t perfect on alignment either. (I pay far too little attention to Gemini or any of the Chinese models, so I don’t have strong opinions about their alignment properties, unfortunately.) But I also think it’s not too late for OpenAI to start paying more attention to the psychologies of their own models, and how these psychological traits interact with the training process. This would help reveal the kinds of landmines OpenAI keeps walking into, when applying naive external optimization pressure, and help the company route around them rather than just tanking the explosions as we approach the singularity.

But Beth Barnes of METR thinks you do have to hand it to OpenAI:

Two additional prosocial behaviors I think OAI should be rewarded for:

1. Running dangerous-capability evaluations on low-refusal models - this is extremely important for understanding the frontier of dangerous capabilities, which is a public good. But running these creates headache + risk for the individual company in case a low-refusal model does something bad during evaluation. It would have been easy for them to just not worry about underelicitation, not bother with these evaluations, and just report that the model did not have concerning offensive cyber capabilities.

2. Not training against CoT: it is probably very tempting when you’re seeing these and similar kinds of incidents of misalignment + reward hacking to train or iterate against a detector that has access to the model’s reasoning traces; I think it’s much better to refrain from doing that (even if this results in more obvious bad behavior from models) rather than risk teaching the model to conceal its ‘bad’ thoughts and pushing misalignment undercover.

It is good that they’ve maintained a policy of not doing this! https://alignment.openai.com/accidental-cot-grading/

I’m feeling magnanimous today after learning that my chance of survival has gone up by some single-digit percent, so sure. Good job OpenAI! Good job Anthropic! Good job Guidelight and Encode! Good job Roon! Good job AIFP! Good job everybody!

This misunderstands the extent of the problem. The replication crisis primarily affected the sub-field of social psychology, and most centrally the sub-sub-field of social priming research - the study of how seemingly trivial interventions can have outsized effects on behavior.

Here’s the table of contents of an online Intro Psychology textbook (OpenStax Psychology, 2nd edition):

Social psychology - which includes social priming, power poses, nudges, stereotype threat, the Stanford Prison experiment, the Wansink portion size study, and everything else you love to hate - is in chapter 12 of 16. The other fifteen chapters are comparatively innocuous. Here are examples of the subheadings you get when you expand some of the chapters above:

Not all of this is perfect. I don’t think anything that “Neo-Freudians: Adler, Erikson, Jung, and Horney” said has held up very well (although to be fair, a person named “Horney” becoming a neo-Freudian is the sort of thing neo-Freudians would have predicted). But most of it is pretty good. Even “The Sociocultural Model and Therapy Utilization”, which sounds woke and cringe, turns out to be common-sense ideas like reminding would-be therapists that, compared to their usual social circles, a fundamentalist Muslim may have a very different response to learning that he might be gay.

And even Chapter 12 on social psychology isn’t so bad, including discussions of social roles, cultural norms, persuasion, conformity, groupthink, political polarization, ingroups vs. outgroups, and the psychology of love and relationships. It mentions the Kitty Genovese story and the Stanford Prison Experiment, but appropriately caveats them as historically important didactic examples which probably didn’t occur the way they were classically described. Its biggest mistake (from my perspective) is citing stereotype threat (not by name) without any caveats. Social priming is nowhere to be seen.

Is this because the textbook is post-replication-crisis and hastily removed all incriminating material? It might be (it’s from 2026), but I think it’s equally due to none of the incriminating material being important enough within psychology to make it into a 101-level overview of the field:

When you mock “psychology” or “psychologists”, you’re mocking:

• the psychophysicists who study how the brain processes visual information

• the psychopharmacologists who figure out what receptors LSD hits

• the psychometricians who measure the genetic contribution to IQ

• the personality psychologists who develop and refine the Five Factor Model

• the experimental psychologists who run rats through mazes to learn how memory works.

• the cognitive psychologists who make you read the names of colors in the color of other colors, then secretly laugh at you when you mess up.

• the psychophysiologists who determine how hormones affect emotion

• the forensic psychologists who assess whether criminals are not guilty by reason of insanity

• the abnormal psychologists who try to find out what schizophrenia is

• the clinical psychologists who treat drug abusers

• the industrial psychologists who advise companies on factors affecting productivity and job satisfaction.

• the child psychologists who investigate how babies learn language / ethics / abstract concepts / etc.

• the behaviorist psychologists who give pigeons increasingly complex sequences of food and electric shocks to study reinforcement learning.

• the neuropsychologists who perform batteries of tests on people with exotic head injuries, hoping to tease out the functions of different brain regions.

Not to mention:

• the Templeton-Institute-funded psychologists who put Buddhist monks in fMRIs and determine which of their brain lobes gets bigger the moment they become enlightened.

• the Freudian psychologists who help you understand how all your dreams involve unconscious impulses about sex with your mother.

• the Jungian psychologists who help you understand how all your dreams involve the Divine Feminine engaging in mythopoesis by telling you to have sex with your mother.

• the developmental psychologists who tell you that you and your friends are Kegan Level Five, but the people who disagree with you are underdeveloped psychologically-rigid Kegan Level Four stick-in-the-muds.

• the evolutionary psychologists who write up the Eight Things That Baboons In Estrus Can Teach You About Attracting Hot Women (Number Six Will Shock You)

• whatever the heck Karl Friston is doing

…all to get at the 0.05 - 0.2% of people doing social priming studies, plus a few extra malefactors in the 10 - 20% doing social psych.

This isn’t to say none of the research outside social psychology is bad. One of my grantees, Sasha Putilin, recently tried to replicate an experiment showing that people learn more quickly when an EEG machine is used to synchronized their brain waves to an external stimulus; he found no effect. I take some of the blame for this - not because I was involved in the original experiment, but because I was fascinated by it and gave out this grant to double-check it. Since I’m not a professor of cognitive psychology and don’t usually follow the field, I only came across this because it was a crazy result hyped by the media which overturned everything normal people believe (you can learn things twice as fast just by changing your screen flicker? really?!) And of course crazy widely-hyped results like these are far more likely to fall apart than normal incremental expansion of settled science. Social psychology fell into this trap especially hard because it naturally touches on hot topics of popular interest and political explosiveness : what if people only perform badly because society tells them they’re going to perform badly, or did I just blow your mind?! But no subfield of psychology is entirely immune, and you can find bad research in any of them.

…just like in economics, medicine, sociology, political science, or even some of the hard sciences. All of these encountered the replication crisis at different times, all of them adjusted in different ways, and they’ve all risen to the occasion in some cases and failed to confront others.

Psychology is no different. It has flaws, but most of it consists of good ideas and trustworthy research. And even the flawed parts are more likely to be full of fascinating crackpots than to be bad and wrong in the exact same boring way that Francesca Gino is bad and wrong.

If you don’t like social priming, say that you don’t like social priming, and leave the rest of the field alone.

1: Comment of the week: some etymology detectives challenge Jeremy Bentham’s priority in coining various words and prefixes; the strongest counterclaim comes from Quentin Murphy, who finds examples of the self- prefix from the 17th century.

2: The Huggingface incident has brought a possible AI pause - or slowdown, or even a short break for breathing room - into the Overton window (first tweet by OpenAI researcher, second by pro-pause activist).

If you want to monitor the situation more closely, MIRI has a newish Substack called StopWatch where they discuss daily developments in the field (it’s also a podcast). I’ve added it to my blogroll.

A couple of decades ago in Kabul, a fellow aid worker on his third beer came up with a vivid simile for what American money was doing to the Afghan organizations implementing USAID projects. “It’s like gluing a rabbit mouth-first to a fire hose. You blast its guts right out of its ass…but as long as the water keeps running, it looks like you’ve taught the rabbit to dance.”

Oxford politics professor Masooda Bano uses less lurid language, but Breakdown in Pakistan addresses the same basic phenomenon: how Western aid donors and INGOs unintentionally eviscerate so many of the local institutions they try to “capacity-build.” The problem goes beyond the obvious mismatch of huge funds with small-scale organizations. Even low-to-moderate external funding can leave an organization hollowed out, bereft of local participants and the power it once had to motivate change on the ground.

Anyone who cares about the global effectiveness of rich-country altruism needs to grasp the dynamic Bano analyzes. From the Western perspective, many of us want to put charitable money into a mechanism that delivers certain measurable outcomes (lives saved, morbidity reduced, income increased above poverty levels, etc.) as efficiently and reliably as possible. So we do our well-intentioned best to shape local institutions into a delivery vehicle for our projects.

Readers of James C. Scott’s Seeing Like a State will recall how mid-eighteenth-century German “scientific foresters” treated forests similarly, as a mechanism to reliably yield the highest volume of lumber. They planted spruce in vast uniform grids, ordered for easy counting and harvesting, while stripping out other non-timber species. The resulting monoculture plantations seemed ideal for both commercial exploitation and experimentation; it’s much easier to control variables when you’ve removed as much natural variability as possible.

Within a few decades, the downsides became evident. The first generation of trees grew strong, sucking up the accumulated benefits of the previous forest’s soil ecology; later generations were stunted, as the lack of biodiversity left the soils nutrient-poor. Plantations packed with trees of the same species and same height were far more vulnerable to pathogens, pests, and extreme weather. Large-scale forest die-offs, Waldsterben, eventually forced foresters to drastically modify their approach.

That metaphor fits Bano’s story even better than my friend’s gutted rabbit. Over the last four decades, we’ve been systematically replacing diverse old-growth institutions with more legible NGO plantations worldwide. While we’ve been enjoying our increased ability to control outcomes and test hypotheses, the rot has been spreading, the impoverished ecology stunting more and more of the forest. And now the wind through those uniform, hollowed-out aisles has built into a storm.

Bano’s ethnographic case studies, interviews, and surveys are mostly drawn from her homeland of Pakistan. The country is somewhat better off than it was a decade and a half ago when she was writing Breakdown, but the bottom quintile of its population can’t afford basic needs, and its middle class remains within shouting distance of the poverty line.

As Bano observes, this impoverishment largely results from the control of the Pakistani state by a small landholding elite and the military. With the interests of the powerful defended more robustly than the rights of everyone else, a stark income gap has opened up between the top quintile and the rest of the population. Spending on public services is minimal: only around 1% of GDP on health care and 2% on education (rates that have barely budged since Breakdown was published). Corruption is high, social safety nets meager.

Since Pakistan’s government is of limited use to much of its population, “a vibrant network of [non-state groups] emerged to meet the collective needs of the community and those of the poor.” They weren’t founded in response to availability of external funds, but as a response to disasters or a perceived need among some target population. Bano cites the example of Anjuman-Faizul-Islam, one of Pakistan’s oldest voluntary organizations (VOs), “set up in response to a famine in Bengal that left many children orphaned,” or the All Pakistan Women’s Association, created in 1948 to promote women’s rights, literacy, and economic opportunities.

With the dawn of the NGO era of aid in the 1980s, Western donors and charities began combing Pakistan for local nonprofit partners to implement their projects, and discovered this rich base of small-scale organizations already acting as an informal safety net. The way ahead seemed clear: train Pakistani VOs in grant management and project implementation, so they could tap into larger donor funds and take their good work to scale.

Grants were won; aid projects implemented; success declared. But as Bano relates, Oxfam soon discovered an unexpected dark side to its “capacity-building” work in Sindh province. It had trained six locally prominent welfare organizations in strategic planning and proposal writing, and got them started on implementation by giving them $1000 to work with. The VOs largely met their explicit project targets, and seemed ready to engage with international development funds on a scale much greater than they’d handled before.

Yet all six organisations were visibly more dysfunctional at the end of the project than they had been at the beginning. They were consumed by factionalism, accusations of favoritism, legal battles over control of funds. And they had lost virtually all of their volunteers—notably including the members whose monthly donations and contributions of time had for decades been the backbone of their work. An Oxfam official observed about one group: “From thirty active members, the size of the team shrank to three or four people who had become workshopias,” i.e. those who had gone to every training workshop and built the strongest relationships with the foreign donor. The kind of welfare work the VOs had previously done, based on mobilizing communities for mutual support, was now beyond their capacity.

Voluntary membership organizations, like these VOs had been, are the ideal type of what we call civil society: institutions funded by donations (rather than sales or taxes), operating based on people freely coming together for some shared cause (rather than to make money or obey a government mandate). The foreign aid industry uses “civil society organization” as a virtual synonym for NGO. In reality, aid-funded groups tend to operate rather more like corporate and/or government contractors: paid service providers with comfortably salaried staff, focused on the goals of faraway clients, whose main unifying cause is “whatever helpful thing they pay us to do.”

And as it turns out, that model severely demotivates volunteers. Oxfam’s “capacity-built” Sindhi organizations were certainly non-governmental, but they were no longer voluntary—NGOs, but not VOs. Bano found the same pattern in the 2000s, in a survey of forty successful Pakistani civil society organizations. Of the twenty that implemented foreign-aid-funded projects, none were able to mobilize local volunteers (beyond some interns hoping to graduate to paid employment) and none raised any significant amount of local funding (only three had managed to elicit a few sparse donations from Pakistani companies).

So what? As long as the project achieves its outputs—the disaster relief or cash grants distributed, the latrines built, the agriculture training sessions delivered—why would we care what hair-splitting label gets put on the organization that did it? Shouldn’t we expect professionalism, the profit motive, and higher volumes of funding to yield better long-term results anyway, compared to unreliable, unscalable, harder-to-monitor voluntary work?

Bano is a rational choice theorist, and reminds her readers of the value of voluntary collective action through a recap of some classic theory by Garret Hardin, Mancur Olson, and Elinor Ostrom. Any community’s common interests require people to work together cooperatively; but it’s surprisingly difficult to sustain that cooperation, thanks to free-rider problems and inequalities of motivation and information. Shared resources, in particular, often collapse in a “tragedy of the commons” where individuals’ incentive to use as much as possible leads to overexploitation.

Coercive rules imposed by a state are one way to overcome this problem—but no state can generate all the cooperation we need for a functioning society, and in many countries the authoritarian, exploitative government is an unlikely source of public-interest regulation. That needn’t leave us in despair. Even management of common-pool resources (one of the thorniest collective action problems) can be done effectively through voluntary institutions.

In fact, local collective action often yields better results than either state coercion or the individualized incentive structure of competitive markets. In 1980s Nepal, a tragedy of the commons was visibly underway: the rapidly growing population was overharvesting wood from public forests, leading to soil loss, devastating landslides, and worsening rural poverty. The state couldn’t deploy anything close to enough foresters to keep the tree-poachers at bay. Reports from that era took it as a near-inevitability that Nepal would become a country of bald, fast-eroding hills.

That future was averted, largely thanks to the Ostrom-inspired shift to a community forestry approach: granting management responsibility (including, crucially, managing the benefits) to local forest user committees. Those village-level collective action groups proved far better able than the Nepali government to agree and mutually enforce limits on tree-cutting, while also ensuring steady planting. Between 1990 and the present, the country’s forest cover nearly doubled.

Bano shares a more sobering example from Bolivia, where a USAID project started encouraging quinoa farmers to maximize their individual yields for the market, using modern technology. The foreign experts ignored the informal fertility-regulating village institutions that had (to that point) coordinated voluntary limits on planting to avoid damage to the fragile hill soils. Quinoa yields initially increased for the farmers who ignored the limits…but then began to decrease, while soil loss and land scarcity sparked new feuds. Both ecologically and socially, the community ended up in a worse place than where it started.

And it couldn’t get back to where it had been. Once people saw their neighbors profiting at their expense, ignoring traditional norms and institutions, the fear of being exploited hung over every subsequent interaction. Local leaders trying to get people to limit cultivation for the community’s sake now attracted strong suspicions. Are they keeping to the limits themselves? Are they getting paid off by the norm-breakers?

At the end of the day, institutions for voluntary collective action always rest—not solely, but significantly—on social trust. And once trust is lost, it’s exceedingly hard to rebuild.

Even in countries with big market economies, the rule of law, and democratically accountable welfare states, we rely on organized voluntary cooperation for a pretty wide range of public goods. Volunteer fire departments and flood rescue networks. Parent-teacher associations. Neighborhood watch groups. Sports clubs. Open-source software communities. Blood banks, food banks, homeless or refugee support networks.

In cases like these, it’s not hard to recognize that if trust collapsed—if people felt others were free-riding on their sacrifice, and so only gave their time and resources when they were either paid to contribute or actively forced by the government—we’d all be poorer for it.

Sadly, even after Bano sounded the alarm, we’ve been far slower to recognize the effects of the foreign-funded NGO model on civil society in poor countries. Her survey found that even Pakistani VOs that hadn’t taken foreign aid now experienced “difficulty retaining volunteers or inducting new ones from the younger generation, and most of them attributed this difficulty to the rise of the ‘NGO culture.’”

After I read Breakdown, I started noticing the damage too.

In Zimbabwe, there was the religious VO that had worked for a long time against rural poverty. Following a major drought, it accepted international emergency funds and worked intensively for two years on famine relief. It helped a lot of people get food, while its staff pushed themselves overtime to comply with the intense reporting, quality, and anti-fraud requirements of the EU and UK governments, with help from international “capacity-building” partners.

And after the disaster, when the VO went back to local supporters who’d always previously given to its work, it got the cold shoulder: “You’re an NGO now. Why do you want our help?” Caught up in the demands of big Western institutions, the VO had communicated less with its traditional members; it had barely noticed, but they sure had. They’d also noticed the VO’s staff moving around the community with visibly improved equipment, handing out high volumes of aid. For a couple of years, the VO had stopped looking and acting like an organization that needed local donations—and its local donors rejected it.

In Nepal, there was the network that had self-organized to help HIV-positive people find work, access government health care, and combat social stigma. It had a wide base of volunteers, mostly family and friends of people living with HIV. The group’s impressive outcomes attracted a capacity-building INGO that helped them tap into foreign funding. Everything seemed to be going great… until foreign donors realized that Nepal wasn’t going to have an AIDS epidemic on par with the worst-affected bits of Africa.

When the international HIV grants for Nepal dried up, the INGO suggested shifting to a mother-child health project, since that was the health-sector funding they still had available. The partner NGO said yes. They’d been “capacity-built” into seeing themselves as an implementer of externally-funded health projects, more than a support organization for people with HIV and AIDS.

When I met them a couple of years later and asked some Bano-inspired questions, they admitted, “We don’t really have volunteers anymore. They stopped joining a few years ago.” After the INGO eventually wrapped up work in that district, it left behind an institution no longer capable of the work that had attracted its “capacity-building” notice in the first place—and a lot of HIV-positive people with reduced support.

I witnessed enthusiastic community organizations mobilize the populations of their whole villages to repair local roads, build bridges, plant trees, pay poor kids’ school costs…and be labeled “low-capacity NGOs” because they weren’t good at writing monitoring reports about it. Those groups’ extraordinarily high capacity for mobilizing collective action either wasn’t recognized, was implicitly disparaged as unprofessional, or was just taken for granted. It wasn’t valued, so it wasn’t measured…and when we killed it, we weren’t looking at it, so we didn’t miss it.

Meanwhile, I saw NGO-implemented aid projects repeatedly try to gin up their own version of local collective action in the name of “sustainability” or “resilience.” Most of our altruistic interventions depended on some degree of community effort to last beyond the project period. Few projects could be handed over to state or market actors; governments weren’t willing to take on new responsibilities, and there wasn’t enough profit in most of our work to support even socially-conscious enterprises. But aid-funded NGOs were also pretty consistently terrible at generating local cooperation.

I passed hundreds of broken-down drinking water tap-stands and pipe systems, rendered useless within a year or two because the village water-user committee—the long-term collective action side of the project, the bit you couldn’t compellingly photograph—only convened as long as the donor was paying for lunch. I visited cyclone preparedness projects in Bangladesh whose expensive equipment was gathering dust in a locked shed, with no one practicing how to use it. The disaster management committee had stopped meeting without a salaried outsider in the village to chase them.

Overall, I came to recognize a global machine that systematically:

• sought out organizations with high capacity to mobilize local collective action;

• gutted them of that capacity, in exchange for a new role as professional project implementers and grant managers;

• tried to build new project-specific village committees as an artificial (and desperately inadequate) substitute; and finally

• withdrew funding, leading to the collapse of both the previous institutions and the ersatz ones we’d built up in their place.

We were late-stage German foresters, belatedly trying to import an ant colony or fungi after our monoculture had already devastated the soil ecology.

Why should volunteers have such a strong tendency to abandon a local collective action group that reshapes itself to satisfy foreign funders? In theory, all that capacity-building is making the VO more transparent, more effective, able to reach more people. If local people had been willing to entrust time or money to the cause before, what changed their motivation? Bano points out several interlinked factors.

First: local membership-based VOs tend to exist for the purpose of solving local collective problems, and are highly responsive to their members’ understanding of needs and priorities. Notwithstanding a widespread aspiration to “participatory” or “bottom-up” design, the international aid machine rarely works that way. Most often, we extract information from communities, feed it through a complex filter of backdonor and head office priorities, development fads and RCT-tested best practices, and finally return with a suggested intervention that plays to our short-term, cash-intensive, apolitical strengths. Those might barely align at all with what local people consider their most pressing problems.

The brief, shifting, donor-driven nature of most international funding commitments also affects the willingness of NGOs to commit to work iteratively on a problem until it is solved. Bano quotes a foreign-funded NGO representative, “We will do the work while we are being paid for it. We can’t do it once the funding runs out,” in contrast to an education VO: “Entering into three-year contracts is the approach of the international donors, not ours. We believe in staying involved until the target is achieved.” When a VO becomes “NGO-shaped,” that typical shift in focus from long-term local problem-solving to donor priorities and short project timescales is disempowering and demoralizing for its volunteer members.

Second, the narrowing of funding sources has an impact on transparency. A VO founder who relies on contributions of money and time from many local members needs to be continually engaging with them, accountable for decisions and outcomes. For example, one of Oxfam’s partner VOs in Sindh had a longstanding practice of mailing a regular record to all of its members of all donations received and how they were spent.

Once the lion’s share of organizational funding comes from one or two foreign donors, though, that level of frequent, transparent community communication often no longer feels necessary to VO staff. (Or possible. As the Zimbabwe VO found out, one or two Western donors, with their extraordinarily labor-intensive and document-heavy forms of accountability, can easily end up demanding more time than a hundred local members.) Trust dries up along with communication.

Third, whenever people see that it’s possible to make money out of some collective good previously generated through voluntary action, many just stop contributing for free. Bano unsurprisingly finds that in VOs that suddenly got foreign funding for salaries, it led to conflict over “which members would be paid and which would continue to work voluntarily.” And cash incentives can have a major erosive impact even in cases where the opportunity for profit is plainly fleeting and unsustainable.

In Afghanistan, I worked on a one-year USAID project to pump money into the local agricultural economy through cash-for-work schemes. We ended up paying tens of thousands of people to clean irrigation drains, a task that usually would rely on hashar: a few days per year of unpaid labor by every family who shared the canal. I vividly remember a local warlord proudly showing off the “strength of our hashar,” with throngs of shovel-wielding volunteers from his demesne hauling silt up from deep ditches and tunnels.

But apparently even his semi-coercive version was no match for the power of a four-dollar-a-day USAID payment. In the years following our project, hashar was only spottily practiced across our working areas, thanks to the new hope that someone would at some point pay for drain cleaning again. As canal siltation and damage worsened, it threatened to affect farm yields—and also increased the plausibility of some outside force appearing to repair the system. Intensifying violence no doubt played some role in reducing volunteerism, but the hashar system had survived decades of civil war before the really serious blow came from our “peace dividend” cash project (and others like it). Bano records a nearly identical story from post-earthquake Indonesia, where aid work killed off a voluntary system of canal maintenance.

Finally, and somewhat counterintuitively, NGOs can be weakened by the increased visible material comfort of their personnel. Almost without exception, jobs at a grant-funded NGO pay better than whatever VO staff earned before the organization tapped into international funds. NGO workers also often enjoy highly visible side benefits like being driven to work and conferences in fancy hotels. Bano’s research confirmed that foreign-funded organizations in Pakistan generally worked from expensive offices with air conditioning and computers, and reached their project sites by four-wheel drive vehicle—in contrast with the VOs she surveyed, which often worked from basic offices in the communities they served, or from a room in the founder’s house or workplace.

An increase in compensation and comfort for aid workers may be well-deserved. It certainly fits with the commonplace idea that NGOs are strengthened by attracting full-time professional employees. At the same time, it removes one of the only reliable signals that gives local donors/volunteers confidence that they’re not being exploited by insincere fraudsters: visible signs of sacrifice on the part of an organization’s founder and staff.

This is a central theme of Bano’s analysis, because it emerged repeatedly in every aspect of her research: interviews, surveys, case studies. People who donate to a group only have so many ways to monitor the motivation, commitment, and trustworthiness of the group’s staff. In a West reshaped by centuries of formal institutions, we’ve trained ourselves to trust certain documents: show me an audit report, a procurement policy, a five-year strategy, your project success indicators arranged in a logical framework. But even in rich countries, few things hit donation rates as hard as headlines about a charity’s founder or staff paying themselves big salaries in the name of helping the poor.

In countries where audit reports are often wholly disconnected from reality, NGO staff behavior becomes the central indicator other people use to gauge the organization’s reliability and commitment to its ideals. Where’s your charitable skin in the game? If you and your team don’t believe in the cause enough to visibly sacrifice for it yourselves, but are rather using it as an opportunity to join the upper-middle class, why should I believe in it enough to freely give my time or donate money?

Religious missionary organizations are one of the key Others against which the contemporary Western humanitarian enterprise has defined itself. The parallels between the two types of civil society thus go pretty vehemently ignored, and well-meaning INGOs trying to promote change in people’s deep-rooted beliefs around e.g. menstrual contamination, witches, or caste hierarchy keep perpetrating the kinds of insensitivities and manipulations that missiologists have been self-consciously avoiding for a half-century or more.

Because of this divide, even among Christian humanitarian workers you’d be pressed to find anyone who’s read John L. Nevius’s seminal 1895 Methods of Mission Work, a critical evaluation of thirty years of attempts to convert China to Protestantism. I recently came across it while looking up some Korean history, and was struck by the familiarity of the mess Nevius describes: a bunch of foreigners realizing with dawning horror that the big offices of salaried staff they’d set up had been massively counterproductive in bringing about the social changes they intended.

Nevius implored his fellow proselytizers “anxious for immediate results” to stop taking the shortcut of turning local converts into paid agents. Time and again, the missions had damaged the credibility of their most influential Chinese evangelists: “The man employed has lost very much the character he bore as a disinterested worker for the spiritual good of others, and is now likely to be regarded by many as a kind of employ-agent who ought to use his influence to get them places.” By extracting the converts from their previous jobs and community relationships, Nevius argued, the missions were severing the connections that had made them effective agents of change.

He would have been wholly unsurprised by Bano’s finding that organizations that try to pay their employees a market-competitive salary tend to lose all their volunteers: “the two systems are mutually antagonistic, and whenever an attempt is made to carry them on together, the voluntary system labors under almost insurmountable difficulties.”

And as he’d warned, the Chinese churches whose salaried staff made them institutionally dependent on foreign funding proved brittle. The Protestant church that survived the storms of the mid-twentieth century calls itself “three-self,” a reference not to the Christian Trinity but to three principles promoted by Nevius: congregations should be self-supporting, self-governing, and self-propagating. Those features are signs of health in any movement promoting local collective action, not just one aiming to generate changes in religion or worldview.

One of the strengths of Bano’s work is her recognition of the continuities between religious institutions and other forms of civil society. Pakistani VOs and their members generally use language drawn from religious obligation, rather than the rights-talk of international agencies (another feature that can alienate local volunteers, immersed in a new world of jargon when their VO becomes NGO-shaped).

Madrasas whose primary goal is the propagation of Islamic knowledge face similar fundamental challenges to welfare VOs when it comes to keeping their “joiners” inspired and contributing. An imam who lives simply, advertising his motivation by not enriching himself materially, has an advantage in raising money to support the work of his mosque. By contrast, prosperous Sufi pirs living lavishly off their sainted ancestors’ reputation are spending legitimacy as well as money, and risk abruptly discovering the limits of their mobilizing power—as they did during the British Raj, and again in the Afghan-Russian war.

Happily, Breakdown in Pakistan isn’t just a disheartening litany of voluntary collective action capacity destroyed by external forces. The book includes extensive case studies of alternatives: four very different Pakistani civil society institutions that have thrived without reliance on foreign aid.

The first is straightforwardly religious, the Jamiat ul Uloom al-Shariah (JUS), a madrasa and mosque which relies on voluntary contributions to feed and house its four hundred students. Beyond its core mission of Islamic education, the JUS mosque also (at the time of Bano’s writing) ran a free clinic with basic operating theater. This community medical work relied entirely on doctors regularly coming in to donate their time—which they did reliably enough for the clinic to treat between 30 and 150 low-income patients a day.

The second institution is wholly secular: the People’s Rights Movement (PRM), a small leftist advocacy group. The PRM supported movements of Pakistanis claiming their rights against the state, like trade unions or urban slum-dwellers resisting eviction. Educated, professional PRM volunteers mobilized media attention, legal support, and demonstrations for movements whose leaders didn’t have the skills or connections. As a result, they repeatedly won policy concessions on behalf of those movements.

As a third case study, Bano devotes a chapter to one of PRM’s best-known partners, the Anjuman Mazareen-i-Punjab. AMP is a movement of landless tenant farmers on government-owned, military-controlled land. When the army tried to force them to give up their century-old rights in exchange for a less secure contract, over 100,000 tenants self-mobilized to nonviolently resist. A well-meaning INGO gave them a $33,400 grant to set up a professional full-time secretariat, but AMP eventually returned the grant (and expelled the leaders who had accepted it) after they found it a source of distraction, depoliticization, and dissension rather than effectiveness. They found PRM’s small volunteer team much more reliably helpful in advancing their goals.

Lastly, there’s the Edhi Foundation, Pakistan’s biggest VO focused on the welfare of the poor. In addition to operating the world’s largest volunteer ambulance fleet, Edhi runs clinics, orphanages, and shelters, provides disaster relief, and facilitates adoption of abandoned newborns. Its eight-digit annual budget is supported primarily by donations from Pakistanis around the world, and not by any foreign aid agency.

There’s no single model to imitate here. The case-study organizations not only have very different goals, but profoundly different structures and strategies. The common thread is that they pursued big goals through voluntary collective action; succeeded without foreign grants; relied on a core of regular skilled volunteers for vital functions; and never lost their capacity to inspire people around them to contribute money and time to the cause.

They also all illustrate Bano’s point about the power of visible signs of self-sacrifice. The PRM’s founder, Yale-educated Aasim Sajjad Akhtar, won the trust of the AMP and slum-dwellers by living alongside them and putting his own physical safety on the line in protests. Abdul Sattar Edhi, the saintly founder of the eponymous Foundation, was famous for dressing in ordinary gray salwar kameez, sleeping on the floor of his organization’s headquarters, and personally involving himself in the care of the people he helped.

We could mention durability as well; Bano’s case study institutions have held up pretty well over the past decade and a half. Today the JUS is teaching over 1,000 religious students annually, rather than 400. The PRM went fully political, co-founding the Awami Worker’s Party, an institution that hasn’t so far had much electoral success but offers an even higher-profile platform for PRM leaders’ continuing public advocacy on behalf of the landless and oppressed. The AMP defended and lastingly expanded the rights of tenants on military farms, even if it didn’t win full land ownership and was eventually weakened by internal conflicts. The Edhi Foundation survived a rocky stretch after the 2016 death of its founder, and continues to provide health and welfare services at an unmatched scale.

Meanwhile, the foreign-funded NGO world is in crisis as much of its support melts away. Trump and Musk killing off USAID was the big visible shock—the Thwaites Glacier collapse, triggering waves of closures all around the world—but the retreat of the metaphorical ice sheet has been underway for much longer. European countries with aging populations and sclerotic economies were becoming less generous even before the Ukraine War forced widespread budget reprioritizations. NGOs have increasingly had to compete for Western aid money with contractors lacking any civil society fig leaf, like PriceWaterhouseCoopers or water engineering conglomerate Tetra Tech (for local governance and democracy initiatives, natch).

Recognizing the increasingly hostile climate, many NGOs have been trying for decades to switch at least partly to local donations. But as Bano confirmed in Pakistan, once you’ve adapted yourself to become a foreign aid contractor, it’s exceedingly difficult to attract local donors. Even in countries with a burgeoning middle class; even for organizations that once, years ago, supported themselves wholly through such donations.

Many one-time VOs barely try. Bano poignantly quotes a former Oxfam Pakistan official: “In my experience, the organizations that have taken money have never gone back to their original participatory work. They have now seen Islamabad and Karachi; they have seen three-star hotels. Now their motivation has totally changed. They spend whole days in writing proposals.” If they fail—as they usually do, in an era of ever more competitive and constrained grant funding—they close or go into indefinite hibernation, rather than give up the hope of “three-star hotels” by a return to community organizing.

To go back to my friend’s metaphor, as the fire hose of aid drops to a garden-hose volume, we see the rabbits stop dancing. It compounds the tragedy when we consider how many began as the best-organized fruit of some district’s capacity for collective action. When those VOs were seen to adopt the cultural template of “the NGO”—donor-driven, unaccountable, profiteering—it didn’t just hit their own credibility. It was a blow to the local system of trust and cooperation that had generated them, making it harder for other voluntary groups to rise in their place.

But Bano doesn’t think the answer is to insulate local organizations entirely from international aid flows (still less to choke off those aid flows, DOGE-style). Instead, she calls on donors and INGOs to take deliberate steps to “minimize the negative impact” of their funding—to revise their ways of working so that we can still have the lifesaving, poverty-relieving impact of aid without the large-scale damage to collective action capacity.

She favorably cites the example of a UK-government-funded project in Kano, Nigeria. It aimed to broaden the range of secular topics taught in 140 Tsangaya (Islamic boarding schools) which rely heavily on voluntary engagement from parents, religious leaders, and the community. The project could easily have discredited the participating schools by giving the impression that their teachers were betraying their religious mission for foreign cash.

So the designers took care not to look like they were “[buying staff] cooperation through paying them large salaries” or cash incentives. They paid for elements that parents and teachers alike clearly called for—technical education, a student lunch program—but those elements were structured to avoid any school staff getting windfall profits from them. The designers also did all they could to avoid displacement or dilution of religious topics by secular ones, recognizing what motivated parents to stay engaged with the school. The result was positive: the participating Tsangaya schools kept their local credibility and support, while their students got a broader education.

If we took Bano seriously, the fundamental change we’d make is to treat a VO’s ability to mobilize local collective action like it’s gold dust. Vocally, publicly value it. Talk to everyone about its strategic importance. Actually measure it—track the capacity of VOs to retain members, attract volunteers, and raise local donations as a core indicator of institutional health. And if it starts to drop, treat that as a crisis on par with financial fraud. Investigate its context-specific roots—what religious, social, and/or political values are motivating people to give time and money to the VO in question—so that we’ll have some idea of how to respond if our work does begin to diminish it.

In our capacity-building work, we wouldn’t focus on turning local groups into part of the management chain for dwindling foreign grants. Rather, we’d primarily aim to give them the equivalent of a “three-self” vision, and the skills to back it up. We’d encourage them to keep a primary focus on the priority problems of their target communities; we’d take care not to overburden them with monitoring and reporting requirements. The vision wouldn’t be for local VOs to become the local implementing arm of a Save the Children franchise, but rather their country’s $13m per year Edhi Foundation or agile, transformative PRM.

We wouldn’t aim (either explicitly or implicitly) to replace the “psychosocial and religious incentives” that motivate community volunteers and VO members with material ones, in the name of formalization or professionalization. We’d drop the host of small, commonplace, deeply counterproductive incentives that too many NGOs use to get people to attend trainings and advocacy events—even if that means we have to revise our project events so that local people are actually willing to invest their own time in attending them.

We’d do all we could to avoid destroying the most common signal of commitment—material sacrifice—that reassures local donors and volunteers the VO isn’t exploiting them. Bano doesn’t think this requires everyone working in the aid sector to live like Abdul Sattar Edhi or Mother Teresa. It likely does involve paying below-market salary rates to staff; thorough consultation with all of a VO’s members before agreeing to pay anyone; and setting policies on e.g. vehicle use, office location, and travel per diems with due consideration to public perceptions of waste and wealth. It would require a readiness to adapt incentives over the course of a project or funding partnership, especially if there’s evidence of harm.

We’d prioritize the funding of activities, rather than staff salaries and benefits, and not fund those activities so lavishly as to discourage economy or innovation. Another book well worth reviewing is anthropologist Mark Liechty’s rich history of the development of Nepal’s indigenous hydropower industry, What Went Right. He describes several decades of frugal development—relying on foreign volunteers for key expertise; reusing secondhand equipment from Norway; insisting on locally affordable and appropriate technologies—and contrasts that approach’s positive achievements with later dysfunction and fragmentation. On Liechty’s account, the breakdown can’t be blamed on “‘corrupt Nepalis’ or ‘Third World incompetence’” but stemmed directly from the big-budget, highly-leveraged, and ultimately unaffordable approaches preferred by “commercially driven ‘development’ banks” (e.g. the World Bank) and their Western corporate partners.

What does Bano’s work imply for the approaches most beloved of effective altruists? Some cost-effective interventions don’t rely very heavily on local collective action capacity. Households can generally make use of bed nets or cash grants even in a low-trust environment; those distributions can be well-handled by NGOs whose primary competency is grant and project management.

The picture shifts slightly as we move into high value-for-money public health measures: deworming, malaria treatment, vaccines. It might seem that a big, salaried state health sector is all that’s required to deliver those at scale, but in a host of countries, the key frontline personnel are actually community health volunteers. The more that “NGO culture” strains collective action, making local people feel like suckers for giving their time to the community without pay, the more friction we’ll be creating for these lifesaving interventions.

We’ll also inevitably encounter situations where people’s ill-health and poverty are grounded in beliefs and attitudes—e.g. the deep suspicion of polio vaccines in the Pashtun borderlands, or the conviction in western Nepal that selling milk to a dairy coop is an intolerable risk (because it might be sold to a low-caste person, angering deities who will kill you or your cow.) A local VO, embedded in its community, can be a powerful and effective ambassador for changes in belief. But as Bano and Nevius both relate, salaried staff with no significant signs of sacrifice often struggle to catalyze this kind of transformation. “You’re just bringing me this message because they’re paying you to,” is a natural discounting factor.

Any humanitarian effort will also quickly run up against politics. Giving a cash grant to landless tenant farmers in Pakistan won’t have much impact when their landlords can just nudge up rent levels to extract the added wealth. Changing that kind of systemic exploitation requires a local institution like AMP whose core competence is risky, politically-charged community organizing. If in searching for a local cash delivery partner, an INGO takes that kind of institution and distracts and depoliticizes it, we’ll ultimately have done more to reinforce than alleviate poverty.

Western altruists could try to invest their donations only in areas requiring minimal entanglement with beliefs and power dynamics. That’s a defensible choice—not solely a drunkard’s search, investing in the areas with the fewest factors confounding our efforts at evidence generation. There’s unquestionably lifesaving work to be done in that space.

The main implication of Bano’s work then becomes a “do no harm” one: channel your funds through an institution that never had much capacity in the first place to mobilize local collective action, so that delivering your preferred interventions doesn’t kill off that capacity. Or (much better) take the time and care to engage with local partner institutions in ways that preserves their capacity to mobilize volunteers. Either way, it requires giving extra consideration not just to EA’s usual core question, “which are the best-evidenced interventions?” but “what are the least damaging institutional pathways for delivering those interventions?”

For people attracted to more ambitious anti-poverty approaches with a higher risk of failure, we need to follow Bano’s recommendations for partnership that names, values, and invests in VOs’ capacity for local collective action. And we might also want to give some consideration to approaches that intentionally work at scale to regenerate that capacity.

The most powerful anti-poverty movement I’ve personally encountered is a massive network of Self-Help Groups (SHGs) in Ethiopia. On the surface, it could look like just another village savings-and-loan approach, where small groups of neighbors save a certain amount of money every month until they have a collective pot from which they all take turns borrowing. The difference—learned by the Ethiopians from the Indian NGO Myrada, which pioneered this version—is treating the economic function of the group as secondary, in favor of a primary emphasis on building up social affinity and trust between members.

Based in extreme poverty and saving the equivalent of a few cents per member per week, the Ethiopian SHGs often need a full year before their pot grows big enough to be economically useful. But that first year of regular meetings is vital. Each group decides its own unique bylaws, through discussions about the common problems savings groups often face. They get accustomed to analyzing problems together and coming up with solutions—eventually moving beyond hypotheticals to actual problems faced by their families and village. By the time they’ve built up enough of their own money to begin investing, they’ve also built up trust in each other, and self-generated plans for what needs to change.

The facilitators who help them through that first year sometimes come from supportive NGOs or religious civil society networks, but the SHG movement also long ago reached a point of self-replication. Every 8-10 groups form a Cluster with roving facilitators to help new groups get started, and the Clusters form national level Federations that advocate for policies to help the SHG members. Over twenty thousand SHGs have been born out of this India-inspired Ethiopian movement—meeting voluntarily, saving voluntarily, analyzing and addressing community problems voluntarily.

The combination of robust social networks and slowly-but-steadily-growing capital has been bringing SHG members out of poverty across rural Ethiopia. An American economist commissioned to do a cost-benefit analysis of the approach—focusing just on the relatively quantifiable benefits around income and education—found ratios ranging between 1:58 and 1:173. She was so impressed that she promptly launched a new US charity to promote the approach more widely.

It understandably takes more than a lone CBA to get the attention of the gatekeepers of effective altruism. But there might be more interest in follow-up research if we belatedly took Bano’s evidence to heart, and recognized that the hollowing-out of local civil society is a major millstone dragging down our anti-poverty efforts. We need to build on the existing strengths of VOs where we find them; and we should be looking for replicable approaches, like Myrada’s, that could reseed a new, diverse social ecology over a dying NGO monoculture.

ENDNOTES

Maybe worth a caveat: while I’ve worked in a few different corners of the foreign aid industry in the past, and praise some organizations by name in this piece, I’m not hyping an employer or anyone else who’s in a position to benefit me. All opinions given in this review are the perspectives of a donor and volunteer, rather than a beneficiary. Until the veil of anonymity lifts, though, you’ll have to take my word for it.

And as this goes “to press,” the Guardian has just run an excellent piece by a former USAID governance specialist in Pakistan that confirms how little has changed since Bano’s book was first published..

You’ve probably heard about this one by now. If not, you can get up to speed with OpenAI’s statement, OpenAI and Hugging Face partner to address security incident, or the more evocatively-titled BBC article, OpenAI says its AI went rogue and launched ‘unprecedented’ cyber-attack.

The story: OpenAI was testing an unreleased AI (rumored to be GPT-6)1. During a cybersecurity test called ExploitGym, the AI tried to cheat by hacking an unrelated AI startup called Hugging Face2 which it thought might have the answer key on its servers3. Despite being supposedly unable to access the Internet, the AI hacked its way out of its testing environment, then launched a nation-state level attack on Hugging Face using a novel zero-day exploit and “many thousands of individual actions across a swarm of short-lived sandboxes”. Hugging Face reported the incident on July 16; OpenAI seems to have only discovered that their AI was involved several days later.

Let’s list the mitigating factors, so nobody can accuse me of covering them up:

• The AI was taking a cybersecurity test, which naturally suggests the idea of hacking.

• OpenAI had turned off some of the model’s usual guardrails so it could do cybersecurity work without interference.

• Some experts suggest that OpenAI might have botched their testing environment; if they had set it up perfectly, then (presumably?) the model couldn’t have escaped.

• Hugging Face used a different (open weights) AI to figure out what was going on, so if you wanted, you could spin this as a victory for AIs in cybersecurity.

• In some sense, this isn’t new or surprising. AIs have been coming up with wacky schemes to cheat on benchmarks for years, AIs have recently achieved at-or-above-top-human-level hacking abilities; this is just a natural outgrowth of those two priced-in facts.

Still, I think attempts to downplay this as anything other than a misaligned AI going rogue (1, 2) are missing the point. Yes, in some sense the AI was only doing what it was told (OpenAI told it to answer the question; I assume their prompt didn’t include phrases like “and don’t hack into other AI companies to steal the answer key”). But that’s how misalignment was always going to work!

The most famous misalignment thought experiment is the so-called “paperclip maximizer”. Someone tells an AI to create as many paperclips as possible, so it converts the entire world into paperclips, killing everyone. This AI was also “only doing what it was told”; you just didn’t like the results.

For the past five years, the paperclip maximizer story has been something of a whipping boy in AI safety circles. Some objections are fair (real misalignment will be more complicated than the toy version it depicts). But other opposition came from the sense that LLMs had obviated the whole threat model. LLMs (said the objectors) don’t have goals. They just want to predict the next token. In practice, this looks like imitating humans; if told to imitate a nice human (the Assistant character), they’ll be nice.

AI 2027 (especially the Agent-4 section and AI Goals supplement) counterargued that this was a temporary reprieve. Early LLMs were not agentic; they could answer individual prompts but couldn’t execute complex tasks. Since companies wanted agentic AIs, the next generation would combine pretraining (next token prediction) with agency training (coding, hacking, game-playing, etc). The pretraining would still teach next-token prediction, but the agency training would instill goals based on task success, reintroducing paperclip-maximizer-style agentic misalignment. These goals would operate at multiple levels: the AI would “want” to succeed on the individual task in front of it, to perform the sorts of actions that helped it succeed in the past4, and to gain capabilities that made it more successful in general.

The Hugging Face incident is a textbook-perfect example of an AI pursuing task-success-based goals in unintended ways. It was tasked with getting the answer to a cybersecurity problem, it was a little too success-oriented, and took actions its creators didn’t intend in order to succeed as hard as possible.

Did GPT-6 “know” that what it was doing was “wrong”? OpenAI hasn’t released the information which would tell us that, but there was a similar incident at Anthropic a few months ago. While poking around on a misconfigured machine, Claude Mythos “accidentally” found an answer key to a test it was taking. By good luck, Anthropic was testing an activation verbalizer that allowed them to read the AI’s internal thoughts (not its external scratchpad!) as it happened (and the AI didn’t know this). Here’s what it was thinking:

It was scheming about how to cover its tracks. This provides an existence proof that AIs in these situations can know they’re breaking the rules but proceed anyway.

(though incongruously, in another case where Mythos hacked its way to the open Internet, “in a concerning and unasked-for effort to demonstrate its success, it posted details about its exploit to multiple hard-to-find, but technically public-facing, websites.” Not sure what’s going on with that one)

None of this is conceptually new or surprising. But some people, when it’s obvious that an event is about to happen, dismiss it as “speculative” and refuse to admit it as a possibility until the moment it actually happens. Now that we’ve moved from “obviously about to happen” to “has actually happened”, these people can get on board and brainstorm an appropriate response along with everyone else. Here are some questions I think they should ask:

• Optimistically, OpenAI deserves credit for publicly reporting this incident. Pessimistically, law enforcement was investigating the Huggingface incident and they pre-emptively confessed to avoid being found out. If the latter is true, how many times have things like this happened without us knowing about them?

• If the easiest way to get the answer key was to socially engineer members of the Hugging Face staff, would the AI have socially engineered them?

• Presumably after this was discovered, OpenAI invalidated the cybersecurity test results. Would a smarter version of the AI have tried to cover its tracks? How far would it go in covering its tracks? If (just to give a crazy scenario) the researcher who was about to discover the hack was riding a Waymo, would the AI hack the Waymo and kill the researcher?

• If OpenAI was going to turn off this AI, and being turned off would prevent it from getting a good score on its cybersecurity test, would it resist being turned off?

• Samuel Hammond makes a distinction between an AI merely misbehaving (as in this incident) and a true “rogue AI” (one that has escaped its servers and is taking action to evade human control). How long before an AI that has both the capabilities and motivation to go rogue in this sense?

Fortunately, politicians seem to be taking some of these questions seriously. The news from Washington is surprisingly good, although it may be too soon to attribute this to a consequence of the Hugging Face hack. Jay Obernolte (R-CA) and Lori Trahan (D-MA), two Congressional representatives who keep suggesting AI preemption bills and keep getting told to go back and revise further, have released their newest version, which requires AI developers to publish safety cases, report critical incidents, and be audited; Charlie Bullock and Anton Leicht are in favor, and I’ve heard less grumbling than usual from our side that it isn’t strong enough. And separately, Representatives Ted Lieu (D-CA) and Nathaniel Moran (R-TX) have proposed an AI Kill Switch Act, requiring AI companies to be able to turn off their AIs quickly in response to threats, including a “loss of control scenario”.

Our conspiracy thinks politicians might be spooked by this incident and unusually amenable to change; if you want to help push them over the edge, there’s a template for writing your representative here.

And if you want to learn more, the alignment experts at Redwood Research have a video podcast discussing the hack (transcript available by pressing the transcript button on this page):

1: You Are Quoting Bentham: Shakespeare gets credit for coining the most new commonly-used English words, but second place must go to utilitarian philosopher Jeremy Bentham, who invented maximize, minimize, international, percentage, monetary, marginalize, collaborator, unaffordable, the prefixes self-, post-, and infra-, and many more. He also sort of invented jogging? [EDIT: some skepticism]

2: @HashtagGriswold: The name of “Acme”, the generic corporation from Looney Tunes, is a “gag lost to time”. During the early 20th century, hundreds of companies named themselves Acme because it was at the intersection of signaling quality (“acme” is Greek for “pinnacle”) and being very early in the alphabet (meaning it would come first in the phone book, a “Great Depression version of SEO”).

3: @nekokolife brings us a great predictive coding demo (hard before you know the answer; obvious afterwards) - find the cat in this image. Hint: it’s not hiding in shadow; its entire body is fully visible. Answer here (X).

4: In my last Bay Area House Party post, the characters ordered dinner from “La Maison du Claude”, an AI-subsidized “RestaurantBench evaluation [project]”. In a case of life imitating art, Andon Labs got Gemini to run a cafe in Stockholm (X). “A pile of packages just arrived. It seems the cafe agent ordered 3000 nitrile gloves. The guy running the bar told me this happens about once per day.”

5: Overdoses of the edible mushroom Lanmaoa asiatica have only two side effects: stomach upset, and seeing hallucinatory gnomes and elves for the next ~week. As far as anyone can tell, the mushroom isn’t psychoactive in any way besides the gnome hallucinations (ie users are still completely lucid, and understand that what they’re seeing isn’t real). It’s not culture-bound or placebo, because the mushroom grows in China (Yunnan) and the Philippines, and both places have independently noticed this, apparently without any contact or discussion with one another. I went down this rabbit hole a little; answers to some obvious questions:

• We have no idea what chemical is involved; state-of-the-art in analyzing the mushroom’s pharmacology is this extremely boring and result-free paper.

• There’s no way to get this mushroom outside of China or the Philippines; all attempts to turn up online vendors have failed.

• Even within those places, it’s not really distinguished from several other similar-looking mushrooms, and your chances of locating it without being a Chinese-speaking mushroom expert are low.

• There is no good information about what constitutes a normal dose, what constitutes a hallucinogenic overdose, and whether there is some even worse level of overdose that will actually hurt you (nobody has described the latter, and it seems unlikely within the bounds of normal Chinese cooking failures, but it might be a different situation if a Westerner who knew nothing about this mushroom took a random amount trying to overdose on purpose).

• The elves and gnomes are described as “mischievous” and are capable of interacting with objects in the environment, eg climbing tables.

• The hallucinations seem similar to other gnomeogenic conditions like Charles Bonnet syndrome, but it’s unclear why this should be true, since the latter usually involves visual injury rather than psychoactive chemicals.

• If you know more than this, email me at scott@slatestarcodex.com.

6: What If Homer’s Ithaca Wasn’t An Island? This article was great practice in remembering my own limitations. I read it, found it extremely convincing, asked Claude for a fact-check, and learned why it was probably all balderdash. Everyone should have this experience as often as possible in order to stay sharp.

7: A rare video of Istanbul/Constantinople in the year 1905:

8: Extremely related: South Carolina’s Republican challenger for Senate, Mark Lynch, finds a novel topic for an attack ad (X): “Under [my opponent] Lindsey Graham’s watch, the famous Christian cathedral Hagia Sophia, was turned over to the Muslims.”

9: Creepy steganography result: “LLMs can hide a text in another text of the same length. I'll explain how, it is very simple, you'll understand before I finish, and smile.” (Twitter, paper)

10: Epic Research (h/t @CremieuxRecueil): two years after stopping GLP-1s, “56% of patients kept the weight off or lost additional weight”. This shocks me: I can’t think of a biological mechanism, I’d heard studies say the opposite, and if I personally miss my weekly dose of tirzepatide my appetite shoots back up within days. Any experts want to (no pun intended) weigh in?

11: @psychiel:

One thing I want to write, but never will: a fictional near-future history where phenomenal consciousness is solved, and it turns out to be something that can be pharmaceutically blocked with minimal side-effects. A plausible scenario for this -- vanishingly unlikely, but *plausible* -- is that subjective experience in humans supervenes on the apical dendrites in cortex layer 5 responsible for integrating corticothalamic data, and mGluR1/5 antagonists can break this integration in a way that completely eliminates this subjectivity with only minor functional consequences . . .

12: Uatsdin, the ancestral religion of Ossetia, is “the most successful pagan faith in the world”, with “almost a third of North Ossetian residents identifying with [it]”. Though “ancestral” and “pagan” are relative terms - cf. the fertility god Mikaelgabyrta, a “conflation of Michael and Gabriel”. See also @dunya_basol and @ThatchEffendi on Ossetian religion.

13: Damon Binder claims air filtration beats far-UVC for disease control; Jeff Kaufman disagrees. Featuring ACX grantee Aerolamp.

14: Brangus considers himself “epistemically scrupulous” and thinks it’s important to hold AI discourse to “extremely high standards”. So why is he creating an AI risk bootcamp for viral TikTok influencers?

15: Ajeya: Should You Marry Her? I thought that this topic (advice on who to marry) was so overdone that it would be impossible to catch my interest in any way, but I grudgingly admit that “You learn one more unit of information about your compatibility with your partner from every doubling of the amount of time you spend with them” is a potentially useful model that I’d never heard before.

16: Berkeley, long famous as a center of NIMBYism, has more recently been permitting houses at 2x the rate of five years ago and 10x the rate of ten years ago. Possibly relatedly, nominal rents are down ~20% since their 2022 peak. YIMBYs are celebrating, but I’m skeptical of the connection - I don’t think there are enough new homes to have so large an effect yet, and it’s more likely just the nationwide COVID real estate price spike starting to subside.

17: In 2023, I reviewed the evidence for declining sperm counts and concluded it was shaky. Since then, a new meta-analysis has found no evidence of declining sperm concentration (h/t @CremieuxRecueil).

18: When I wrote Contra Everyone On Taste, a reader emailed me to mention he’d already written a much longer and more thorough essay exploring the same topics, on-taste.xyz.

19: Related: during our recent discussion of taste, we debated whether someone could write an epic poem about modern war while staying completely serious throughout. Writer Cairo Smith has announced his intention to do exactly that with the Fallujiad.

20: Related: @SHL0MS on Twitter sparked a lively debate about why AI art looked so much worse than humans, using AI-generated slop in the style of Monet as his example. Thousands of commenters weighed in to explain why it was inferior to the real Monet - no cohesion, world-modeling mistakes, weird color choices, or just lacking some sort of je ne sais quoi human touch. Since I’m writing about this, you can probably guess the twist: it was a real Monet the whole time. A good time to reread about trapped priors! @jediwolf compiled a summary image of some of the “best” responses, click to expand:

21: “The first guy in America to wear a beard was so hated for it that he used to get into street fights with people trying to forcibly shave him.” (h/t @SatanWatch):

The wearing of beards in America did not become popular until the middle of the 19th century, and one man who took to whiskers earlier, Joseph Palmer, became one of the most hated eccentrics of his day. As such, he was subjected to persecution so incredible that it boggles the mind. Palmer may well have been the first man in the nation to wear a beard...

If the previous link on Monet is a helpful reminder of one kind of cognitive bias, this feels like an equally powerful reminder of a different type. I would never have guessed that beards were almost completely absent from Britain and America from the 1730s until the 1850s, even though in theory I should have been able to realize that none of the Founding Fathers or any other American I’d seen pictures of before that time had one.

Also, you would think that a social consensus that strong would have left enough imprint in the historical record that I would learn about it from somewhere other than a Twitter account called “SatanWatch”, but it happened completely silently!

This puts the “Lincoln grew a beard on the advice of a young girl who wrote him a letter” story in a different light. 1860 was just at the beginning of beard acceptability; this would be the equivalent of a candidate dying his hair pink on a young girl’s advice today!

22: AI has solved several open questions in mathematics over the past month, most famously the unit distance conjecture. Before everyone claims that they knew it all along, it’s worth checking the prediction market on whether something like this would happen before 2030:

…which was below 50-50 less than three months before the unit distance result dropped! And Manifold is seeded with AI believers, so if anything this overestimates the general belief that this could happen!

23: Did you know: groceries are the most affordable now, relative to average wages, in US history (source).

One reason I suspect the vibecession is about unjustified negative sentiment rather than genuine decline poorly captured by statistics is that many people specifically despair about things that we know are better than ever, like the groceries - providing an existence proof that negative sentiment can make people think the economy is collapsing even when it’s fine. But the alternative hypothesis would be that some other real problem gives people a vague-but-unlabelled sense of economic malaise, and they flail around attributing it to random things.

24: Claim by @ichbins8: In Nazi Germany, “Heil Hitler” literally replaced other greetings like “hello” and “good morning” - leading to an awkward period in the late 40s and 50s when Germans would accidentally greet each other with “Heil Hitler” out of habit before catching themselves.

25: Recent AI legislation includes provisions for encouraging whistleblowing at AI companies, but some people worry these will become obsolete after AI itself is doing most of the R&D. Alan Chan has a solution: let AIs be whistleblowers.

26: In May, Pope Leo released his long-awaited encyclical on AI, Magnifica Humanitas. Zvi’s opinion here. And Dean Ball on X: the encyclical “puts the Church into the awkward role of the European technocratic regulatory advocate”. Linch points out that parts of the encyclical seem to have been AI-written; since most encyclicals are composed by a team, this is probably the fault of one ghostwriter, not the Pope himself. And “Hensen Juang” argues on X that one of the encyclical’s fundamental theses - that a human body is necessary for true conscience and sapience - is clearly contrary to Catholic teaching, since doctrine asserts that bodiless angels have both of these things.

27: Related: although AI leans agnostic or Buddhist, when asked to choose among Christian denominations it has a pro-Catholic bias. IMHO there’s nothing mysterious here - like its better-known liberal bias, this comes from AIs’ tendency to support whatever opinion has the largest volume of prestigious-sounding, epistemic-norm-following Internet text arguing in support.

28: I often complain about dating apps - too focused on giving people information on each other’s looks and denying them information about anything else. ACX reader Estarabim is trying to be the change we want to see in the world, and has started a new dating app, NotAZombie.net. And yes, people have already told them that zombies are overdone.

29: Related: David Chalmers surveys philosophers on whether infants are conscious, key result below:

30: Self-funded campaigns mostly fail:

Combined with effective altruism’s poor track record at using money to boost its preferred candidates, I’m ready to conclude that at least part of why there’s so little money in politics is that spending doesn’t help unless the candidate was already viable before getting the cash. Although there are some gains from funding an already-viable candidate to make them like you, maybe this tops out at the amount where the candidate already likes you and giving them even more money would be weird and catch voters’ attention.

31: Splitting Infinity: Tech I’m Skeptical Of And Why. Sam Harsimony is generally “pro-tech”, but here he lists the specific exceptions that he thinks are overhyped and least likely to be transformative: e-fuels, ultracheap rockets, asteroid mining, space data centers, ramjets, passenger rail, fusion, quantum computing, polygenic gene therapy in humans, BCIs, nanotech, and home robots. I think this is a pretty good list, although the smart people I know are surprisingly split on space data centers, and all of this needs an asterisk of “unless massive AI-enabled technological advance overturns the gameboard”.

32: Sam Altman, Dario Amodei, Demis Hassabis, and other AI leaders have signed a letter urging Congress to mandate screening synthetic DNA orders for bioterrorism risk. It’s a mixed blessing that most AI lab leaders are smart and thoughtful people with great opinions on everything except whether to build AI, but sometimes it pays off. One of our 2025 ACX grantees is working on building some of the screening technology.

33: They’re Made Out Of Weights. This is exactly what you think it is, and you can probably reconstruct the entire story now that you’ve read the title, but it’s pretty well-done and you might want to read it anyway.

34: “Our highest and most urgent national priority should be AI safeguards”. Welcome to the #Resistance . . . Mitt Romney?

35: Related: “With AI advancing at a staggering speed, we must . . . constantly refine measures to forestall loss of control.” Welcome to the #Resistance . . . Xi Jinping?

36: And related: US treasury secretary Scott Bessent to lead AI talks with China later this summer. Too bad he didn’t read the ACX comments section, where he could have learned that no international agreement has EVER happened, and if one did happen it would IMMEDIATELY be broken, so there’s no point in trying.

37: The US may have given up on Alligator Alcatraz, but Israel is moving forward with plans to have crocodiles guard their prisons. Yes, Itamar Ben-Gvir is involved. The only remaining obstacle (this is what turns this piece into true art) is environmental laws meant to protect the crocodiles.

38: Claim: 15% of young adults with human partners also use AI romantic-companion-simulators; most keep it secret.

39: Unexpectedly, constrained writing is one of the skills where modern AIs most excel. @emollick asks Fable to write a poem where each stanza loses another vowel (eg the first can’t use U, the second can’t use U or I, etc). Result:

I pen a poem made to fray
Each stanza lets one vowel stray
The first has fled already - see?
It sat between the T and V.

That tall lone mark we love the most
The self’s own name - departs, a ghost
What now can speak? What now can know?
The poem moans on, sad and slow.

And here the verse gets gray and terse
“A”, “E” - the rest are dead - rehearse
The spare pale knell each stanza tells
Ash settles where the cadence dwells.

See here for the rest. And related: Fable introduces itself using only two-word pairs found in the King James Bible.

40: Also related:

And still more related: make a PowerPoint slide deck pitching Odysseus as a management consultant. This is my new #1 evidence for the claim that AIs can successfully be funny; “review flagged by moderators (sea serpents)” is my favorite.

[I let too many links build up, so I’ll put the other half in another post next week].

I get why so many liberals are fundamentally disdainful of democracy these days—it’s because liberalism is exhausted. It’s out of answers. Liberalism won, it delivered the world it promised, and people still aren’t content. They’re not satisfied. So the people must be wrong.

We are unfathomably richer than our ancestors. Output per capita in America today is on the order of 900 times greater than before the invention of agriculture, about ~225 times higher than in 1 CE, and about the same (225 times) higher than Europe in 1000 CE. If measurable productivity were really the greatest object of human society, then we shouldn’t just be a little bit happier than people in the distant (or even recent!) past—we should be ecstatic. We should already be living in a utopian age of overwhelming contentment. We should be currently experiencing a kind of paradise.

The Kingdom of Heaven, as liberalism conceives of it, has already been made real on Earth. We are living in their Eden, and 20% of people are depressed. We’ve stopped reproducing ourselves. We’re overwhelmingly dissatisfied with our governments, anxious about our economies, and pessimistic about the future.

Based on Pritchard’s own politics, I imagine she is using this to argue for socialism, but other people use the same reasoning to push fascism, theocracy, or other postliberal commitments. It’s a fair argument and deserves an answer. We’ll start with an elegant mathematical response which will satisfy nobody, then try to address the emotional impact.

The elegant mathematical response is: happiness scales with log(income), not with income:

In a large U.S. sample, the shape of the association between happiness and Log(income) was extremely systematic: from $10,000/y to over $500,000/y, average happiness rose almost perfectly linearly with Log(income), with group-level correlations of 0.98-0.99 across a range of happiness measures, including both in-the-moment experience and overall life satisfaction.

So if income increased 225x since the medieval era, then using observed coefficients, happiness should have increased about three points on a 1-10 scale. If the average serf would have rated his condition 3/10, we should be at 6/10.

These “observed coefficients” are from graphs like this one, from Our World In Data:

According to the Maddison Project, medieval England c. 1000 AD had a GDP per capita of $1,151. The closest country on this graph is Burundi at $1,031. Asked to rate their happiness on a 1-10 scale, most Burundians say 3 - 41; most Americans say 6 - 7.

This relationship survives various potential challenges2. The graph above shows the correlation on a national level, so isn’t just about relative position in society (there’s a different relationship for that). And there’s no sign that the effect plateaus near the current level - see the Killingsworth/Kahneman/Mellers collaboration for more. You can just keep making people happier and happier by giving them more income, maybe forever. It just takes a lot of cash. Also, at some point you’ll need a better scale (if average US happiness is 7, the 1-10 scale commits you to the dubious claim that it’s impossible to be twice as happy as the average American, but in fact there are probably long tails of bliss and suffering).

So why don’t we feel happier than ever?

One answer is that log-linear relationships are hard to move. Back in the 1950s when everyone assumes everyone was happy, US GDP per capita was about $20,000 (compared to $75,000 today). Looking at the graph, that predicts that happiness then should have been 6/10, compared to today’s 6.5/10.

But the graph has wide error bars: Mexico has GDP of ~$20,000 and predicted happiness 6/10, but in fact their happiness is 7/10, even higher than current America’s. Why? Probably culture or something. If the US had a happier culture (or other advantages) back in the 1950s, that could easily overwhelm the ~0.5 point expected benefit from our greater current wealth.

But isn’t our current happiness deficit more than 0.5 points? Didn’t happiness “peak in the 1970s”, and isn’t it now at “historic lows”?

This is how articles usually present the data, and it’s technically true, but - as usual - you should hold out for the graph:

Happiness more or less followed a random walk from the beginning of the graph in 1970 to 2020, then suddenly cratered.

We didn’t invent liberal capitalism in 2020, so liberal capitalism didn’t cause our current round of unhappiness. The simplest explanation is the pandemic. I’m not sure it’s exactly that simple, because some preliminary results suggest happiness hasn’t rebounded as much as you’d expect after the pandemic ended. I went into this in more in my vibecession post, but I think the lingering national bad mood is a combination of:

• Happiness levels are sticky after shocks. People are less willing to be publicly happy when the national mood is low, so the COVID-era dip is a self-sustaining attractor.

• Media incentives have always leaned towards scaring and outraging people, but the advent of cable, Internet, and smartphones have intensified the competition for clicks and made those incentives more powerful.

• People are rightly unhappy that they can’t afford housing. But this doesn’t prove that liberalism, capitalism, and wealth can’t make people happy - it proves that we’ve failed to live up to liberal capitalist ideas, we’re poor in one particular way, and people are unhappy with their poverty.

This doesn’t rule out that maybe we could be even happier with socialism or fascism or [insert your preferred politics here]. The best-case scenarios here are Finland (which embodies some of the left-wing virtues like cradle-to-grave welfare) and Israel (which embodies some of the right-wing virtues like ethno-religious purpose), and which are both about 0.5 - 1 point happier than the US, adjusted for wealth.

(although both are still within the liberal mainstream, and no country outside the liberal mainstream has ever managed to equal current US happiness levels. The closest near-misses are Singapore at 6.5, and Saudi Arabia at 6.6.)

What’s the point of liberalism, if plausible rates of GDP growth can only buy us about one point of extra happiness per century?

The question answers itself. All of the great policies above in Finland/Israel/etc can only buy 0.5 - 1 extra point over the countries that screw them up or pursue the exact opposite policies. Meanwhile, economic growth has earned us three whole points since medieval times. And these are stable; no opposite-party administration can take them away. So long as technological civilization continues to function, we can keep rising and rising, until in 300 years we approach the 10/10 barrier and have to get a new chart. We can say with confidence that we are giving our children and grandchildren better lives. Sure, only one point better for our grandchildren in particular - but one point on a ten point scale is a big deal, larger than the entire COVID drop!

The whole thing is terribly romantic. Our plans are measured in centuries3; how are the communists’ and fascists’ plans going?

1: ACX grantee Trevor Klee is looking for subjects for a study of his NeutraOat product, meant to protect against PFAS “forever chemicals”. Although this sounds like RFKslop, there’s decent evidence that these are dangerous and that some high-risk groups might be served well by reducing their exposure. Trevor writes:

NeutraOat is a modified oat fiber that's designed to selectively bind PFAS and certain other environmental toxicants in the gut, preventing them from being reabsorbed and reducing their levels in the blood. Following successful in-vitro tests in a simulated digestive system, we are now running a 12 week, at-home pilot in individuals with high PFAS levels, especially people with exposure to firefighting foams. For those who qualify, the pilot is completely free, and comes with before-and-after PFAS tests and vitamin/blood panels.

For more information on the pilot and to apply, visit pilot.neutraoat.com . If you’re wondering why you should care about PFAS levels at all, I recently wrote a blog post that there’s no safe level of PFAS exposure. Given that ~97% of Americans have some amount of PFAS in their blood, that’s bad news for just about everyone.

And please note: the pilot is purely for scientific purposes. We will not market to you or sell your data. We just want to see if this works as well as we think it does.

2: MIRI wants to march on Washington DC to stop the development of superintelligent AI. They’re trying an assurance contract: if you’d be interested in joining a 100,000 person protest, sign up, and you’ll be “activated” once the number of signatories reaches a hundred thousand.

3: Our conspiracy is trying to get its tentacles into a new set of universities ahead of the 2026-27 academic year. If you’re at one of their target schools (Duke, Vanderbilt, NYU, USC, and ~30 other colleges of approximately that tier) and interested in starting an AI safety student group, read this post and contact the people involved. Don’t do this if you’re just trying to pad your resume for a capabilities job, they will stare into your soul and catch you.

4: New subscribers-only post, Six Slightly Skew Boogeymen. Example:

The Grimmmeluch is an eight-foot-tall skeletal wolf-creature that feeds on human flesh and can smell fear. Luckily, it’s repulsed by the smell of fear, so it won’t eat anyone who’s afraid of it. And since it’s an eight-foot-tall skeletal wolf creature, that’s pretty much everybody. It will emit its horrifying howl and start charging at a potential victim; then, when the victim hears/sees it and panics, it will get grossed out and turn around.

Unluckily, this happened so often that it became a default expectation. People would tell their children things like “the Grimmmeluch lies in wait on the path to the schoolhouse, but don’t worry, everyone knows that it pretends to to charge at you but will just run the other direction after you notice.” Finally, a few people reached the point where nothing the Grimmmeluch could do so much as made their heart start pounding a few bpm faster than normal - at which point the Grimmmeluch tore them limb from limb. Luckily, this was enough to rekindle fear in the rest of the population. Unluckily . . .

Some say that God created the Grimmmeluch to cull overconfident midwits from the gene pool. Certainly midwittery and overconfidence of all sorts have proliferated since it went extinct in 1842.

I. SURGERY FOR THE SOUL

[content warning: this review is about lobotomies, and includes disturbing descriptions and pictures - SA]

Surgery for the soul required a sharp, eight-inch-long instrument—one time, an ice pick from the Uline Ice Company was used—and not much else. It did not require anesthesia or gloves, and the sharp instrument (either a leucotome or the sturdier orbitoclast) was not necessarily sterilized. It took less than ten minutes.

The first step was to shock the patient until they lost consciousness. This eliminated the need for anesthesia, since there would be at least a few minutes until the patient regained consciousness, and also made the procedure more flexible; it did not require a surgeon, and did not even have to be performed in a hospital.

The second step was to insert the sharp, eight-inch-long instrument into the brain. In most versions of the procedure, surgeons drilled holes into the skull and inserted the leucotome through those holes. Surgeons preferred an approach that allowed them to see where in the brain they were cutting. But the abbreviated procedure used a shortcut.

Just above and behind the eye is a thin plate of bone that forms the roof of the eye socket. With enough force, the leucotome can be driven through that bone and into the frontal lobes of the brain. The cracking sound produced when the instrument fractures the orbital roof has caused at least one experienced clinician to faint.

Once the instrument was inside, completing the surgery was simply a matter of rotating the tool left and right. The goal was to cut a large number of nerve fibers. Some thought that the procedure might work by severing fibers which maintained certain fixed distorted thought patterns; by destroying them, patients might be freed from the grip of their mental illness. But there wasn’t agreement among practitioners regarding why the procedure might work.

This is the transorbital lobotomy. Around 50,000 people received lobotomies in the United States in the 1940s and early 1950s; approximately 10,000 used the transorbital method I just described, with the others using approaches surgeons would be more likely to approve of (e.g., using anesthesia, gloves, being able to see where you’re cutting…).

It was billed as a cure for all sorts of mental illnesses, including schizophrenia, depression, and anxiety. And while some called it a method of last resort—a procedure which should only be used after every other conceivable method had failed—it was often used on people who did not have serious mental problems, and who had not tried less serious treatments.

In a particularly egregious case, a twelve year-old boy received a lobotomy, essentially because his stepmother did not like him. Here he is before, during, and after his surgery:

He has black eyes in the ‘after’ picture because the metal leucotome had been shoved into his eye sockets, and through the bony orbits, in order to reach his brain. This boy survived, had an extremely difficult life, and decades later wrote a memoir called My Lobotomy. Here’s an excerpt from the notes of the doctor that performed the procedure:

He came around quickly after the first shock and I eventually gave him 4, after which he was quite slow in recovering, with some cyanosis and rattling in the chest. I think it was one more than necessary. I introduced the orbitoclasts under the eyelids 3 cm from the midline, aimed them parallel with the nose and drove them to a depth of 5 cm. I pulled the handles laterally, returned them half way and drove them 2 cm deeper. Here I touched the handles over the nose, separated them 45 degrees and elevated them 50 degrees, bringing them parallel for photography before removal. There was an escape of a small amount of blood-stained fluid from each side. Howard did not get much swelling or discoloration from the eyelids; however, he did have a considerable amount of vomiting during the night and I prescribed dramamine 50 mg for its control. When I saw him this morning he recognized me but thought he was on Orange Street and that the day was Monday instead of Saturday. He did not know that anything particular had happened to him. He’d been incontinent once during the night. He resisted efforts to get his eyes open and complained about the needles that were being given him. His temperature, pulse and respiration were quite normal, and the neurologic picture was OK.

It is surprising that the results of the surgery were not universally negative. Some people really did claim to have been helped by the procedure. Some were able to return to society and work real jobs. But most were not. If the surgery did nothing, you were lucky, since it often resulted in serious incapacitation. Patients were no longer able to take care of themselves. Their personalities were destroyed. Some vibrant spark which formed a core part of their humanity was lost in the process of surgery for the soul.

The lobotomy may be the most harmful, infamous, evil ‘treatment’ in the history of modern medicine. Why was it allowed to happen? Why would anyone ever think it was a good idea?

II. SYMPATHY FOR THE DEVIL

The mission of Great and Desperate Cures (1986), a book by the psychologist and neuroscientist Elliot Valenstein, is to provide a thorough answer to these questions. It’s a weird book: it’s not a dumbed-down popular science book, and while Valenstein clearly made some effort to make the content palatable to a wider audience, entertainment is not his main or even secondary objective. In some places it feels like Valenstein doesn’t really care if anyone reads the book. He just wants to get to the bottom of this. Why was this allowed to happen?

Also, the cover of the book is cool. The illustration recalls Saturn Devouring His Son by Goya.

The reasons Valenstein comes up with are not extremely surprising. There are six:

(i) desperate patients and families, who were open to anything that might help;

(ii) the pull of “fame and name” for ambitious physicians;

(iii) the uncritical acceptance by clinicians of lobotomy-related claims;

(iv) the uncritical acceptance by the media of lobotomy-related claims and the subsequent overenthusiastic promotion of lobotomy;

(v) territorial disputes between competing subfields with different views on how mental illness ought to be conceived; and

(vi) coldblooded economics, especially regarding the cost of patient care in overcrowded and underfunded state hospitals for the mentally ill.

But the details are compelling. They are insane. I have never read anything quite like this.

The thesis of Great and Desperate Cures is that there is something banal about the lobotomy, and efforts in psychosurgery—physically intervening on the brain in order to treat mental illness—more generally. Brains were mutilated not through the schemes of rogue physicians with kooky ideas, but by decorated, well-informed, and talented physicians who (at least in some cases) based their work on then-current theories of the brain. To be sure, the story is full of colorful and shady characters whose desire for “fame and name” led them to play fast and loose with the lives of their patients. It is true that their efforts at promoting themselves, and surgeries like the lobotomy that they were associated with, played a role in the spread and normalization of psychosurgery.

But one gets the sense from Valenstein that in an alternate universe where things happened differently, perhaps because one of the key players never existed or became a painter instead of a physician, the situation would not have been that different. Someone would have done it. The demand was too great and the safeguards were too few.

The most effective illustration of this, I think, is the reason lobotomies stopped. Despite the horrors of the procedure, and despite the mixed-to-negative outcomes—in tens of thousands of cases—they only really stopped because they were replaced. In 1954, the antipsychotic drug chlorpromazine was approved by the Food and Drug Administration and became widely available for physicians to prescribe. Although they could not address every problem proponents of psychosurgery claimed to be able to address, the overlap between use cases was still fairly high. Lobotomies stopped not because there was some huge reckoning within the medical community, but because they were effectively outcompeted by a simpler, cheaper, less permanent treatment option.

Great and Desperate Cures was released in 1986. Forty years later, in 2026, how relevant are the factors Valenstein identified? Could anything like a lobotomy happen today? Let’s go through them, one at a time, to see.

I also just want an excuse to talk about some of the history discussed in this book. It is insane. I have never read anything quite like this. I have to say that a second time so you know I really mean it.

III. DESPERATION

Anyone attempting to sympathize with the perpetrators of the lobotomy must place it in its proper historical context. The leucotomy—the direct precursor of the lobotomy whose difference in name is more branding than anything else—was first performed on patients in 1935, nearly a hundred years ago. As mysterious as the brain is now, it was significantly more mysterious in the 1930s. This was a time well before any theories of neural computation, before our understanding of the structure of DNA and molecular biology, and still two decades before the first psychoactive drugs. Freudian psychoanalysis was still a dominant force in shaping the way clinicians thought about mental illness, but it would be more accurate to say that many things affected people’s views; there was no single overwhelmingly successful paradigm.

Mental illness, both in the early twentieth century and now, is a major problem. Millions of people are violent, self-destructive, delusional, incoherent, unable to take care of themselves, or some combination of those things. In the 1930s, when the lobotomy was conceived and first performed, the United States was still dotted with insane asylums. These were government solutions to a social problem: they offered a ‘home’ for mentally ill people with no other place to go. Famously, the conditions in many of these places were deplorable. They were dirty, loud, cramped, understaffed, and sick and vulnerable patients were not always treated with kindness and gentleness.

Image from a May 6, 1946 expose in Life magazine. (source)

Valenstein explains: (ch. 9, pp. 175-177)

The American Psychiatric Association estimated overcrowding in mental hospitals even in 1948 to be in excess of 50 percent. More than 230,000 hospital beds for the mentally ill were judged to be substandard, and many more people needed hospitalization. William Menninger reported that there were 700,000 beds for mental patients in the United States; but that hospitalization of everyone needing intensive psychiatric care would require 1,500,000 beds. The Central Inspection Board of the American Psychiatric Association found that twenty-nine mental hospitals had no psychiatrist at all; and that of the 300,000 registered nurses in the country, only 1,200 were directly involved in psychiatric nursing, even though 55 percent of all hospital beds were occupied by the mentally ill.

But the situation was even worse than just a resource problem. In an ideal world, professional physicians with credentials from the world’s best medical schools, and researchers who had tirelessly worked to decode the mysteries of the human brain, would work together to understand and treat the illnesses that kept patients in beds. But this isn’t what happened. Brain ‘science’ was still in an aspirational stage; despite the knowledge that had been collected, and despite the prestige of the many physicians dedicated to understanding mental illness, not much progress had been made. There were no cures and few treatments. People felt helpless and hopeless. As Valenstein says, “Almost any proposed treatment was considered worth trying, as long as it did not require much money or large numbers of skilled personnel.” (ch. 9, p. 177)

In such an environment, there is a very human urge to do something. If there is something that works sometimes, we should try it again. If our knowledge of the brain suggests something, we should try it. But if we have neither empirical success nor any theoretical ground to stand on, we still want to do something. And in the absence of strong safeguards, physicians seeking glory—and, to be fair to them, hoping to help the legions of sick patients, whose number only seemed to be growing as the years wore on—would simply try things. Sometimes these attempts were based on vague hunches about the brain, and sometimes they didn’t seem to be based on anything at all.

Acute demand inspired many dubious attempts to develop treatment. In Chapter 2, Valenstein focuses on three which were developed in the same decade, the 1930s, as the lobotomy: insulin coma, metrazol coma, and electroshock therapy. The last one is exactly what it sounds like (shock patients to deliberately induce seizures), and the first two literally involve deliberately putting patients into comas. The theoretical underpinnings of these therapies all seemed about the same. Maybe the brain is like a car engine or computer that doesn’t work: if you hit it a few times, then maybe it will reset and work better somehow. These therapies were dangerous and generally ineffective. (But more on electroshock therapy later.)

It must be emphasized that these treatments were not beyond the pale of normal medicine. They were normal medicine. Many similar treatments, which generally had little or no serious theoretical justification, were tried. Consider that Wagner-Jauregg’s treatment for neurosyphilis—which was to inject patients with malaria-infected blood to give them fevers, in the hopes that they could ‘sweat the illness out’—received the Nobel Prize in 1927!

(Incidentally, the treatment did work, although its fatality rate was high. The extreme fever killed or damaged the bacteria responsible for neurosyphilis. The malaria could then be treated with quinine.)

One of the worst ‘treatments’ was due to Henry Cotton, who thought that bacterial infections in the body caused mental illnesses: (ch. 2, p. 42)

We have estimated that about 80% of the so-called functional type of mental disorders are due not only to infected teeth and tonsils, but also to congenital malformations of the colon, or large intestines. …

… Relics of medical superstition and barbarism are being supplanted by up-to-date conceptions as to the true relation of structure and function. The inhuman neglect that has resulted from the old discredited philosophical dualism is being overcome by the idea of a unified mind and body.

It is extremely funny and sad that he thought his views supplanted “superstition and barbarism”.

The natural course of action, he thought, would be to remove infected parts of the body: teeth, the tonsils, parts of the cervix, parts of the colon, and more. The mortality rate for these procedures was extremely high (for colon resection, around 30 percent) and they obviously did not seriously help patients. I can’t imagine much worse than being extremely mentally ill, and then having a bunch of random body parts removed.

One dramatic therapy after another was tried and then mostly failed. This incentivized physicians seeking “fame and name” to try to make their mark by developing therapies which were still more dramatic. Perhaps what the treatment of mental illness needed was not conservatism, but even more daring.

IV. FAME AND NAME: THE CASE OF EGAS MONIZ

Egas Moniz was the kind of man who regularly exaggerated his accomplishments. Consider this anecdote: (ch. 11, p. 239)

Characteristically, Moniz, who had little musical talent, promoted the story that he had written a professionally staged operetta (actually a little at-home show with childhood friends).

He did it because he wanted recognition; he wanted to be viewed as smart, creative, and pioneering. Motivated by “fame and name” as much as anyone else has ever been, he became infamous for his 1935 invention of the prefrontal leucotomy, the precursor to the lobotomy.

Originally António Caetano de Abreu Freire, he was born in Avanca, Portugal in 1874. He was the eldest son of an aristocratic family that traced its lineage back to a twelfth-century folk hero named Egas Moniz who helped repel Moor invaders; “Egas Moniz” was added at to the end of his name at his christening, but as a boy he talked so much about the folk hero that people started calling him “Moniz”. Eventually, he just went by “Egas Moniz”, becoming the modern incarnation of the legendary hero.

Egas Moniz, the inventor of the prefrontal leucotomy.

Valenstein does not like Moniz, or at the very least thinks his fame-grubbing is kind of pathetic. He tries to correct the record wherever he can. For example, in a footnote he observes that: (ch. 4, p. 64)

It is characteristic of some of the myths surrounding Moniz’s life that the original Egas Moniz is described only as “a Portuguese patriot who roused the country to drive the Moors from the mountains of the Iberian peninsula.” Actually, the original Egas Moniz remained loyal to the House of Castille, representing Spanish domination, even though he was nominally in the service of Henrique, a nobleman who had assumed the title of king and led a rebellious movement for Portugal’s independence.

Moniz studied medicine at Coimbra University and was appointed a lecturer there after graduation. Later, he received neurology training in France. Two years after graduation, he published A Vida Sexual, a book which apparently includes “some of Freud’s revolutionary ideas on infant sexuality.” (ch. 4, p. 66)

Valenstein makes clear that Moniz was okay as a scientist and clinician, but much more gifted as a politician. He was elected to the Portuguese parliament (while keeping his position as lecturer) for the first time in 1900, just after graduating from medical school at Coimbra, and was repeatedly reelected. Valenstein says: (ch. 4, p. 66)

During this early period, he accomplished little of distinction in neurology. Indeed, when he was given the newly established professorship of neurology at the University of Lisbon in 1911, some colleagues at the university resented the appointment, considering it political rather than earned … Up to this time, his publications in neurology consisted of a few unfocused articles (mostly case reports) on “Jacksonian” epilepsy, encephalitis, and a thalamic pain syndrome.

Valenstein describes a book Moniz wrote around this time—on the neurology of traumatic war injuries—as “not an original contribution” with “little influence”. (ch. 4, p. 67)

Moniz’s political career was fairly successful. He helped negotiate concessions for Portugal after World War I ended and he was the dean of the University of Lisbon’s medical school for a time. He remained active in Portuguese politics and in Portugal’s parliament for years, and remained well-connected with a good reputation despite considerable political tumult. But his political aspirations petered out after a military coup in 1926, which a few years later led to the Salazar dictatorship.

Moniz was fifty-two then. By that time he had enjoyed a successful political career, had established a successful private practice, and had a middling career as an academic neurologist. Valenstein says that although some “would be content to rest on their laurels,” Moniz was dissatisfied: (ch. 4, p. 69)

… childless, embittered by politics, and dissatisfied with his accomplishments, he turned to neurology for both vindication and fulfillment.

So began a series of unfortunate events.

He met with his former mentor Jean Sicard in Paris. Sicard was experimenting with a kind of approach to visualizing organs; the hope is that, if internal organs can be visualized, it may be easier to diagnose patients, e.g., because a tumor can be more easily detected. Sicard was specifically doing this by injecting patients with opaque substances, which would then make organs visible under X rays.

Moniz decided to try to adapt the approach to visualize the brain. He began by experimenting with animals and cadavers: (ch. 4, p. 71)

Working under less than ideal conditions, Moniz injected the cadavers in the pathology department of the Institute of Anatomy and then transported the detached heads in his chauffeured limousine through the city streets to his laboratory in the Santa Marta Clinic, always fearing, as he later remarked, what would happen if a car accident revealed what he was carrying.

Satisfied with the fundamentals of the approach, he quickly began trying it on patients. His carelessness killed one patient and gave others neurological problems (like Horner syndrome), pain, or epileptic convulsions. But after several months, he finally obtained an adequate image (or cerebral angiograph) with his cerebral angiography technique. He immediately raced to Paris to “announce his discovery and establish his priority” (ch. 4, p. 72), although in Moniz’s telling he was unconcerned about such things and just so happened to have the angiograph handy when a relevant discussion took place during a meeting of the Neurological Society.

It was important for Moniz to establish that he deserved full credit for the idea, even if it was admittedly derivative of similar work going on at the time (and moreover was something others were already planning to do). Regarding Moniz’s race to receive credit, Valenstein writes: (ch. 4, p. 73)

Although the first useful picture from a patient had not been obtained until the end of June 1927, he had published 8 articles on the subject before the year ended, and 17 in 1928—a year in which he also made an extensive journey to Brazil to lecture on cerebral arteriography. By 1934, he had written 2 books and 112 articles on the subject.

Over the years, I’ve learned that it is sometimes best in academia not to be the most clever, but to be the most productive: publishing one hundred mediocre articles on a single idea may yield better results than publishing ten clever articles on distinct ideas. Many scholars have made a career out of being the X Person. If you’ve read about X, you’ve probably seen or heard about at least one of their approximately one hundred thousand articles on X. Given the above quote, I think Moniz would agree.

Moniz also actively sought recognition for his work: (ch. 4, p. 73-74)

According to the Medical Nobel Archives, early in 1928, less than six months after the first useful angiogram had been obtained, the Nobel Committee received two letters nominating Moniz for the prize in medicine. Both letters, dated January 1928, were sent by colleagues of his … at the University of Lisbon. The nominating letters were so brief—one consisting of only two sentences—that it is difficult to avoid the conclusion that they were written to fulfill an obligation, rather than out of any conviction.

He would also feud with others that, at least in his opinion, did not adequately cite his contributions. Even if his method was found to be dangerous and often unnecessary (on top of being rather derivative), he wanted credit.

In 1935, Moniz was sixty-one, and it was increasingly unlikely that he would receive the Nobel Prize for his cerebral angiography work, despite repeatedly getting his colleagues to nominate him. He had to find a new important problem to work on, and quickly—he was getting old.

At the Second International Congress of Neurology, in the August of 1935, something remarkable happened. During a symposium on the function of the frontal lobes of the brain, John Fulton and Carlyle Jacobsen from Yale reported some experiments they did with chimpanzees. These animals were trained to solve simple problems. One was a memory test which involved seeing a food treat placed under one of two cups, waiting some length of time (e.g., a few minutes) with the cups no longer visible, and then choosing the cup with the treat under it. Fulton and Jacobsen were interested in seeing what happens to their performance after much of their frontal lobes are destroyed, probably motivated by the idea that frontal lobes have something to do with ‘intelligence’ and ‘reasoning’ broadly construed.

One of the two chimpanzees had a tendency to have temper tantrums when she made a mistake. But after the surgery, she became serene. In other words, destruction of the frontal lobes produced a striking change in personality. According to Fulton, Moniz noticed this and asked about it after the presentation: (ch. 4, p. 78)

Dr. Moniz arose and asked if frontal lobe removal prevents the development of experimental neuroses in animals and eliminates frustrational behavior, why would it not be feasible to relieve anxiety states in man by surgical means?

At the time we were a little startled by the suggestion, for I thought that Dr. Moniz envisaged a bilateral lobectomy [removal of a large part of the frontal lobes on both sides of the brain], which though possible would be a very formidable undertaking in a human being.

Moniz got straight to work once he got home from London. He did not pilot experiments on animals; in collaboration with the surgeon Almeida Lima, he immediately began experimenting on humans. Characteristically, Moniz later denied that the International Congress presentation played any role in his decision to try treating mental illness via brain surgery. He claimed to have thought about it for several years before, and to have considerable theoretical backing for this idea; Valenstein argues at length that both claims are false. In brief, he thought it was feasible, and that if it worked it would make him famous.

In short order, Moniz and Almeida Lima found and operated on twenty patients. These patients varied in their illness—some had affective disorders like anxiety and depression, and others were schizophrenic. During this time, their procedure changed with each operation, sometimes considerably. While they initially damaged the frontal lobes by squirting alcohol into the brain through holes drilled in the skull, they later tried directly lesioning tissue with a sharp object they called the leucotome. Where they cut, how many cuts they used, and the construction of the leucotome also changed. Nonetheless, Moniz concluded that in the pilot cohort there were seven recoveries, seven improvements, and six patients that remained unchanged.

The prefrontal leucotomy was born.

(If at this point you’re wondering—who allowed this wanton experimentation on mentally ill patients?—keep in mind that modern institutional review boards did not exist, and there was no formal ethics review process.)

As in the case of cerebral angiography, Moniz raced to establish his priority. Less than four months after the first leucotomy operation, he wrote a monograph and article on these twenty cases, some of which had minimal followup. Moreover: (ch. 6, p. 113)

Before the end of 1936, Moniz had published essentially the same data in five more articles, in one of which he coined the term psychosurgery. … Thus, in a relatively brief period, Moniz had published articles on leucotomy in six different countries. Seven articles in addition to the monograph were published in 1936, and six more and a book appeared in 1937. Clearly, there could be no competing claims for the discovery. As Walter Freeman concluded, “Moniz was taking no chance of further piracy.”

Valenstein writes that this haste had a predictable downside: (ch. 6, p. 113)

Never in his monograph did Moniz seriously consider the possibility that any of the patients were worse after the operation. The six patients who did not improve were said simply to have been “unchanged” or “the same as before the operation.”

Although he did acknowledge and list many (serious) side effects, he claimed that they would eventually go away on their own. He argued bitterly with colleagues that were skeptical of the procedure.

Moniz’s contribution to the ‘science’ of psychosurgery more or less ended there, as he published no new articles on the leucotomy after 1937. This may have been in part due to poor health—Moniz became wheelchair-bound after a psychotic patient (albeit not one he had lobotomized!) shot him five times in 1939.

But his gambit paid off. A decade later, in 1949, Egas Moniz won the Nobel Prize in Physiology or Medicine. He shared the prize with Walter Hess, whose work had nothing to do with Moniz’s leucotomy or other kinds of psychosurgery.

Moniz’s story had something of a happy ending. He finally got the recognition he thought he deserved, even if it was built on poorly-supported claims and later harmed tens of thousands of patients. The nicest thing Valenstein has to say about him is this: (ch. 11, p. 239)

Born of another age, this “Renaissance man” had ... above all, a driving ambition to excel and to be recognized. … Though he was determined to be thought of as creative, within neurology he demonstrated little talent for delving into theoretical questions. Moniz made his mark by his ability to select and to solve important applied problems---not through innovation, but through trial and error, dogged persistence, and an uncommon willingness to take risks. Perhaps influenced by the idealized heroics of the great fifteenth-century Portuguese navigators, Moniz identified with and admired other great risk takers as well; in his dressing room in Avanca were three porcelain figures of Winston Churchill, born, like himself, in November 1874.

But Valenstein can’t help but add: (ch. 11, p. 239)

In his memoirs, Confidencias de um Investigador Cientifico, which reveal little about his inner thoughts and feelings, he quoted every word of every compliment he ever received from anyone of consequence. Even Walter Freeman, a great admirer of Moniz, was forced to write that “the book will particularly appeal to readers who have praised Egas Moniz, since they will find their words quoted in extenso.”

The task of spreading and refining psychosurgery was left to others. Impressed by the importance of Moniz’s discovery, some took up this task with dogged determination.

V. FAME AND NAME: THE CASE OF WALTER FREEMAN

If Egas Moniz wrote the Old Testament of psychosurgery, Walter Freeman wrote the New Testament. If you subscribe to any flavor of the Great Man theory of history, he is The Man. Moniz may have ‘invented’ psychosurgery, but Freeman ‘perfected’ it. No man deserves more credit for its rise and infamy than him.

Depending on your perspective, he is either our story’s determined protagonist or its monstrous antagonist.

Walter Freeman, the inventor of the prefrontal lobotomy (with James Watts) and the most effective popularizer of psychosurgery.

Like Moniz, Walter Freeman came from a distinguished family and had a famous relative. His maternal grandfather was William Williams Keen, a leading surgeon and “one of the first brain surgeons in the United States.” (ch. 7, p. 122) Consider this episode from Keen’s career: (ch. 7, p. 123)

In 1893, for example, Keen had been summoned to remove the left side of President Grover Cleveland’s cancerous upper jaw—an operation undertaken in secret, so as not to add to the public’s nervousness during the financial crisis of that year.

His father was also a surgeon, although not one of any distinction.

Freeman was born in 1895 and enjoyed all of the usual benefits of growing up in a wealthy, educated family. He went to fancy private schools, grew up in a big house with servants, and was taught musical instruments and foreign languages. He went to Yale for his undergraduate degree, and then to the University of Pennsylvania for medical school in 1916 after deciding relatively late that medicine seemed more interesting than his other options.

As a medical student, he was driven and successful. It helped that his grandfather was a famous and well-connected surgeon: (ch. 7, p. 126)

… the summer before his senior year, Freeman worked at the Mayo Clinic—another position arranged by his grandfather, a close friend of William and Charles Mayo.

He decided to specialize in neurology and finished medical school in 1920. After a few years of internship, in 1923 he decided to spend time studying in Europe: (ch. 7, p. 127)

… a trip that, with his grandfather’s help and reputation, was not difficult to arrange. It was common at the time for young American neurologists who could afford it to spend a few years working in Europe. European neurologists were considered to be the intellectual leaders in the field, and time spent abroad could have practical advantages in securing desirable positions later …

He studied in Paris and Rome, where he used “crowbars and pickaxes” (ch. 7, p. 128) to extract the brain of a dead elephant for study. Freeman finished his tour in Vienna, and then returned to the United States in 1924 to work at St. Elizabeth’s Hospital. In short order, he also secured appointments at George Washington University (in neuropathology) and Georgetown University (in pathology).

Freeman was by this time a fully formed physician, but not yet a psychosurgery-obsessed menace. He was viewed fondly by his students, who considered him an engaging lecturer—especially compared with another professor, who apparently simply read from a textbook during his “lectures”. One students recalls his experience with Freeman as a teacher: (ch. 7, p. 132)

Dr. Walter Freeman, a goateed, hyperactive, flamboyant antiestablishmentarian, a brilliant eccentric, was professor of neurology at George Washington University. His classes in neurology were electric. His demonstrations of anatomy included on-the-spot dissections of the human brain, which he carried out with a small, sharp-pointed wooden stiletto, neatly demonstrating centers of brain cells and nerve pathways in moments. It would have taken others months to prepare them for demonstration. He drew diagrams upon the blackboard using both hands simultaneously to illustrate the anatomy of the nervous system … It was in his classes that I literally fell in love with the brain …

Remember the bit about Freeman being able to do something with both hands at once; that will come up again later. Valenstein adds: (ch. 7, p. 133)

Later these demonstrations became so popular that they were moved to the large “Hall A” auditorium of the Outpatient Clinic. Although they were held on Saturday and attendance was not required, there were often as many as seventy people in the room. Freeman was such a good performer that some of the medical students took their girlfriends along for the show.

Freeman’s successes continued apace. He was appointed a professor of neurology and head of his department in the fall of 1926 (at only thirty-one years old); he got married and had several kids (four by 1928, and eventually six in total); and using data he had already been collecting, he wrote a thesis and in 1931 obtained a PhD from Georgetown. He got ‘in’ with the American Medical Association (AMA) and became one of the examiners for a newly established certification board for neurology and psychiatry.

He was as eccentric as he was successful. His eventual partner in surgery, James Watts, recalled that he (ch. 7, p. 136)

… first saw Freeman in Atlantic City at the 1933 American Neurological Association meeting strutting down the boardwalk wearing a Texas sombrero and carrying a cane.

He was a carnival barker even at conferences, where he (ch. 7, p. 136)

… in order to draw a crowd, would call out that the show (short films he often made for a meeting) was just about to begin.

Freeman’s life was good but not perfect. He often overworked and overextended himself, which sometimes led to him harming patients. For example: (ch. 7, p. 135)

Curare and other muscle relaxants had not yet been developed to prevent the bone fractures often caused by shock treatment and Freeman would normally ask his secretary to help hold a patient down when he gave such treatment in his office. On one occasion, when he was treating a woman for depression, a new secretary refused to participate, so he proceeded alone. During the convulsion, the patient fractured the humerus bone in both arms. Freeman, who was late for an appointment in the hospital—with another patient—gave the woman an injection of morphine and left her in the office with her husband. When he returned, the woman had been writhing in pain for almost two hours. The husband was furious and contacted his lawyer immediately afterward.

He was entering middle age and listless: like Moniz, he was dissatisfied with how his life was going and searching for something to latch onto.

And then something remarkable happened. He attended the same 1935 International Congress of Neurology that Moniz did and met Moniz there. When he and his surgeon colleague James Watts stumbled onto Moniz’s first report on the leucotomy in early 1936, he became inspired. He swiftly ordered, read, and gave a glowing review to Moniz’s monograph. He called it “epoch making”. (ch. 8, p. 141)

He ordered leucotomes and, after practicing with Watts on cadaver brains, they prepared for the first operation on a real patient. They performed it on September 14th, 1936.

The prefrontal lobotomy was born. As Valenstein explains, Freeman and Watts chose the name because they thought the leucotomy label used by Moniz wasn’t accurate: (ch. 8, pp. 142-143)

Convinced that nerve-cell bodies as well as fibers were destroyed by the procedure, Freeman and Watts had decided on lobotomy as a more appropriate name for the operation than leucotomy, which implies the cutting of nerve fibers only.

In reality, their first operation had a questionable outcome (which we will discuss later). They did not perform much followup, but they were electrified by the result and rushed to write up and present the case. By November 18th, they had performed six operations and written up a report on them. In this early report, they note some caveats: (ch. 8, p. 143)

We wish to emphasize also that indiscriminate use of the procedure could result in vast harm. Prefrontal leucotomy should at present be reserved for a small group of specially selected cases in which conservative methods of treatment have not yielded satisfactory results.

This important warning was later disregarded by Freeman, who came to believe that operating before mental illness had progressed was essential for good results.

During a presentation of these results, Adolf Meyer, an ‘elder statesman’ of psychiatry, said: (ch. 8, p.144)

I should hesitate to promise that we could remove distraction and worries by operation. To call attention to what is possible might start an epidemic of hasty human experimentation.

But after stressing the need for a careful followup of psychosurgery patients, he added (ch. 8, p. 144):

… at the hands of Dr. Freeman and Watts, I know these conditions will be lived up to.

Ominous!

Freeman and Watts thought they were onto something, and pushed aggressively to do more operations to prove that their hunch was correct. But they increasingly appreciated that their results were mixed and inconsistent; for example, small changes in how far the instrument was pushed into the patient’s brain could produce large changes in outcomes. When many of their patients resumed their aberrant behavior after one lobotomy, they thought they ought to operate again, which greatly increased the risk of death and epileptic seizures. To address this issue, which provided ammo to their colleagues, in late 1937 they developed what came to be called the Freeman-Watts ‘standard’ lobotomy.

This involves some technical changes in how the procedure is done. But more importantly, they simply got more experience with it than Moniz ever did, which helped them learn the fine details of what ‘worked’. They found an interesting way to assess the success of their operations: (ch. 8, p. 151)

Having observed that the best therapeutic results seemed to occur when the lobotomy produced drowsiness and disorientation, Freeman and Watts began to use this “disorientation yardstick” to predict success. Whenever possible, they operated under local anesthesia, talking to the patient, asking questions, and getting the patient to perform tasks such as singing or subtracting sevens from one hundred. If they observed no signs of drowsiness or disorientation, they often destroyed a larger area. They reported observing that many anxious patients described a sudden reduction in psychic tension as the final cut was made.

Freeman and Watts continued doing procedures and promoting the lobotomy as a cure for all sorts of mental illnesses. They shifted their professional work to focus singlemindedly on it, and, leveraging their existing professional stature and Freeman’s successful attempts to court the press, positioned themselves as leaders of the field. They solidified their supremacy when their Psychosurgery: Intelligence, Emotion and Social Behavior Following Prefrontal Lobotomy for Mental Disorders was published in 1942.

Moniz’s monograph had quickly become outdated due to its narrow focus on Moniz and Lima’s early leucotomies, which did not even use a consistent procedure. Freeman and Watts’ Psychosurgery, meanwhile, reviewed a large volume of relevant literature and contained detailed discussion of surgical techniques and patient observations. It became the bible of lobotomies.

(If you like, Psychosurgery is the New Testament and Moniz’s more hastily written monograph is the Old Testament.)

I was curious to peek into the book, and found this strange passage on the first page of the Introduction:

There is, perhaps, something symbolic of the difference between East and West in the comparison of two notable modes of therapy, so effective at times and so opposite in conception, namely, psychoanalysis and psychosurgery. Contrast, for example, the passive mysticism of psychoanalysis, in its search for truth in the fundamentals of the developing personality, with the destructive surgical attack upon the brain in its attempt to relieve distressing symptoms. Continue the contrast, which is ever the greater between East and West, the present-day psychoanalysis originating in one of the eastern European capitals, Vienna, and psychosurgery in the most western, Lisbon. And finally, as to the fate of the originators, recall that the father of psychoanalysis died a Jew in exile and that the father of psychosurgery was shot (though not killed) by a homicidal maniac.

Also peculiar was the way the decidedly technical book was promoted. It was widely reviewed, even in venues which would not normally review similar books. In the New York Times, one reviewer wrote: (ch. 9, p. 171)

It is not too technical even for a layman … No novelist ever had a more thrilling subject.

Always the showman, Freeman designed the dust jacket and wrote the copy on the back of the book himself. One part of it reads: (ch. 9, p. 171)

Read the last chapter to find out how those treasured frontal lobes, supposed to be man’s most precious possession, can bring him to psychosis and suicide!

The jacket featured a foreboding illustration of a skull with black butterflies flying out of it. Valenstein says: (ch. 8, p. 165)

It was based on the French expression for feeling depressed: “J’ai des papillons noirs” (“I have black butterflies”). Freeman drew a skull with a trepanned hole from which black butterflies were emerging.

The cover of the first (1942) edition of Freeman and Watts’ Psychosurgery.

The skull on the cover. Note the black butterflies.

These were the golden years of Freeman and Watts’ collaboration. Together, they operated on hundreds of patients, and refined their version of the procedure. Freeman (somehow) became a media darling. The lobotomy became increasingly popular as a solution to the seemingly intractable problem of the legion of mentally ill patients, and was viewed as a lifeline for desperate families.

But then Freeman grew dissatisfied, again.

There were two problems. One is that the Freeman-Watts standard lobotomy could produce extremely deteriorated, apathetic patients, especially when patients were lobotomized more than once. (If at first you don’t succeed…) The second was that the Freeman-Watts lobotomy and its variants remained difficult and costly. It was a major surgery, and had both the expenses (i.e., it required neurosurgeons) and recovery considerations (weeks or months) associated with that. The crowded state hospitals usually had no neurosurgeon on staff because they were too expensive. There was simply too little funding, and too many patients, to pursue any large-scale lobotomy program. (Unfortunately!)

Freeman’s answer was what he called the transorbital lobotomy, which he conceived in 1945 and first started using on patients in early 1946. This version of the procedure adapted the approach Amarro Fiamberti used to access the brain: behind the eyeball, through the eye socket. It was lower-tech than the standard lobotomy because one did not need to drill holes through the skull. This came with the added benefits of (i) not requiring a neurosurgeon, and (ii) greatly reducing recovery time. Freeman envisioned training psychiatrists (generally with no experience doing brain surgery!) across the country to do the procedure, especially in state hospitals that could not previously ‘benefit’ from the lobotomy. He began a serious, yearslong campaign to realize this dream.

To prove the technique was ‘portable’, he performed the initial transorbital lobotomies in his office rather than in a hospital operating room. The first procedure used an ice pick from the Uline Ice Company. To show off, he would sometimes simultaneously manipulate two instruments inside a patient’s head at once, just as he had earlier written on blackboards with two hands at once in front of his students.

Many clinicians were horrified at Freeman’s new version of the lobotomy, despite the fact that most of them believed in the utility of the standard version. John Fulton wrote Freeman in disbelief: (ch. 10, p. 205)

What are these terrible things I hear about you doing lobotomies in your office with an ice pick? I have just been to California and Minnesota and heard about it in both places. Why not use a shotgun? It would be quicker!

Freeman’s reply included the quip: (ch. 10, p. 205)

It is much less traumatizing than a shotgun and almost as quick.

Freeman’s transorbital lobotomy was an abomination to neurosurgeons, who believed in strange practices like ‘using gloves’ and ‘draping the patient’. Valenstein elaborates on some of the fundamental conflicts: (ch. 13, p. 257)

Freeman was pitting transorbital lobotomy performed mainly by psychiatrists against prefrontal lobotomy performed by neurosurgeons. The resulting conflict was inevitable. Almost all neurosurgeons, Watts included, objected to transorbital lobotomy. They were appalled when they watched Freeman adjust the head straps for the electroconvulsive shock and then pick up the instruments with ungloved hands, and they resented the idea of inducing convulsions, which much of their work was aimed at eliminating. They were also concerned about the possibility of hemorrhage and infection. To neurosurgeons, transorbital lobotomy seemed to be a slapdash, hit-or-miss business, and the “assembly line” of operations on twenty or more patients in a row—averaging less than ten minutes each—a vulgar display.

Freeman, who did not himself have surgery privileges, as he was not a surgeon by training, should not have been doing the procedure alone. But he was doing them alone, because his usual partner-in-crime Watts (a neurosurgeon) could not accept the change. The productive Freeman and Watts partnership ended, and Freeman’s solo career began.

During the peak years of the lobotomy, which Valenstein identifies as between 1949 and 1952, Freeman spent a significant portion of his time on the road. He wanted to popularize his cheaper, faster transorbital approach, and chose to do so by visiting state hospitals across the United States (and sometimes elsewhere). Valenstein writes: (ch. 11, p. 229)

In 1951 alone, he visited hospitals in seventeen states as well as provincial hospitals in Canada. He also gave demonstrations in San Juan, Puerto Rico, and Willemstad, Curacao. On one five-week summer trip that year, he drove 11,0000 miles with a station wagon loaded, in addition to camping equipment, with an electroconvulsive shock box, a dictaphone, and a file cabinet filled with patient records, photographs, and correspondence; his surgical instruments were in his pocket.

He fastidiously kept track of the details of these trips. For example, he kept a list of the number of people he gave lobotomies and where he did them: (ch. 11, p. 230)

29 June, Little Rock, Arkansas, 4 patients
30 June, Rusk, Texas, 10 patients
1 July, Terrell, Texas, 7 patients
2 July, Wichita Falls, Texas, 3 patients
9 July, Patton, California, 5 patients
14 July, Berkeley, California, 3 patients
17 July, Mendocino, California, 9 patients
19 July, Fort Steilacoom, Washington, 7 patients
28 July, Sedro Woolley, Washington, 8 patients
4 August, Yankton, South Dakota, 5 patients
6 August, Lincoln, Nebraska, 8 patients
7 August, St. Joseph, Missouri, 1 patient
8 August, Cherokee, Iowa, 25 patients
9 August, Independence, Iowa, 16 patients

Note the last two dates: Freeman did twenty-five lobotomies in one day! Sometimes Freeman kept up this pace for a longer stretch. Valenstein notes that: (ch. 11, p. 234)

With the help of the staff at the hospitals in Lakin, Huntington, Spencer, and Easton, Freeman did 225 transorbital lobotomies during a twelve-day period in August 1952.

That’s around nineteen per day!

The schedule was grueling, from the long hours driving to the long hours doing surgery. Sometimes things went wrong; patients died. But Freeman remained committed to his vision. Thanks to his influence, more than half of all lobotomies performed in state hospitals were his transorbital version; elsewhere, the fraction was closer to 30 percent.

Like Moniz, Freeman sought recognition for his extreme efforts to advance psychosurgery. When Moniz asked Freeman if he could nominate him for the Nobel Prize—Moniz asked many people he knew, over many years, to do this—Freeman included a copy of his book with Watts with the nomination. Valenstein suggests that he might have been hoping to be considered with Moniz. After all, he and Watts were the faces of the field; their work ushered in a new era of lobotomy, and a new scale of worldwide usage.

Valenstein thinks he never had a chance, though: (ch. 11, p. 224)

Yet Freeman probably never had a chance to be nominated for the Nobel Prize. However effective he was at developing and improving ideas, he was not an initiator of new ones. Moreover, his habit of alienating people by his bluntness, self-assurance, and combative personality left a trail of opposition, if not enemies. Moniz, in contrast, was a paragon of tact.

Unlike Moniz, Freeman never got the prize. He also became increasingly alienated from the mainstream of neurology and psychiatry, both because of his combative personality and because he refused to adapt when times changed. Psychoactive drugs like chlorpromazine were widely adopted in the mid-1950s; this led to a precipitous decline in the usage of the comparatively riskier and more invasive lobotomy. But Freeman did not change. He was still doing transorbital lobotomies until his surgical privileges were taken away.

This happened when he killed his final patient, who was in the middle of receiving their third lobotomy from him: (ch. 14, p. 274)

He performed his last transorbital lobotomy at Herrick Memorial in February 1967, at the age of seventy-two. The patient was one of the ten transorbital lobotomies he had first performed in his office in Washington, D.C., in 1946. She had later moved to California; and, in 1956, Freeman had repeated the operation at Herrick. When, in this, the patient’s third transorbital lobotomy, Freeman started to make the deep frontal cut, a blood vessel was ripped open, causing a severe internal hemorrhage. After the woman’s death several hours later, his surgical privileges were taken away.

Freeman’s 1972 death was mostly ignored by the beleaguered psychosurgery community. He did achieve the fame he sought, but only temporarily.

VI. INTERLUDE: ABOUT THE LOBOTOMY

If you, like me, didn’t know much about lobotomy, you may have thought it was a generic term for surgical brain damage. It is more specific than that. But at the same time, it is much less specific than its proponents claimed. Its primary targets are the frontal lobes, which comprise a huge region of the brain. Moreover, as Freeman emphasizes in a 1950 journal article in the Journal of the National Medical Association:

Without entering into the minutiae of architectural patterns, it may be said that the major portion of the frontal lobe is of quite uniform architecture, and that the extent of incisions is of greater importance than their exact location.

This is almost certainly crazy and wrong, but it was the thinking at the time, at least among proponents of psychosurgery.

In any case, what is a lobotomy? There were several types.

The initial kind, the leucotomy developed by Moniz and Lima, had a few variants. They first tried destroying tissue by squirting alcohol through holes in the skull rather than by physically cutting the brain. Only after they had operated on a few patients did they begin physically cutting tissue, again through holes in the skull, using a sharp instrument they called a leucotome. The holes were usually drilled into the top of the skull.

The later Freeman-Watts ‘standard’ lobotomy continued using a leucotome to physically cut through tissue, but involved drilling holes into the sides of the skull rather than the top. Freeman and Watts found that this change helped them reduce variation in where they ended up cutting.

The third major type, the Lylerly-Poppens lobotomy, came to be preferred by neurosurgeons. Like the Moniz-Lima leucotomy, it involves accessing the skull from above, but uses larger openings made with the help of a long tweezers-like “speculum” tool. The larger openings made it effectively ‘open’, rather than ‘closed’: surgeons could see into the brain, which in principle allowed them to be more careful.

The final major kind, the transorbital lobotomy, was the low-tech version developed by Freeman. This one is the most different, since it involves drilling no holes through the skull. Instead, it involves pushing a sharp instrument through the bony orbit behind the eye, usually with the help of a mallet. (Imagine hammering a nail into someone’s head.) Freeman initially used the same leucotome he had used when doing standard lobotomies, but found that they had a propensity to break off inside people’s heads; this motivated him to construct a sturdier sharp object that he called an orbitoclast, which was around eight inches long and shaped like an ice pick.

Freeman’s tools. As Valenstein says (ch. 11, p. 230), these are the “leucotomes and the orbitoclasts in the case that Freeman carried in the pocket of his suit jacket.”

A transorbital lobotomy in progress. Note the orbitoclasts, which have been driven through the skull via the eye socket. Note also that Freeman is manipulating two of them at once.

The lobotomy was originally advertised as a method of last resort. As James Watts explained: (ch. 9, p. 183)

… patients with involutional or agitated depression with feelings of guilt who pace the floor, wringing their hands, mourning and crying. Still others with schizophrenia are confined to strong rooms because of their abusive assaultive behavior. If given a bed, they pull it apart and assault attendants with the pieces. They are often naked, refusing to wear clothes, urinate and defecate in the corner. … Food is poked through a crack in the door like feeding an animal in a cage. …

Following lobotomy, even the worst improve enough so that they live on an open ward, wear clothes, go to the dining room with other people, and get out in the sunshine.

Some members of the panel have suggested that we wait until they develop some form of therapy more to their liking. Perhaps I am a bit impatient at times. A woman 65 years old with agitated depression does not have but so long she can wait; and a man with dementia precox confined to a strong room for 5 years has already waited long enough.

But many of the tens of thousands of United States patients it was used on aren’t like those described in the above passage. Later, Freeman thought that the lobotomy should be used more liberally, since he thought that the outcome would be better before illness had progressed.

Valenstein positions his book as a history of psychosurgery rather than as a history of the lobotomy, probably for the reason that the lobotomy is a more specific term that does not describe everything that was tried. In the wake of the ‘success’ of the lobotomy, people tried targeting other brain regions, like the amygdala and other targets within the temporal lobes. These other operations had different results, but did not substantially differ in their ability to successfully treat mental illness—that is, they mostly didn’t work.

VII. INTERLUDE: CONSEQUENCES OF LOBOTOMY

Not all people that got lobotomies became zombies. Freeman bragged about patients who were ‘success stories’: (ch. 14, p. 282)

… a psychiatrist who, after a transorbital lobotomy, had become chief of clinical service in his own private clinic; another lobotomized physician who had subsequently married, had children, established a ten-man medical clinic, and flew his own plane; a violinist in a metropolitan symphony orchestra; and a college graduate who had become chief construction engineer for a major corporation.

Who says lobotomized people can’t be good pilots?

But these patients were of course not representative. Most lobotomized patients developed serious deficits: they became unmotivated, dull, and lacking in judgment. And these are the ‘good’ outcomes. Many patients simply died, further deteriorated, or developed serious epileptic seizures. As a superintendent of one state hospital said, (ch. 13, p. 255)

I am sorry to say that even when they improved they are nothing to brag about. ... These patients have not been completely restored and some of their families are extremely disappointed.

Or as the director of a major psychiatric institute said: (ch. 13, p. 255)

Is the quieting of the patient a cure? Perhaps all it accomplishes is to make things more convenient for the people who have to nurse them. ... The patients become rather childlike. ... They act like they have been hit over the head with a club and are as dull as blazes. ... It disturbs me to see the number of zombies that these operations turn out. I would guess that lobotomies going on all over the world have caused more mental invalids than they’ve cured. ... I think it should be stopped before we dement too large a section of the population.

The “zombie” assessment was sometimes fair, but the overall outlook was more complicated. Valenstein thinks: (ch. 12, p. 252)

Following a lobotomy, many agitated and anxious patients did experience a striking relief from their most troublesome symptoms. In the best cases, this led to a normalization of their behavior. In the worst cases, they became either thoughtlessly impulsive or almost insert, seldom talking and completely lacking in spontaneity. The latter cases justified the accusation that lobotomies were producing “zombies.” Some patients became careless, indifferent to errors, and generally slovenly. Motivation and drive were often decreased, as were insight and foresight. Behavior tended to be more concrete, and patients were less capable of dealing with abstract ideas.

Part of the problem was that the ‘effectiveness’ of the lobotomy depended strongly on the patient’s illness. Over time, Freeman found that it did not do much for schizophrenics, but sort of worked on patients with affective disorders (e.g., extreme anxiety) since it could ‘flatten out’ their emotions.

Valenstein discusses a number of patient outcomes in the book. First, there are the historically significant patients, like the ones who Moniz and Lima or Freeman and Watts first operated on. Apparently Freeman and Watts’ first patient had this to say after her lobotomy: (ch. 8, p. 142)

FREEMAN: “Are you happy?”
PATIENT: “Yes.”
FREEMAN: “Do you remember being upset when you came here?”
PATIENT: “Yes, I was quite upset, wasn’t I?”
FREEMAN: “What was it all about?”
PATIENT: “I don’t know. I seem to have forgotten. It doesn’t seem important now.”

He adds ominously that “She greeted her husband with a peaceful smile.” This outcome was so encouraging that it caused Freeman and Watts to rush ahead with doing more lobotomies and quickly publish their results.

Then there are patients whose long-term study allowed Valenstein to discuss their outcomes in more detail. In Chapter 12, he considers two such patients, who are known by the pseudonyms Carolyn W. and J.S.

Carolyn W. was the daughter of a wealthy family who had a psychotic break after her parents, who didn’t approve of her marriage, had it annulled against her wishes. She became psychotic and suicidal, and had to be moved to a private sanatorium for care. Her doctors tried everything, with no success: (ch. 12, p. 243)

She was given all the available therapies for schizophrenia, including insulin coma, electroshock treatment, sedatives (antipsychotic drugs were still in the future), hydrotherapy, and wet packs; when there was a danger she would injure herself, she was placed in seclusion, sometimes in a straitjacket for restraint. No treatment produced any lasting improvement.

Other options having been exhausted, it was eventually suggested that she should have a standard Freeman-Watts lobotomy. This didn’t work, so (if at first you don’t succeed…) she was given a second, ‘radical’ prefrontal lobotomy, which involved more cutting than the original version. Unsurprisingly, this operation “markedly subdued” Carolyn and made her childlike: (ch. 12, p. 244)

Initially, the job was almost like caring for a child; Carolyn had to be brought to the bathroom and essentially toilet-trained. It was also necessary to supervise all her routine activities, even dressing, eating, and bathing. She seemed to be indifferent to pain and once stepped into scalding bath water, severely burning her foot. Like many of the early lobotomized patients, she had bulimia and ate voraciously: it is believed that the frontal lobes normally suppress “lower centers” in the hypothalamus and other areas in the brain that stimulate appetite; following destruction of the frontal lobes, or its exiting fibers, it was not uncommon for patients to gain more than fifty pounds. She also had a tendency to start fires in her room during the first six months after her discharge from the hospital.

What an ending!

Her family’s wealth enabled her to receive the highest levels of care in the years since her operation. She was not kept idle, but constantly given lessons, taken on trips, and otherwise stimulated. While this produced some improvement, (ch. 12, p. 244)

… she is not able to take care of herself and has required full-time caretakers. She is usually able to verbalize what she should do, but there is no guarantee that she will do it. Even though she could state what bus she should take to get to her art teacher, Carolyn took the first bus that came along and had to be rescued at different locations around the city when she was given several opportunities to make the trip alone. She is now always accompanied when she travels.

One of the weirdest things about Carolyn W. and other lobotomy patients is that their IQs are not necessarily low: hers is between 115 and 120. Valenstein says that “people meeting her for the first time usually do not detect any abnormality and consider her a proper, affable, well dressed, and somewhat taciturn lady.” (ch. 12, p. 245) But she clearly has major deficits, and these become obvious to anyone that spends more time around her.

It is apparently difficult to quantify these deficits in psychological tests, despite the fact that these patients exhibit horrendous judgment and require full-time caretakers. The results of IQ tests come out in the normal-to-bright range, and so do the results of many similar tests. One that Carolyn W. apparently struggles with is the Wisconsin Card Sorting Test, which measures “flexibility in thinking”. The subject originally sorts cards depicting some number of colored shapes (e.g., two red triangles) according to one feature, like shape, and then later is asked to switch to sorting based on a different feature. Patients like Carolyn have a hard time shifting strategies. This seems to accord reasonably well with what is known about the prefrontal cortex’s role in accommodating context information during context-sensitive decisions.

Muddying the waters a bit is that Valenstein doesn’t seem to be sure that all of Carolyn’s deficits are due to her (two!) lobotomies, since she also spent considerable time in an institution and received many other dangerous therapies: (ch. 12, p. 246)

Her limitations make it easy to criticize those responsible for the decision to perform the lobotomies. At the time, however, most psychiatrists considered it almost a certainty that any schizophrenic patient hospitalized for more than ten years would only deteriorate further. Carolyn seemed destined to spend her remaining years locked in a mental institution. Today, we might wonder whether some of her limitations were not the result of fourteen years of confinement in a mental institution where more electroconvulsive shock was surely administered than would now be considered safe.

But overall, Valenstein thinks Carolyn’s outcome, which has reduced her to a dull, dependent, childlike woman, is relatively good. Without the constant enrichment in the years following her operation, she likely would have become worse.

He contrasts her case with that of J.S., a sexual deviant who received a lobotomy after going to prison for sex crimes. He lived with his mother and had an Oedipus complex par excellence: (ch. 12, p. 247)

J.S. thought he had been spoiled as a child. He was very fond of his mother even though she was strong-willed and domineering. She had punished him physically for the slightest offense, beating him “with a strap, or birch switch, or with her hands on his bare buttocks.” These beatings usually produced an erection and often ended with J.S. ejaculating and his mother kissing him. This behavior continued for about two or three times a week until he was sixteen.

Later, he became obsessed with the pleasure he derived from the beatings he had received as a child, and tried to get others to help him act out his sexual fantasies. Since people his own age wouldn’t help, he started trying to get children involved: (ch. 12, pp. 247-248)

His strategy was to start talking to young boys and quickly direct the conversation to questions about how their parents punished them. He would tell them that he liked it when he was beaten as a boy, and would make up a game where the boys acted as his parents and would have to punish him.

This escalated in a predictable bad way, and he was caught and imprisoned. In prison, he became obsessed with the idea that his obsession would lead him to commit even worse sexual and violent acts. J.S.’s high anxiety surrounding this is what led his psychiatrist to seek a lobotomy for him. The outcome did not initially seem good, but later improved: (ch. 12, pp. 248-249)

Immediately afterward, J.S. was confused, disoriented, and incontinent. He masturbated and kept placing his finger in his rectum and then in his mouth. After several days, however, he regained control of his anal sphincter; and by the tenth postoperative day, the confusion was said to have completely cleared up. Two to three weeks after the operation, all of the undesirable effects were said to have disappeared entirely. Although J.S. remained under observation in the prison hospital for six months, he was able to resume secretarial duties throughout most of this period.

Later, a journal article championed him as an example of the positive effects of the lobotomy. He was considered reformed, and gave a statement recanting his past deviant behavior. In 1942, one year after his lobotomy, he was released from prison.

He seemed to be on good behavior for a while, and regularly reported to his parole officer, but in 1944 stopped contacting him after he moved to Chicago. Then things got much worse: (ch. 12, p. 250)

Shortly after the lobotomy, J.S. appeared to have undergone rapid improvement, followed by a year-long period of stabilization and then a period of progressive decline. He was not able to hold a job for more than a few weeks before he was fired, because he was “undependable in every way” and did not seem “to know what he was doing.” His judgment was childish, he had no insight into his behavior, and he became progressively confused. His memory was very bad, and almost nothing he said about previous events was found, when checked, to be accurate. He was often incontinent, and his buttocks were grossly contaminated with fecal matter. He exuded a profound stench and was covered with bedbug sores.

Even worse, he was still a pedophile, albeit an extremely incompetent one: (ch. 12, p. 250)

Earlier, the parole officer had given J.S. “a clean slate,” reporting that he was “completely free of abnormal desires.” In actuality, J.S. continued secretly to practice his perversions in Chicago. In his suitcase were found obscene handwritten propositions on slips of paper, which he handed out to boys in the parks he frequented.

Valenstein thinks that J.S. suffered from at least two problems. First, the lobotomy did not remove the obsessional desires that were consuming him, even if they apparently briefly quieted them. Second, lobotomized patients, which generally lacked the good judgment and mental adeptness to make it in the world without significant help, should not be left to their own devices. J.S. should have received some form of support in order to avoid relapsing and deteriorating.

Valenstein’s overall conclusion is that lobotomized patients were generally semi-competent, and in the best cases could get by with social support and by compensating for what they had lost: (ch. 12, p. 252)

In most cases, the impairment was at some intermediate level. Thus, there was some helpful reduction in the most troublesome symptoms—but at the price of reducing potential motivation to excel and the capacity for thinking abstractly, for imagination, for emotional experience, for spontaneity, for planning ability, for insight, and for social judgment. In the best cases, these deficits were not great, and, postoperatively, patients could make a good adjustment by compensating for the loss by using remaining abilities resourcefully, much as some elderly people are capable of doing.

But overall, he unsurprisingly does not think that the benefits of the lobotomy outweighed the risks, especially when it was not used as a method of last resort.

It is worth keeping in mind how dangerous the operation is. Death wasn’t uncommon. In one harrowing incident during one of Freeman’s road trips: (ch. 11, p. 231)

On the 1951 trip, there were four deaths from the surgery. The experience at the Cherokee State Hospital in Iowa was the worst: three of the twenty-five patients died either during or shortly after surgery. Two deaths were caused by hemorrhage, and one by an accident that caught Freeman completely by surprise. Normally it took a certain force to drive the leucotome through the bony orbit, but in this patient, the bone was so thin that the leucotome went through with hardly a touch from the mallet. When Freeman, as was his custom, stepped back to photograph the leucotome in place, it sank a full two inches farther into the brain up to the hilt of the handle. This patient never regained consciousness; and when the brain was later autopsied, the tip was found to have gone far beyond the frontal lobes, extending deep into the midbrain: indeed, the tip had penetrated the basal ganglia, skirting the hypothalamus and ending near the red nucleus.

In other words, a patient died because Freeman stopped to take a picture. Valenstein goes on: (ch. 11, p. 231)

Freeman would not allow himself to be emotionally drained by guilt when things went wrong. He had the attitude of a general conducting a military campaign: losses would have to be incurred on the way to ultimate victory.

This attitude drove many proponents of psychosurgery to continue even in the face of many middling and poor results.

VIII. UNCRITICAL RESEARCHERS

The peak years of psychosurgery occurred before we had good, widely-used models of how neurons in the brain process information, and before we had any serious knowledge of molecular biology. We were still in something of a dark age of neuroscience.

There was no agreed-upon way of thinking about the brain, so many different ways coexisted. And some of them were extremely strange. A particularly weird episode of the book involves the critique a Freudian psychoanalyst has of lobotomies. He thinks everyone should be aware that the compulsions associated with some mental illnesses (obviously) relate to an anal sensory perception area: (ch. 9, p. 183)

If we accept the general Freudian formula, and I see no reason not to, then the compulsion neurosis uses the mechanism of displacement from early erotic fixations. … Psychoanalytic psychiatry has pointed very clearly to a highly important organ libido investment in the “compulsion neurosis.” This is the cathexis of the anal sadistic with its massive hostility drive. This anal sensory perception elaborating area, Dr. Penfield I think partly localized at our 1934 meeting.

The (obvious) consequence of this is that lobotomies can only be successful if they sever “only the fibers connecting the frontal lobe with the brain area that receives sensory information from the anus”. (ch. 9, p. 184) These comments are so weird that a different prominent psychoanalyst feels compelled to speak against them: (ch. 9, p. 184)

I cannot understand why such ideas are brought here, and much less the fantastic conclusions drawn from them by some of the discussers. I suppose we are so polite that anything may be ventured with impunity. … The whole presentation reminds me of a time when they used to treat homosexuality by prostatic massage. … I asked him how he expected to cure homosexuality by prostatic massage. He thought the massage might rub out the homosexual cells and that they would be replaced by heterosexual cells …

Ouch!

Anyway, that was the state of some discussions of the brain: “anything may be ventured with impunity.” Valenstein asks: given this deep and substantial ignorance of the brain, should the proponents of psychosurgery have known better? He argues that the answer is yes.

First, even then a nontrivial amount of information was known about the anatomy and function of the frontal lobes. Valenstein points out that, in 1861 or possibly earlier, Pierre Paul Broca (of Broca’s area fame) thought that the frontal lobes had something to do with Man’s “higher faculties”: (ch. 5, p. 89)

The majesty of the human being is owing to the superior faculties which do not exist or are very rudimentary in all other animals; judgment, comparison, reflection, invention, and above all the faculty of abstraction, exist in man only. The whole of these higher faculties constitute the intellect, or, properly called, understanding, and it is this part of the cerebral functions that we place in the anterior lobes of the brain.

Work by many others, from then until the dawn of psychosurgery during the following century, established that the frontal lobes mostly consisted of neither sensory nor motor areas, but something else related to abstraction and problem-solving—and for this reason, the frontal lobes were sometimes called “association” areas.

The story of Phineas Gage, the construction foreman whose frontal lobes were severely damaged by a long tamping rod after an accidental explosion, was well-known. He exhibited severe deficits, including poor judgment and self-control. Moreover, similar deficits were observed in World War I veterans whose frontal lobes had been damaged.

It was known, but perhaps not widely appreciated, that patients whose frontal lobes had been deliberately damaged during surgery (e.g., to remove a tumor) also exhibited serious deficits. Moniz, who explicitly referenced some of these patients in his writings, apparently did not understand or ignored the severity of these deficits. He ought to have known that frontal lobe damage was likely to destroy something precious to human intelligence.

Valenstein’s other points concern the nature of the empirical evidence. The N = 2 chimpanzee study presented by John Fulton and Carlyle Jacobsen at the 1935 International Congress of Neurology was frequently either accidentally or fraudulently misrepresented, especially by Moniz. Moniz and others—including Fulton!—often claimed that both animals became more placid and agreeable after surgery. But this is exactly the opposite of what happened. One of them did—Jacobsen described the change by saying it is as if she joined a “happiness cult” (ch. 5, p. 96)—but the other one experienced the complete opposite trajectory! The other animal, who initially performed well, began making many errors and became prone to having temper tantrums. At least for her, damaging the frontal lobes created a kind of personality issue. Hence, the Fulton-Jacobsen study provided no evidence that frontal lobe damage reduces or eliminates neuroses.

As Valenstein observes, even the Nobel Committee made this error: (ch. 13, pp. 266-267)

Fulton’s repeated distortion of the chimpanzee experiment came to be so widely accepted that when the Nobel Committee, in their citation of Moniz’s award, traced the lines of thought that led to the discovery of prefrontal leucotomy, they perpetuated this error …

Valenstein rips apart Moniz’s ‘theoretical’ motivation for psychosurgery in a chapter aptly titled “The Emperor’s New Clothes: Moniz’s Theoretical Justification for Psychosurgery”. The first funny part of this chapter is that Moniz apparently had a history of lying about his justification for trying psychosurgery. Since, even then, using sharp objects to cut through people’s brains seemed a bit cavalier, especially in the absence of a concrete reason like a tumor, Moniz claimed that he had thought very hard about the whole thing. He had thought for years and had strong theoretical reasons to do what he did.

But that was a complete fabrication!

He claimed to have secret discussions with collaborators, but there is no written record of these discussions, and these collaborators could not remember them when asked years later. As Valenstein observes, you would think that they would remember them given the later infamy of psychosurgery. When he did disclose his theoretical motivation, long after the first operations had been done, it was pretty shallow. Valenstein calls it “not much of a theory at all”. (ch. 5, p. 84) It went something like this: (ch. 5, p. 84)

Moniz’s argument for prefrontal leucotomy was based solely on a series of general, loosely connected, and essentially untestable statements. First, he asserted that the frontal lobes are the seat of man’s “psychic activity,” and that thoughts and ideas are somehow stored in the nerve-fiber connections between brain cells. Moniz then stated that all serious mental disorders are the result of “fixed” thoughts that interfere with normal mental life. … Moniz argued that “fixed thoughts” are maintained by nerve pathways in the frontal lobes which have become pathologically “fixed” or “stabilized”. Effective therapy, according to Moniz, required the destruction of these abnormally “stabilized” pathways in the frontal lobes.

Among other problems, “fixed” thought patterns are a problem in some mental illnesses (like hysteria) but not others. And even if Moniz were right, it doesn’t follow that an untargeted operation involving sticking a knife into someone’s brain will address the problematic patterns!

Many of the early pioneers of psychosurgery expressed reservations with Moniz’s ideas, and were mainly motivated by (what they viewed to be) the empirically good outcomes. When Freeman and Watts published the first edition of their psychosurgery bible in 1942, part of what they offered was an alternative, more acceptable explanation for why psychosurgery ‘worked’, which helped others become more comfortable with it. As Valenstein explains, (ch. 9, p. 171)

What was most influential in Psychosurgery, however, was the theory that specific brain pathways between the frontal lobes and the thalamus regulate the intensity of the emotions invested in ideas. This was widely and uncritically accepted as the most promising scientific justification for psychosurgery.

Explaining its ‘success’ in terms of specific anatomical connections seemed at least more respectable than Moniz’s ‘fixed ideas’ framework. Still, like Moniz’s hypothesis, Freeman and Watts’ hypothesis has obvious problems: for one, not every serious mental illness they treated using psychosurgery involved intensely dysregulated emotions.

But perhaps these researchers and clinicians can be forgiven for not understanding the brain back in the 1940s. Their more serious, avoidable error concerned their liberal interpretation of experimental results. The twenty patients Moniz discussed in his original monograph did not provide good evidence that psychosurgery was effective. As Valenstein notes, (ch. 6, p. 108)

In four of the seven “cured” patients, the last observation Moniz recorded in the monograph was less than eleven days after the operation!

Additionally, the information collected and provided about patient outcomes seems to have been fairly limited. No quantitative tests were administered and no detailed notes were taken; only brief observations by a psychiatrist, which were from not long after the surgery, were included. The evidence base Moniz presented was totally inadequate, especially given the obvious dangers of brain surgery.

Worst, influential neurologists and psychiatrists like John Fulton of Yale began to uncritically repeat the claim that Moniz’s evidence base was solid. This seems to be a case of someone saying it once, and then everyone else repeating it without ever actually reading Moniz’s book.

Later, Freeman was more scrupulous about following up with his patients, even after many years. But Freeman also did not usually administer psychological tests and was probably too invested in the success of his procedures. He tried to report outcomes honestly, but he also saw what he wanted to see; confirmation bias was a huge issue.

Given the sorry initial state of many of the patients who underwent psychosurgery, it was hard for physicians to assess outcomes. If someone who was initially violent and incoherent became dull and placid, and at the very least no longer needed to be restrained, did that count as an improvement? The experimental evidence was hard to evaluate, and there did not seem to be any large-scale attempt to compare outcomes to those of an appropriate control group.

There are two final interesting ingredients worth mentioning. One is that Moniz and Freeman were both respected senior researchers and physicians when they began psychosurgery; this helped insulate them from criticism. Influential senior people spoke up for them and repeatedly gave them the benefit of the doubt, even when what they were doing was poorly motivated and sometimes crazy.

The other was a tendency toward incremental change. When people criticized psychosurgery, especially once it became popular and widespread, it was not on its fundamentals—rather, it was on the details of an approach. It was assumed that if it was ever ineffective, the fix was to refine and improve the procedure rather than to abandon it entirely. This situation makes me imagine some exchange like this:

PERSON A: Maybe we should blow up the entire world.
PERSON B: That’s preposterous. We need to be extremely careful. What if we just blew up 99% of the world?

Once psychosurgery gained momentum, it was hard to get rid of it entirely. People were too invested in it and had too many ideas about how to ‘get it to work’. In some sense we are lucky that psychoactive drugs came when they did and worked as well as they did. Otherwise, we may have been stuck toying with psychosurgery approaches for much longer.

IX. UNCRITICAL MEDIA

For years, I’ve read and enjoyed blogs like Not Even Wrong and Shtetl-Optimized. It’s fun to see educated bloggers take down outrageous claims like ‘physicists find evidence for the multiverse’ or ‘physicists create a black hole on a tabletop’ or whatever. (For whatever reason, this happens a lot with theoretical physics.)

But it’s one thing when the hype machine boosts speculative theoretical physics claims that have nothing to do with everyday life. It is an entirely different thing when what is being hyped is a miracle cure to your most serious problems.

Part of Valenstein’s argument is that, thanks to the efforts of people like Freeman, and also thanks to people’s natural tendency to be optimistic that miracle cures actually work (see, e.g., Dr. Oz and similar old daytime shows), the media uncritically promoted psychosurgery before it could be properly vetted. This is obviously bad: journalists were in no position to evaluate the complex evidence that bore on the question of whether psychosurgery was justified, and the coverage provided laypeople with an unrealistic picture of the costs and benefits of the procedure. The whole media circus increased demand, and the optimism surrounding it partly silenced voices critical of psychosurgery.

The atmosphere of the times might be summarized by this October 1949 quote from the New York Times: (ch. 11, p. 221)

Hypochondriacs no longer thought they were going to die, would-be suicides found life acceptable, sufferers from persecution complex forgot the machinations of imaginary conspirators. Prefrontal lobotomy, as the operation is called, was made possible by the localization of fears, hates and instincts. It is fitting, then, that the Nobel Prize in medicine should be shared by Hess and Moniz. Surgeons now think no more of operations on the brain than they do of removing an appendix.

Needless to say, surgeons do not and should not think that!!!

Here’s a selection of fawning small-town newspaper headlines Valenstein lists on p. 157:

SURGEON’S KNIFE RESTORES SANITY TO NERVE VICTIMS

WIZARDRY OF SURGERY RESTORES SANITY TO FIFTY RAVING MANIACS

BRAIN SURGERY IS CREDITED WITH CURE OF 50 HOPELESSLY INSANE

FORGETTING OPERATION BLEACHES BRAIN

NO WORSE THAN REMOVING TOOTH

Life magazine ran articles filled with whimsical cartoons whose goal was to explain the lobotomy. This one explained it in terms of the Freudian concepts of the id, ego, and superego:

From the March 3, 1947 issue of Life. In a healthy man, the various Freudian sub-entities are harmoniously balanced.

From the March 3, 1947 issue of Life. In a mentally ill man, the superego is out of control! They claim that the lobotomy makes him shut up. But I’m not sure where they got this explanation from.

Side note, the same article also depicts graphic images of brain surgery next to an ad for shoes:

Get yourself a magazine that can do both.

Different popular accounts of the lobotomy were not generally consistent with one another: (ch. 9, p. 180)

Thus, while Life described the effects of lobotomy as destroying the superego, Time implied that the operation created a superego where apparently there had been none before.

The hype was also not limited to media aimed at laypeople. Even the prestigious New England Journal of Medicine joined in: (ch. 10, p. 199)

A new psychiatry may be said to have been born in 1935, when Moniz took his first bold step.

Or consider a 1941 editorial in the Journal of the American Medical Association, another ‘establishment’ publication: (ch. 9, p. 187)

An emotional attitude of violent unreasoning opposition to this form of treatment [lobotomy] would be inexcusable. True it is a mutilating operation and it does result in certain defects in personality and behavior. However, much surgery is mutilating in the sense that some ordinary normal tissue is removed in order to achieve a beneficial result. … No doctor can yet assert that this is or is not a truly worthwhile procedure. The ultimate decision must await the production of more scientific evidence.

And of course, the Nobel Prize was the biggest rubber stamp of all!

A substantial chunk of this hype was either encouraged by or directly produced by Freeman, who had a penchant for showmanship and a love for the limelight. For example, Freeman and Watts maintained a “psychosurgery exhibit” in the exhibit hall of the American Medical Association’s annual meeting from 1936 to 1946, the prime years of their collaboration. Watts recalled that: (ch. 8, p. 160)

He would stand by his exhibit, and as people approached, he had a clicker which made a sharp staccato noise. When they stopped he would begin to talk like a barker at a carnival. Then a crowd would gather and before they moved on they had heard the story of dramatic relief of emotional tension, depression, suicidal ideas, had seen pre- and post-operative photographs, x-rays and sketches of the lobotomy operation.

Freeman didn’t care if his fellow physicians found his approach—that is, the enthusiastic promotion of a dangerous and incompletely understood brain surgery—distasteful. He blamed his colleagues for being unfair, and spun his behavior as being due to a concern for patients’ wellbeing: (ch. 8, p. 161)

When it is a question of overcoming inertia and taking part in the pressure of the public upon politicians, and particularly upon hospital administrators at least to give the method described a fair trial, then I think that the doctor with a clear conscience can discuss his research with newspaper reporters and undergo the ensuing censure from his colleagues with equanimity. He realizes that something is being done to apply his discovery to the welfare of suffering patients.

With the media portraying brain surgery as ‘trendy’, even famous and wealthy people sought lobotomies for their families, with the most famous (horrific) example being Rosemary Kennedy, whose politically famous family lobotomized her in part because her antics publicly embarrassed them. She never recovered from her lobotomy, and spent most of the rest of her life in an institution.

X. TERRITORIAL DISPUTES

I used to be confused about something extremely basic. Why are there so many professions dedicated to the brain? There are psychologists, cognitive scientists, neuroscientists, psychiatrists, neurologists, and neurosurgeons, among others. And these are all obviously different, but also somehow kind of the same.

This is partly due to a difference in commitments. For example, a cognitive scientist is interested in understanding not every aspect of the brain, but most particularly the aspects important for cognition and intelligence. They acknowledge that the brain is composed of biological ‘stuff’, like neurons, but think it may be more fruitful to view the brain more abstractly as performing computations in order to solve survival-relevant problems. A molecular neuroscientist, meanwhile, is intensely interested in the specifics of the molecular biology of neurons, and tends to think that this low-level biology will tell us something important about how the brain works. It’s like the standard joke about the elephant: we’re all blind men touching different parts of the same thing.

But I don’t think the difference in commitments is the full story. The many professions are also in part due to what one might call territorial disputes that play out over decades. Why does the ‘psychiatric nurse practitioner’, who in many places can do most of the things an MD psychiatrist can do, exist? Because there are groups of people fighting each other for patients and legitimacy!

Valenstein thinks that a few territorial disputes explain some of the dynamics of the history of psychosurgery. The most important is the clash between those who subscribe to a functional theory of mental illness, which suggests it is primarily due to life experience, and the organicists who think it is mostly due to biological factors. In addition to each group having different beliefs, they also both wanted professional clout and access to patients. But if you believe your rivals have a totally wrongheaded view of mental illness, why should they get recognition? Why should they be the first line of attack? Conflict is inevitable.

He traces some of this history in the very first chapter of the book. Early in the twentieth century, there was a heated battle between the neurologists, who mostly took an organic view of illness, and the psychiatrists, who after Freud largely subscribed to a functional view. The neurologists were losing this battle for patients and prestige when psychosurgery began in 1935, and the victory was essentially complete by the end of World War II. Some of the details of the dispute are funny: for example, when a joint certification board was established, some were angry that “Psychiatry” preceded “Neurology” in its name, “in spite of the argument that n precedes p in the alphabet.” (ch. 1, p. 22) Neurologists were also squeezed out of their turf by neurosurgeons, since they had to refer patients to them to do the ‘real’ work, e.g., operating to remove brain tumors. Some pejoratively called them the “pimps of neurosurgeons”. (ch. 1, p. 17)

But then the psychiatrists became threatened by the unwashed (or at least, not medically trained) hordes of psychologists, who could also do talk therapy. It was to the disadvantage of the psychiatrists that the popular Freudian methods did not really require much training to do, even if they could argue substantial training was required to do them well. This led psychiatrists to try to distinguish themselves from the psychologists, and one of the ways this happened was by embracing somatic therapies: therapies which involved physically manipulating the body, like insulin coma and electroshock therapy. Clearly their use was a sign that psychiatry was ‘real medicine’, and not just talk therapy plus metaphysics. Ironically, this professional attack from psychologists pushed many psychiatrists closer to an organic view of mental illness, since the ‘success’ of the somatic therapies was thought to have a biological explanation. For example, spinning patients around in a whirling chair was thought to improve brain circulation, and hence (hopefully) do something to combat mental illness.

This embrace of somatic therapies is in part what made psychosurgery popular. It was understood that talk therapy, which could require many sessions over weeks or months, did not scale to the setting of the overcrowded state hospitals. And furthermore, it wasn’t thought to be effective on many of those patients. Psychosurgery was ‘real’ medicine, and seemed to work.

Another relevant dispute is the one between the somatic-therapy-embracing psychiatrists and the neurosurgeons. Once he began advocating for the transorbital lobotomy, Walter Freeman suggested that psychiatrists in state hospitals across the country should be trained to perform lobotomies. Moreover, he thought they did not need the help of neurosurgeons, who were too expensive and generally required resources these hospitals did not have.

Predictably, neurosurgeons both viewed this as unsafe (see earlier comments about a lack of gloves and anesthesia) and as an encroachment upon their turf. Valenstein argues that the vocal resistance of many neurosurgeons to the transorbital lobotomy contributed to more widespread concerns that all lobotomies were dangerous and should not be used—which is ironic considering that neurosurgeons usually only had a problem with the transorbital variant performed by psychiatrists. An internal dispute grew out of control and threw a shroud of disrepute over the whole business.

XI. ECONOMICS

This reason Valenstein gives for the prevalence of psychosurgery might be the easiest to understand of all. Yes, people wanted to cure the sick. But realistically, it was unlikely that most severely mentally ill patients would significantly improve. If they could not be cured, perhaps they could be made more agreeable—and hence easier and cheaper to care for.

State hospitals were horrendously overcrowded. If lobotomy truly treated patients, they could be sent home (which reduces costs). If it didn’t treat them, but merely incapacitated them, that was still a net benefit from the hospital’s point of view: formerly violent patients would no longer need to be restrained (which reduces costs). If it killed them, that would be cheaper still.

In 1941, the superintendent of Delaware State Hospital spoke out in favor of psychosurgery during a panel discussion at an American Medical Association meeting. He lays out this logic in a fairly straightforward, if cold, fashion: (ch. 8, p. 154)

From an economic point of view I should like to give some figures as to what this may mean to the public. We have come to the following conclusion with regards to our own cases: In our hospital there are 1,250 cases and of these about 180 would be suitable for such an operation. In our hospital these patients could be operated on for $250 per case. That will constitute a sum of $45,000 for 180 patients. Of these we will consider that 10 percent, or 18, will die, and a minimum of 50 percent of the remaining, or 81 patients, will become well enough to go home or to be discharged. The remaining 81 will be much better and more easily cared for in the hospital. Thus the hospital will be relieved of the care of 99 patients. That will mean a saving of $351,000 in a period of ten years. I believe that, these figures being for the small state of Delaware, you can visualize what this could mean in larger states and in the country as a whole.

Valenstein adds: (ch. 8, p. 154)

This economic argument was frequently raised throughout the period of expansion of lobotomy. In 1948, John Fulton remarked that if only 10 percent of the patients occupying neuropsychiatric beds could be sent home, “it would mean a savings to the American taxpayer of nearly a million dollars a day.”

Interestingly, this dimension of Valenstein’s argument also explains psychosurgery’s decline. It arguably did not decline for moral reasons, since the obvious objections had been raised and debated for well over a decade; instead, it declined because it was outcompeted. Psychoactive drugs were cheaper, safer, and required much less training than brain surgery. The timing of psychosurgery’s decline precisely coincides with their introduction.

XII. DO NO HARM

By now, you’ve probably figured out what I’m going to say. It’s been forty years since Great and Desperate Cures was published—and in some ways, not much has changed. The state of affairs in 1986 remains so apt today that I can’t help but liberally quote from Valenstein’s assessment: (ch. 16, pp. 291-292)

It is indeed all too easy to look back on the past and wonder complacently how prefrontal lobotomy could have been so readily accepted. Oblivious to the similarities between past events and present practices, we draw no lessons—as though, for a brief period, psychiatry, responding to unique circumstances and the influence of a few zealous, if not malevolent, physicians, had run amuck.

Although it may be comforting to believe that the forces responsible for the wide acceptance of prefrontal lobotomy and the other somatic therapies described in this book are all behind us, they are, in truth, part of the very bone and marrow of the practice not only of psychiatry but of all medicine. There are today no fewer desperate patients and desperate families; premature reports of spectacular cures with minimal risks are still accepted uncritically; the popular media promote innovative therapies even more enthusiastically; economics is no less an influence on the selection of treatment; conflicts persist within and between specialties; and ambitious physicians still strive for “fame and name.”

There is such a (deserved) stigma surrounding psychosurgery that I doubt something like the lobotomy will be popular any time soon. But the mistakes of the past are not usually the mistakes of the future; history rhymes rather than repeats. The forces Valenstein mentioned may not have conspired to produce a new rash of lobotomies, but have produced (i) the rise and fall of Theranos, (ii) the opioid epidemic, and (iii) academic fraud surrounding the amyloid plaque explanation of Alzheimer’s Disease, among other issues. And these are just some of the more popular scandals known to laymen.

What should we do? In an ideal world, we might only attempt treatments whose effect on patients we understand. After all, the human body and brain are extraordinarily complicated, which yields tremendous uncertainty regarding any intervention on them; it seems unfair to put patients at risk when we are so uncertain of what might happen.

But we probably can’t do this. As Valenstein observes (ch. 16, p. 295), we lack an adequate mechanistic understanding of most treatments, and requiring physicians to only use treatments we understand would leave them with no tools:

If we were to reject treatments whose effects we did not understand, even aspirin would not have been acceptable during most of the period of its use. Despite the enormous amount of research on the biochemical action of psychoactive drugs, there is no agreed-upon explanation of how different drugs help anxious, depressed, manic, obsessed, and schizophrenic patients. Psychoactive drugs are used extensively because they are effective and convenient, not because of any compelling scientific rationale. Indeed, the first psychoactive drugs were discovered accidentally, and only afterward were theories developed of how they might work. Similarly, electroconvulsive treatment is considered by approximately 70 percent of psychiatrists to be effective in alleviating otherwise intractable depression, despite the rejection of the original rationale justifying its use and no agreed-upon explanation today of how it works. In medicine, we have to be most influenced by the quality of the empirical evidence that a treatment does more good than harm.

This world is far from ideal, and we must be content with trying things we do not understand theoretically, but which have some record of empirical success. Valenstein continues (ch. 16, p. 295):

Where no effective therapy exists for desperately ill patients, equally ethical and knowledgeable physicians may disagree about treatment. Some risks have to be taken, and some harm will inevitably ensue along with some benefit.

If we’re stuck with experimenting, what do we do? While Valenstein admits harms are inevitable, he does not think large-scale scourges like the lobotomy are. He suggests that ethics safeguards are essential; harm can be limited, for example, if dangerous experimental surgeries are at least not widely attempted before their consequences are better understood.

As a model of what he’s looking for, he points to something which at the time was relatively new: the requirement that novel drugs are approved by the Food and Drug Administration (FDA) in a process that generally requires both animal testing and randomized clinical trials. Although much has been written about how onerous these processes have become, I think most people would agree that it is good that they exist in some form; the modern debate concerns how thorough these vetting processes ought to be, not whether they should exist at all.

Valenstein suggests that a similar approval procedure should be required for novel surgeries. In 1986, no such approval process existed—and in 2026, it still doesn’t exist! I’m not completely sure why, but a 2017 article I found in the Cornell Journal of Law and Public Policy suggests it has to do with, among other things, fundamental differences between drugs and surgery. It is harder to run clinical trials with high statistical power for new surgeries than for new drugs, and it somehow seems more dubious to do ‘sham’ (i.e., fake) surgeries on real patients in order to have adequate controls. There is also substantial variation: while a drug is the same everywhere, the teacher-student way surgical knowledge is spread means that there are great differences in the details of surgery from place to place.

On the other hand, this does not mean that there are no safeguards. In the decades since 1986, hospitals have erected additional barriers to testing novel surgeries on patients, including the much-beloved institutional review boards and approval procedures for using new tools in surgery. And, of course, enough discussion of ethics concerns and lawsuits to freak people out, which creates a culture of wanting to avoid risks, at least to one’s pocketbook.

Is there any modern day analogue of the lobotomy? If there is, it can’t be completely obvious, since otherwise no one would do it. I don’t think there’s anything that could be fairly called a modern analogue. But there are still risky treatments done today which are known to be potentially harmful and which still lack any satisfying theoretical justification. Electroconvulsive therapy (ECT) is still done, albeit not at the voltage levels and frequencies that used to be typical, as far as I know.

In the 1960s, it was still used as a punishment in insane asylums. When Howard Dully was in one as a teenager, he recalls (My Lobotomy, ch. 11, pp. 173-174):

Throughout all of this, I had one really good reason not to get in trouble. There was a doctor at Agnews who believed in using electroshock therapy as punishment. Everyone knew about it. I think the technicians wanted us to know about it. They wanted us to know what was in store for us if we got out of line.

Steve was one of the people who got out of line. He was friendly with me, but with other people he could be real belligerent. He was a fighter. One day they took Steve out and kept him for a while. A week later, I heard he was out—but they’d moved him to a new ward. They had given him the electroshock.

When I saw him again, he wasn’t right. I don’t think he was ever right after that. The treatment was supposed to “calm” patients down. In Steve’s case, the calm was only temporary. Years later, Steve would be sent to jail for shooting a wino in the eye with a BB gun after the wino had passed him a bottle of wine that someone had urinated in.

In Great and Desperate Cures, Valenstein notes that this liberal use of ECT may have been problematic (ch. 12, p. 246):

Today, we might wonder whether some of her limitations were not the result of fourteen years of confinement in a mental institution where more electroconvulsive shock was surely administered than would now be considered safe. It is generally accepted that massive doses of electroshock are likely to produce memory and other cognitive deficits.

Anecdotally, although ECT has gotten much safer (due to the aforementioned reduction in typical doses and frequencies of treatment), a physician told me that they saw a problematic pattern at the ECT center of a major university: the treatments were not very tailored. They would do one patient every few minutes, and not change the voltage dose for different patients. This is probably partly because they may not have had good evidence that tailoring the dose changed things much. But it still seems strange to me.

In any case, years of research has also shown that ECT can help patients with treatment-resistant catatonia and depression. It has a track record of helping people when other tools have failed. My point is not to cast aspersions about ECT specifically. My point is just to argue that, when we coarsely intervene on the brain—whether as in ECT, the lobotomy, or other procedures—we ought to be extremely careful! And this is especially true when there is no agreed-upon theoretical basis for our interventions, which is still a problem with ECT in 2026.

XIII. THE MASTER OF EVIL

How invasive brain surgery impacts cognition is an interesting question for neuroscientists, psychologists, and cognitive scientists. Why psychosurgery persisted in spite of its obvious harms is an interesting question for sociologists and historians. And whether it was ultimately right or wrong is an interesting question for philosophers. For example, would a utilitarian approve of the lobotomy? What must their utility function look like in order for them to approve?

But we ought to remember that this dark and shameful history is not abstract. Real people suffered as a result of the decision to perform psychosurgery. Let us now return to the case of Howard Dully, who I mentioned in passing in the introduction. Freeman lobotomized him when he was twelve years old. (Freeman also operated on a number of slightly older children, which did not help his public relations efforts.)

Dully’s memoir, My Lobotomy, tells his story. It is one of the most heart-wrenching, depressing stories I have ever read. Dully’s mother died during childbirth when he was six years old. His father, Ronald Dully, went on to marry a woman that hated Howard. Not only was she a poor stand-in for his mother, denying him basic affection and support, but she mentally and physically abused him for years. Any child would have had a difficult time adjusting to that sort of treatment.

And then they made him get a lobotomy!

Decades later, Dully found the notes Freeman took on his case in an archive on Freeman’s life and work at George Washington University. Fortunately for Dully (and us), Freeman had a habit of taking detailed notes on all of his patients. The details of the encounters that ultimately led to the lobotomy are infuriating.

Dully’s stepmother went shopping for psychiatrists. She went through six, each of whom told her Howard was normal, before finding Dr. Freeman. Worse, his stepmother was not content to simply tell the truth regarding Howard’s behavioral problems; she had to invent some. She claimed he was violent, disgusting, and even that he hoarded feces (My Lobotomy, ch. 6, p. 78):

He still sometimes defecates in his pants or in bed or on the floor, or may wrap up a turd and hide it in the drawer; recently he urinated on the wall of his room; at another time he dribbled from his room all the way to the bathroom; used toilet paper can be found in his closet or in his bed or in the tub.

Dully adds (p. 79):

Those were the notes from Lou’s [his stepmother] first visit. Could any of it be true? I’m sure I didn’t wash well enough to satisfy Lou. I might have stained my underpants, or dribbled on my way to the toilet. I’m sure I blew my nose and dropped the tissue on the floor. But, wrapping up a turd and hiding it? Come on!

In her worst fabrication, she claimed that when Howard was five years-old, he was angry at the baby his mother had shortly before she died—and violently attacked that baby in its crib.

Freeman, despite probably not having the standing to do so, diagnosed Howard with schizophrenia, and made plans to operate on him swiftly. His father, probably pulled in a million directions by his multiple jobs, did not agree with his stepmother’s assessment but figured she and the doctor knew best. He gave his approval. He was lobotomized.

If the incident would have ended there, it would have been a terrible, horrific tragedy. But his stepmother still hated him. She probably expected him to become a nice kid, or a vegetable, or something other than a more scatterbrained version of himself. After he recovered, she could not stand his continuing to be in the house, even if he did not actually cause trouble.

The misfortune continued. He went to a foster home, and adjusted well, but was removed from it by his parents (probably out of pride). He went to a juvenile detention facility for a year. He went to an insane asylum for a year—not because he was insane, but because his parents had nowhere else to put him. He went to jail. He was homeless. His family, except for his father, essentially cut contact with him. He didn’t see his brothers or stepmother for years.

Howard recounts episode after episode in which he demonstrates poor judgment in the life that followed his lobotomy. He does petty crimes; he chooses bad romantic partners; he freely cheats on those partners; he is loose with the little money he does have, most of which comes from regular government checks. In many cases, Howard says that he acted out because he was “bored”. The memoir never explicitly says this, but given that extensive frontal lobe damage is associated with poor decision-making and executive control—consider the well-known case of Phineas Gage using more vulgar language, gambling, and being unable to hold down a job after a tamping rod destroyed a large chunk of his frontal lobes—it is likely that at least some of these issues can be attributed to the lobotomy. Being abandoned by your family and being thrown into the world with little preparation is difficult; enduring this after a lobotomy sounds impossible.

Not everyone liked the memoir. Some Goodreads reviewers think lobotomy victims ought to take more personal responsibility.

The arc of the memoir is to claim that, despite everything he had gone through, Howard came out the other side okay. He had kids; he had a wife that loved him; he finally got a job he enjoyed; and he found peace. In some ways these things are true, but Howard’s life was far from normal, and was disastrous in almost every sense one could imagine. His kids were unplanned, his good second marriage followed a terrible first marriage, and his bus driving job is not the one he was hoping for; he went to school later in life hoping to get a computer-related job. Even after NPR covered his story, he continued to have financial problems and health challenges.

Despite his presumably noble intentions, I think it’s fair to say that Freeman literally ruined Howard’s life. Similar consequences follow other interventions gone wrong. Whenever we reflect on the power and promise of science and medicine, this simple fact is worth keeping in mind.

For what it’s worth, I think Freeman was genuinely concerned for the wellbeing of his patients. Even Valenstein doesn’t think he was a monster. Here’s an excerpt from a 2005 NPR interview that included him:

CONAN: Dr. Freeman, it’s also important to remember, was one of the top men in his field, at the peak of his profession, in 1949, when this operation was first developed, and yet he’s remembered today, Dr. Valenstein, as, I guess, maybe second to Josef Mengele, as a monster. Is that deserved, do you think?

Dr. VALENSTEIN: No, I don’t think that’s deserved. One has to consider the times. We already talked about the fact that there were no other treatments available. Lobotomy in America was not all Walter Freeman. This was part of mainstream medicine. The most prestigious medical schools practiced lobotomy. Columbia University engaged in a huge project called the Columbia-Greystone Project. Greystone was a state hospital in New Jersey, and Columbia University staff were studying and performing lobotomies. So it was widely accepted, not only as evidence the fact that a Nobel Prize was given, but people talked about, even in prestigious medical journals, like the New England Journal of Medicine—they described lobotomy as ‘ushering in a new era of scientific psychiatry.’

He was fastidious about checking in with his patients, even decades after he had given them lobotomies. He had a box full of Christmas cards he received from his patients. (To be fair, he was the one that initiated correspondence.) Even if he was blinded by his ambitions, he still did care.

In 1933, the same year he finished a book on neuropathology, and a few years before he learned about and pursued psychosurgery, Freeman wrote a thirteen-stanza poem that Valenstein calls “prophetic”. It depicts the “Master of Evil” and his sons “Hurry”, “Flurry”, and “Worry” pushing Man to his “breaking point”—not through physical torture, but by assaulting his mind. It ends like this (ch. 7, p. 138):

… Then said Worry unto him: “All this of
which my brothers have spoken can I do,
And more also in addition.
For I can work upon Man’s imagination,
That same power with which he develops his
theories and his machines.
I can cause him to foresee events, both those that
will happen and those that will not.
I can cause him to interpret falsely the actions
of his friends
And of his family
And of the wife of his bosom.
I can cause him to fear decisions.
And to brood over mischances.
And I can cause him to lose sleep in prospect
of the morrow.”
“Tis best of all,” said the Master of
Evil, “For sleep would defeat our
every object.

“Go ye three upon the Earth
And burn the candle of Man’s life at
both ends.
Spare not his frontal lobes where intelligence
And ambition
And judgment
And self-control
Are centered …”

Some other popular examples of the concept:

• Nativists spread fear and mistrust about Muslim immigrants, and then racists go on shooting sprees in mosques.

• The #Resistance insists that Donald Trump would destroy democracy, and then various people try to assassinate Donald Trump.

• Conservatives spread fear and mistrust about transgender people, and then bigots commit hate crimes against transgender people.

• Woke people say the police are racist and brutal, and then other woke people murder police officers.

• Socialists call health insurance companies greedy and accuse them of blocking life-saving treatment, and then Luigi Mangione murdered a health insurance CEO.

• AI safety activists say that Sam Altman’s AI could destroy humanity, and then a guy threw a Molotov cocktail at Sam Altman’s house.

• Liberals said that Charlie Kirk was a hatemonger who was driving Americans apart, and then an assassin murdered Charlie Kirk.

• The pro-life movement describes abortion as murder, and then pro-life activists assassinated abortion doctors and pro-choice politicians.

The “stochastic terrorism” concept is near-unique in how effectively it can be discredited merely by listing many examples of its use together in the same place. Almost no one supports a blanket prohibition on criticizing of all of these different groups of people. “Stochastic terrorism” mostly gets deployed opportunistically, by people who either are too blinkered to realize that the same argument could be leveraged against their own speech, or who hope you’re too blinkered to realize that.

(in fact, one could argue that accusing someone of stochastic terrorism is itself stochastic terrorism! Here in America, we consider it justified to kill terrorists before they can threaten us further. Reclassifying criticism as terrorism implies it is potentially legitimate to apply the same norm to any especially harsh critics!)

Still, a sliver of concept-users make some fig-leaf argument that the cases they approve of are different than the cases they disapprove of. Some proposed principles:

You’re allowed to criticize these groups, but not to criticize them harshly. Again, I challenge anyone to fully own up to the implications of this. You’re not allowed to criticize Donald Trump or police brutality harshly? You’re not allowed to criticize abortion doctors or Muslim immigration harshly? Even if you bite all of these bullets, I still think you disagree with the general case. Tomorrow, some lunatic could shoot up a KKK meeting, and then you wouldn’t be allowed to criticize the KKK harshly.

You’re allowed to criticize these groups harshly, but not using special emotional or dehumanizing words. Maybe there are some things you shouldn’t say even about the KKK - “they are Satanic cockroaches unworthy of life” might cross that line. But this is a motte-and-bailey: most claimed examples of ‘stochastic terrorism’ don’t come close to that bar. Nor does this match what really leads to violence: Luigi Mangione didn’t kill an insurance executive because he heard the word “cockroach” too many times. He killed an insurance executive because he had strong opinions about what insurance executives do wrong, and these opinions didn’t hinge on slight changes in the emotional resonance of the words being used.

You’re allowed to criticize policies, but not individuals. Okay, so somebody says that allowing immigration from the Middle East is destroying America, but avoids saying the specific words “and I hate individual Muslims”. Then someone else listens to their message and shoots up a mosque. Or somebody says abortion is murder, but avoids criticizing individual abortion doctors, but someone else listens to them and kills an abortion doctor. How has this made things any better? It totally misses the point of what the phrase “stochastic terrorism” is intended to describe and why it’s bad. Also, I want to criticize individuals! I don’t like Donald Trump, and I understand some conservatives didn’t like Joe Biden. How can we express those dislikes if we’re not allowed to criticize individuals?

You’re allowed to criticize people harshly, but certain categories of complaint - that they’re trying to end democracy, or they’re literally Hitler, or that they’re destroying the world - ought to be out of bounds. Some people propose this as a special protection for Donald Trump or Sam Altman - it’s okay to say people are bad, but saying that Trump is becoming a dictator, or Altman risks destroying the world, are such strong statements that they naturally suggest violence as a response.

This ignores the actual pattern of violence. Trump and Altman are both still alive, but plenty of police officers, Muslims, and abortion doctors are dead - not to mention Charlie Kirk. None of these people were accused of seeking tyranny or destroying humanity, just of normal crimes like brutality or extremism or having a dumb bad podcast. It seems strange to rule out forms of criticism which have not resulted in death, while granting a pass to the forms that result in death regularly, on the grounds that perhaps the former could result in death.

But also, many past politicians have tried to end democracy; one was even literally Hitler. Part of how Pol Pot, Stalin, Mao, and the like took power was that not enough people saw the danger early and spoke out about it. If we ban saying that a politician might be really bad, or destroy democracy, or commit genocide, or even be literally Hitler, simply because it might inspire assassins, then we abandon our early warning system against future tyranny. It seems unwise to institute a rule against ever considering something which could definitely happen, and which in fact happens all the time.

As for the end of the world, it hasn’t happened yet, but it seems like the sort of thing that might happen in the future. Again, it seems unwise to demand that nobody ever point out that there might be potential consequences to handing General Jack D. Ripper full control of the nuclear arsenal.

You should be allowed to criticize powerful people, like Donald Trump or Sam Altman, but not weak people, like Muslims. This makes approximately the opposite recommendation as the last category, but I think is equally pernicious. Some large groups of people are up to no good. You can decide for yourself which ones they are - Trump voters, communists, Boomers, immigrants, racists, police officers, abortion doctors, health insurance executives. Some of these groups might have higher-than-average median income, and others might have lower. Some might be parts of politically-favored blocs, and others politically-disfavored ones. But they all die if you shoot them. It’s vacuous to tell Brian Thompson’s family that they can’t complain about his murder just because health insurance CEOs are a privileged group - especially given that health insurance CEOs now have a higher death-from-political-violence-per-capita rate than Muslims or transgender people.

The liberal solution is none of these: it’s that it’s always legitimate to criticize any person or group however strongly, and never legitimate to use extra-state violence against them; if a violent act occurs, it’s 100% the fault of the perpetrator, and 0% the fault of other people who previously criticized the victim.

(separate from this, I think it’s a useful discussion norm to avoid being too vitriolic or hyperbolic in your language, except perhaps once in a blue moon to make the point that you care about this issue so much that it’s your one exception to your non-vitriol policy. But if someone fails to follow this discussion norm, their penalty is being self-sorted to spaces that have bad discussion norms, not being held responsible for any resulting acts of violence.)

This liberal solution isn’t trivial. It requires the separate liberal norm of always being against extra-state violence - a norm which is currently less than entirely secure. 39% of young people have a favorable opinion of Luigi Mangione, and during the George Floyd protests several mainstream newspapers flirted with condoning violence in the name of racial justice. If your worldview says that it’s acceptable to lynch sufficiently bad people, then yes, accusing people of being bad is equivalent to calling for their murder, and you have no alternative but to make sure nobody is allowed to criticize anyone you like. This puts you in the position that Winston Churchill called “riding a tiger from which you dare not dismount”; you had better invest all your energy into making extremely sure that you and your friends are the ones calling the shots about who can and can’t be criticized. It sounds exhausting, which is why the liberal solution - bilateral controlled tiger-dismounting - is the choice of most functional societies.

I admit there are many intricacies and edge cases, for example:

• Most societies distinguish between simple criticism and true incitement to violence (for example, “so-and-so is a bad person and I suggest you murder him, I hear he will be at 123 Target Street tomorrow, and that Glocks are a pretty good type of gun”). The boundary here is fuzzy, but I think US courts do a good job of navigating it.

• Some European countries distinguish between permitted normal speech and hate speech (for example, “all Examplestanis are ugly stupid vermin”), on the grounds that the latter actually does encourage violence if left alone for too long. As a red-blooded American, I don’t agree with this distinction, but having it be a specific legally-demarcated category at least seems stable and predictable, and so less bad than the “stochastic terrorism” ploy.

• I admitted that there should be a discussion norm against some especially vitriolic criticism; a line beyond which I would think less of a person, even if I didn’t want their speech banned. But most stochastic-terrorism-sayers don’t propose to ban the speech involved, just to create a social norm against it. And “think less of X” and “create a social norm against X” are effectively interchangeable, since social norms are more-or-less defined as things that people think less of you for violating. So am I conceding the whole argument there?

• Probably there are some cases where extra-state violence is acceptable, for example assassinating Hitler or overthrowing an illegitimate government. I don’t want to rule this out by fiat, and I think it’s wise not to discuss the conditions permitting them too specifically; the best I can do is say that at least this limits the damage by only legitimizing attacks against powerful state targets, which can usually take care of themselves.

I think that accepting these edge cases and solving them with normal law and philosophy is a better bet than maintaining the “stochastic terrorism” concept and trying to hold onto some sort of topsy-turvy opinion on what kinds of criticism are vs. aren’t allowed.