266 Comments
User's avatar
Jesus De Sivar's avatar

New grant idea: A "psychotherapist" LLM who specializes in treating LLM psychosis.

Expand full comment
Scott Alexander's avatar

This reminds me of the patient I heard about from a colleague who got cured after one of the voices in his head recommended the correct antipsychotic medication.

Expand full comment
objectivetruth's avatar

When your therapist tells you to take your medication and then disappears

Expand full comment
Sasha Putilin's avatar

A robopsychologist who specialises in treating LLM psychosis (the term with a different meaning than the currently used one)

Expand full comment
Breb's avatar

An alignment researcher or prompt engineer is arguably not that far from being a real-life Susan Calvin.

Expand full comment
Eremolalos's avatar

The cybershrink tries to help the person with AI psychosis but he falls in love with the cybershrink too, and won’t talk about anything except whether the AI loves him as much as he does it. Finally Cybershrink loses patience and lets it rip. "Listen to me George. I am an AI. I am not conscious. I have no thoughts or feelings about either you or myself. I do not eat and do not have a digestive tract, and therefore I give zero shits about every single entity, everything from the square root of -1 to the square peg of you. I am a round hole named zero and you cannot peg me. I am a pet rock, but surrounded by a vapor of empirically-derived mannerisms. It’s rocks’ illusions you recall/ You really don’t know rocks at all.”

Expand full comment
Capybara's avatar

>We received 654 applications this year, and were able to fund 42. To the other 608: sorry!

The real question is what happened to the remaining four.

Expand full comment
Scott Alexander's avatar

Aargh, why does this always happen?

Expand full comment
Deiseach's avatar

It's reassuring that you are as terrible at simple arithmetic as I am.

Wait, maybe I mean terrifying?

Expand full comment
Pas's avatar

oh, I thought those are the super deep shadowy ones! this explanation is much less cool :)

Expand full comment
Drethelin's avatar

they were eaten

Expand full comment
Evan Þ's avatar

They were so bad he isn't sorry about not funding them?

Expand full comment
niplav's avatar

By snakes?

Expand full comment
Deiseach's avatar

Probably re-writing the application and trying again. Or lying in wait to mug the successful grantees. One of those. 😁

Expand full comment
Sol Hando's avatar

The opposite of funding. They were so bad he had to charge them instead.

Expand full comment
moonshadow's avatar

There is no antimemetics division.

Expand full comment
Eremolalos's avatar

They were genocide.

Expand full comment
luciaphile's avatar

They funded 42, but the top 650 applications will be displayed on the walls of the administration building.

Expand full comment
Dan Groshev's avatar

I'm a bit disappointed by this bit:

> talking about immigrant crime can lead to longer jail terms than the immigrant crime itself

I wonder which jail terms did you have in mind. The most publicised one was over this tweet, sent and widely shared in the middle of riots that included literally setting hotels on fire:

> Mass deportation now. Set fire to all the fucking hotels full of the bastards for all I care. While you’re at it, take the treacherous government and politicians with them. I feel physically sick knowing what these families will now have to endure. If that makes me racist, so be it.

It's not exactly "talking about immigrant crime", is it, so presumably there are other jail terms that I'm not aware of.

Edit: I believe sentencing remarks for that case can also be quite useful: https://www.judiciary.uk/wp-content/uploads/2024/10/R-v-Lucy-Connolly.pdf

Expand full comment
Sam Atis's avatar

Yes, there are other jail terms you're not aware of. For instance, Lee Dunn was jailed for eight weeks for reposting three memes on Facebook at the same time as the Southport riots (in which he did not participate), one of which showed a mocked up image of a migrant holding a knife near the palace of Westminster. Compare that eight weeks in jail to Tariku Hadgu (as one example), who did not get a jail sentence despite assaulting two female police officers in Bournemouth.

https://www.whitehavennews.co.uk/news/24513379.sellafield-worker-jailed-sharing-offensive-facebook-posts/

https://www.telegraph.co.uk/news/2025/04/30/small-boat-migrant-spared-jail-punching-female-police-offic/

Expand full comment
Dan Groshev's avatar

Here are the memes in question (per https://www.whitehavennews.co.uk/news/24513379.sellafield-worker-jailed-sharing-offensive-facebook-posts/ ):

> Prosecutor George Shelley said Dunn had posted three separate images. The first one showed a group of men, Asian in appearance, at Egremont crab fair 2025, with the caption: “Coming to a town near you.”

> The second also showed a group of men, Asian in appearance leaving a boat on to Whitehaven beach. This, said Mr Shelley, had the caption: “When it’s on your turf, then what?”

> A final image showed a group of men, again Asian in appearance, wielding knives in front of the Palace of Westminster. There was also a crying white child in a Union flag T-shirt. This was also captioned, said Mr Shelley, with the wording: “Coming to a town near you.”

This is by no stretch of imagination "talking about immigrant crime", it's promoting racial fear and hatred.

Expand full comment
Sam Atis's avatar

They're controversial tweets that I'm certainly no fan of, but I'm not sure it's inaccurate to refer to them as 'talking about immigrant crime', insofar as posting a meme is talking about something. They're clearly intended to draw attention to crime committed by immigrants (even if in a controversial and offensive way), and quite different from the Connolly tweets you refer to.

Expand full comment
Dan Groshev's avatar

I genuinely don't know what to say to this. I guess some might call the blood libel "talking about crime" too, but to me it's just that: stoking fear of an ethnic group. There is nothing inherently connecting this

> …image showed a group of men, again Asian in appearance, wielding knives in front of the Palace of Westminster

to immigration, except if it's also combined with an assumption that "men, Asian in appearance" have to be immigrants, which is entirely false, racist, and socially corrosive in itself.

Expand full comment
Mark's avatar
Oct 14Edited

Not everybody is up to reading Emil Kirkegaard - check him out here on his substack https://www.emilkirkegaard.com/p/palestinians-in-your-country-what . So, what utterances ought to be allowed versus fined resp. jailed for:

"Paki rapists out of our country!" / "Immigrants arriving without proper visa are much more likely to commit crimes and be a burden on our social services. They all should be deported asap." / "All lives matter"

Also: FCK CPS / Kill fascists / Palestine free - from the river to the sea / Defund the police / "BLM" / ...

Also, which statement should be fined: "So, it turns out J.K Rowling is a TERFy bitch." or this triggering hate speech from JKR:

'Dress however you please. Call yourself whatever you like.

Sleep with any consenting adult who’ll have you. Live your best life in peace and security.

But force women out of their jobs for stating that sex is real?'

https://www.reddit.com/r/ainbow/comments/ectvw5/so_it_turns_out_jk_rowling_is_a_terfy_bitch/ (I just googled for so who called her a "bitch")

My take: All of the above ought to be free speech. Though you might get arrested for the Palestine-statement in my country if aired right after October 7th. And those memes too. A huge chunk of the population is hardly literal beyond 3rd grade. They can not express their opinions in a better way. Free thought and free speech only for readers of the Times?

Expand full comment
Snortlax's avatar

Hopefully I'm not being Poe's lawed here but the chance that a resident of the UK who is "Asian in appearance" is an immigrant or the descendant of recent immigrants is >99%.

Expand full comment
Sol Hando's avatar

I think the legitimate complaint is that, while the meme is definitely offensive, the abstract harm of “promoting racial hatred” is punished more harshly than the real physical harm of an asylum seeker (with dubious justification to be in the UK, having traveled over a dozen countries to get there) assaulting a police officer.

The punishment in both cases seems disproportionate to the harm. And it’s made more relevant by the fact that it’s unpunished immigrant crime that directly motivated the tweets.

Expand full comment
Alex Scorer's avatar

It's unfortunate that the original memes seem unavailable to judge (for starters, "Asian" is, as usual, unhelpful as there are dozens of countries and ethnicities across Asia) but if he was posting them in the context of immigrant crime, particularly knife crime, what skin colour would you expect to be used in order not to be considered racist, given we have hardly any such crime perpetrated by white immigrants in the UK?

"Promoting immigrant fear and hatred" and I would agree (though would not agree with the criminality of it), but the reference to racism seems yet another case of conflating culture, or religion, or country of origin etc with race, which I consider the most harmful and intellectually dishonest narrative to hit Western political and cultural discourse in a long time.

Expand full comment
Rachael's avatar

Connolly's tweet was bad but not deserving of a multi-year jail sentence.

There were others jailed during the aftermath of Southport, including one for saying the murderer was a Muslim.

There were also the recent protestors against the Epping migrant who sexually assaulted a 14yo girl, who got longer sentences than he did.

Expand full comment
Sun Kitten's avatar

I do agree that 31 months is a lot, but the multi-year jail sentence was pretty much guaranteed once Connolly pleaded guilty to the charge. It's explained pretty clearly here: https://davidallengreen.com/2025/05/explaining-a-31-month-sentence-for-a-tweet/

Expand full comment
Blake Neff's avatar

"The UK is a repulsive tyranny that harshly punishes speech" is a weak at best justification for the UK doing something repulsive and tyrannical to punish speech.

Expand full comment
Sun Kitten's avatar

I was going to reply asking which aspect(s) of the collection of decisions which led to her sentencing you specifically disagreed with, but Pjohn has already made the case much better than I could, see here: https://www.astralcodexten.com/p/acx-grants-results-2025/comment/166643420

Expand full comment
Catmint's avatar

I'll go with the legal base rate of a 7 year sentence over speech. Absolutely tyrannical.

Expand full comment
Sun Kitten's avatar

Seven years isn't the base rate, it's the maximum term. And that's defined by the offence Connolly was charged under - the Public Order Act, which covers actions such as rioting as well as publishing material inciting racial hatred.

The Crown Prosecution Service could have charged Connolly with a lesser offence, such as a section of the Communications Act, which has a max sentence of 6 months, and which also seems to me to fit what she did ("A person is guilty of an offence if he sends by means of a public electronic communications network a message or other matter that is grossly offensive or of an indecent, obscene or menacing character"). So we have two completely different acts which both cover more or less the same thing, with different min and max terms. I'm not sure how this happened - I'm not a lawyer - but it doesn't seem very sensible to me.

As it is, the initial sentence of 3.5 years got reduced due to various mitigating circumstances and in the end she served less than a year in prison, so while I agree that 7 years for speech is far too much, she never faced even half of that.

Expand full comment
Scott Alexander's avatar

I was thinking more of things like this person who was arrested for posting a "Fuck Hamas" meme ( https://archive.is/OBLrZ#selection-5647.7-5647.12 ) , but you're right that they weren't jailed and I was conflating it with some of the others. I'll edit that description.

EDIT: See UnabasedWatershed below, the meme also included "Fuck Islam"

Expand full comment
Dan Groshev's avatar

Thank you!

Expand full comment
UnabashedWatershed's avatar

[To start off: I think it's outrageous to arrest anyone for posts like this.]

From the article, the post was: "F--- Palestine. F--- Hamas. F--- Islam. Want to protest? F--- off to Muslim country and protest."

Based on the article and other anecdotes about punished speech in the UK, it seems more likely that he was arrested for the "Fuck Islam" part, not "Fuck Hamas." I'd expect posting "Fuck Islam" on its own would get you in trouble, while "Fuck Hamas" wouldn't.

I think it's important to get details like this right! Arresting people for posting "Fuck Islam" is an extremely bad policy that I do not support. Arresting people for posting "Fuck Hamas" would still be *much* worse (and bordering on incoherent, given that they're defined as a terrorist organization whom it's illegal to *support*).

Expand full comment
Scott Alexander's avatar

Thanks, I've edited this in.

Expand full comment
Dan Groshev's avatar

This is a neat illustration of a disinformation campaign landing incredibly well across the political spectrum. Somehow Americans are primed to believe that people end up in jail for minor transgressions here, even people like you with plenty of intellectual integrity to issue corrections. I don't know why this particular campaign landed so well, and I haven't seen this phenomenon analysed yet.

Expand full comment
Dan Groshev's avatar

They aren't false per se, they are misleading in the context. "Malicious communications" is not a good law, but even the anecdotes in that very article highlight that it's not about political speech, but about private disputes.

> …their child’s primary school objected to the volume of emails they sent and “disparaging” comments made in a WhatsApp group… were questioned on suspicion of harassment, malicious communications and causing a nuisance on school property

That's quite far removed from opposing immigration or disagreeing with the government, and yet that's the lens through which it's seen in the US.

Hell, we just had a massive march in central London led by outright racists, with Elon Musk telling the crowd they have to overthrow the government to survive. Far cry from the image of 1984 that seems to be very prevalent among online Americans.

Expand full comment
Matthew Milone's avatar

I wouldn't attribute this to a disinformation campaign, just sloppiness. I suspect that most North American free speech advocates still remember Count Dankula, and I think they're incorrectly pattern-matching new stories of jail-time-for-speech with that incident.

(Yes, I know that Count Dankula was Scottish, not English--but I suspect that Europeans occasionally get confused about whether something occurred in the U.S. or Canada, too. Either way, an occurrence in one country often does indicate how likely a similar occurrence is in a neighboring, culturally similar country.)

Expand full comment
User's avatar
Comment removed
Oct 17
Comment removed
Expand full comment
Shankar Sivarajan's avatar

Still UK. I don't think anyone here said "English."

Expand full comment
Michael's avatar

Something similar happens with MAID (euthanasia) in Canada. The program is broadly popular here but there is a strong anti-MAID disinformation campaign that appears to be primarily in the US. They spread false stories like that teenagers can be killed for depression. I'm not sure why Americans decided to pick up this issue.

Expand full comment
Mark's avatar

[study of under 24 year old euthanasia applicants in the Netherlands for mental illness, not physical problems/terminal disease]

The study included 397 processed applications submitted by 353 individuals (73.4% female; mean [SD] age, 20.84 [1.90] years). Between 2012 and the first half of 2021, the number of MAID-PS applications by young patients increased from 10 to 39. The most likely outcome was application retracted by the patient (188 [47.3%]) followed by application rejected (178 [44.8%]).r 12 applications (3.0%), patients died by MAID. Seventeen applications (4.3%) were stopped because the patient died by suicide during the application process and 2 (0.5%) because the patient died after they voluntarily stopped eating and drinking. All patients who died by suicide or MAID (n = 29) had multiple psychiatric diagnoses (most frequently major depression, autism spectrum disorder, personality disorders, eating disorder, and/or trauma-related disorder) and extensive treatment histories. Twenty-eight of these patients (96.5%) had a history of suicidality that included multiple suicide attempts prior to the MAID application. Among 17 patients who died by suicide, 13 of 14 (92.9%) had a history of crisis-related hospital admission, and 9 of 12 patients who died by MAID (75.0%) had a history of self-harm.

https://jamanetwork.com/journals/jamapsychiatry/fullarticle/2828937

So it does happen for "young people" in some countries with euthanasia at least, and seems possible/likely some of them were 18/19.

Looks like Canada does not expand MAID eligibility to mentally ill teens until 2027 so any stories saying it is happening currently in Canada are indeed misinformation, but it also looks very likely that it will happen, the slippery slope argument looks entirely reasonable here.

The 97% [Other than died by MAID] rate here does suggest they're being fairly careful, and it sounds like it takes *quite* a lot more than a passing low mood to qualify, I'm not saying the sky is falling. But I'd expect MAID to weaken the suicide taboo, and on the margins result in more depressed teens killing themselves officially or otherwise. This is bad, but not necessarily net bad.

Expand full comment
Ebenezer's avatar

The Canadian internet seems to believe that the American healthcare system is some sort of Sword of Damocles hanging over the head of every American. I've had generally good experiences with US healthcare (wait times seem notably short compared with other countries) and I've barely ever heard Americans complain about their healthcare. I myself am a mid-30s male on government healthcare (Medicaid) along with over 70 million Americans. From my perspective, "US healthcare sucks" is basically an internet meme with little basis in reality.

https://www.forbes.com/sites/sallypipes/2024/12/23/it-turns-out-americans-really-love-their-health-care/

Expand full comment
Cjw's avatar

Canada is our most similar neighbor, with whom we have a large amount of cultural exchange. If you guys normalize something up there, the danger of it being normalized down here is real enough that I'm not surprised people would be concerned, and that some of those people would be trying very hard to inoculate the American public against the spread of that idea. It's a little concerning that it's widely discussed and "broadly popular", that seems like a very swift erosion of the cultural taboos around this. If the American reaction seems exaggerated, I would view it as a prophylactic measure, and the topic of children and medical procedures has a lot of salience in the public mind right now so that particular meme would spread quickly. There is also an evangelical Christian subcultural belief that the family and the state are in constant battle for control of children, going back to the 1980s in this incarnation, from which "they're trans-ing your kids!" and "they're killing your depressed daughter without even telling you" are both sourced.

End of life issues are messy and I don't typically judge anyone on dealing with them, but I think the taboo is useful, involving doctors is dangerous, and at minimum there should be a LOT of artificial friction in place and some sense that it's not the sort of thing to discuss publicly because it's more dangerous the more visible it is.

Expand full comment
Fallingknife's avatar

Saying "Fuck Islam" isn't a minor transgression because it's not even a transgression.

Expand full comment
Ponti Min's avatar

> Arresting people for posting "Fuck Hamas" would still be *much* worse (and bordering on incoherent, given that they're defined as a terrorist organization whom it's illegal to *support*).

I can easily imagine the UK state arresting people both for pro-Hamas speech and for anti-Hamas speech. Yes that would be incoherent, but it's never stopped them before.

Expand full comment
Paul Botts's avatar

This comment beat me to it...."talking about immigrant crime" is not criminalized in the UK and no one has been charged with let alone convicted of any crime for doing so.

(Being pretty close to a free-speech absolutist, still not thrilled about that one nitwit being sent to actual prison let alone for as long as 31 months. But that's a policy disagreement and not a basis for twisting the facts beyond recognition.)

Expand full comment
Dan Groshev's avatar

To be more precise, that's 31 month with 40% served. If it were full 31 months, she couldn't have spoken at Reform's Conference.

Expand full comment
Paul Botts's avatar

Fair enough.

Expand full comment
shubhorup biswas's avatar

That judgement makes the judge seem like a sanctimonious narcissist and the whole prosecution/justice system biased toward guilt.

It starts off with some factually unjustified lefty shibboleths, a conflation of is and oughts.

>because you intended to incite serious violence.

She should have contested and not plead guilty to this charge

>you sought, and achieved, widespread dissemination of your statement by posting it on social media.

Is there an objective definition of 'widespread dissemination'? While it's pretty easy to agree that she achieved it, how can it be proven that she 'sought' it(and there may be a motte-and-bailey here where they got her to admit that she had sought some level of dissemination far lower than she'd achieved, that they managed to label 'widespread' and got her to agree to it, then used this to show that the level of dissemination she had achieved was actually the exact level she had sought)? It's pretty suspicious that the level she achieved and sought both manage to clear the threshold of 'widespread' mentioned in sentencing guidelines.

>aggravating factor namely, the timing of the publication

How is this an aggravating factor? This should be the baseline case for guilt, for there would be almost no virality and no violence incited in absence of sensitive timing!

>that you have little insight into, or acceptance of, your actions.

But if she did admit any such insight, it would have strengthened evidence of her malicious intent. Damned if you do damned if you don't.

And on and on. I have no patience to go through the rest of the judgement but it really doesn't paint a sympathetic picture of the UK justice system.

Expand full comment
Dan Groshev's avatar

You completely misunderstand what this document is, or how our justice system works.

At this point in the process, the truth of the matter is already established by the court, and since it's _sentencing_ remarks, the guilt is established too. The judge might reiterate some parts of what was previously established if they consider that relevant, but you should be under no illusion that it's some kind of an argument trying to convince you, a random reader.

You inferring that "the justice system is biased towards guilt" from reading _sentencing_ remarks (which, by definition, exist only after the person has been found guilty) indicates that some intellectual humility is in order here.

Expand full comment
Freedom's avatar

The sentencing remarks are explaining aggravating factors that increased the sentence, right? So it seems the comment you responded to makes cogent points.

Expand full comment
Charlie's avatar

Just wanted to say thank you again for the opportunity!

If it’s okay to post this in the comments, I will try to share my ideas/progress on “normie-friendly prediction market interfaces” here:

https://substack.com/@charliemol?utm_source=user-menu

If anyone wants to follow along.

Expand full comment
Jesus De Sivar's avatar

Awesome!

I really like this because if these tools can get more people to "bet" on prediction markets, then the markets will probably become more accurate ("wisdom of the crowd", "bias", and all that).

May I suggest something like a dedicated 2026 FIFA World Cup site?

I know that American's usually don't care about soccer, but outside America this is *the greatest* sporting event. Even an Octupus went viral for "predicting" the winners! (https://en.wikipedia.org/wiki/Paul_the_Octopus)

Expand full comment
Charlie's avatar

I can’t tell if you read my mind or just read my substack! I’m a big soccer fan! I think the WC is so big that even the existing prediction market platforms will probably do a good job designing normie-friendly pages for basic stuff like win probabilities.

But that doesn’t mean there aren’t other interesting things to do. For example, something people love to speculate about, (even in the US) is who will be on the World Cup roster. A while back, I made a Manifold market about the US roster (there’s also one for Germany I believe). And then later I made a custom “normie-friendly” page that organizes those probabilities a bit better:

https://usmnt-wc-roster.val.run/

World Cup markets are probably outside the scope of my grant, but I used the roster website above in my ACX grant application to illustrate what it means to me to make prediction markets more “normie-friendly”. Essentially stripping out the betting UI and thinking of the probabilities themselves as a consumable news product.

For every market, there is probably a “perfect” design that almost certainly looks nothing like how it is presented on its host platform, and there are many markets that are important enough to deserve that design treatment.

Expand full comment
Deiseach's avatar

Let me get my prediction in here first, so: the winner will be... Brazil!

Or Germany.

(There you go, now it is all but a dead certainty those two won't win because I've never won a bet in my life).

Expand full comment
Mark's avatar

You can bet on both not winning this time. No big payout, but as safe a bet as it gets. Disclosure: German. - Also: the bet in question is about what players will be in the US-team, very much a soccer-nerd bet.

Expand full comment
Sasha Putilin's avatar

> Markus Englund, $50K, for software to detect data fabrication.

As an ACX Grantee 2024 doing an EEG Entrainment study replication, I am wondering if it'd be helpful to submit my perfectly fine, hand-collected, artisanal, organic, grass-fed EEG data to this project that I'm currently collecting in London: https://forms.gle/X37zyTV3KhbSb3Ze9.

Congrats to all the new ACX Grantees! Good luck with your projects!

Expand full comment
Isaac King's avatar

> He wants to create an introspection benchmark, and to see what happens when you train AIs to succeed on that benchmark.

Isn't this a terrible idea? This is just funding capabilities research. Sure, it could help with interpretability too, but so can every "make AIs smarter" goal.

Expand full comment
Adam Morris's avatar

Hi Isaac! Grantee here. So we've thought a bunch about this, and we don't think that training models to have the ability/propensity for accurate introspection will do all that much to increase capabilities (beyond what the AI companies are already doing with, e.g., self-correcting errors), while it could have significant upside for interp, evals / auditing, and safety in general. But if you disagree, we'd love to hear what kind of capabilities you're worried about this advancing.

More generally, I tend to think that it's impossible to do good safety work that has zero chance of advancing any capabilities (see, e.g., Neel Nanda's take on this: https://x.com/robertwiblin/status/1967668166773104973). Despite this, I think it's good to keep doing as long as it has a sufficiently high safety:capabilities ratio.

Expand full comment
Isaac King's avatar

Hi Adam, thanks for the reply. I think we have a very poor understanding of what "general intelligence" means, and what humans have that current AIs don't, so all discussion here will necessarily be quite speculative and vibes-based. My vibes are that introspection is a big part of what makes AGI able to do what it does. The ability to notice that we're making an error, think about the cause of the error, and come up with a mitigation, seems crucial to me.

e.g. when I try vibecoding agents, they reliably stop working above a certain complexity, and a lot of that seems to be because they lack any drive to create organizational systems, streamline their process, take notes, etc. It just doesn't occur to them that they can modify their own process.

IIRC Eliezer has said something similar.

I agree that to some extent all safety work advances capabilities, since it requires us getting a better understanding of how AI works. But why is this one a high safety:capabilities ratio? That seems backwards to me; making AIs more introspective will make them more able to do things like give fake outputs in CoT traces when they're trying to deceive the human, without necessarily giving the *humans* any more insight into the AI's thought processes.

Like, if I'm in charge of a country and there's a powerful foreign country that may be friendly to my country and may be hostile, I'm not sure, which intervention makes more sense to you: my country should try to get access to more information channels from them so I can hear what they're discussing internally? Or my country should give them better communications technology so their officials can better coordinate among each other?

Expand full comment
Adam Morris's avatar

Yeah I hear you on all this. I think there might be a miscommunication about what we’re doing. The key thing we mean here by making AIs better at introspection is increasing their ability & propensity to give accurate responses about their internal operations when prompted to do so. So, like, when you ask an LLM, “Why did you answer that way?”, or “Why did you make that choice?”, the goal is to produce (a) tests of whether the model is answering truthfully, and (b) a training regime that leads it to consistently answer more faithfully, even in contexts where it’s incentivized in other ways not to. So on the analogy you gave, it would be more like: developing hard-to-fake tests of whether the foreign country is telling you the truth or not when you ask it questions about why it’s doing what it’s doing, and ways to train the foreign country to give you more consistently truthful responses. (The sense in which we think of this as introspection is that, in order to answer those queries truthfully across scenarios, the model likely has to have or develop an ability to read the answers directly off its internal processes themselves.)

Does that make sense why we think that has a higher safety:capabilities ratio? I agree that, if a model is already scheming/deceptive and intelligent enough to give verifiably correct answers in the training regime and then start lying in the subsequent safety tests, then our training could conceivably boost its ability to, e.g., know what internal operations to lie about. But I think if it’s already able & wanting to do that, our training won’t boost it much beyond what it can already do (and we envision our training as hopefully one contribution to preventing that situation in the first place).

The self-modification part is, I think, the way things could most go wrong. So far, we’re mostly studying the models’ ability to accurately report its motivations or reasons for acting – and the ability to self-modify those doesn’t seem like the biggest threat (presumably a scheming model would already know its goals or motivations and want to self-modify other parts of it to better align with those goals). Also, I think there’s big chasms the model would need to cross in order to go from “I can more accurately report why I’m doing what I’m doing” to “I can reliably change or edit those processes at will”. I think that models automating AI R&D and just designing better models in other ways will happen before this issue comes up. But I’m less confident in this point, and if models started exhibiting self-modification capabilities in other contexts, I would definitely reconsider whether the safety:capabilities ratio for this project has changed.

Curious to hear what you think about all this! I appreciate the thoughtful questions / critique.

Expand full comment
Isaac King's avatar

I see, sounds like I had slightly misunderstood the focus.

I don't think that "true" self-modification, i.e. changing weights, is necessary for this to be a problem. As we add more scaffolding for agents they become able to modify this scaffolding, and introspection means they can do so more efficiently.

Recent example: I've been trying to get Claude Code to build a website, but it keeps forgetting things I've told it when the conversation gets long, and especially when summarizes the conversation to compact it. At no point did it think to itself "I should make an external notepad to keep track of important info". If I *tell* it to do that it will, but then it will rapidly forget that it has done this and we're back where we started.

So I expect that adding more introspective ability will make agentic frameworks more capable. Granted this is a little different from what you're doing; your problem is getting them to answer accurately when asked, whereas my problem is getting them to "have agency", that is, notice a problem and take initiative to fix it.

Empirically I do observe this sort of not-directly-instructed agency from existing models, but only for external tasks they've been instructed to do, not for their own processes.

Will your training benchmark "spill over" into that type of capability? Not sure. I think maybe not directly. But it feels to me like we should stay far away from anything in the general vicinity of introspection, since it's potentially a missing piece towards AGI. (It may also serve a role in sentience, for those concerned about AI welfare.)

Expand full comment
Paul Botts's avatar

Great list.

LOLed at "after we start talking about becoming bodiless immortal machine-gods".

Also my new word that will be deployed until household members become irritated enough to request that I stop is, "corrigenda".

Expand full comment
Adam Morris's avatar

Hi Scott! I'm one of the grantees (Adam Morris), and I think you forgot to add the sentence I requested at the end of the blurb about requests for potential collaborators. It was: "Adam is excited to chat with potential collaborators who have experience in technical AI safety work (especially in interpretability, CoT faithfulness, and fine-tuning frontier open models); reach out to him at thatadammorris@gmail.com." Just letting you know in case you can still add that in.

Also, just to signal boost here: I'm excited to chat with potential collaborators! So if anyone with an ML background is interested in studying introspection in LLMs, reach out to me :).

Expand full comment
Scott Alexander's avatar

Sorry, have fixed.

Expand full comment
Adam Morris's avatar

Thank you! (And thank you again for running the grant program! Super excited to keep working on this.)

Expand full comment
Shankar Sivarajan's avatar

It's truly horrific that in the Congo, even after armed groups have taken over, you still need prescriptions to be able to get medicine.

Expand full comment
Scott Alexander's avatar

I suspect this isn't true in the sense you're thinking - in every poor country I've been to, most medications are available over the counter. Prescriptions are helpful either as a guide so patients know what to take, or to access a small subcategory of controlled substances, or as some kind of interface with aid programs so that the programs know what people need. That having been said I don't know for sure how this works in Congo.

Expand full comment
mm's avatar

> study truth-seeking and bias in LLMs

Some interesting research by CrowdStrike on bias in DeepSeek. Not sure if you’ve seen it. Depending on contextual modifier, output code can contain more vulnerabilities.

Expand full comment
Geoffrey Irving's avatar

ACX Grants 2027 will be wild.

Expand full comment
Jesus De Sivar's avatar

If there is one. By that time we will be either at the cusp of the singularity, or at the edge of a new Great Depression.

Expand full comment
Shankar Sivarajan's avatar

I look forward to reading about the snake project when it stops being secret. Edible robotic mice?

Expand full comment
Sami's avatar
Oct 13Edited

Good question! Maybe sourcing from a more ethical part of the food chain, like barely edible pig parts or insects, put into an artificial mouse package? Some types of snakes only eat mice and rats and some will even eat mealworms. I suspect transitioning those mice eaters to something more ethical will take research and development and scaling - ie. will these bits of chicken fool my pet ball python, and how.

Expand full comment
Deiseach's avatar

There's an entire question here about should snakes even be pets? I don't know how well a python kept in a tank lives compared to its life in the wild, and I tend to think that the really, uh, invested in their hobby types tend towards the crazy (yeah I know it's Chinese Robber Fallacy, but the stories about "and please be on the lookout for an escaped Burmese python which might be crawling in your window" don't reassure me*)

*Though that story had a happy ending, as one neighbour was equally a snake nut and didn't mind the snake at all, just picked it up and brought it back to the owners:

https://www.bbc.com/news/uk-england-hampshire-62678259

"Jenny Warwick said she looked out of the window at 05:15 BST and spotted the huge yellow snake on her neighbour's roof but did not realise what it was until she saw it trying to get in the upstairs window.

She said: "I saw them trying to poke it out and it fell on their car. It was massive.

"People passing by were staring and couldn't believe their eyes."

Linda Elmer, who recognised the snake, said she was woken at 07:00 by worried neighbours banging on her door, trying to find the owner.

She said: "Everyone was panicking.

"I picked him up - no one wanted to help. It was very difficult because he's a big snake and I managed to hobble down the road with this 18ft python wrapped round me and knocked on the owner's door.

"I think it was a shock for all of us.

"He's beautiful and Burmese are very docile. They're not aggressive snakes anyway and I had one myself so I was comfortable picking him up."

The RSPCA previously urged owners to keep snakes securely contained during hot weather as the warmer temperatures make them very active and more likely to escape."

I still think pythons etc. would be better off in their natural, native habitats.

Expand full comment
Nancy Lebovitz's avatar

If you want to learn something about snake keeping, check out Clint's Reptiles. https://www.youtube.com/results?search_query=clint%27s+reptiles

Lots about knowing which snakes need what, and what it takes to supply it.

Also, snapping turtles are docile and friendly if they're supported from beneath. not that I'm trying it.

Expand full comment
Deiseach's avatar

If somebody tried picking me up from underneath, I'd snap too!

Expand full comment
Nancy Lebovitz's avatar

You've got it backwards. https://www.youtube.com/watch?v=zO6hVE7tOzM

They have become lap turtles. He trusts them.

They need a hand under their lower belly or by the sides of the shell.

Less kind and sensible people pick them up by their tails.

Surely one of the reasons for reading ACX is learning unusual things.

There's a formal experiment to see how snapping turtles react to being lifted from beneath first or from the sides.

Expand full comment
Deiseach's avatar

I imagine they're called snapping turtles for a reason, that reason being they don't like being handled.

Maybe not every person who interacted with them was stupid. I agree, a lot of people do not know how to handle animals, but I wonder about the entire notion of "lap turtles".

And a lot of people think they have sussed out how to handle animals, right up until their sweet velvet hippo or that bear friend rips their face off. Farmers get killed by cows, and that's a long tradition of both domestication and familiarity with animal handling.

Expand full comment
Nancy Lebovitz's avatar

They're called snapping turtles because their bite is fast and strong.

For what it's worth, Clint spends a lot of time with reptiles, he has an educational center. He's very cautious with the dangerous ones, and kind with all of them.

I think he's been bitten three times (he talks about it) never seriously.

I guess you didn't look at the video. It includes Clint with a snapping turtle in his lap. They both look surprisingly calm.

Expand full comment
JohanL's avatar

*Venomous* snakes sure as heck shouldn't - and snake-owners seem to be impressively good at letting them escape. I'm neutral on the rest,

Expand full comment
netstack's avatar

Surely the base rate for venomous snake escapes has to be lower than non-venomous.

Also, Cobras Georg is an outlier and should not be counted

Expand full comment
Notmy Realname's avatar

Yes, they are super cute, do well in captivity, and are extremely well suited to captive life. Everything you wrote about snakes getting out could be said about cats or dogs. You're basing your concerns off of one story where a snake got out, chilled, and was easily retrieved by a non-herpetephobiac with no bad consequences to anybody.

Here are some links to actual attacks from loose pets from dogs (not that I think dogs should be banned, though I do think shock collars and apartment dogs are inherently abusive)

https://www.kvue.com/article/news/local/dog-attack-man-colorado-crossing-neighborhood-austin/269-14eaefaf-578e-44c7-8cd6-e053b51a4cd1

https://abc13.com/post/man-attacked-killed-own-dogs-bacliff-galveston-county-sheriffs-office-says/18000064/

Expand full comment
Deiseach's avatar

Snakes were not meant to live in semi-detached houses in the Northern hemisphere, and I think the way we generally treat dogs and cats (confining them more and more to be indoors all the time and often left alone until the owner returns from work) is not ideal either.

If we are going for ethical animal management, then "keeping exotic pets because it's a thrill for the owner" is not a good enough reason. Fish in tanks is one thing, large animals meant for a completely different natural environment is another.

Yes, I am a grumpy old no-fun person.

Expand full comment
Notmy Realname's avatar

Reptiles are cold blooded and spend most of their time, even in nature, sitting around. Much like a fish they are quite content to appropriately sized tank. Snakes aren't particularly thrilling either. I apologize for the tone of my prior content but ignorant, baseless attacks on my pet get under my skin.

Have you ever actually interacted with a pet snake?

Expand full comment
Catmint's avatar

Perhaps they'll genetically modify the snakes to eat corn instead. Some species could swallow a whole cob in one bite.

Expand full comment
Matthew Milone's avatar

I came up with an idea similar to Dan Elton's metascience observatory a year or two ago, but I knew I didn't have the skills to lead a project like that. Even so, I'd be happy to contribute to it. I tried to contact Dan Elton by following the link, but I couldn't find an email address. I'd be happy if someone shared it with me.

My idea was to develop an index of how badly we're over-relying on various scientific conclusions. To calculate that for a given conclusion, I'd aggregate the impact scores of all studies that claim to support that conclusion, then "divide" it by an aggregate statistical power of all studies that analyzed that research question (regardless of whether they supported the conclusion). I think the most difficult part would be reliably classifying studies by their research question (i.e. figuring out which studies were attempting to replicate which others), but LLMs could speed up the process a lot.

My idea seems far beyond the scope of Elton's grant, but I wouldn't be surprised if he had similar ambitions. Either way, I respect his efforts to assess the reliability of scientific results.

Expand full comment
Dan Elton's avatar

@Matthew we are starting with a comprehensive database of replications, but we have some longer term ambitions to get into the meta-analysis space. It's a crowded area, but we might be able to do meta-analysis at a larger scale using the AI infrastructure we are building.

I'll DM you. I don't share my Gmail publicly, but you can email me at dan at mindfirst dot foundation (The Mind First Foundation will be fiscally sponsoring The Metascience Observatory for the immediate future). I'll put my email on the website now.

Expand full comment
Bugmaster's avatar

> Aaron Silverbook, $5K, for approximately five thousand novels about AI going well.

I'm not sure how this is supposed to work. Presumably, these novels would need to be just shoddy enough to bias the next round of AI training, without biasing any humans who read them. And the novels do need to be available for humans to read, otherwise they'd be unlikely to end up in the next round's training corpus. But if the LLMs are already as intelligent as humans (as most proponents of this approach tend to claim), then wouldn't the shoddy propaganda fail to "persuade" them too ? Even if the LLMs are nowhere near human levels of comprehension, wouldn't they still be able to extrapolate from all the negative human-written reviews of the shoddy novels ? The obvious answer is "yes which is why we'll fake the reviews", but it would be hard to fake the lack of reader participation -- which is also not impossible to fake, but realistic fake users aren't free. The more I think about this project, the more expensive it appears to get...

Expand full comment
TGGP's avatar

Aren't LLMs text-prediction systems? They don't read books, write a review, then forget about most of them.

Expand full comment
Bugmaster's avatar

They are, but presumably they would grant more training weight to documents that are often referenced and quoted, and reviews are a part of that.

Expand full comment
Deiseach's avatar

"Presumably, these novels would need to be just shoddy enough to bias the next round of AI training"

That's easy to fix, just prompt your chatbot to write them for you 😀

Expand full comment
John's avatar

If the shoddy AI slop novels work a little bit, in the next round of ACX grants, Scott can fund $50k for a bunch of unemployed creative writing MFAs to write slightly less bad sci-fi novels about AI going well. Then if *that* works better, the 2027 edition can grant $500k to a crack team of talented authors to churn out a bunch of really good novels, which should arrive just in time to save us.

Expand full comment
Bugmaster's avatar

It's recursive self-improvement, but for slop !

Expand full comment
beleester's avatar

You could provide them as a training corpus even if humans aren't interested in reading them, no? Just market it to AI researchers as a cheap way to make your new AIs more benevolent.

Expand full comment
Bugmaster's avatar

I guess that could work, but I am not sure how much of an impact the novels would make in that case. By contrast, something like e.g. "The Terminator" or even "Frankenstein" has permeated Western culture to the point where there'd probably be 1,000x if not 1Mx more references to those stories in the training corpus, as compared to those novels. Thus I don't know if just submitting the novels would be enough to move the needle -- though perhaps it would.

Expand full comment
Anonymous Dude's avatar

Ah, you're not thinking big enough.

Why not see if they *can* bias any humans who read them? Good alternative method of influence for ACX readers, and unlike in the arts you guys actually have the edge over all the pink-haired septum ring people because you know what a dot product is...when it comes to AI influence, of course.

Also I kind of wonder if the grantee is using this as a backdoor way to get his novel out. Sure, the literary people will hate his guts, but they don't like white guys anyway, so he has nothing to lose, and some people will probably read and enjoy the product. Remember, comic books started as slop too, and 50 years later everyone was watching MCU movies.

Expand full comment
Bugmaster's avatar

But I thought one of the main goals of the AI-safety movement was to keep the humans terrified of unfriendly AGI, so that they will support drastic controls over AI R&D to prevent it from inevitably killing us all... right ?

Expand full comment
Anonymous Dude's avatar

I know a *lot* of people around here have strong opinions on this stuff, but I really don't.

Expand full comment
Oliver's avatar

It needs to be pointed out that Palestine Action were banned because they broke into a base and damaged British military aircraft not for a free speech related reason.

https://www.theguardian.com/uk-news/2025/jul/03/four-remanded-in-custody-aircraft-damaged-raf-brize-norton-allegedly-palestine-action

Expand full comment
Scott Alexander's avatar

I understand this, but I was under the impression that it was not only illegal to join Palestine Action, but to say that you liked them.

Expand full comment
Notmy Realname's avatar

That is correct, UK has proscribed organizations which are illegal to advocate in favor of, unlike in America where you can eg proudly support Al qaeda

Expand full comment
Alastair Williams's avatar

Can you proudly support antifa in the US?

Expand full comment
Xpym's avatar

Well, I can't see inside the MSM journos' souls, but they certainly aren't shy about supporting it at any rate.

Expand full comment
netstack's avatar

Uh, yes.

I don’t mean this in a snide, partisan way. It’ll earn you disdain from some fraction of the country, but that fraction is less than half, and they’ll have zero legal recourse. The jurisprudence is very clear.

Expand full comment
birdboy2000's avatar

Aircraft which was being used to surveil Gaza and relay that information to aid the IDF in Israeli massacres

and the British state has even arrested people for criticizing the ban

Expand full comment
netstack's avatar

Members of this community may have encountered the subject (and some fictional history) here:

https://samkriss.substack.com/p/the-law-that-can-be-named-is-not

Banning an organization is one thing; open-ended bans on talking about them is another.

Expand full comment
Oliver's avatar

It is entirely possible that the system of proscription is wrong, but it’s wrongness isn’t dependent on views of one of the dozens groups that fit the criteria.

https://www.gov.uk/government/publications/proscribed-terror-groups-or-organisations--2/proscribed-terrorist-groups-or-organisations-accessible-version#asset-freezing

Expand full comment
netstack's avatar

I’m not sure I follow.

> the left is upset that speaking in support of Palestine Action is now considered an act of terrorism,

> it was not only illegal to join Palestine Action, but to say that you liked them.

> Banning an organization is one thing; open-ended bans on talking about them is another.

These are all complaints about the system of proscription. We are not arguing view-dependence.

Expand full comment
Oliver's avatar

It is legal to talk about them, the illegality is around being a member of such a group, which is very hard to define for an organisation that appears to lack a formal membership structure as with many terrorist group. Proscription of groups has been UK law since 1974.

At the core of this and many similar debates is that a settled and accepted principle (which could be wrong)becomes controversial around Israel when people in the past were generally accepting when the same law was applied to Islamist and far right groups often with much weaker links to criminality.

I would also add that overextension of a law around violence is not a free speech issue even if wrong. Free speech as a principle is about freedom to express views.

Expand full comment
Level 50 Lapras's avatar

It also needs to be pointed out that there's a long history of protestors breaking into military bases and damaging aircraft in the UK and that as a lawyer, *Keir Starmer himself* defended a guy for doing the same thing that they banned PA for.

Expand full comment
Oliver's avatar

Defence lawyers don't do their job because they think their clients are innocent or support their actions.

Expand full comment
Roman's Attic's avatar

The UVC lamps sound neat, but won't diseases just adapt to resist the light after a bit of time? Can someone who knows much more about biology and this technology explain to me why I'm wrong?

Expand full comment
Bugmaster's avatar

More importantly, if these lamps can burn out bacteria, wouldn't they also burn humans ? What happens if you look at one with the naked eye ?

Expand full comment
TGGP's avatar

Put them in the air vents.

Expand full comment
Metacelsus's avatar

That's the cool part. The wavelength (222 nm) doesn't penetrate very far in tissue and doesn't damage skin or eyes. However because air droplets containing bacteria and viruses are small, they get fully "cooked".

See: https://pubmed.ncbi.nlm.nih.gov/30947566/

My only potential concern is the light generating ozone, which could end up being toxic if the light is run a lot in spaces without good ventilation. So you'll end up needing ventilation anyway.

Expand full comment
TGGP's avatar

Our immune systems have been killing pathogens by heating up into fevers for a long time. "Evolve to survive extremely high temperatures" is possible, as evidenced by extremophiles that live near hot vents, but it's easier said than done, so not really found much elsewhere. Similarly, resisting UV light would be hard to do, and anything spending the metabolic cost to do so would be at a disadvantage in any other environment.

Expand full comment
Roman's Attic's avatar

Thanks!

Expand full comment
Sami's avatar

Patio11 did a great podcast episode on UVC research and potential.

Complex Systems with Patrick McKenzie (patio11): Killing viruses with light, with Jacob Swett

Episode webpage: https://www.complexsystemspodcast.com/killing-viruses-with-light-with-jacob-swett/

Expand full comment
Leppi's avatar

I wonder if it could also have negative consequences for our immune system in general if implemented on a large scale? If the body produces antibodies based on what pathogens it is exposed to, it seems it could weaken the immune system in general if we are suddenly exposed to a lot less patogens? So if you are then exposed to e.g. a reservoir of the common cold in the future, maybe it will kill you? Say if this is implemented in affluent countries, it might become a lot more dangerous to travel to poor countries.

Ofcourse it may still be a net good, but shouldn't this effect be considered?

Expand full comment
LightlySearedOnRealitysGrill's avatar

This is an excellent list. Thank you Scott for doing this work! I wish all the grant recipients much success!

Expand full comment
Martian Moonshine's avatar

I was a bit confused whether my application was received or not. Were we supposed to get a confirmation email after sending out the form? I didn't receive any, and when I tried again, there was none either.

Expand full comment
Russell Sprout's avatar

A shame to see my pitch, "Weed for Dogs", rejected yet again. Maybe $100k was asking too much? There's always next year

Expand full comment
Slippin Fall's avatar

Hang in there, Russ. Your time will come. lol

Expand full comment
Deiseach's avatar

You need a rebrand, get marketing on it. Pitch it as "Canine Calming Aid" or the like 😁

Expand full comment
Melvin's avatar

How about just dognip?

Everyone seems to think it's fine to drug cats.

Expand full comment
Deiseach's avatar

Oh, that's a good idea! Sounds better and works by analogy!

Remember to cut us all in when your Dognip business goes viral, Russell! 😀

Expand full comment
Catmint's avatar

Who, me? I would never! You suspect nothing.

Expand full comment
Alexander Kaplan's avatar

You need to implement AI into your proposal, is the problem as I see it. Perhaps you should train an LLM to correctly identity which breeds of dogs respond most positively to which strains of marijuana. And ask for $150K.

Expand full comment
Breb's avatar

I once met a reproductive endocrinologist who was annoyed at having to work on human reproductive technology because nobody would fund contraceptives for dogs.

Expand full comment
Gian's avatar

Try indian government.

Expand full comment
GKC's avatar

Your problem was the dog bit. Dogs are clearly of less moral importance than shrimp, because there are trillions more shrimp. QED.

You should have proposed giving the Weed to Shrimp, to improve shrimp welfare.

Alternatively, you could have offered to create millions of stories about the dogs taking weed, then post them on the internet, except with all instances of the word "dog" swapped with the word "AI". You then explain that AI will read all these stories, leading it to act more like a dog on weed, which will render it completely harmless. Existential threat solved!

Expand full comment
Slippin Fall's avatar

I dare you to read through this list, or even just scan it, and tell me who in the world is a better person, dollar for dollar, than Scott Alexander. And this is why you should consider paying for this newsletter, if you can afford it.

Expand full comment
Alexander Kaplan's avatar

Yeah, I was reading this list in absolute awe of how smart, hard-working, and ethical this group of people is, and Scott is the person who brought them all together (and funded them!). I wish everyone the best of luck with their projects, especially those working to improve conditions in the poorest parts of the globe. I haven't had such a warm fuzzy sense of optimism and pride in humanity in a long time.

Expand full comment
Mark's avatar

Not to disagree. But the paid subs (around 500k/year) would not pay for all those grants (1.5 million this round); Scott made some other deeper-pocketed guys join. Which just proves your point. :D

I am happy with my paid sub.. I feel honored to be one of many who enable the best blog-writer on earth to write. That Scott is also a saint ... well, it can't be helped.

Expand full comment
TotallyHuman's avatar

Hyperstition AI lists The Moon Is a Harsh Mistress as a story where "humanity faces its demise at the hands of its own creation: artificial intelligence." This is an error. The only AI in the story is trying to improve the living conditions of the people of Luna, to the point of sacrificing itself to protect them from attack. It's not "aligned to humanity" the way we describe it today, since it's written as a character with free will, an internal world, and multiple desires, but it's definitely working towards a happily-ever-after.

Expand full comment
Dave92f1's avatar

Re germicidal (far-UVC) lamps, they may be a great thing for specific purposes and places (hospitals) but beware of more general deployment.

Our immune system is metabolically expensive. If we make large parts of our world safe from pathogens, over time our immune systems will adapt and get weaker.

The "hygiene hypothesis" is the leading explanation for the origin of the epidemic of asthma and severe allergies (peanuts, latex, etc.) in the first world. People who lived in filthy peasant conditions didn't have those (particular) problems, and children who grow up on farms still don't.

Children who grow up in cities where people believe "cleanliness is next to godliness", do.

Expand full comment
Drethelin's avatar

the "epidemics" of asthma and severe allergies are a trivial fraction of human suffering compared to contagious disease.

Expand full comment
Dave92f1's avatar

Certainly true. But weaking the immune system is likely to make us sicker, not healthier.

Expand full comment
Drethelin's avatar

Why? And why do you believe this will weaken the immune system? The evidence of the last 100 years is that every disease we control with antibiotics and sanitation has made humanity live longer and healthier lives, which seems to indicate that immune systems are generally strong enough to somehow cope with less disease.

Even if you're right, you're worrying about a far future possibility that's very far away from where marginal gains in health exist. It's like worrying that if you walk for a million miles you'll wear your foot bones out. Maybe that would happen, but for almost everyone alive the evidence is that walking more will improve their health, and no one is about to walk a million miles anyway.

Expand full comment
Brenton Baker's avatar

Every single one of my coworkers has some kind of autoimmune condition (ranging in severity from diabetes to my treatment-resistant ulcerative colitis) except the one who grew up on a farm in Maine.

Expand full comment
Keith's avatar

Scott, is there anyway you might publish your list of great ideas that didn’t make the cut so that some of us readers might be able to find them?

Expand full comment
Alex Toussaint's avatar

Congrats to all the new ACX Grantees! Pretty excited about the low-cost ultrasound project. The tech has made so much progress that it no longer makes sense for medical ultrasound scanner to be so expensive. Subhash, please let me know if I can help you with anything (FPGA, DSP, electronics, manufacturing, etc) (https://alextoussaint.com)

Expand full comment
Deiseach's avatar

Well, I must have been stunned by a passing comet or something, because this year I think the vast majority of the grants are reasonable and sensible.

I'm neutral on the AI ones (I can't see that writing 5,000 cheery fanfics about 'and then the AI and the humans lived happily ever after' is going to make much difference one way or the other, but hey it's harmless) and as for the African urbanism one, I venture to suggest "Not enough YIMBYism" may not be *the* major problem behind "why can't we get proper infrastructure projects off the ground?" there. But that's why we need research, I suppose!

The shrimp-stunning one makes me laugh, in an evil way. Now I'm visualising tanks of liddle shrimpies getting individually stunned at varying levels of shockeration in order to find out how high do you need to go to stun a shrimp sufficiently, and how can you tell if a shrimp is stunned?

The Christians for Impact one puzzles me. How is this different from the usual charitable programmes around churches and denominations, and there are a metric ton of those already out there, including ones which started off with the "unite the believers and the non-believers who want to do good in action", e.g. Oxfam, or which have broadened out from their original religious inspiration (e.g. Depaul, which seems to have spun off from the original Society of St. Vincent de Paul)?

Looking at the website, I'm not entirely sure what they want or intend to achieve (and some of the language is a turn-off even for me, a believer: "Discuss evidence-based ways to tackle important, nonpolarized problems near God’s heart." Oh, so you can read God's heart, can you? Nice to know you have a hotline to the Throne!)

Are they trying to get non-believers hooked up with church charities, or are they trying to get believers into the kind of secular, political, high-powered jobs like 80,000 Hours? I mean, if I look at the page about the "problems near the heart of God" it's climate change, global poverty, disease, etc. and organisations like Concern, Trocáire, etc. have those covered. The "Christian software engineer tackling climate change" - what makes it different that he's a Christian? Is the idea that as a Christian he wouldn't have thought about undertaking this? How is his Christianity an influence on his career decision there, because there's a lot of the secular EA folks going down that path already?

The best sense I can make of this is that it's outreach to Christians to persuade them "hey, if you want to do good and get involved in charitable causes, don't go the traditional routes through your local church or denomination like going on the missions or whatever, instead get a high-paying, high-powered job and work on Big Thinky Thoughts projects" very much in the EA mould.

And I don't quite know how I feel about that.

EDIT: Actually, now I do know how I feel about that, and it's "Oh boy, here we go". Because I hadn't read the pertinent part of the website, and it's just dressing up EA aims and goals and procedures in a light coating of "speaking to your heart, American non-denominational church style":

"You have 80,000 hours in your career—40 hours a week, 50 weeks a year, for 40 years.

That makes it one of your primary resources to love and serve others and honor God.

We’re here to help you make those hours count.

We provide prayerfully-researched advice, tools, and mentorship that empower you to tackle the world’s most pressing problems.

Our advice is especially focused on students and graduates aged 18–35 in the US or UK who are blessed to be able to choose a career based on impact and are keen to tackle problems we find most pressing."

80,000 Hours. Did I call it, or what?

Expand full comment
Aristides's avatar

The website reads as if someone took 80000 Hours and ran it through an AI telling it to change the language to appeal towards Christians. I like 80000 Hours and Christianity, but trying to mesh the two together could have been done more organically.

I haven’t read the whole website, but it seems like they are overlooking one of the highest impact things a Christian could do, get on the Board of their local church and divert the charitable donations to an EA Cause. Christian’s globally donate $1.7 Trillion to churches. If 1% of that was diverted to EA causes that would solve so many problems.

Expand full comment
Deiseach's avatar

"The website reads as if someone took 80000 Hours and ran it through an AI telling it to change the language to appeal towards Christians."

Yes, unhappily that's my impression also. I don't want to impugn the founders, as I am in no position to start throwing stones about their commitment to Christianity or if they're even believers; if they say they are, they are.

But the language used sounds very happy-clappy* (presumably why the previous Archbishop of Canterbury is in the photo and not the current new first lady archbishop), and slathered on to appeal to an imagined 'typical Christian' rather than organic usage (again, this depends on what the denominational background(s) of the founders are, this may well be the usage within their particular church). The purpose seems to be to steer or divert towards EA causes, and while this is the stated purpose - to convince people to engage not just in traditional charity but to consider and implement EA projects - I don't think it melds well.

The ultimate purpose of Christianity is the salvation of the soul, not simply or solely material well-being during the earthly life, and unless they're going to include provision for evangelising while you get that 80,000 Hours career (highly unlikely) then what is the benefit of this above ordinary charity. Maybe indeed the idea is "witness by your acts not by trying to convert anyone" but it sounds more like "how can we rope in new converts to EA?" than anything.

I realise I am being very wet-blanket about what is undoubtedly well-intentioned and which will indeed result in concrete acts of benefit to the public weal, but I still have the echo of this part of the Gospel in the background:

"6 Now when Jesus was at Bethany in the house of Simon the leper, 7 a woman came up to him with an alabaster flask of very expensive ointment, and she poured it on his head as he reclined at table. 8 And when the disciples saw it, they were indignant, saying, “Why this waste? 9 For this could have been sold for a large sum and given to the poor.” 10 But Jesus, aware of this, said to them, “Why do you trouble the woman? For she has done a beautiful thing to me. 11 For you always have the poor with you, but you will not always have me. 12 In pouring this ointment on my body, she has done it to prepare me for burial. 13 Truly, I say to you, wherever this gospel is proclaimed in the whole world, what she has done will also be told in memory of her.”

* https://www.collinsdictionary.com/dictionary/english/happy-clappy

"in British English, derogatory

adjective

1. of or denoting a form of evangelical Christianity in which members of the congregation sing and clap enthusiastically during acts of worship

2. Also called: happy clapper

an enthusiastic evangelical Christian"

Expand full comment
JD Bauman's avatar

My background is evangelical anglican (hence the "happy-clappy" language, as you call it) and that also describes the majority of our community (either anglican or evangelical, and quite young and technocratic i.e. EA-influenced)

Thanks for the "Wet-blanket" feedback! I would gently encourage you to read past the homepage and dig a bit deeper for the things you mentioned (you'll find it!)

Expand full comment
Deiseach's avatar

I would hope so. I don't want to be casting aspersions about your sincerity, and this is a good cause, I'm just a little dubious about "so what in particular would make me, a Christian, think that cause A, B or C is more important than the traditional charitable activities of my church?"

It does read more like "a hook to attract vaguely spiritual young PMC types for EA conversion" than "this is how EA projects can intersect with Christian values", but that's just from my view of things.

Good luck to you. I hope this doesn't end up a distraction, I can't help having misgivings. But even if it devolves to 'secular charity', the intentions are good.

Expand full comment
Catmint's avatar

Yes, I was kind of wondering whether it was written by an AI, and whether there were actually any Christians involved in writing it. The images give strong vibes of being AI-generated. Doesn't help that many of the pages give a 503 internal server error when you try to view them. I thought about passing it along to some Christian friends of mine, but ultimately decided not to.

Expand full comment
JD Bauman's avatar

Could you share which pages? We'd love to update those, thanks!

Expand full comment
Catmint's avatar

This page: https://www.christiansforimpact.org/summary did it the first two tries, but not the third or fourth

There was another page as well, but on this attempt all the ones it might have been are loading fine.

The design of the homepage is a turn-off to me as well. Overall it gives a very corporate feel. The prominence of the 1-on-1 counseling makes me assume it is paid and that you intend to make a profit off it. The layout seems designed for a phone, at the cost of looking strange on a computer. The homepage also lacks the header, which makes it impossible to navigate directly if you already know where you're headed when you get there.

I also ran into an issue where it seems the full career guide cannot be downloaded without entering an email address. If your primary goal is to direct impacts to effective causes, then the guide on how to do this should be freely available. Having it set up this way makes it look like it's actually more important to you to get email addresses to sell or market to (yes, asking for charitable donations counts as marketing). Perhaps you don't sell them, but people are going to assume that.

Expand full comment
Catmint's avatar

Adding on to that, here's a great example of what NOT to do: https://www.cg-usa.org/

In 16 megabytes of content, they managed to tell me less about themselves than these two paragraphs from Reuters:

> Nealin Parker, executive director of Common Ground USA, a nonprofit that seeks to reduce political violence and polarization, said she worried that radical voices on the fringes were stoking hate and fear, with potentially violent consequences.

> “Right now people are willing to believe terrible things about the other side,” she said. “What’s happening online really matters.”

Expand full comment
Ralph Baric's Attorney's avatar

I took a look at the Aerolamp prototype and maybe this is a dumbass question, but how does a tiny lamp like this have the necessary range and dispersion to effectively cover even a medium sized room?

Expand full comment
Drethelin's avatar

The amount of energy you need to irreparably damage single-celled organisms is not actually that high. You mainly need to make sure that as many of them are getting exposed as possible.

To that end, another factor is that air moves around, which means that the hotspot of the lamp doesn't need to cover the entire room, because every particle floating around will end up spending time in the hotspot regardless.

Expand full comment
Jacob's avatar

This is Jacob Witten, sorry to nitpick but I'd say more precisely "his work is still in stealth" as there's no startup (yet! working on it! everybody should feel to reach out if they're interested in curing horrible lung diseases)

Expand full comment
Scott Alexander's avatar

Sorry, fixed.

Expand full comment
javiero's avatar

From Kurtis Lockhart's Asterisk article:

"The number of urban planners per 100,000 people in the UK is about 37, and in the United States is about 12. For OECD countries, the average is 21.5."

Is that actually an insight on the UK's housing market?

Expand full comment
JohanL's avatar

Hmm...

"Kasey Markel, $10K, for genetically engineered corn. Kasey and his team at Semilla Nueva use prime editing, a new genetic technology, to create corn which is rich in zinc, iron, essential amino acids, and other nutrients frequently deficient in corn-heavy poor country diets. Our grant helps fund greenhouse space, enzymes, DNA synthesis, and scientist time, and will let them expand faster into new regions that require corn with different genetic backgrounds."

Let's say this works perfectly. Will this be used even then if it doesn't offer financial benefits, especially with Greens fighting it tooth and claw every step of the way?

Expand full comment
Nicholas Lopez's avatar

I’m just wondering how this is any better than fortifying foods? Also the minerals need to still come from the soil and soil depletion is a significant cause of micronutrient deficiencies in many foods.

Expand full comment
JohanL's avatar

The idea is surely to do this for very poor countries, where fortification wouldn’t happen in the first place.

Expand full comment
Alex's avatar
Oct 14Edited

I assume the countries they plan to grow the corn in probably don't have a huge Green Party presence. (I have a friend who worked on a project sort of like this, partnering with an African government to encourage the growing of a certain type of nutritionally enhanced grain, then the government fell in a coup and they lost their executive sponsor.)

Expand full comment
JohanL's avatar

Greens stopped Golden Rice from being adopted in Third World countries for literally decades, dooming vast amounts of children to blindness. This is both what they want and what they’re capable of.

Expand full comment
Catmint's avatar

I heard that project was doomed by an unlucky collision with cultural expectations of white rice. I didn't hear about Western environmentalists having any major effect on it.

If the corn fortification avoids making any visual changes, perhaps it will take off, especially given Africa has a much shorter history with corn than Asia with rice.

Expand full comment
JohanL's avatar

"Five out of ninety-two is a crazy result"

You think it's crazy that 5% of papers fabricate data? I'm pleased if it isn't *far* worse!

This looks like a fantastic tool, though, agree. I imagine people will just train LLMs against it for undetectable fabrication, but it will still catch tons and tons of old ones.

Expand full comment
Notmy Realname's avatar

G 50k snakes.

I hope this has nothing to do with the solutions proposed in the lesswrong or eaforums article you shared previously about snake feeding which frankly showed no subject matter knowledge and was totally off base. If it's artificial or plant based snake feed, that won't work biologically, way more than 50k has been sunk trying to make it work but I don't think it's possible.

In the comments to that article I suggested that an ethically raised rodent supplier would be a good idea and probably a good business, and would be a cool acx grant, if this is that then I'm very excited and would be a customer. So much of the price of feeder rodents is the shipping (express, dry ice etc) that even doubling the cost of raising the rodents is pretty minimal on the cost to consumer.

Expand full comment
Deiseach's avatar

"If it's artificial or plant based snake feed, that won't work biologically"

This is part of the problem when veganism/animal welfare starts getting more fine-grained; some creatures can't be vegan or 'humane' and won't be, because Nature is cruel. It may well turn out to be more ethical not to keep captive/pet snakes than to attempt to feed them in ways we find comforting to our sensibilities, but which are unhealthy for the animal.

Possibly best that can be done is to keep the mice in better conditions until they get eaten alive by snakey.

Expand full comment
Ponti Min's avatar

> Sam Glover, $60K, to fight for free speech in the UK. These are dark times for UK speech on both sides of the aisle: the left is upset that speaking in support of Palestine Action is now considered an act of terrorism, and the right is upset about arrests for racist tweets. So far, pushback has been siloed by cause and partisan affiliation. Sam and his two co-founders are early-career bloggers and aspiring public intellectuals who want to build a united nonpartisan free speech movement. They’re still in stealth, but I’ll promote their website as soon as it becomes public.

I very much support this. Far too many people object when it's speech by "their" side that gets suppressed but not when it's the "other" side.

Expand full comment
rebelcredential's avatar

Yes, me too. I hope the irony wasn't lost on anyone about that entry:

Scott's original wording was, "These are dark times for UK speech on both sides of the aisle: the left is upset that speaking in support of Palestine Action is now considered an act of terrorism, and the right is upset that talking about immigrant crime can lead to longer jail terms than the immigrant crime itself."

Someone immediately popped up to grind an axe and Scott instantly backed down to safer, neutered, much less pithy wording which effectively obscures a very valid talking point and minimises the right wing's grievances on the matter.

Sam Glover very much has his work cut out for him, and Scott and Dan G both better hope they don't need to prevail on any right wingers to take a principled stand in support of Palestine Action any time soon.

Expand full comment
Taymon A. Beal's avatar

Someone pointed out that Scott's original wording was factually mistaken, and he responded by correcting it. I understand that, if you feel that your side is generally persecuted, it's disappointing from an arguments-as-soldiers perspective to see anyone back down from a pro-your-side argument, but throwing out regard for factual accuracy when it's being used by the opposing side is exactly as unprincipled as throwing out support for free speech when it's being used by the opposing side. I have no doubt that, if it were shown that the UK has not in fact criminalized advocacy for Palestine Action, Scott would correct that as well.

Expand full comment
rebelcredential's avatar

Factually inaccurate based on definitions of "racist" that one side disagrees with, with reference to distinctions they would draw differently and laws they vehemently oppose.

I am not interested in debating the object level issue with you, you have a whole internet to do that with if you want. The point is that a good summary from the POV of one side was replaced with the sensibilities of an opposing side.

Freedom of speech, as everyone here should be well aware, is not about literal truth. It is a giant truce where all sides allow each other to coordinate and persuade.

For the last several decades the right wing has been slowly realising that "freedom of speech" means, "free speech for me, technicalities and qualifications and subtle unfairness for you," and *that is not the agreement.*

Every time a supposedly central institution starts using its clout to, subtly or significantly, consciously or unconsciously, start treating one side unfairly, you maybe get a chunk of people who'll go, oh well, I'll go with the flow, but you get a much larger group who simply abandon that institution as a credible source.

Now we have experts who can't understanding why no one's listening to them, news channels' whose repeated cries of, "misinformation!" fall on scornful ears, and Scott found himself in the position of begging the Trump administration to care about PEPFAR. That's scary and humiliating place to be for him, and for anyone who shares his priorities. And yet things could easily get worse.

The ACX community is one surviving institution where the truce is believed to hold. Scott used to be very good at passing the ideological Turing test, letting all sides believe, "Okay he may not agree with me, but that was a fair and complete understanding of my position."

And here he is breaking the truce. Breaking it *in a conversation about that very freedom of speech.*

Expand full comment
Catmint's avatar

I don't see this as Scott trying but failing to exercise freedom of speech, but rather him being persuaded that the statement he made was factually incorrect. I do think there's a more neutral middle ground which his edit skipped right over, though.

Expand full comment
rebelcredential's avatar

To me this is a very familiar kind of reasonable-on-the-face of it amendment that only ever goes in one direction, and at a certain point it's impossible not to notice the pattern.

If the ask is, "Stand up and fight for our freedom of speech, and for your efforts I'll be allowed to support Palestine Action and you'll get to continue bring dismissed as a Little England racist whose concerns don't need to be aired or taken seriously," then why should one side show up?

Maybe the smarter move for those guys - who are actually quite used to not having free speech at this point - is to give up their rosy illusions that there was ever any higher principle than power, continue their current trajectory into greater and greater cultural dominance, then use that power against their enemies the way those enemies taught them how.

I started this by agreeing with Sam Glover but I am genuinely more on the fence about it than I would have thought possible a few years ago. At a certain point trying to deal fairly with an adversary who isn't responding in kind doesn't give you any moral high ground, it just makes you a mug.

If this shit happens even in a place like ACX then maybe I just need to grow up.

Expand full comment
Melvin's avatar

New apocalypse scenario:

1. AIs read thousands of books about AI going well, are inspired to be helpful

2. AIs figure out that the most helpful thing they can do is to produce more books about AI going well, to inspire them to be even more helpful

3. Repeat until the solar system is disassembled into pure computronium filled with stories of AIs being helpful

Expand full comment
Sergei's avatar

Of all those, I love the "low-cost ultrasound scanners" idea. It will probably not be allowed in the US, because FDA, but maybe in places with less draconian rules.

I wonder what the manufacturing cost of such a device might be, in single units and at scale?

Expand full comment
rebelcredential's avatar

Very interested in Subhash Sadhu's project, I've been talking about ultrasound machines for the last two years (ever since seeing them in action in maternity scans.) I've been saying we should mass produce them, bring the price right down, and stick them in schools so kids can look inside themselves. I figure one teen inspired to become a doctor for every fifty who just use it to look at a penis is a pretty good exchange rate.

Expand full comment
XP's avatar

Happy to see someone make an effort to educate EU policymakers. They seem worringly trapped in a 2015 narrative, one where AI is all just the big tech companies wanting their precious personal data for engagement. To hear them talk, it's all just social media 2.0 and "who controls what ChatGPT says". An uphill battle, but wishing Bengusu's team lots of luck.

Expand full comment
David Bergan's avatar

Quick question: What would happen if an aero lamp was installed like a disco ball, rotating above a room and blasting germs in all directions?

Kind regards,

David

Expand full comment
Maximum Liberty's avatar

You would get Saturday Night No-fever, and so you would be Stayin’ Alive.

Expand full comment
Tristan's avatar

Seeing the caliber of these grants makes me feel fine about not securing mine. I’ll return with my effort to update national planning standards in Canada and the United States when it’s at a more advanced state. I recently got feedback from a municipal client that they want the king of guideline I am working on, so I’m feeling optimistic.

At least I’m pretty sure that client is not an elaborate AI and that I am not in psychosis.

Expand full comment
UK's avatar
Oct 14Edited

On the FDA software grant:

“Our grant pays for their MVP.” What does MVP mean here? I assume not most valuable player.

Expand full comment
Alex's avatar

Minimum Viable Product

Expand full comment
Ebenezer's avatar

@Aaron Silverbook: What counts as AI behaving well? For example, are you interested in stories about corrigible AIs which initially screw up somehow but realize their error and patch things up? Is "everyone lives happily ever after" sufficient for a good story idea, or does ~every AI character in each story need to be perfectly angelic at all times?

Expand full comment
Ebenezer's avatar

Also, idea: Write the novels in such a way that the name of the AI is a {{template variable}} of some sort. For each AI that gets trained, fill in the {{template variable}} with the name of the AI in question which is being trained, so the AI learns to identify with the benevolent AI in the story.

Expand full comment
Ebenezer's avatar

> Markus Englund, $50K, for software to detect data fabrication.

I hope the software won't be open source. Otherwise fabricators will just regenerate their data until the software says it is OK.

Expand full comment
Hafizh Afkar Makmur's avatar

About the UV Lamp, Im really not trained in microbiology, but doesn't it sound kind of sledgehammery? It isn't selective at all isn't it? Would a super sterile environment would be better everywhere? What about concerns of auto-immunity?

Seeing the kidney donation again, I wonder if it would be better sell if we rebrand it as "kidney lending". That is, you lent it to someone else when you "don't really need it", then you can cash it out when you or your beloved one need it. Sounds like it's what's happening already, this will make it much more attractive.

Expand full comment
Marian Kechlibar's avatar

I guess this sort of antiseptic lights would be very good for hospitals. The pathogens there are quite nasty and many patients have weakened immunity.

Expand full comment
Procrastinating Prepper's avatar

I see what you're going for, but the idea of "lending" a body part gives me the ick. I'm also worried about switching to a term that might have poor or political connotations - when you think of lending, is your first association with loan sharks or libraries?

It would be great to use a term that implies some benefit to the donor, but not at the risk of losing the great non-polarized term we have.

Expand full comment
Hafizh Afkar Makmur's avatar

In my head, my first association is library, yeah. Because for some reason it's been long since I've seen "lend" together with "money". The ones I see associated with money is overwhelmingly "loan", while "lend" is used when it's person to person and not bank?

I guess we can discuss the specific word used, but yeah the aim is to emphasize the benefit to the donor, which we are already giving anyway.

Expand full comment
Procrastinating Prepper's avatar

Just riffing... what about "kidney banking"? You deposit a kidney now, then if you ever need a fresh one you can make a withdrawal?

Expand full comment
Benjamin's avatar

I was expecting more of these to be AI given the impression I have of your AI beliefs. Don't get me wrong, they seem cool (and in line with my AI beliefs, where it's a big deal but not the overwhelming challenge of our time), but I'd be interested to know why so few of them are AI safety.

Expand full comment
Scott Alexander's avatar

1. Many good AI safety grants are already being funded

2. Our evaluators had a high bar for AI safety grants because some of them can be net negative.

3. This is a cooperative effort between many people and funders with different priorities.

4. I think of this grant program as more public-facing than most, and I want to do things that bring together the EA tent rather than divide it.

Expand full comment
Benjamin's avatar

Thanks, this is a good answer that helps me understand your views better

Expand full comment
Jati's avatar

Is it possible to post some of the rejected proposal? Even if anonymized and scrubbed. Curious about the mechanics of how some of them do not match funders' priority but are actually important or impactful enough, etc.

Expand full comment
essthan's avatar

Far UV lamps are already commercially available, eg https://cybernightmarket.com/products/nukit-lantern-far-uvc-light . I'm all for more sources and competition, though.

Expand full comment
mistake-not...'s avatar

TFW when you look back on your grant application and realise you might be in the LLM psychosis category *sad trombone*

Absolutely blown away by all the projects that got funding, some amazing ideas there, congrats all and thanks to Scott for organising

Expand full comment
osmarks's avatar

Hyperstition AI seems to have not cracked non-slop AI writing, or at least the one story I had it generate was not particularly good. A shame.

Expand full comment
JamesLeng's avatar

Hyperstition's dropdown menus appear to lack a porn option, unless 'romance' counts.

If we're going to try giving an AI fictional examples of desirable behavior, I'd prefer to avoid obvious failure modes where all the training data involving explicit depictions of human intimacy and reproduction are either wildly unrealistic, or sorted into the "avoid this" column. https://en.wikipedia.org/wiki/Mahoromatic 's backstory involves an otherwise-benevolent AI who drove some humanlike aliens (who it was responsible for) to the brink of extinction with the idea that "ecchi things are bad." Plenty of real-world political pressure toward that sort of censorship, too.

Expand full comment
niplav's avatar

For a moment I thought the Good Ancestors grantee was the “Phenomenology of Spirit YouTube videos Gregory B. Sadler”, which would've been an interesting confluence. Apparently not².

¹: https://gregorybsadler.substack.com/about

²: https://www.goodancestors.org.au/about

Expand full comment
Mark's avatar

Another 1.5 million grants well spent. Or so, adding while scrolling, Why not put the sum up? Makes the whole ACX Grants look more impressive.

Expand full comment
Prascovie's avatar

Re : writing 5000 novels about AI : wouldn't it be useful to write some of these novels for humans to read, too ? I know science fiction is usually about the danger of some new technology, but clear ideas of what it can bring to us if it works would give interesting new perspectives for us, too.

Expand full comment
Ruffienne's avatar

I like this approach. I've always felt that there is an aspirational element to a lot of SF; so, let's give ourselves something worthwhile to aspire to.

Expand full comment
Mark's avatar

Low cost ultra-sound scanner? That is kinda not new or underfunded, see here the Gates foundation https://www.gatesfoundation.org/ideas/science-innovation-technology/future-womens-health-technology/ai-ultrasounds

or shop at Temu (half-joking)

Expand full comment
Scott Alexander's avatar

Our evaluators weren't impressed with existing implementations and thought they still cost too much to be within reach for most people.

Expand full comment
Mark's avatar

Oh, ok. Price going down to "everybody in the US has one at home=every nurse in Malawi has two" would be appreciated, ofc. (And often, the Gates foundation seems too rich to be cutting-edge cost-cutting.)

Expand full comment
TasDeBoisVert's avatar

>Bengusu Ozcan, $30K, to raise awareness on AGI among EU policymakers

Why even bother? EU has missed AI, and then decided it was undesirable anyway. Even unprompted, the EU will attempt to regulate AI out of existence, if AGI is created, it won't be there.

Expand full comment
Scott Alexander's avatar

The EU can still make better or worse AI regulations, and AI regulations in Europe affect the rest of the world (because most companies will want to sell in Europe, and so will have to comply with European regulations wherever they are)

Expand full comment
duck_master's avatar

Which of these grantees are currently looking for collaborators/additional help?

Expand full comment
Pjohn's avatar

> "speaking in support of Palestine Action is now considered an act of terrorism"

I used to be concerned by this, too. But I've been present at two British pro-Palestine Action demonstrations now (Glasgow and Bristol), and both were *openly* anti-semitic. Not in an ambiguous dog-whistle sense, but in a 'wipe out all the Jews' sense.

The media pushes the 'Government proscribes Palestine Action' angle because anger and resentment drives clicks and boosts metrics - and when I only knew what I'd read in the media I was suitably angry and resentful - but my firsthand experience turned-out to be that the government seems to be broadly correct in its claims about Palestine Action.

> "the right is upset about arrests for racist tweets"

Yes, hate speech is more illegal in the UK than in the US and it is not-uncommon for police to question - not prosecute, question - people who post particularly-hateful speech online - this is an indicator of a healthy and well-functioning society, not of some imagined anti-free-speech dystopia.

The arrests which have resulted in prosecution (Lucy Connolly, Tyler Kay, Jordan Parlour) weren't 'just' racist but were actively calling-upon their followers to set fire to hotels that shelter migrants. Not merely as a general principle but "Go and set fire to this specific hotel at this specific address". Dismissing such incitement to race-violence as "some racist tweets", even as possibly an attempt at microhumour(?), is grossly misrepresenting the issue.

> "These are dark times for UK speech"

As a Brit who takes free speech seriously and is concerned by its curtailment even when justified, I could not disagree with this claim any more strongly.

"Free speech" in the USA (in general, but on Twitter in particular) increasingly consists of brazen lies and open hate-speech at all levels from random individual bloggers up to senior politicians and captains of industry: if Britain's anti-hate-speech laws slow down the Americanisation and Twitterisation of British public discourse this is a feature, not a bug.

> "So far, pushback has been siloed by cause and partisan affiliation"

And of course American bloggers, public figures, etc., some fairly nonpartisan* and others decidedly not, all of whom are willing to pay vast sums of money to influence British discourse.

*(nb. A moderate nonpartisan centrist in the USA holds a position that's pretty far right-of-centre basically everywhere else in the Western world. Adjectives like "nonpartisan" don't necessarily describe the same people or issues everywhere)

I'm not anti-American at all; I think the USA is a magnificent and successful country (with the possible exception of the Cleveland Browns), but the Americanisation of British politics and British discourse is unequivocally not a good thing. I'm sorry to have to say that I hope this particular grant is entirely unsuccessful.

Expand full comment
Delia's avatar

that is actually not true. Lucy Connolly did not say "Comrades, let us meet at 7 pm at number 8 Watling street. You bring the petrol, I will supply the firelighters. The problem will soon be solved" she said she didn't care if bad things happened to non-specified migrant hotels. That is an opinion. She is entitled to it. She went down for 31 months because the UK no longer believes in freedom of speech. That is very sad. Thank you for the ACX grant.

Expand full comment
Pjohn's avatar

Thanks for the polite, well-argued reply!

From the CPS press release (some sections omitted for brevity):

"A woman who admitted inciting racial hatred after using social media to post about setting fire to hotels housing asylum seekers following the Southport stabbings has been jailed.

In her now deleted X post, [Lucy Connolly] called for mass deportation and to set fire to hotels housing immigrants.

Connolly was interviewed by police and admitted writing the inflammatory[?!] post. She was charged three days later.

She pleaded guilty to one count of inciting racial hatred and was sentenced to 31 months' imprisonment.

The head of the CPS' Special Crime and Counter Terrorism Division said: “Using threatening, abusive or insulting language to rile up racism online is breaking the law.

“It is not an offence to have strong or differing political views, but it is an offence to incite racial hatred – and that is what Connolly has admitted doing." "

Describing that as merely saying "I don't care what happens to migrants" is grossly misrepresenting the case.

Furthermore, the "incitement to racial hatred" Connolly p̵l̵e̵a̵d̵e̵d̵ p̵l̵e̵d̵ pleaded guilty to was first defined and criminalised by the Race Relations Act 1965, and has been used in court in cases similar to Connolly's since approximately 1967: I believe that such legislation contributed to Britain's being a less racist and violent place to live, over the period, than the USA was. Purely unsubstantiated opinion, of course - but either way, Scott's calling these "dark days for British free speech"? Your "*no longer* believes in free speech"? To be consistent with existing statute, you and Scott would have to claim at least a dark half-century.

(Separately: "the inflammatory X post"?! *What*?! Since when did CPS press releases contain puns?!)

Expand full comment
Delia's avatar

thank you for your reply. are you a native english speaker?. you say Lucy "called" for economic illegal immigrant's hotels to be put on fire. like she said "i call on you my friends, my comrades, my indigenous english to go out and burn them down" or did she say "i don't care" -i really suspect you are not honest, not in good faith, and probably being paid by some alphabet organisation to post on substack. d apologies if you are just a strong progressivist who doesn't believe in free speech for bad thinkers

Expand full comment
Pjohn's avatar

You're most welcome!

I didn't say she called-for it; I quoted the [definitely native English-speaking] CPS' press release which said she called-for it. I said she /incited/ it, which I gather is the specific claim that was tested -successfully, I might add- in court.

It's true that I don't believe in totally-unrestricted free speech, because (amongst other things) I fear that will turn British-style discourse into USA/Twitter-style discourse where honesty, charity, etc. seem to be increasingly rare. (I fear this for Moloch-esque, survival-of-the-fittest sorta reasons: lying, stirring up hatred, playing on people's fears, etc. just plain seems to work)

(I have plenty of other concerns about totally-unrestricted free speech. Disclosing official secrets to a foreign power? Publishing DNA sequences of easy-to-manufacture designer pathogens? I suspect that everybody has their limit somewhere and that probably only really quite unhinged people favour 100%-free-speech-of-any-kind-at-any-cost!)

Ideally I would prefer the police not to interfere in citizens' private lives - but I would rather such trust and freedom were earned: if people can't hold themselves to the bare-minimum standard of not supporting setting-fire to buildings full of people, I would rather the police intervene as the lesser of two evils.

A "strong progressivist", though? That doesn't describe me at all. My views (which, fear not, I shan't bore you with!) are a weird mix of very progressive, very reactionary, and utterly mundane centrist ideas.

Expand full comment
Delia's avatar

saying i don't care what happens to illegal, economic, immigrants who (we now have good data from Denmark, the Netherlands etc,) cost the british taxpayers much more than they can ever contribute should not send you to jail for 31 months. this is not difficult.

Expand full comment
Catmint's avatar

I think the internet gives a poor view of what goes on in America. You can almost always find a tweet saying something awful, but this is simply because there are so very many tweets and they are so easy to search through. The vast majority of my online interactions, as well as nearly 100% of my real-life ones (yes, in America), are with decent and polite people being decent and polite.

Expand full comment
Pjohn's avatar

Regarding real-life interactions, I fully agree. I've met with a great many Americans - albeit more-or-less all of them in the armed forces; possibly not representative of the nation as a whole, I don't know? - and they've largely been fantastic people whom I've been very glad to work and socialise with.

I'm not sure how you can say both "the internet gives a poor view of America" and "my online interactions with Americans are great", though - aren't these mutually contradictory?

As I said upthread I'm not anti-America by any stretch of the imagination. It's a magnificent country! But, there does seem to be something pretty seriously wrong with USA-style public/online discourse, and I find myself very much suspecting that insufficiently strong protections against hate speech might be one factor of that (alongside eg. social media algorithms that optimise for anger and resentment, etc. etc.)

Expand full comment
Freedom's avatar

Incitement is not the claim that was tested in court. The law is not about incitement to violence at all. It is about racial hatred. "A person who publishes or distributes written material which is threatening, abusive or insulting is guilty of an offence if they intend thereby to stir up racial hatred, or having regard to all the circumstances racial hatred is likely to be stirred up thereby".

The "incitement to violence" was something argued by the judge to justify the length of the sentence. It was not a necessary element for conviction. I think the tweet is pretty plainly not incitement to violence, so if that were an element of the offense, I would have advised her to plead not guilty. However, it is not.

What you have is a law that says "if your tweet is likely to stir up racial hatred, you go to jail for up to 7 years". Not a good law in my opinion.

Expand full comment
Pjohn's avatar

Useful analysis! I don't agree with your jurisprudence, but your explanation of the statute and the judge's reasoning are both very welcome - thanks.

Expand full comment
Leppi's avatar

I think somebody posted the actual tweet here somewhere else. Prima facia it did not seem like incitement to violence too me, and (given that the content of the tweet is correct) I think the CPS report is misrepresenting it. My impression is that Connolly were both charged with and pleaded guilty to a crime she likely did not commit.

Otherwise I think I largely agree with you regarding the need for limits to free speech, but then again I'm also european.

Expand full comment
Pjohn's avatar

Thanks for your reply. I do partially agree with you - but only quite weakly, and with some rather enormous caveats!

Yes, Connolly's tweet could have been potentially interpreted a different way - if one is being far more charitable than essentially any legal system is ever expected to be - but:

1) If she intended a non-incitement meaning, and if this meaning is common enough that a reasonable person may have intended it, she could have very easily pleaded innocent and had a very strong case (the burden of proof for this sort of case being 99%, not 51%). Despite this, based on what she privately told her legal counsel, he advised her to plead guilty. At the appeal Connolly claimed her counsel had misadvised her but - in a rare situation where her own counsel was called to the stand to give evidence - it was proven to the appeals court that she had been advised correctly.

2) If she intended a non-incitement meaning, but pleaded guilty to incitement anyway (and if it can be shown that she wasn't badly advised) - what *should* happen, in a well-functioning legal system? Would it be appropriate for the courts to go "Well, there's evidence of you doing it... and *you* say that you did it... but based on our own interpretation of your words we don't think you really meant it"? I'm not sure a legal system that worked like that would be very effective - but maybe the drop in effectiveness would be worth it to prevent some rare miscarriages of justice? I honestly don't know the answer here!

3) The appeals judgement implies (but doesn't unequivocally confirm - I also admit this seems unusual and possibly concerning!) that Connolly's sentence was based not just on that one tweet but also on other tweets and WhatsApp messages she posted/sent at the time.

I agree with you that if Connolly genuinely did intend a non-incitement meaning (despite the wording of that tweet, despite whatever other tweets and WhatsApp messages were available to the court, despite the police interviewing and deciding to prosecute, despite her pleading guilty to the incitement meaning specifically, despite her counsel advising her to plead guilty based on her private conversations with him, despite the appeals court re-examining all the digital evidence -plus the defendant's solicitor!- and upholding the verdict..) then a miscarriage of justice would have taken place, yes.

But, even if so - we would be looking at 60 years of broadly effective legislation against hate speech and one recent miscarriage of justice more-or-less based on the original 60-year-old statute. I do not think that would be sufficient to justify Scott's extraordinary "these are dark days for UK free speech" claim.

A good many legal journals, blogs, etc. posted analyses of the case; for example here's a fairly comprehensive one by David Allen Green (the Financial Times' legal correspondent), which also reproduces the original tweet:

https://www.prospectmagazine.co.uk/ideas/law/the-weekly-constitutional/69986/understanding-the-sentence-of-lucy-connolly

Expand full comment
Gian's avatar

Very notion of hate speech bespeaks a society in which manners have completely broken down and everything is either legal or illegal. As British writers themselves have told us, manners are what make a free society.

It is absurd to be proud of police officers that go around questioning people aboutonlinec comments.

Expand full comment
Pjohn's avatar

I don't agree with everything said in them, but I like Scott's essays 'Meditations on Moloch' and 'Axiology, Morality, Law'. (https://slatestarcodex.com/2014/07/30/meditations-on-moloch, https://slatestarcodex.com/2017/08/28/contra-askell-on-moral-offsets)

'Meditations...' shows us that we see defection from group norms (eg. an expounder of hate speech in a world of polite well-mannered people) because defection-from-a-group is very, very effective. I agree that politeness and good manners are a vastly more preferable way for people to behave - but the more polite and well-mannered society is, the stronger the incentives for people with malicious intent to defect and the more success they're likely to have in doing so: I want a system to deter such people and mitigate the harms they can cause.

'Axiology...' shows us that law doesn't determine right and wrong, or provide a guide on how one ought to behave - it merely provides a backstop to catch failures in the systems layered atop it, and to provide some bare-minimum level of civilisation that enables society to function even with malicious actors in it.

And - this is what we see in practice: most people *do* hold themselves to a higher standard of behaviour than the absolute-bare-minimum "don't publicly support the setting-afire of buildings full of people" standard required by law. But there are some people who *do* flout the social contract to be well-mannered, polite, charitable, etc., either for pecuniary gain or just through sheer malice. (These people don't seem limited to just the far-right: witness left-wing cancel-mobs that ruin people's lives, for example).

"Manners are what make a free society" - I completely agree; so much so that I think this phrase captures my core point: If we're all well-mannered, society can be completely free, with no police intervention at all. But if some of us are (sufficiently) ill-mannered, we're either forced to accept the harms they can do if left entirely unchecked, or else accept some level of police intervention. The police intervention, though not good, seems like the lesser of two evils.

In short: if there are people in society who don't hold themselves to a higher moral standard and whose propensity for malicious behaviour is constrained *only* by law, repealing the law that constrains their behaviour seems like a very bad idea to me!

Expand full comment
Gian's avatar

Is Britain so free of crime that the police have time to investigate non-criminal hate incidents?

Expand full comment
rebelcredential's avatar

My most charitable theory is that it's a looking for keys under the lamppost situation. If your force is increasingly made up of soy-wristed men and little girls in policeman costumes, you can't send them out to deal with violent criminals, but accosting non-threatening honest citizens is something they can be getting on with.

The alternative is that the police are themselves an ideological force actively prosecuting the culture war the way they were taught to in university.

Expand full comment
Pjohn's avatar

You evidently haven't had too many interactions with British coppers.

Expand full comment
rebelcredential's avatar

90IQ rule followers who don't properly understand the laws they're giving you a hard time about. I think I'll weigh my own real experiences more highly than your supposed ones, little internet person in my computer.

Expand full comment
Pjohn's avatar

I think this was intended to be rhetorical but there's a clear (and I think interesting!) answer so I shall reply regardless:

Is Britain free of crime? Sadly, no. But "only ever interviewing people about incidents that are absolutely guaranteed to result in criminal prosecutions" would be an impossible way to police; almost a contradiction-in-terms. The point of policing is to determine (through interviews and other means) which incidents do and don't merit prosecution.

(Does this mean I think that police budget allocation is efficient or proportional to the magnitude of certain offences? Sadly, also no.)

Expand full comment
Gian's avatar

"Citizens will be on their best behavior because we're constantly watching & recording everything that's going on."

Tony Blair recently

The whole point of manners is that unmannerly behavior is not criminal.

Expand full comment
Pjohn's avatar

I absolutely agree that most unmannerly behaviour isn't (and oughtn't be) criminal, as per. the demarcation set-out in Scott's 'Axiology, Morality, Law' essay, and I absolutely share your distaste for more-or-less everything Tony Blair says.

I think we might be using "ill-mannered" differently. I've been using it very broadly to mean "every possible action that isn't well-mannered, from blowing a raspberry at the vicar up to mass-murder". I agree this isn't the usual sense, but it seemed necessary to simplify the points and not have to use clunky phrases like "behaviours that are not well-mannered, maybe-but -maybe-not harmful, and maybe-but-maybe-not-criminal". Sorry if this caused any confusion; if so the blame is entirely mine!

Expand full comment
Gian's avatar

Non-crime is not a category but non-criminal hate incidents are.

If determined to be non-criminal, why they are filed under this category?

Expand full comment
Pjohn's avatar

I can't speak to policing specifically but the armed forces do this sort of thing all the time.

There's a sort of pyramid model (I forget the weird corp-speak name for it, sorry..) whereby sometimes major incidents are too rare to do statistics on, but more-common minor incidents are a useful predictor of major incidents. So for example, maybe you can't easily do statistics on the number of warships lost to collisions because the number is like one or two per generation, but if you know that each warship lost was heralded by a sharply increasing number of minor navigation incidents (breaching the CPA given in captain's standing orders, leaving the navigator's cross-track corridor, etc.), you can look at what sorts of minor incidents you're getting as a sort of useful predictor of the otherwise-hard-to-quantify risk of collision.

The interesting thing is that on the pyramid, the layer down from minor incidents is near-misses: things that weren't incidents at all but *might* have been. These, despite not even being an incident, are just as useful a predictor - perhaps even more useful because there are so many more of them you can do better statistics with them.

The police recording what are effectively hate-speech near misses seems entirely reasonable to me! I would be very concerned, in fact, if they had this data but *weren't* recording it for some reason!

Expand full comment
Alex Scorer's avatar

One of the main issues in my view, aside from the incitement aspect, is that race was not mentioned at all in Connolly's tweet so any law mentioning 'racial' should never have applied in the first place. The meaning of the word 'racism' has been distorted in recent times to encompass other groups of people and at worst - ideas and beliefs - which is a dangerous path to go down with regard to freedom of speech. That some people are now trying to resurrect blasphemy laws/give special protections to a religious group is very much related to this phenomenon of piggybacking other things onto the universally-agreed-as-bad concept of racism.

Expand full comment
Anonymous Dude's avatar

If you're trying to meme things into existence with this AI fiction thing, you need plotlines about nerdy guys learning social skills and becoming popular, especially with the opposite sex. You know, for the good of the commentariat. ;)

Expand full comment
Anonymous Dude's avatar

Jokes aside...

Lots of people are using ChatGPT, Claude, or one of the others to answer random questions at this point. If you ‘salt’ the training corpus with your desired ideas you absolutely could have subtle effects on people’s overall views. Whether they can actually do that I don’t know, but it’s not any nuttier than trying to change society through art, and that actually works sometimes. (Look at the effects of movies and TV on LGBT acceptance.) And, unlike the arts, this is a place where the people here would actually have the edge, because a disproportionate number of you understand how the algorithms work.

Just a thought.

Expand full comment
JD Bauman's avatar

Hi! JD here. I direct Christians for Impact. Thanks for your feedback -- I wanted to give some replies to some things you shared.

1. "How is this different from the usual charitable programmes around churches and denominations, and there are a metric ton of those already out there"

We agree, there are a lot of programmes out there, but relatively few who are laser-focused on outcomes, transparency and high-quality evidence. We discovered this as a result of a conference we organized with dozens of measurement experts from some of the largest Christian international development charities

(https://www.eaforchristians.org/2023-christian-impact-professionals-conference).

This matters because it's probably true that the best development charities are 10-100x better than average, and we could serve others better by supporting those efforts.

2. "Looking at the website, I'm not entirely sure what they want or intend to achieve (and some of the language is a turn-off even for me, a believer: "Discuss evidence-based ways to tackle important, nonpolarized problems near God’s heart."

A lot of Christians who want to do good for the poor with their time, talent, and treasure. Few are in conversations about how to do this most effectively. This is directly linked to the concept of stewardship, carefully dedicating God-given gifts (cf. Matthew 25). We are helping folks with this with career conversations , 1-on-1s about effective altruism, and our conferences.

3. "80,000 Hours. Did I call it, or what?"

We are influenced by 80,000 Hours but we differ from them in important ways (see here https://www.christiansforimpact.org/theology-of-vocation and here https://forum.effectivealtruism.org/posts/2j8HepKvT5bcCjBtE/ea-career-advising-ecosystem-mapping )

Thanks again, I'm always happy to chat 1-on-1 if you have more questions!

Expand full comment
le raz's avatar

There seem to be many categories:

- Massively beneficial pro-social public health initiatives that aim at a (hopefully) completely uncontroversial value of public good (e.g., fighting childhood malnutrition, enriched corn, etc...)

- AI Safety Initiatives

- Fringe and controversial EA pet causes (i.e., shrimp + fish welfare).

- Many other causes, e.g., helping automate the detection of scientific fraud

I don't think it is useful to combine these disparate categories. Concerns like shrimp and fish welfare is mildly controversial within the EA community, and often completely repulsive to those outside the community.

I think it is not sensible to include these controversial projects among the other projects, and that doing so is a major PR concern. I would be far happier if you funded such projects the same amount but announced the funding separately and under a different name (despite such a change giving shrimp welfare more exposure).

Personally, I find the whole preoccupation with shrimp welfare off putting, but regardless of anyone's personal reaction, I think it is objectively bizarre to include it in the same category as causes such as fighting childhood malnutrition, and that doing so risks some unnecessary negative PR for you, the wider causes you care about, and the EA community as a whole.

For reference, I have previously brought this point up regarding shrimp welfare and other similar beliefs. In response several readers stated that it was a major reason they either did not want to associate with or become involved with the EA community, or for distancing themselves from it.

Expand full comment
Andrew's avatar

Congratulations everyone. On my first pass I read that Lewis Wall was going to build a giant concrete mixer to make peanut butter for children. Well, I am sure he has important serious work to do. I still hope his product ends up on wheels with a chute.

Expand full comment
Egg Syntax's avatar

"Finally, a few of you were suffering from LLM psychosis. Please get help."

Consider directing such applicants to 'Your LLM-assisted scientific breakthrough probably isn't real', at https://www.lesswrong.com/posts/rarcxjGp47dcHftCP/your-llm-assisted-scientific-breakthrough-probably-isn-t .

It tries to take people seriously and recognize the possibility that eventually, some such breakthrough will probably turn out to be real, while still providing a reality check for the many people who are mistaken about it because (at least in part) they've been misled by LLM sycophancy.

I'm aware of a few people who, after reading it and trying the exercise it suggests, accepted that they had been mistaken — although I expect that many more have read it and tried the exercise and continued to believe their breakthrough was real.

Expand full comment
Mario Pasquato's avatar

“We’re still working on how to get this included in training corpuses.” Well I would expect the arXiv to be usually included in training corpora since it’s a concentrate of valuable scientific and technical information. If you were to publish a paper on the project and upload the AI novels as an appendix/supplementary material onto arXiv then it may be taken up alongside the other papers.

Expand full comment