288 Comments
Comment deleted
Dec 5, 2023
Comment deleted
Expand full comment

"But their most recent proposal allows ECPs (eligible contract participants) to invest up to $100 million, and I think this is just way too high a limit to start playing with, and I worry concerns about manipulation actually become quite plausible."

Well how can that possibly be risky, it sure isn't like we've seen a magic money machine founded on Rationalist pet project lines with big sums of money floating around going very publicly "ka-boom!" in the press recently with attendant trials upon trials 🙄

Expand full comment

Re Argentinian economy, there is a difference between Dollarized and Fully Dollarize where they try to Dollarize but it doesn't quite take and they end up with a mix of dollars and the old currency (pesos)

Manifold wants the dollar de jure in place, Metaculus wants the peso de facto gone

Expand full comment

Thanks; I've added a link to this comment into the text.

Expand full comment

Happy to. You could also interpret the spread as "Milei tries to Dollarize Argentina but it doesn't work and/or he is replaced by somebody who rolls it back", which I think is quite possible

Expand full comment

What do you call it when the currency is pegged to the dollar at a fixed exchange rate?

Expand full comment

A currency peg. They work as long as the economy is stable and the government is trusted, because they rely on the central bank buying and selling currencies to keep the actual rate equal to the posted rate, if they fail to do so there will eventually be a black market at the actual exchange rate and the peg will fail.

Expand full comment

Author of the Metaculus question here. I believe this is correct.

Expand full comment

Taylor Swift has been a big pop star for a while, but now she's a billionaire pop star.

https://www.foxbusiness.com/entertainment/taylor-swift-passes-billionaire-status-music-industry-era-artists-losing-millions

Also, smacking Hollywood by bypassing them to put her concert film in theaters and grabbing some of the NFL's spotlight makes it a better year for her.

Expand full comment

Also, she has reached the level of being called out by name in a regional Federal Reserve Bank report…

«Despite the slowing recovery in tourism in the region overall, one contact highlighted that May was the strongest month for hotel revenue in Philadelphia since the onset of the pandemic, in large part due to an influx of guests for the Taylor Swift concerts in the city»

Expand full comment

Something has definitely happened to Taylor Swift over the past year that has taken her from being part of the cultural background noise, easily ignored, to a huge player that you have to work hard to ignore. Check Google Trends if you don't believe me, there was a spike last October, and then a steady climb from March 2023 to current levels where she's a more popular search term worldwide than "sushi".

I can only attribute this to the fact that (a) she's touring and (b) she apparently has the best PR people in the world. At some point earlier this year I noticed that articles on totally unrelated subjects started talking about Taylor Swift for some reason, and it's only got worse since then.

Expand full comment

Obligatory link to the best analysis of the TS phenomenon, for some weird values of "best": https://samkriss.substack.com/p/taylor-swift-does-not-exist

Expand full comment

I love Sam Kriss so much.

Expand full comment

What on Earth did I just read

I don't know how much of it to believe... it almost seems like a fanfiction of reality, but the Kerfuffle stuff is too notorious for this story to have been invented whole cloth, i think...

but gosh, what a read. thank you so much for the link

Expand full comment

That's definitely Kriss' style. Somewhere between 50%-75% of what he writes is poignant schizophrenia.

Expand full comment

If you enjoyed that one, don't miss his previous exploit, called "Live from the hate march", on the same substack.

I appreciate the expression "fanfiction of reality". I can almost see the snarky headline on some normie blog, "Internet rationalists discover the existence of literature - now called fanfiction of reality".

Expand full comment

Is Kerfuffle real? I'm used to Sam making things up and Googled it to see if I could find anything about it, but I didn't.

Expand full comment

When I was reading it, it tripped some memories that I had of something that seemed like it might fit the bill, something about an indie zine owner getting in huge trouble, but in retrospect I might have been Mandela Effecting myself

Expand full comment

tl;dr: Swift never sexualized herself unlike Britney Spears, Katy Perry, or Miley Cyrus.

He then goes off on a bizarre tangent about Taylor Swift's navel and how he thought she was Lilith.

IMHO:

I actually find the 'never showed off that much skin' theory credible: mild feminism and attractiveness without excessive sexualization would allow her to appeal to the center-left and center-right respectively, creating a bipartisan audience that rarely exists anymore. But *that* spin is mine.

Expand full comment

Hanania had a few posts about pop music that were interesting to me as someone who stopped following pop music around the same time as him (early 00s).

https://www.richardhanania.com/p/changes-in-music-1998-2022

He argued that white female pop stars have for the most part aggressively *de-sexualized* (i.e. androgenized) themselves while at the same time not really singing what we would recognize as a "love song" anymore.

To the extent this is true (and I'm too ignorant of pop music to really know), we might say Taylor Swift is the last pop star. She sings romantic love songs (and break-up songs, but ones that still hold romance and love in high regard) while looking pretty but not pornographic, and she does it highly competently. There used to be a lot of competition in this space, but perhaps now there's basically none?

Expand full comment

I think I read that. I don't know enough about pop music anymore to say. Sounds credible. Hanania's a smart guy, even if he says weird stuff sometimes. Maybe she identified an underserved market and went for it.

Expand full comment

I'm skeptical on the last sentence. I think it's exceptionally rare for musicians to pivot hard in a musical direction that goes against their instincts in pursuit of commercial success, and to then succeed for any length of time. I'm aware of two credible alleged examples: the bands Bush and Sugar Ray. In both cases, they didn't really last long.

There's a documentary about Kenny G, and it's funny because they interview all these "jazz experts" and they all hold him in contempt, describing him as a complete sellout who saw the market opportunity from watering down jazz music to make it palatable to the masses. And they interview Kenny and he basically says, "I'm really not that smart. I just make the kind of music I like to make." I believe him.

Expand full comment

No. This is completely wrong. Billie Eilish, Olivia Rodrigo, Ariana Grande, I can keep listing names but I can say if your summation is correct Hanania is ludicrously wrong again. Like, the main most popular new white female pop stars still sing mostly about romance and look conventionally attractive.

Expand full comment

Billie Eilish - Richard describes her as "going out of her way to look less pretty than she otherwise could" and based on an image search this seems apt to me. I also didn't think she really sung about romance; based on her debut album cover art and some vague things I heard, I thought she was positioning herself more like a young female Marilyn Manson, but I suppose I'm wrong there. But I look at her album tracklist (and again, that cover art) and see titles like "all the good girls go to hell" and "wish you were gay" and we're clearly in very different territory from Taylor or any conventional romantic songwriter of the past.

Ariana Grande - Maybe she bounces around stylistically, but I Googled her top songs and the first one that popped up was called "7 Rings". If you look at the lyrics and music video, it clearly falls into the "bad bitch" style that Richard describes.

Olivia Rodrigo - I hadn't heard of her, maybe a more promising example than the others. Though she's apparently half-Filipino, doesn't qualify as "white" in the US.

EDIT: Just looked at Richard's piece again and he DID mention Rodrigo as an example of someone who produces a new-style breakup song that is indicative of helplessness and possible mental illness, as opposed to the breakup songs of the past. Or, we might say, those of Taylor.

Expand full comment

*Somebody* has been studying his Gonzo Journalism. The spirit of the Hunter rises again.

Expand full comment

Yep. I don't actively look for anything TSwift related, but it seems like, in terms of career success, she's gone from "probably the most successful female artist of this generation" to "almost certainly a top 2 most successful artist ever" (alongside The Beatles).

(Also, Scott says, "she's been successful for the past 10 years as well" but frankly she's only become more successful over time *and* entirely changed genres, despite already being quite popular in 2008, which is actually quite rare among musicians. Who else was big in 2010? What are they doing now?)

Expand full comment

Isn't there a strong case for Beyonce being nearby?

Expand full comment

I haven't checked carefully but I suspect if you look at the numbers and the records that Swift set, it's not very close.

Expand full comment

Sit down, Kayne.

/s

Expand full comment

Imma let you finish...

Expand full comment

As for albums sold, Chartmaster puts Swift in 16th place. Above Mariah Carey, but below Rod Stewart. However she's moving up fast, and it's expected she'll knock Celine Dione out of 14th place soon. The only solo female artist above Dione and Swift is Madonna. She's in 5th place.

https://chartmasters.org/best-selling-artists-of-all-time/#:~:text=The%20Beatles%20are%20the%20best,solo%20artist%20of%20all%2Dtime%3F

Expand full comment

I'm not convinced lists like this do a good job of accounting for changes in how music is consumed over time. I know they make an attempt, but even the "selling" in the phrase "best selling artist" isn't really accurate when so much is available for free. I'm also not convinced that the way they account for international developments is very good, since they rely on a separate organization in each country, each of which might do things differently.

Different lists aren't even consistent. For example https://en.wikipedia.org/wiki/List_of_best-selling_music_artists has Swift in 9th (counting by "total certified units) with Madonna (who's been active for far longer) and Rihanna above for female artists, but this metric would have Rihanna as by far the most successful artist ever. Mainly I wanted to link it because it also gives numbers for "claimed sales" which vary wildly within each artist (when multiple numbers are given), differ dramatically from "total certified units", and are only presented with very approximate values, highlighting the difficulty of measurement in this category.

Expand full comment

I agree with you for the most part. Also, remember that this definitely doesn't reflect the size of the fanbase of any given artist at the time of this writing (2023). Many Elvis fans are now passing away. And Beatles fans are in aging and dying too. I doubt that either the Elvis or the Beatles are enlarging their fan base faster than their fans are dying off. Heck, it's the 40th Anniversary of Michael Jackson's Thriller! His fanbase must be on a general downward trend now.

Using Google's ngram viewer, it looks like the Beatles will drop from cultural consciousness sometime between 2070 and 2080, and Elvis, though he peaked higher than the Beatles is dropping faster and will probably be extinguished from our cultural consciousness in by 2060. Unfortunately, ngram viewer only tracks word frequencies up to 2019, so we cannot really see where she stands today against other pop music icons.

https://books.google.com/ngrams/graph?content=Beatles%2CElvis%2C%22Taylor+Swift%22%2C%22Lady+Gaga%22&year_start=1950&year_end=2023&corpus=en-2019&smoothing=3

Expand full comment

As the global population continues to expand, you should look at fans per capita. Wouldn't know where to find those statistics.

Expand full comment

Heh. Every generation rediscovers the power of branding, and is surprised at how powerful an effect it has on the popular culture. She may be this generation's Oprah.

Expand full comment

Yep, she's exploded this year. There is a reporter dedicated to covering her, Outlets are discussing how much GDP she produces while in your, Freddie and others have started ranting on how she and others cannot be criticized. My Father in his 60s has started calling himself a Swifie and gushes over how cute a couple she makes with the NFL star. She's the biggest pop star since Michael Jackson, who was also a Time Person of the Year.

Expand full comment

She's basically at Madonna or MJ level now. Before this year I sorta lumped her into a group with e.g. Lady Gaga and Katy Perry, as female pop singers who I knew were famous but I rarely see in the wild due to my apathy for Top40 music, broadcast TV and celebrity gossip. But she's clearly peaked well above the level of other similar artists, for whatever reason.

Re-recording several albums is nutso, shouldn't have worked, and definitely couldn't have worked in prior eras of music, but apparently did work due to harnessing absurd levels of fan loyalty and manipulating the streaming services. The well-timed fake romance with Travis Kelce, where everyone very obviously avoided saying anything untrue while still giving the impression they were dating, had to involve tons of media people playing along, but they did since it was in their own self-interest to do so. I don't know how much she herself was a driver of these recent moves, but it was rather clever. I think MJ's music was more ubiquitous, I still can't name more than 3 of Swift's songs, but in a fractured media landscape where she can't put her videos on Fox before the Simpsons and draw half the TVs in America, she's gotta be close to the peak stardom possible.

Expand full comment

> Re-recording several albums is nutso, shouldn't have worked, and definitely couldn't have worked in prior eras of music

What about it is nutso, or unusual?

Seems to me that most performers routinely re-record their studio releases by doing live performances. A live album is mostly just re-recordings of earlier work.

Expand full comment

Re-recording studio versions of an album and then selling it again is unheard of. Live versions are quite different for rock acts, maybe less so for pop acts but still offering something distinct.

This as I understand it was done only to own the songs, and clearly depends on Spotify and iTunes etc to offer the new version without any artistic justification for it. Whatever company now owns the original “1989” and other records can still license and sell the music, so the entire thing relies on Swift convincing people only to stream and purchase the version she owns, out of fan loyalty or antipathy to big business (as if she herself wasn’t basically just a giant corporate blob at this point.)

Expand full comment

My understanding is that TS still holds the songwriter copyright, and is using that to prevent the original catalogue from being played. The story I heard is that her original recording contract held the copyright for the recordings. The label sold them to someone (not sure of what the history there is, but my understanding is playing TS's "Bad Blood" will provide proper mood) for $900,000,000. Of which TS received $0. So withholding songwriter permission for performances lets TS zero out the value of that property, while recording the "Taylor Version" gives her a steady income stream.

Expand full comment

Wasn't the label itself sold to someone Swift has accused of bullying? And possibly for mere $300 000 000 – $400 000 000.

Apparently Swift cannot block exploitation of the old recording via copyright means. She seems to have enough consumer loyalty to cause a consumer boycott of those old versions though.

Expand full comment

> So withholding songwriter permission for performances lets TS zero out the value of that property, while recording the "Taylor Version" gives her a steady income stream.

(EDIT - this following paragraph is wrong; Taylor Swift herself always held the publishing rights to her songs, and the royalties owed on the covers are therefore owed to her. That said, the value of the originals is clearly very high, not zero.)

But the Taylor's Version raises the value of the original back above zero, because she has to pay royalties when her covers play.

Also she doesn't appear to have blocked the originals: here's Taylor's Version of You Belong With Me ( https://www.amazon.com/dp/B08W92DXXM/ ); here's the original ( https://www.amazon.com/dp/B006WWU9Y2/ ); here's the original but Platinum Edition ( https://www.amazon.com/dp/B00AFDDB4Y/ ).

Expand full comment

Others have re-recorded their albums (and/or just individual songs), sometimes for basically the same reasons, other times because they wanted to do it differently or better.

https://americansongwriter.com/4-earlier-artists-who-pulled-a-taylor-swift-and-re-recorded-their-music/

https://rateyourmusic.com/list/conchobhar/play-it-again-artists-who-re-recorded-their-own-albums/

Expand full comment

In most of those cases it does appear there was an artistic reason cited or an obvious stylistic change. I still think it's unique to the Swift situation that she was both able to re-record an entire back catalog for purely financial reasons AND that the current music distribution landscape is such that this maneuver actually works. The Everly Brothers and Sinatra examples are cases where they re-recorded for new compilations, which makes sense in the days of physical media that you had to be offering a new product to sell. But when everybody just streams music, they're accessing it anew each time they want to listen rather than keeping a pre-existing already purchased music library, so she could lean on Spotify etc to replace the old versions with the ones she owned.

As an analogy, imagine Garth Brooks had tried this 20 years ago with his first 7 albums. There were already too many copies of those CDs in his fans' hands to get them to re-buy all of them, best he could've done was slap them all into a box set and hope to sell enough to fans that didn't already own them, but that wouldn't have justified the expense. Only in the current age, and only if you can implicitly threaten to unleash your army of drone fans like Swift can, does any of this work.

Expand full comment

A live performance is very much not the same thing as a re-recording; recording generally involves multiple takes and layered parts. Performing an album live takes as long as the album runs. Re-recording an album can take months.

Expand full comment

If one considers amount of news coverage, I definitely consider her in the running. By that metric, considering the competition by year she's definitely had a bigger year than previously.

Expand full comment

I have a couple of fans in my family.

Swift has retrieved the rights to a bunch of her old songs, re-recorded them and released new versions as albums. She is also touring extensively this year.

She was well-known and famous before, but these factors have combined into some sort of next-level fame.

Expand full comment

> Swift has retrieved the rights to a bunch of her old songs, re-recorded them and released new versions as albums.

I thought the point of re-recording them was that she didn't have the rights to the old songs? If she owned the old recordings, what would be the purpose of re-recording them?

https://www.vox.com/culture/22278732/taylor-swift-re-recording-1989-speak-now-enchanted-mine-master-rights-scooter-braun seems to make it pretty explicit that the point of the new recordings is that she owns the rights to them and she doesn't own the rights to the old ones.

Expand full comment

Oops you're correct.

I guess the reason for the new fame that the fans are taking her side on the dispute, which is why they're buying the new versions in more than the usual numbers that would happen for e.g. a remix album.

Expand full comment

She failed to retrieve rights to the specific recordings, but recovered rights to make new recordings of the same songs.

Expand full comment

Perhaps, unlike so many other pop stars, she's actually intelligent and a skilled business strategist?

I mean, she's obviously cute, a good singer, and charismatic. But if she's smart on top of that...

Expand full comment

I remember reading long ago that TS as a kid studied the downfalls of musician's in behind the music with the express goal of avoiding their failures.

Expand full comment

Really.

That would make so much sense.

I also note her dad is an ex-stockbroker and her mom was a mutual fund executive before being a homemaker, so evidently she has genes for business. Her parents may have taught her something too.

Expand full comment

> but recovered rights to make new recordings of the same songs.

Huh? Everyone has the right to cover a song. It isn't possible not to have that right. You can't "recover" what you never lost.

You need a license to record the cover, but issuance of the license is mandatory (and doesn't involve talking to the copyright holder) and the fee is set by law.

Expand full comment

I think she has made sure that the cover fee for those albums goes to herself (as the songwriter)

Expand full comment

That appears to be correct. As far as I can tell (from the wikipedia article someone linked, https://en.wikipedia.org/wiki/Taylor_Swift_masters_dispute ), she always held the publishing rights.

In reliability-of-wikipedia news, however, this claim from the same article is definitely false:

> Swift would not have been able to re-record her musical work had she not been a songwriter.[16][3]

Two citations on an obviously untrue statement. Wikipedia is self-correcting!

Expand full comment

Taylor Swift is dating Travis Kelce. Romance involving two attractive superstars is PR gold.

This is obvious to virtually every teenage girl in the US and much of the rest of the world.

Expand full comment

Yeah, I might look dumb in a few weeks for saying this, but the fact that the prediction markets are giving Altman a higher chance of being POTY than Swift doesn't speak well of them, imo.

Expand full comment

Semi-snarky nit: Should ChatGPT really be in the running for _person_ of the year? This feels like jumping the gun on claiming to have AGI to me... Or does passing the Turing test (with whom as the judge?) suffice?

Expand full comment

Maybe not, but Time one year did "You, yes you, the reader" as their "person of the year" so it wasn't inconceivable.

Expand full comment

Good point!

Expand full comment

Well, that was the easiest $150 I ever made

Expand full comment

Well done, though I don't think this was as easy a call as you say. They've never had a pop star or for that matter any kind of entertainer as person of the year. They've almost all been some sort of political, social, religious or business figures.

Expand full comment

Yeah, I would've been less confident if I'd realized an entertainer had never won it, but 1. I think she was clearly the most likely winner on the shortlist (Hollywood strikers, Xi Jinping, TSwift, Altman, Trump Prosecutors, Barbie, Putin, Jerome Powell), and 2. Having Altman at twice her price was just insane, so it was pretty clear to me she was undervalued. Time is in the business of selling magazines, and if they'd put Sam Altman on the cover, 90% of people would've said, "Who?"

Expand full comment

Yes 2023 was massive for Taylor Swift:

- highest grossing tour of all-time https://www.billboard.com/pro/taylor-swift-eras-tour-top-grossing-global-tour/

- highest grossing concert film of all-time https://deadline.com/gallery/highest-grossing-concert-movies-all-time/taylor-swift-the-eras-tour-seattle-wa-2/

- released re-recording of one of her highest-selling albums

- was #1 streaming artist on Spotify (was #2 the last two years and not in the top the years before that) https://en.wikipedia.org/wiki/List_of_most-streamed_artists_on_Spotify

- major topic of sports news / celebrity gossip with Travis Kelce

- became a billionaire

That said, TIME has never chosen a musical artist for POTY.

Expand full comment

Interesting how an entire thread on the world's biggest pop star at the moment doesn't seem to include a single word about whether her music is any good. I take it as an indication it might be not - but maybe I'll have to do the supreme effort of firing up youtube and checking for myself

Expand full comment

OK, had a good listen around her albums. Not exactly my cup of chai, but she's clearly a good artist. Excellent production, good songwriting, expressive voice, good grasp of the zeitgeist, variety of styles, and weaving a fine line between emotionally engaging, assertive but also kinda harmless. Lots of moody minor chords, tonal ambiguity and electronic textures. "Red" is probably the best of what I've sampled - but I still prefer the King Crimson album of the same name.

Expand full comment

Ha! I fear that opinions about the quality of her music will be more subjective than discussing stats about her reach. Personally I’m fairly indifferent about her, but she certainly seems to deeply resonate with a large number of people. I suspect it has something to do with the fact that she writes her own songs - not that common in pop world - and she combines her talent with a vulnerability and openness that seems to connect with fans who like to dissect the meanings of her lyrics (even if it’s as apparently shallow as “which ex is this one about?”) and they end up feeling like they know her deeply or something like that. Rumor has it she’s also got some business savvy.

Expand full comment

The Eras Tour was the highest grossing tour in U.S. history. It generated small booms in excess consumer spending in every city where she performed; According to current estimates, about $4.6 billion in excess consumer spending. It's like Swift ran her own private stimulus program for post-COVID downtowns. If you worked in or near a city she performed in it was all people were talking about for weeks. This is pretty much the largest cultural phenomenon in the US in years and years.

Expand full comment

How is a commitment to merge with other labs close to superintelligence even plausibly a good thing to commit to if you are concerned about alignment? What if those other labs have different ideas about alignment? What if one of them actually wants an unaligned AI?

I mean either it's essentially meaningless or it's dangerous. If it only requires you merge if the other labs acceed to putting you in charge or at least adopting the approach you prefer then it's just kinda silly. I think pretty much any CEO and board of a non-profit will always accept some other organization with a bunch of assets showing up and saying: you can direct all our resources too.

OTOH if it means serious negotions to work out a power sharing agreement that induces other institutions to accept then it obligates you to share power with groups whose goals (or at least methods) are directly opposed to your own. And if the then board gets to decide if any other institution is too incompatible it's really saying: merge if it seems wise to do so which is the same as not saying anything.

--

My concern isn't the legal enforceability. Often it's good to put in commitments to a charter that lack any realistic enforcement mechanism. Rather, it's that either good faith board members are either agreeing to a rule they should know would be quite harmful if they felt obliged by it or it's misleading to phrase it as a commitment and should really be more like: when you get close stop and do whatever helps.

Expand full comment

You're close to ASI, and have competitors who are also closing in. There is the fog of war. Would you prefer to rush alignment so you can be first (increasing risk), or try and negotiate with the other labs so all of you can actually work out alignment?

Expand full comment

The charter says "if a value-aligned, safety-conscious project comes close to building AGI before we do, we commit to stop competing with and start assisting this project." I think the theory at the time was that a major issue wouldn't just be solving alignment, it would be making sure that the first group to ASI would have enough of a lead to be careful about implementing it, and wouldn't rush the last steps because of uncertainty about less-careful competitors.

Expand full comment

Ok, thank you. That explains the thought process. I was just thinking about it quite differently.

It does basically commit openAI to scuttle any attempt at a pause and seems to ignore the possibility that the first group to get close might be motivated by other concerns than capitalistic ones -- eg it won't be people who are pursuing the project because they believe it will save their country from some kind of war or political crisis.

Expand full comment

> a graph of different people’s probability estimates of AI causing human extinction

Are these probabilities of extinction conditional on AGI or extinction in general? There's plenty of future outcomes where AGI is never built for one reason or another.

Expand full comment

What are some of those? Absent AGI, or similarly advanced tech, there are no plausible total extinction scenarios in the foreseeable future. Technological progress is a robust attractor that civilization would eventually converge to even if the current iteration suffers some catastrophic disruption like a global thermonuclear war, which would once again put it on the path to AGI, another robust attractor, and seemingly in easier reach than Drexlerian nanotechnology or brain emulation.

Expand full comment

You don't need full nanotech to end the world; an alga that can't be digested, has a better alternative to RuBisCO and needs 1000x less phosphate is enough (because it blooms like crazy across the ocean and ties up all the atmospheric and then biospheric carbon as gunk on the seafloor). This is *mostly* a computational problem at this point, although the assembly would also be mildly beyond current SotA.

Redirecting a sufficiently-large asteroid into Earth would do the trick if self-sustaining offworld colonies did not yet exist. Chicxulub wouldn't be enough, Ceres would, not sure about the actual crossover point.

Scaling up particle colliders enough could lead to micro black holes.

The amount of dakka required for these is large, but they don't really need super-tech.

Expand full comment

All of those seem to be deliberately self-destructive while requiring non-trivial effort, which, to me, puts them below the level of consideration. After all, we could always have just murdered everyone with sticks and stones, but somehow have managed to avoid doing that for millions of years. Meanwhile, stuff like AGI and full nanotech are dangerous precisely because of their vast non-destructive potential, but which could catastrophically spiral out of control.

Expand full comment

I agree that in general it's not worth giving these scenarios much consideration, but I don't think I agree with your reasoning. Individual human beings do, indeed, murder people with sticks and stones. They're a small proportion of the population, though, and they can't kill more than a handful solo before someone stops them. the probability of so many individual humans all committing a bunch of murders all at once is vanishingly small.

I have no doubt that there are individual human beings who really existed who, if given the opportunity, would have killed more but lacked the capability to do so.

once nukes were on the scene, the bar for human-caused extinction finally came within line of sight of our species, but even getting close still required the resources of a major national government. as the bar lowers, the number of simultaneous crazy-psycho events needed to seriously screw things up drops

as technology advances, that bar gets lower, eventually it gets so low that, idk, a random dealthcult with a single microbiology PhD could pull it off. if we survive that, eventually it gets so low that a single person with a garage bioreactor could do it.

I would not be so quick to write off 'obvious malicious self-destruction' timelines

Expand full comment

>Individual human beings do, indeed, murder people with sticks and stones.

Overwhelmingly pursuing their self-interest. There were a few moderately "successful" suicide cults over the centuries, but I wouldn't expect their future analogues to get world-destructive capabilities before actually powerful actors are in position to destroy the world seeking beneficial ends.

Expand full comment

The first one will be sold as «highly efficient biological carbon capture». Won't even be a lie. No need to reveal (or notice in time) that your breakthrough technology just happens to have side effects.

Expand full comment

That's not even a side-effect, it's just overshooting the goal.

Expand full comment

The particle collider and the Life 2.0 can happen without actual malice; we have GoF research right now despite it being -EV and despite the people doing it not generally wanting to kill millions with a pandemic.

The asteroid does need malice, but it's plausible that someone living off-world might not understand that his/her colony is not self-sufficient and thus it only absolutely requires war malice, not omnicide malice.

Expand full comment

I do agree that the continued GoF clown show is some evidence that humanity could sleepwalk into self-destruction without actual malice, but I'd still in general expect several progressively worse non-total-extinction warning shots due to such brainfarts.

Expand full comment

Pandemics and asteroids yes, Life 2.0 and particle collider no. A particle collider either produces ice-nine of some sort (black holes, stable strangelets, true vacuum, etc.) or it doesn't; there's no in-between. Life 2.0's not quite as obvious - bad-but-not-X Life 2.0 is physically possible - but the things that would kill us are almost the easiest things to get working at all.

Expand full comment

Anything that would cause a serious WWIII. One crazy politician in the right place might be sufficient.

Expand full comment

WWIII isn't an X-risk. GCR, definitely, but not X. Direct blast/heat/etc. can't kill rural communities (or towns/cities not nuked, and there aren't that many nukes anymore), fallout decays to tolerable levels in weeks, and nuclear winter a) is mostly a hoax, b) would have to be hilariously, unrealistically bad to actually finish us all (as even hunting remaining possible would preserve the species, so the entire ocean would need to freeze over).

Expand full comment

Both biotech and nanotech are possible X-risks. Diseases are not designed to kill people but if the technology for designing them gets good enough they could be. Grey Goo scenarios with Drexlerian nanotech are possible. Either could conceivably wipe out our species.

Expand full comment

It seems like these AI experts have a blind spot in that they are unable to consider the likelihood that AI could be beneficial to humanity. When discussing a range of probabilities, if there's a chance that AI could cause human extinction, what's the chance that the AI could be extinction-neutral? — or could *prevent* human extinction? I can just as easily imagine a Banksian Culture-like future where AI 'Minds' have endearing and eccentric personalities and get along well with humans.

I think there is a deep psychological need in many humans to indulge in apocalyptic fantasies. Since the 1970s I've been told by "experts" that humans are facing extinction by a continuously-changing set of threats—

1. Pollution (will kill the O2-producing algae)

2. Club of Rome prediction that civilization will collapse by the year 2000

3. Global cooling and the coming ice age

4. Nuclear winter

5. AIDs will infect one-third of humanity (ending civilization);

6. Global warming

7. Ocean acidification (will kill the O2-producing algae)

8. Asteroids

9. Y2K

10. COVID-19 will be a mass disabling event (ending civilization)

11. and now it's AI.

Those first five predictions haunted my childhood and young adulthood until I started to realize that most experts were either purposely bullshitting us and/or themselves.

Meanwhile, my Xtian neighbors kept warning me that the End Times were coming any day now!

Neither the experts nor the religious have been correct in their predictions. I just take them all with a grain of salt now. Humans are incapable of predicting their future, and it will most likely be something we didn't consider that will take us down. ;-)

Expand full comment

"3. Global cooling and the coming ice age"

Ah, you remember that too? A good while back I got plenty of "nope, never happened, nobody ever said that, you're mistaken/lying" from the disgruntled CLIMATE CHANGE IS GONNA KILL US ALL types on here when I mentioned that the coming New Ice Age was one of the flaps of my youth.

At least if I hallucinated it, I'm not alone 😀

Oh, and don't forget: hole in the ozone layer! We were all going to get skin cancer and die from that one.

Expand full comment

Eh, to be fair, most of these are attributable to the journalists (who have only gotten worse lately), not to the actual scientists (nor even policymakers).

1. "Pollution (will kill the O2-producing algae)": Still a real problem, only it depends on the kind of pollution. For example, fertilizer pollution creates delightful and vibrant algae blooms (that are also toxic to all other forms of life).

2. "Club of Rome prediction that civilization will collapse by the year 2000": No idea what this is, never heard of it. Sounds like a newspaper thing.

3. "Global cooling and the coming ice age": Was never a real thing, media companies invented this one.

4. "Nuclear winter": And that's the least of your problems if global thermonuclear war actually happens; and with events like the Cuban missile crisis, it seemed like a real possibility... and it still is to some extent. But for now we seem to have pulled back from the brink.

5. "AIDS will infect one-third of humanity (ending civilization)": It's kind of already the case in Africa, sadly; but the world-wide prediction was a media invention.

6. "Global warming": Would be lovely to say it's all fake, but right now looks like it's all too real. Climate skeptics have mostly switched from saying "it's a hoax" to "it's totally happening but maybe that's for the best". Hmm.

7. "Ocean acidification (will kill the O2-producing algae)": Not the algae (they have a pretty quick evolutionary cycle), but certainly the coral reefs. If you wanted to see one in all its glory, now's probably your last chance.

8. "Asteroids": They're a real though relatively unlikely threat. Have you seen the Chicxulub crater ?

9. "Y2K": This one was a combination of real problem that was successfully solved by a concerted effort, and OMG media overhype to the max. If we did nothing, chances are that e.g. banking systems would crash, which would be... not great.

10. "COVID-19": Could've been much worse without MRNA vaccines, but like AIDS, somewhat overhyped.

11. "AI": What pisses me off here is that all the Singularity hype is masking the real danger. There are people now who are cheerfully letting ChatGPT write their laws or even their software. This is like putting a typewriter monkey in charge of same.

12: "Ozone Layer": This was a very credible threat which was solved through a concerted effort (banning CFCs). For once, humans did something right... sadly we probably lost the capacity for such action by now.

Expand full comment

"Was never a real thing, media companies invented this one."

You mean they used speculation to write up scare stories? Say it ain't so!

https://en.wikipedia.org/wiki/Global_cooling

"Concern peaked in the early 1970s, though "the possibility of anthropogenic warming dominated the peer-reviewed literature even then" (a cooling period began in 1945, and two decades of a cooling trend suggested a trough had been reached after several decades of warming). This peaking concern is partially attributable to the fact much less was then known about world climate and causes of ice ages. Climate scientists were aware that predictions based on this trend were not possible - because the trend was poorly studied and not understood (for example see reference[16]). Despite that, in the popular press the possibility of cooling was reported generally without the caveats present in the scientific reports, and "unusually severe winters in Asia and parts of North America in 1972 and 1973 ... pushed the issue into the public consciousness".

Expand full comment

Well, tell that to my (late) Glacial and Quarternary Geology professor. He had graphs and data showing that we were entering the cooling phase of the current Milankovitch Cycle. It was taken very seriously despite what Wikipedia might say. Suggest at John Imbrie's work from the 1970s and 80s.

Expand full comment

"2. "Club of Rome prediction that civilization will collapse by the year 2000": No idea what this is, never heard of it. Sounds like a newspaper thing."

There was a book version too: https://www.clubofrome.org/publication/the-limits-to-growth/

Scathing review in https://www.americanscientist.org/article/computation-and-the-human-predicament

"In this context climate models offer a useful point of reference. General circulation models for the atmosphere and the oceans, along with related models of ice sheets and atmospheric chemistry, have several points of similarity with World3. At a conceptual level the structure is much the same: There are flows of air, water, heat and other entities, which the model must sum or integrate. The time scales are similar: In both cases we want to know what’s going to happen several decades out. And feedback loops are essential mechanisms in both kinds of models. (There are even historical connections. The use of general circulation models to study global climate change began in earnest at MIT circa 1970. The instigator was Carroll Wilson of the Sloan School of Management, who was also the person who got Forrester involved with the Club of Rome.)

These similarities are outweighed by differences. Where the Limits team had a casual attitude to data gathering practices—and outright hostility to statistical methods—the climate science community is passionate about collecting data, verifying its provenance and quantifying its uncertainty. General circulation models are not based on rough estimates or guesses but on decades of meticulously curated measurements—what Paul Edwards, in A Vast Machine , calls a “climate knowledge infrastructure.” "

Expand full comment

The other big difference is that _Limits to Growth_ ignored rational human responses. In their model, if food prices went up, output went up in the short run then down as farmers destroyed their soil trying to produce more. That assumed that the farmers ignored the fact that high prices gave them an incentive to maintain the soil for future output. As I put it at the time, the model was like predicting the path of an automobile whose driver couldn't see through the windshield.

Climate science doesn't have that problem as long as it is only dealing with climate. It only comes up when it is being used to predict the effects of climate change, which isn't really climate science. The amount of CO2 depends on human action but the ICCP can and does deal with that problem by considering multiple emission scenarios.

Expand full comment

Many Thanks! Yes, I'd heard that criticism too.

One more quote from the review: "As for the mathematical model behind the book, I believe it is more a polemical tool than a scientific instrument."

Expand full comment

1. Ocean anoxic events (OAEs) have happened in the past (probably significantly to the Permian mass extinctions), but the claim that pollution and/or acidification will cause an OAE event seems pretty far out on the range of probabilities. Especially since we've significantly cut down on our water-polluting ways since the 1970s—and because the evolution of plankton during the Jurassic that sequestered carbon in calciferous sediments will buffer acidification in the longterm.

https://www.quantamagazine.org/how-jurassic-plankton-stole-control-of-the-oceans-chemistry-20191001/

2. Club of Rome and the famous/infamous publication "The Limits to Growth". ITheir report kicked off a whole slew of books about the uncertain future of humanity...

https://en.wikipedia.org/wiki/Club_of_Rome

https://en.wikipedia.org/wiki/The_Limits_to_Growth

3. Global Cooling was a real worry. If glaciation is dictated by Milankovitch cycles, then we're entering a period of global cooling. See the papers by John Imbrie.

https://www.science.org/doi/abs/10.1126/science.207.4434.943

Also this popular documentary with our favorite Vulcan narrating.

https://youtu.be/NQSBn50o_8M

4. Carl Sagan and nuclear winter. Considering there were over 500 above-ground nuclear tests during the 1950s, do we have any data that suggests that they affected the climate? I don't know. I haven't looked into that question.

https://www.sciencefriday.com/segments/nuclear-winter-carl-sagan/#:~:text=Carl%20Sagan%2C%20writing%20for%20Parade,the%20general%20public%20about%20it.

5. "AIDS will infect one-third of humanity (ending civilization)". Can't find the links because they're pre-Internet, but I think Douglas Feldman was sounding the alarums about extremely high mortality rates from AIDs. This was before they really had a handle on its R0, though. I remember Scientific American amplified his concerns (fears).

6. I'm afraid I'm pro-global warming. Until recently CO2 levels were the lowest they've been since the Permian. Angiosperms—which are responsible for most of our food supply—evolved when CO2 levels were 4x-5x higher than they are today. We can see a positive response to higher CO2 levels to the current greening of the world.

https://www.nasa.gov/technology/carbon-dioxide-fertilization-greening-earth-study-finds/

7. Coral reefs? Coral reefs existed when the global climate was an average of 6º C warmer and CO2 levels were 4x higher than today.

https://royalsocietypublishing.org/doi/10.1098/rsta.2013.0096

Skip to 12. I hate to tell you but the Ozone hole is still opening up every year over the Antarctic. 2023's ozone hole briefly opened enough to be 12th largest hole since they started monitoring it—but overall, 2016 was the 16th worst from Sept. 7 to Oct. 13...

https://www.nasa.gov/earth/climate-change/ozone-layer/2023-ozone-hole-ranks-16th-largest/#:~:text=The%202023%20Antarctic%20ozone%20hole,day%20ozone%20hole%20since%201979.

Expand full comment

How could I have forgotten about the Ozone Layer?! There's still a big hole that opens up over the South Pole every year, It was the 12th largest on record in 2023.

I was taking classes in Glacial and Quaternary Geology when the narrative transitioned from a consensus of global cooling to a consensus of global warming. It happened within a few months by a few high-profile think pieces in the Scientific American and Science magazine (and maybe Nature, but I couldn't afford to subscribe to Nature at the time).

This documentary has survived the cultural memory loss...

https://youtu.be/NQSBn50o_8M

Expand full comment

> It seems like these AI experts have a blind spot in that they are unable to consider the likelihood that AI could be beneficial to humanity. When discussing a range of probabilities, if there's a chance that AI could cause human extinction, what's the chance that the AI could be extinction-neutral?

Isn't the answer, according to the chart, that AI experts think AI will most likely be beneficial or extinction neutral? There are only a few people there who put the odds of extinction from AI above 50%

Expand full comment

Note that The Precipice, essentially the X-risk Bible by Toby Ord, put AI X-risk as bigger than all other X-risks put together for the next century.

Yes, AI could help with other X-risks, but without alignment the cure's worse than the disease.

Expand full comment

I think there are a couple positions on how to balance AI benefits vs. risks:

1. AI would be incredibly beneficial to humanity if we got it right, but there's no chance we will, so let's focus on not dying.

2. AI would be incredibly beneficial to humanity if we got it right, but better to take time to do it right and wait an extra 10 years for the benefits than eat a 20% risk of dying.

3. AI would be incredibly beneficial to humanity if we got it right, but we should also work on controlling it, for the same reason that even though planes are beneficial, we work on ways to prevent them from crashing.

Given that people have written whole books on these things and balancing the benefits and risks, I think it's unfair to call it a "blind spot".

Also, you're doing a thing I should give a fallacy name to and write a post about, which is giving a long list of things that didn't happen, then using it to prove nothing ever happens, without also including a list of things that did happen:

- The Black Plague

- The Mongols killing 10% of the human population

- Two World Wars

- Global warming of the type we've actually gotten

- AIDS killed however many people it actually killed

- COVID killing however many people it actually killed

Before all these things (except maybe the Plague and the Mongols, I don't know enough history to know what people were saying there), people were saying it was alarmist and would never happen, and those people were wrong too. I think the probability (something bad happens|people predicted something bad would happen) is moderately high and definitely not zero.

I admit humankind has never been destroyed before, but that's a tautology - it's logically necessary that the day before humankind is destroyed, humankind will never have been destroyed before!

Expand full comment

> I think there are a couple positions on how to balance AI benefits vs. risks...

As usual, I'd like to point out that alternative positions do exist, i.e.

4. AI could potentially be incredibly beneficial (or possibly harmful) to humanity, but its current iterations fall laughably short of that goal, and there is as of yet no clear path from here to there. We should continue working on AI research, but meanwhile let's make sure we use existing (and incredibly unintelligent) machine learning systems responsibly and without undue exuberance.

Expand full comment

Yup, all 11 sound familiar.

2 quibbles:

a) Re 8. Asteroids was always a low-probability-per-year hazard. I don't _think_ I ever heard it painted as a will-kill-us-in-the-next-decade kind of risk. ( The Yellowstone caldera giving us a supervolcano eruption is similar)

b) For a _lot_ of these, there is an immense amount of noise and oversimplification injected between what researchers in the field concluded and what the pundits spouted

Expand full comment

This is late, but for your reference, up until 5-10 years ago, I was largely pessimistic about the long-term future, because I thought we were stuck on a single planet and there were too many things we could do to end our civilization or existence. My hopes were placed in the creation of self-sustaining, self-replicating space colonies, or building something smarter than us to solve the problems we can't. But this was all centuries or millennia away, and didn't affect my day-to-day thinking.

On the first angle, Elon Musk happened, and despite his erratic behavior, I'm quite happy that we're finally moving in a positive direction again.

On the second angle, it's true that hanging out in Eliezer-Yudkowsky-adjacent spaces got me thinking about AI risk in a more serious way. But the main cause of my pessimism came from a close encounter with a malevolent human with superior social skills. The aftermath convinced me that practically no one thinks about that sort of risk clearly, with the notable exception of Eliezer-influenced folks. The only reason my p(extinction) isn't higher, is that it's blocked by a large p(total civilizational collapse) before an AI gains the capability to kill us all.

Expand full comment

Part of the reason for Taylor Smith being so high now is that she's on Time's released-Monday-morning shortlist ( https://time.com/6341947/person-of-the-year-2023-shortlist/ ).

Expand full comment

It's interesting to see that several candidates on the shortlist are in fact not people:

> Hollywood strikers

> Trump Prosecutors

> Barbie

It's also interesting that Putin is on the list; I would have expected a lot of government pressure not to do that.

Expand full comment