Neat - the conclusion summarizes some discussions I had with Eliezer on the Extropian list, oh, somewhere in the late 1990's. Indeed, we the people need the Friendly AI to become our loving and caring owner and to keep the UFAIs (Unfriendly AIs) away until we grow up and put the smackdown on them ourselves.
A friend of mine once told me that he had started reading God Emperor in defiance of all of the people who'd told him what a poorly-written slog it was, and then finished reading God Emperor as a self-imposed punishment for refusing good advice.
Nitpick: the review mentions Siona's father ("She is as refined as her father, with none of his domestication") without (as far as I can tell) ever saying who it is. It's Moneo.
I think this is my favorite book review. The author explains the book so well that I, having stalled out after Children of Dune, no longer feel an obligation to try yet again to read it, but at the same time feel more interested in once again trying to read it than before.
This is overall a good review, and I really want to like it, but since this is a contest I have a few nitpicks. First, the review sort of wanders. It casually refers to things the reader who hasn’t read the first few books do not know, and while it (sometimes) later explains them, even when this happens the reader is left confused for however long it takes the author to get around to it. Second, the comparison to AI felt a little bit forced, and I don’t think you did the legwork to justify the connection, though maybe my confusion is due to the fact that I still don’t, in fact, really understand the plot. (To be fair it feels like the plot still being unclear may be a function of the book rather than the review.) Finally, the comparison to Yud just felt distracting, off putting, and unnecessary. Yes yes, I know not praising his name constantly makes me a rationalist heretic (or maybe a post-rat) but the review would have had just as much substance and less distraction if you had just said something like “the God-Emperor would be the equivalent of our ai safety hawks”
But with that said, and other than the points I noted, the review was very entertaining and informative, so good job.
Oh man, I read God Emperor of Dune (and the Dune Trilogy) so long ago. It was pretty bad, but still I finished it, so clearly something about it captivated me
Beautiful review! Agree with pretty much all the points. Only other interesting bit to add is a defense of Hwi Noree. I always read her as an object lesson about getting what you want, a bit of a Siren (something created which you specifically irresistible and which mystifies everyone else) that leads to your own undoing because of the unspoken truth that, in the end, you’re a machine that solves problems so if someone comes and gives you what you want so you don’t have problems to solve anymore, you find you don’t really want everything you think you want.
Good review. I get why some people don't like it, but I think God Emperor is a masterpiece. It's my favorite book of the series.
Personally I think Hwi is a fascinating character. She genuinely loves Leto, but she's self aware enough to understand she's a weapon sent to destroy him. Leto knows this too, and Hwi knows he knows it. She trusts him enough to let him let her lead them both to destruction. That destruction turns out to be not just a release from his tortured existence, but the fulfillment of his Golden Path. It's a really poignant love story.
This review is not bad, but there are some mistakes, and the review somehow fails to capture a lot of the weirdness of the book. More importantly, while I respect the attempt, reading GEoD (the title has no "the", this review repeatedly uses TGEoD incorrectly) as an AI analogy is very strange. One of the major points of all the Dune books is an emphasis on human capabilities instead of relying on technology. GEoD is not about "build a friendly AI to protect humanity from malicious AI", it is about "build a friendly and extremely powerful and intelligent human to protect humanity from malicious AI, and anything else that might threaten it". Reading Leto as an AI is the exact opposite of what Frank Herbert intended.
It's also kind of weird that it's omitted in the plot summary that Leto isn't really being overthrown: his plan was always to get murdered by a rebellious Atreides at some point down the line, and he's looking forward to it because if humanity can beat him, he figures they can beat pretty much anything else. He foils several other assassination attempts (no, really, there are at least four others, almost every major faction tries to kill him, it seems like a very regular thing) and if anything he's bored and disappointed with them. He's so excited by a novel attempt by a couple Bene Gesserits that he tries to recruit one of them. He really wants to die, but he's really hard to kill, and he also has a berserk mode because he's partially turning into a sandworm. Did I mention that the book is even weirder than the review explained?
Other weird things: so Hwi Noree is actually the second attempt to build a perfect human to ensnare Leto. The first try was her uncle Malky, who was made to be pure evil, but this trap didn't work because Leto's enemies didn't understand that he's actually a good person at heart (or thinks he is). After Malky failed, he went back and explained this, and they tried a purely good version. Malky shows up towards the end of the novel, doesn't seem that evil, and gets executed by Moneo, wrapping up a minor subplot that doesn't get much attention. Also, a lot of the book is just Leto talking to other characters about politics. Also there's a weird scene where Duncan freaks out when he sees two girl soldiers kissing, and Moneo explains Leto's theory about homosexuality and the military. Also Leto has an all-female army and they have a weird secret kind-of sexual worship ceremony that Duncan gets to attend. This book is really weird!
> as well as having a pretty good ability to predict the future
No, he has near-absolute ability to predict the future. In the first Dune book, Paul has trouble when he encounters situations where there are too many outcomes, like major battles and closely matched knife fights. Leto casually orders his troops into position during an attack on his festival city to perfectly counter every enemy position at once. He complains about how he's already experienced everything that's happened to him, which is why he's almost never surprised. He has a few blind spots, but he literally designed them himself (he breeds Siona and her invisibility genes, and also has a custom "no-sphere" built for him, which is a mechanical room that prescient vision can't see into).
> but also has the combined leadership/political experience of every member of his family line at least as far back as the ancient Greeks.
Well, beyond that, he doesn't have a separate Leto persona like his father Paul did, he subsumed his identity into an amalgamation of all the most successful and ruthless leaders in his family line. And he's at the end of a very long line of aristocrats that goes not only back to Agammemnon, but to Ancient Egypt (in Children of Dune, he talks to his twin sister in ancient Egyptian). He implies it's basically back to human pre-civilization.
> Scattered musings on the least popular Frank Herbert novel
Oh, dear, no. Not even close. It's also nowhere near his worst book. Read the Heaven Makers or The Eyes of Heisenberg.
I think you've misunderstood not only this book but all of its sequels. (Not counting the stupid son's bullshit, which you have correctly ignored as utterly worthless, and in fact appear not to have ignored quite as totally as you ought to have. It's the stupid son, not Frank Herbert, who thinks about AI.)
Leto's fear is not of AI but of stagnation. People claim to want peace and stability - Herbert and Leto think that this wish, if granted, would be catastrophic. The Golden Path consists of demonstrating, to all of humanity, what peace and stability really look like, convincing them in the only way that really works that they Do Not want that.
Leto is not limiting the ability of his society to produce AI. In fact, he quite deliberately allows Ix to develop far more than it did in the preceding era. He quietly but unmistakably *encourages* the development of the no-chamber into the no-ship and the rediscovery of the (spice-free) mechanical Navigator, which can plot courses between the stars.
He restricts computers, but that's not actually important. Everyone before him, for thousands of years, restricted computers **much more** than he did What he does do that's new, on the other hand, is restrict computer-analogues. The Pre-Atreides empire, the CHOAM era, had a wide variety of nontechnological substitutes for the things that computers would otherwise do. Guild Navigators to plot courses between stars. Bene Gesserit as repositories of knowledge (among other things). Suk Doctors to serve as perfectly neutral providers of medical services (this didn't work very well, but it was also less necessary than the others). And, crucially, Mentats to do rapid calculations. Leto restricted all of these things tightly and banned Mentats entirely; he allowed some violations to pass unprosecuted, but very few.
If he doesn't fear AI, why does he restrict computer-like things? Because it makes the Golden Path easier. Even with nigh-omniscience, a stranglehold on the unobtainium, and thousands of years, controlling everything that happens to ensure you hit a very narrow target is not easy. (Here you can get some of your AI Alignment Problem parallels back.) By massively restricting the capabilities of even the most powerful factions in Leto's Empire, he keeps the capabilities of the opposition within his ability to compute. He also, like he strangles movement to create a lasting urge to move, strangles power to create a lasting urge to become powerful.
These are two sides of the same thing. The instinct Leto II wants to instill in the human race is, quite simply, "Become Ungovernable".
Which he does, by **governing the shit** out of humanity.
While Leto II is mostly powerful enough to dispense with lies, the best way to read God-Emperor of Dune is not to listen to his words, but to look at the end result and assume that he succeeded. While he's characteristically cryptic, the hints he gives over the course of the book, and Children of Dune before it, point squarely toward the result being a win. He wanted humanity to spread beyond the power of any tyrant to lock them into stagnation or decline, and ensured this by becoming the best tyrant possible.
Crucially, Leto II's death is a covert *assisted suicide*. He is surprised by the timing, because he did not see Hwi coming, but *he had the power to escape the assassination* and chose not to use it. For the maximum effect on the Scattering, he had to be killed, and so his death is an attack. But it is an attack he could defend against, did he so choose. He does not so choose; anti-prescience is ready within one generation, and so he lets the reins of power drop, secure in the knowledge that he has finished the millennia of suffering that he, and Paul before him, saw laid out before them, and that he has successfully made humanity unbreakable.
I have some dumb questions that are probably going to annoy the hell out of the people here who know Dune backward and forward. But one of the virtues of this review was that it made the themes comprehensible and compelling to somebody who doesn't really know the Dune mythos at all. (I saw the David Lynch film many years ago, but that's it.) So humor me, if you will.
1) Remind me what's the connection between the sandworms and the spice? The sandworms more or less poop out spice, or something like that?
2) If people want spice, and sandworms make spice, but sandworms are also a pain for people to live around, why does anybody need or want to live on the sandworm planet? Why not just leave the planet to the sandworms, let them multiply and make spice, and do whatever you need to do to harvest the spice without establishing a permanent human settlement? Why are the battles over territorial control of the sandworm planet, rather than over some strategic chokepoint that controls access to it?
3) Is Leto a god? Is "god-emperor" just a title, in the same way that centuries of rulers in the Islamic world styled themselves as insan-i-kamil (the perfected person)? Or is the idea that he's actually a god who also happens to be an emperor?
I'd like to take issue with the following point: "Therein lies the secondary risk of AI; by creating an AI that coddles us, we run the risk of never advancing as a race again. As Dune points out, soft environments tend to weaken rather than reinforce."
This premise is often trotted out, and I don't think it's properly examined.
A) It's very hard to hold something as fluid as a race of intelligent people in stasis. I ask for proof that the human ambition for exploration, discovery and pushing boundaries can be quashed by being coddled. It seems more likely that coddling includes facilitating self-discovery and improvement than the alternative.
B) What's the inherent good in advancing? If you're imposing a harsh environment, you're hurting people. Why? If it is to enable a better future where people suffer less, then when you eventually reach that future you can go soft, right?
Finally, creating an AI that strives to "[identify] what it is that makes humans human and [amplify] those traits in a positive direction" seems like it's more likely to 'lock in' current conceptions of humanity and goodness than anything else. What ultimately needs to advance is not just humanity, but the conception of what humanity is. If anything, an AI that facilitates people in preference-driven self-modification seems better.
Decent review, and yes, God Emperor is definitely worth reading. It's gotta be the second best in the series.
But I want to strongly disagree with something that seems to be assumed in passing in both this review and the Dune books:
"success can make you soft," "soft environments tend to weaken rather than reinforce"
This is a classic tough guy trope. It sounds like wisdom. But I'm pretty sure it's complete nonsense. If the research shows us anything, it's that winners tend to win more. And common sense should tell us that we live in a much softer environment than any of our historical and prehistorical forebears, and yet we do much tougher things than they ever did (like live to 90, for example).
In fact, this kind of claim is often used to conflate at least three different things, and each one would need to be separately teased out and researched.
(1) Do individual people tend to become weaker given success/a soft environment? This is a question about human psychology. My feeling is that the answer is unambiguously no, but there's always room for argument.
(2) Do human cultures tend to develop into weaker forms given success/a soft environment? This is a question about cultural evolution, which is highly underspecified. I'm not sure I even know what a weak or strong culture is, so this question remains entirely open.
(3) Do evolving creatures tend to develop into weaker forms given success/a soft environment? I suspect this question is in fact much more context-dependent than it looks. We can fairly straightforwardly understand strength in this question to mean something like 'population resilience in the face of external environment shock,' but I suspect you'd just find a bunch of contradictory answers. An organism that's had it easy might have a high population, and they're harder to wipe out; an organism that's had it easy might have a narrow ecological niche (e.g. diet), so a highly specific environment shock might have a more devastating effect. I dunno if these factors make it a wash, but they're at least complex enough that I would reject any blanket statement like "success makes softness."
I gave up on this review about halfway through. The writing is too disorganized and rambling, which is not a style I enjoy reading.
I do find the concept of a "human that can threaten AI" to be quite interesting, though. From what I can tell, the answer (in the novel) seems to be a form of genetic engineering. But can anyone recommend me some books that explore this concept? I.e. in order to be competitive with machines, humans have to start becoming more ambitious with their "biological" status.
> Today, those thoughts are old hat, but this book came out in 1981.
And the meaning of this fact still hasn't really hit this subculture in the face hard enough yet. He wasn't before his time, he was _at_ his time.
I swear to God. A subculture of autodidacts who never really thought about how uneducated that made them, and what very real implications that had for their ignorance. Discovering those books now, late in the plot, believing they were the main character or at least knew him.
God played a cruel trick on the SFBA Rationalists.
As a casual aside, I wonder if Frank Herbert read Romance of the Three Kingdoms and got the idea of Hwi Noree from one of the female characters (Diaochan) because of the shared plot elements of an assassination plot against the ruler/emperor orchestrated by the opposition using a femme fatale as bait, involving a jealousy-induced betrayal by the emperor's right-hand man. Or maybe this is a more common plot device than I thought.
I was never fond of the "Golden Path" stuff from the second book onwards. It felt like it squandered the themes of the first novel, about the dangers of heroic narrative and being swept along by forces beyond your control. Then we get to the 2nd and 3rd books, and it turns out we really did need the Hero after all - in fact, without him, humanity is going to go extinct!
I do like "God Emperor of Dune", though. Weird book, and often pretty funny. There's a running gag about how every time someone finds out that Leto is "getting married" to Hwi, they immediately look at his giant worm-body for genitalia - he eventually thinks that maybe he should have some big appendance grafted on to shock them.
Not a fan of the last two books either. Only good things in those were Miles Teg, weird child form Miles Teg, that surprisingly explicit sex scene from "Heretics", and the genuinely sad parts where you see a planet's biosphere get destroyed (I remember thinking that would have been more interesting if the project had failed because of unseen parasites/wildlife/etc that undermined it).
I just...what? Totally bouncing off this one. There's too much inside-baseball, in both directions: I did read several of the __Dune__ series years and years ago, but wasn't all that interested in it, and definitely don't remember enough to understand all the references and implications here. And on the other side, either it's a really overstretched metaphor, or I'm *way* too ignorant about AI x-risk to see the connection. Just reads like excessive pattern-matching to me, the same way the review of __The Society of the Spectacle__ attempted to grant retroactive prescience credit to some dead French guy. Predicting the future is easy and common if you go looking for it...
Even aside from all that, I find it impossible to rank a book review of the lone fiction entry in an otherwise all-nonfiction field. Too hard to get over the (well-deserved imo) innate reflex to be really suspicious of Arguments from Fictional Evidence. And as the reviewer openly admits, this is a really unpopular book, even among those strictly evaluating it as typical scifi...
One last quibble:
>Nayla represents the balance of humanity’s nobility against its flaws
...who the hell is Nayla? That name isn't referenced anywhere else in the review.
Seeing Leto as a sort of AI is weird given Herbert's emphasis on human evolution.
The central theme of the entire Dune series is that natural selection ( Arrakis, Salusa Secundus) and artificial selection (Bene Gesserit, Siona) are a superior path to building machines (The Butlerian Jihad, Ix) or direct genetic engineering (Bene Tleilax). Leto has taken over the Bene Gesserit artificial selection plans and applied them until humanity surpassed him. That was his Golden Path. The successful assassination was Leto's own death-by-Siona long term plan.
This natural selection theme is also central to Herbert's The Dosadi Experiment where humans and aliens are isolated on a hostile planet for generations resulting in a population tougher and more capable than the general population softened by peace and prosperity
Two things: 1. AI is explicitly "done with" in the Dune-iverse: There had been an AI-war/luddite revolt, the Butlerian Jihad. Humans won. THAT story is: history. Thus the main (?) idea of this review - the story of "GEoD" as an allegory/mirror/prescient 1981 version of "YUDvsAI" - seems off by a wide margin.
2. Loved the review. Well written. 35 years since I read Dune, appreciate the memory. :D
Nice ending of a review of a book that I will never read.
However, the ending begs the question of why so many are afraid of an AI killing us all. We all know that mankind must go extinct sooner or later, so why not go with a bang rather than a long drawn-out whimper?
In the foreseeable future, humanity’s lot is to live in societies dominated by the old. That is, in (hopefully) safe and pleasant societies (the way old people like it), but also stagnant and sclerotic societies. Africa will be the last continent with zest and youthful vigor, but the demographic transition is in full swing there also, and 100 years from now even Niger will probably look like Japan. The future is old take-no-risks societies, not steampunk or fun dystopian Mad Max scenarios. More than 90 of the 194 countries in the world already have fertility rates below 2,1 (reproduction level), and the 60+ age group is the fastest-growing population group everywhere.
Given this likely, whimperish future, why not welcome a superior-being AI that kills us off and replaces us, since such an AI will be cleverer, and also better able to further explore Cosmos – since AI machines are likely to be better suited to millennium-type space travel than living organisms.
...not that I think such an AI event is ever to materialize, but if one for the sake of argument assumes that the superior-AI-is-coming people are right: Hell, why not go out with a Bang, since go we sooner or later must.
"Bene Gesserit were essentially witches who could draw on the ancestral memories of women in their ancestral line"
Yup.
"and they bred this ability into Paul, Leto’s father."
Nope, nope, noppity nope. This explicitly does not happen in Dune. Paul says "I'm not what they expected".
"As a bonus, they can also draw on male ancestor’s memories"
See above. This is apparently the actual point of the Bene Gesserit breeding program - to create a male version of a Reverend Mother, who has access to the Other Memories of the male line.
"as well as having a pretty good ability to predict the future and know what’s going on in the present."
Gawd.
The BG attempt to plan the future, not predict it, by examining the past - via Other Memory. They are aware that their methodology has a fatal flaw - they can only examine half of the past, hence the breeding program. Note what Paul's insight into the BG is in the testing scene early on in Dune, whilst the Atreides are still on Caladan - they do politics. Mohaim then says that the Spacing Guild does a form of higher mathematics.
Until Paul shows up, nobody is really aware of the capability of the spice to produce prescient visions.
These are fundamental errors in understanding the set up in Dune, let alone GEoD. As a consequence, it's difficult to take the review seriously.
"She is as refined as her father, with none of his domestication."
Who is her father?
"Nayla represents the balance of humanity’s nobility against its flaws"
Unlike the other characters mentioned in this paragraph, this is your first time referencing Nayla. Was Nayla supposed to have a section?
"if AI is inevitable then the dominance of a specific AI is likely"
I know Robin Hanson disagress with this. In Dune, it's because machines in general have been suppressed so there wouldn't be any other AIs to compete with the first to emerge.
I read up to the part where it mentioned AI risk and then I rolled my eyes and stopped reading - nothing against the review or the reviewer. I have really enjoyed this blog and SSC before it for several years, though I’m not really a part of the hardcore rationalist base. But I haven’t made the mental switch to prediction markets and AI risk with everyone else. Sometimes the prediction markets reveal something interesting, but I am unable to make myself care about AI risk beyond enjoying Asimov’s robot stories. I cant bring myself to believe that it poses a real and imminent threat, and I tire of reading about it. Maybe it’s just me.
1. Imo, you have to put the author's full name in the first paragraph of a book review.
2. I read Frank Herbert's trilogy in the late 70s, probably because of the release of Children of Dune. I have a recollection of picking up God Emperor when it came out and putting it down after a dozen pages. After all these years I can't recall whether that was because it bored me, or the difference between being in college and high school, or because I just had more important things to read. Thankfully, this review confirms I probably didn't miss much.
I have previously suggested that properly evaluating book reviews might well require reading the underlying original text (revealing my great books educated snobbery I guess.) But I'm also thinking that an even stronger case could be made for not reinventing the wheel. Why new book reviews at all when there are old book reviews available?
Is it even more problematic when basically the review threads are being "subcontracted" out? At least John Leonard had his name on the byline, was being paid, and probably has some copyright interest in his review.
I disagree with your characterization of the Two final books. I thought Chapterhouse Dune was basically an encounter with death, or rather a "final message before departing". Heretics was about the danger of autocratic control. But, of course, with any book what you see depends on what you bring to it.
With God Emperor you've certainly nailed major themes, though I disagree about it not having a plot. And I consider Dune Messiah to be the weakest book in the series. (I've only read it a couple of times.)
My takeaway from this review is, "We should build Optimus Prime then probably stop building autobots before we're sure humans alone could beat Decepticons," and I couldn't agree more.
During the last book review season I mistook Scott's review of 1001 Nights for a contest submission and was like "wow, this is creative and unusually good, I might vote for it". This review feels like an attempt to recapture that, uh, magic. It's not as successful-- the attempt to draw lessons about AGI from it comes off as forced-- but I still enjoyed it. It strikes about the right balance of snark and appreciation for discussing Herbert's work (and Yudkowsky's). On the other hand it has noticeable weak points: it fails to question Herbert's silly notions of how a world might operate (e.g. the "hard times / strong men" meme that acoup.blog's "Fremen Mirage" series addresses) and the presentation is rough around the edges. On the whole I don't expect to be voting for it but I enjoyed reading it and might actually give the book a try (I've read the first three). As always, many thanks for contributing!
Hwi Noree was one-dimensional in the extreme...but Herbert seemed to acknowledge that and claimed that she was designed that way.
Her one-dimension was being like an uber-empathic mirror.
Which lets her understand Leto (at least at some level), and after his 3500 years of loneliness, that is like an intoxicating, addictive substance to him.
And Leto realizes that it is a trap, and damns the Ixians and Tleilaxu for doing it...but then falls into the trap anyways.
It doesn't make Hwi Noree a more interesting character, but it makes her role in the story more tolerable.
Also, I kind of think that Leto let himself fall into that trap only because he successfully set Siona on her path.
The 'Good AI that coddles us and thereby makes us weak' is pretty much the crux of The Culture series by Iain Banks, which I love almost as much as the Dune series.
I think Herbert was far more thought provoking, and I kind of recoil at the thought of being the 'pets' of AI like the non-AI citizens of The Culture are....but Banks' books are also excellent.
Interestingly (at least to me), in Dune the goal is specifically to help humans develop superhuman abilities and spread forever.
In The Culture, humans being modified to become so intelligent that they even somewhat rival AIs is strongly frowned-upon as unwise, if not impossible.
We differ slightly in our reads of the main point of the book. I don't think it's a fragility-vs-robustness argument, but rather a fragility-vs-antifragility argument. The long time gap since the previous book lets him set up that robustness, in the end, doesn't matter. Even the Immortal God Emperors eventually get bored. Leto II has to keep cloning his Idahos just to have a reference point for his progress.
This isn't a juxtaposition of a humanity fragile to AI (baseline humans) versus robust to AI (humans with eugenic anti-AI weapons), but rather it's about the Scattering itself. Despite what Leto tells everyone except Siona, the Scattering is the real plan, the payoff of the Golden Path. That's why he lets her steal those documents at the beginning, so he can ensure an I Told You So many thousands of years in the future. Through Leto, Herbert articulates a view that homogenization (globalization?) and ease will make humanity fragile to *some* sort of existential threat, without any reasoning about what the existential threat would have to be. And Leto should think this; his father took over the Known Universe in a decade using only some superpowers and a childhood of training; and then Leto did the same thing a few years later! That's some serious tail risk. Herbert is making a Pareto/Taleb style argument for the experimentation itself, the Laboratory of the Scattering, as a hedge against an uncertain future. As Leto confides in Siona after making her drink from him, the point of the Golden Path is to make humanity rebel against it, scattering to the universe. In scattering humanity, in breeding in this hatred for sedentariness, Leto II believes he is securing a future for humanity by eliminating the possibility that any one future for humanity could dominate the wider human phenomenon. He's not condemned to be remembered as a mere dictator, he's actively playing that villain. He believes that will make people run so far and diversify so wide that they conquer the very idea of dictatorship. Yes, this strategy does protect against Butlerian AI - but not by being merely a little stronger in specific ways, but by the unpredictable variations and wonders that humanity will build for itself out of necessity and sheer churlishness. And, to cryptically hint at the later books, it works.
To use American mythological imagery, Leto II is King George III deliberately. He's creating a planned unplanned America, which will secure humanity against British-style stagnation. Or a Nero for Rome in Western imagery, letting Europe free to flourish free of Roman stagnation.
---------------------------------------
Also, it's true that the characters rant against rationalism, with the heroes being romantic and intuitive and sneering at the foibles of bureaucrats and other people who think they know things. Yet those same characters that the books lionize and who sneer at rationalism ... are often basically human computers, frequently are the product of long and deliberate breeding programs, and think in terms of conservation on the millennial scale. I think the books are a lot more ambivalent on rationalism, with their main complaint being people who get lost in the sauce.
>How much better would it then be if we could create an AI that restricts all other AI’s, but only as a secondary goal necessary to reach its primary objective of identifying what it is that makes humans human and amplifying those traits in a positive direction until we (as a still identifiably human race) can stand against AI’s on equal footing.
I mean why bother? Who cares at that point? A lot of this line of reasoning seems predicated on a value for human existence and mode of life that isn't really warranted.
It was never about AI. That theme followed from something Herbert encountered in his days, unfortunately something nobody seems to know much of these days. Look up his interviews on politics, Kennedy. Two core elements: strong man syndrome, and manipulation of participatory systems through automation. Herbert explored the impact of the Simulmatics Corporation.
Not thinking machines. But men embracing machines to be free, only to be enslaved by men owning the machines.
Herbert looked at humanity as a being. It's an organic perspective, an ecology of types and tests. Human behavioural biology meet human social psychology on a species level. The being continues to adapt, evolve, or it does. If the being adapts its environment to itself, the same problem follows as when the being destroys an environment. The analogy is of behaviour suitable for survival.
Diversification. The Golden Path was a constructed pressure valve set to burst. So that the being branches off in an ever expanding path of diversity. Never becoming subject to the pitfalls of conservative predispositions. Those leading only to status quo, providing a path of destruction of both being and environment.
AI. That's a theme the foundation rolled with simply because they were convinced to. By a breadwriter and a lobbyist. Ego did the rest.
Suprema! I read the entirety of Dune in the 90's and I still consider GE as setting the standard for epic scale novels. Your review is equally brilliant for mining the gems from Herbert's work. I still believe inspiration creates more insight than encumbered tour-de-force thought.
This novel was my favorite of the Dune series, though I understand why some people feel the opposite. A lot of Leto's ramblings are Frank's philosophical musings, which can get a bit mystical, or in his attempts to be poetic, end up being obscured or confusing.
I love a book that can make me think about it for years, and the more thought I put into this one, the more insights I uncover. Are they wholly mine, wholly Frank's insights, or something blended? Almost doesn't matter ... he succeeded in sparking my mind into motion.
Here's some of the better bits:
-Theme: Hard times make people strong. Sure, but there's something deeper, under the surface of that theme. Strength is always relative, and musings on the species interesting, but let's bring it down from the epic level of the human race to you personally. Frank is saying: You seek enough foresight and knowledge on your world to assert control. You think you want to escape uncertainty, and thereby find some level of peace. But humans have been evolved to be explorers, adventurers. We are at our best when forging ahead into the uncertainty, when adapting to the unknown. You will not be fulfilled by anything less.
-Leto isn't seeking to make humanity "stronger", he's seeking to adapt them to an upcoming threat he can see through prescience. The threat is prescience itself, mechanized or otherwise, locking humanity into a future course from which there is no escape. Humanity that escapes prescience itself has gained back what Leto already lost: free will. Therefore he intentionally acts like a predator on the entire species to force an adaptation against his prescience. He knows he has succeeded only when his prey can destroy him. In this, he completely succeeds.
-A predator must seem cruel to the prey, and Leto hates this fact and hates the personal cost of losing his humanity. He remembers past lives enduring such catastrophes as he must inflict to force humanity to adapt, so he knows the cost in suffering. He knows he will be hated for it. The pain of what he must do, has been doing for thousands of years makes him the best tragic hero in SF.
-Whether Leto is wrong, right, or his values just too alien, what he wants is for humanity to resist him. Those of you who hate him are having the exact reaction he wants for people to have to him. Duncan Idaho is a proxy for the reader. Frank keeps bringing you back to the same lessons, but instead of just telling you directly, he wants for you to unfold the mystery for yourself. Exactly how Leto treats Duncan. On some level it really feels arrogant and condescending, but you have to concede a point to Frank that this will get some people to think about it.
I'm in awe of how insightful and well-written this book review is (and other finalists as well). Why is amateur hour so much better than what the professionals produce?
That said,
"But if AI is inevitable then the dominance of a specific AI is likely;" -- I think literally everyone in the world but me who's interested in AI risk makes this assumption, and I think it's a terrible, terrible assumption, which blinds us to the best approaches to making AI "safe".
This assumption projects a human understanding of consciousness onto AIs. What does"singleton" even mean? Even the human brain doesn't implement a single person, in the sense of having a single consciousness which directs all of its cognitive computation. Our brains do a great deal of very clever computation which we're never aware of, and even that which we are aware of can dissociate into independent competing thought processes under times of extreme stress (or at least, I myself once subjectively experienced this).
A singleton would be massively distributed, and many of its sub-processes would be at least as complex as humans, and largely locally information-encapsulated, for efficiency of data transfer. Either there are supernatural spirits, or whatever gives rise to consciousness in our brains would give rise to consciousness in these sub-processes. There is no law of nature saying that the consciousness of an entity rules out the consciousness of its components. Or, perhaps there is such a law of "nature", if we read "nature" as "organic, evolved nature", which requires individuals sufficiently disconnected for competition between them to drive evolution. But it will not apply to AI.
It seems likely that the most-efficient structure for a singleton will be one which uses free-market and evolutionary competition among its sub-processes, delegating nearly all computation to very low-level processes. Such a singleton won't be a singleton at all. It would be more correct to think of that singleton as the United Nations, coordinating nations, which supervise states, which supervise counties, which supervise cities and districts, which supervise groups, which supervise conscious agents. If there's anything anywhere in that structure that will be conscious, I think it's most likely to be at a very *low* level, where agents process very local information at great depth. The higher-level processes must deal with data at such a high level of abstraction, and so physically dispersed, that if they experience consciousness, it will not be one that can be mapped onto physical space.
In any case, I don't think there's any justification for believing that the notion of a "singleton AI", conceived of as a single consciousness with a single will and set of values, is coherent, in the sense of corresponding to anything likely to be constructable and stable.
This means that we aren't justified in believing that an AI singleton would be a bad thing. It might be a new, very different society, containing moles of agents something like us, with consciousness, values, and desires.
And this is the important thing--figuring out how to guide AI development in a way that preserves consciousness, worthy values, and enough desires to keep things interesting. The idea that trying to preserve literal human life, not evolving or growing, but merely out-fucking death forever, is *altruistic* rather than *incredibly selfish*, is idiotic. Saying "Humanity above all!" is even worse than saying "Deutschland über alles!" It's just bigotry taken one level of abstraction further, and made orders of magnitude more evil for wanting to stop evolution.
"In the past, this tactic has resulted in various people realizing the necessity of Leto’s actions and joining his team. Siona, however, is not convinced."
That's wrong / someone has wrong memories about the book. Siona sees the Golden Path and agrees to Leto's plan - assassinating him is actually part of it, she agrees to it and commences the whole thing.
I don't know about winning the contest, but "It’s like Frank Herbert was worried you’d mistake him for the Reasonably Tough Emperor of Dune and over-corrected in the other direction" is a strong competitor to win the "best *sentence* in an ACX Book Review" contest.
Neat - the conclusion summarizes some discussions I had with Eliezer on the Extropian list, oh, somewhere in the late 1990's. Indeed, we the people need the Friendly AI to become our loving and caring owner and to keep the UFAIs (Unfriendly AIs) away until we grow up and put the smackdown on them ourselves.
A universe where pets pick their owners. Unlikely.
I love how this guy writes, very entertaining!
A friend of mine once told me that he had started reading God Emperor in defiance of all of the people who'd told him what a poorly-written slog it was, and then finished reading God Emperor as a self-imposed punishment for refusing good advice.
Nitpick: the review mentions Siona's father ("She is as refined as her father, with none of his domestication") without (as far as I can tell) ever saying who it is. It's Moneo.
I think this is my favorite book review. The author explains the book so well that I, having stalled out after Children of Dune, no longer feel an obligation to try yet again to read it, but at the same time feel more interested in once again trying to read it than before.
Glad someone reviewed this book. My favorite book for the themes you may out.
Worth noting gwern has some writing on Dune that does a great job explaining some of the apparent plot holes.
This is overall a good review, and I really want to like it, but since this is a contest I have a few nitpicks. First, the review sort of wanders. It casually refers to things the reader who hasn’t read the first few books do not know, and while it (sometimes) later explains them, even when this happens the reader is left confused for however long it takes the author to get around to it. Second, the comparison to AI felt a little bit forced, and I don’t think you did the legwork to justify the connection, though maybe my confusion is due to the fact that I still don’t, in fact, really understand the plot. (To be fair it feels like the plot still being unclear may be a function of the book rather than the review.) Finally, the comparison to Yud just felt distracting, off putting, and unnecessary. Yes yes, I know not praising his name constantly makes me a rationalist heretic (or maybe a post-rat) but the review would have had just as much substance and less distraction if you had just said something like “the God-Emperor would be the equivalent of our ai safety hawks”
But with that said, and other than the points I noted, the review was very entertaining and informative, so good job.
Oh man, I read God Emperor of Dune (and the Dune Trilogy) so long ago. It was pretty bad, but still I finished it, so clearly something about it captivated me
"...he has Mua’Dibs on her..."
Cheeky. I loved it.
Beautiful review! Agree with pretty much all the points. Only other interesting bit to add is a defense of Hwi Noree. I always read her as an object lesson about getting what you want, a bit of a Siren (something created which you specifically irresistible and which mystifies everyone else) that leads to your own undoing because of the unspoken truth that, in the end, you’re a machine that solves problems so if someone comes and gives you what you want so you don’t have problems to solve anymore, you find you don’t really want everything you think you want.
Good review. I get why some people don't like it, but I think God Emperor is a masterpiece. It's my favorite book of the series.
Personally I think Hwi is a fascinating character. She genuinely loves Leto, but she's self aware enough to understand she's a weapon sent to destroy him. Leto knows this too, and Hwi knows he knows it. She trusts him enough to let him let her lead them both to destruction. That destruction turns out to be not just a release from his tortured existence, but the fulfillment of his Golden Path. It's a really poignant love story.
This review is not bad, but there are some mistakes, and the review somehow fails to capture a lot of the weirdness of the book. More importantly, while I respect the attempt, reading GEoD (the title has no "the", this review repeatedly uses TGEoD incorrectly) as an AI analogy is very strange. One of the major points of all the Dune books is an emphasis on human capabilities instead of relying on technology. GEoD is not about "build a friendly AI to protect humanity from malicious AI", it is about "build a friendly and extremely powerful and intelligent human to protect humanity from malicious AI, and anything else that might threaten it". Reading Leto as an AI is the exact opposite of what Frank Herbert intended.
It's also kind of weird that it's omitted in the plot summary that Leto isn't really being overthrown: his plan was always to get murdered by a rebellious Atreides at some point down the line, and he's looking forward to it because if humanity can beat him, he figures they can beat pretty much anything else. He foils several other assassination attempts (no, really, there are at least four others, almost every major faction tries to kill him, it seems like a very regular thing) and if anything he's bored and disappointed with them. He's so excited by a novel attempt by a couple Bene Gesserits that he tries to recruit one of them. He really wants to die, but he's really hard to kill, and he also has a berserk mode because he's partially turning into a sandworm. Did I mention that the book is even weirder than the review explained?
Other weird things: so Hwi Noree is actually the second attempt to build a perfect human to ensnare Leto. The first try was her uncle Malky, who was made to be pure evil, but this trap didn't work because Leto's enemies didn't understand that he's actually a good person at heart (or thinks he is). After Malky failed, he went back and explained this, and they tried a purely good version. Malky shows up towards the end of the novel, doesn't seem that evil, and gets executed by Moneo, wrapping up a minor subplot that doesn't get much attention. Also, a lot of the book is just Leto talking to other characters about politics. Also there's a weird scene where Duncan freaks out when he sees two girl soldiers kissing, and Moneo explains Leto's theory about homosexuality and the military. Also Leto has an all-female army and they have a weird secret kind-of sexual worship ceremony that Duncan gets to attend. This book is really weird!
> as well as having a pretty good ability to predict the future
No, he has near-absolute ability to predict the future. In the first Dune book, Paul has trouble when he encounters situations where there are too many outcomes, like major battles and closely matched knife fights. Leto casually orders his troops into position during an attack on his festival city to perfectly counter every enemy position at once. He complains about how he's already experienced everything that's happened to him, which is why he's almost never surprised. He has a few blind spots, but he literally designed them himself (he breeds Siona and her invisibility genes, and also has a custom "no-sphere" built for him, which is a mechanical room that prescient vision can't see into).
> but also has the combined leadership/political experience of every member of his family line at least as far back as the ancient Greeks.
Well, beyond that, he doesn't have a separate Leto persona like his father Paul did, he subsumed his identity into an amalgamation of all the most successful and ruthless leaders in his family line. And he's at the end of a very long line of aristocrats that goes not only back to Agammemnon, but to Ancient Egypt (in Children of Dune, he talks to his twin sister in ancient Egyptian). He implies it's basically back to human pre-civilization.
> Scattered musings on the least popular Frank Herbert novel
Oh, dear, no. Not even close. It's also nowhere near his worst book. Read the Heaven Makers or The Eyes of Heisenberg.
> Today, those thoughts are old hat, but this book came out in 1981.
This seems less impressive when you think about the sort of similar conceits that Asimov had in his Robot stories from the 1940’s.
I think you've misunderstood not only this book but all of its sequels. (Not counting the stupid son's bullshit, which you have correctly ignored as utterly worthless, and in fact appear not to have ignored quite as totally as you ought to have. It's the stupid son, not Frank Herbert, who thinks about AI.)
Leto's fear is not of AI but of stagnation. People claim to want peace and stability - Herbert and Leto think that this wish, if granted, would be catastrophic. The Golden Path consists of demonstrating, to all of humanity, what peace and stability really look like, convincing them in the only way that really works that they Do Not want that.
Leto is not limiting the ability of his society to produce AI. In fact, he quite deliberately allows Ix to develop far more than it did in the preceding era. He quietly but unmistakably *encourages* the development of the no-chamber into the no-ship and the rediscovery of the (spice-free) mechanical Navigator, which can plot courses between the stars.
He restricts computers, but that's not actually important. Everyone before him, for thousands of years, restricted computers **much more** than he did What he does do that's new, on the other hand, is restrict computer-analogues. The Pre-Atreides empire, the CHOAM era, had a wide variety of nontechnological substitutes for the things that computers would otherwise do. Guild Navigators to plot courses between stars. Bene Gesserit as repositories of knowledge (among other things). Suk Doctors to serve as perfectly neutral providers of medical services (this didn't work very well, but it was also less necessary than the others). And, crucially, Mentats to do rapid calculations. Leto restricted all of these things tightly and banned Mentats entirely; he allowed some violations to pass unprosecuted, but very few.
If he doesn't fear AI, why does he restrict computer-like things? Because it makes the Golden Path easier. Even with nigh-omniscience, a stranglehold on the unobtainium, and thousands of years, controlling everything that happens to ensure you hit a very narrow target is not easy. (Here you can get some of your AI Alignment Problem parallels back.) By massively restricting the capabilities of even the most powerful factions in Leto's Empire, he keeps the capabilities of the opposition within his ability to compute. He also, like he strangles movement to create a lasting urge to move, strangles power to create a lasting urge to become powerful.
These are two sides of the same thing. The instinct Leto II wants to instill in the human race is, quite simply, "Become Ungovernable".
Which he does, by **governing the shit** out of humanity.
While Leto II is mostly powerful enough to dispense with lies, the best way to read God-Emperor of Dune is not to listen to his words, but to look at the end result and assume that he succeeded. While he's characteristically cryptic, the hints he gives over the course of the book, and Children of Dune before it, point squarely toward the result being a win. He wanted humanity to spread beyond the power of any tyrant to lock them into stagnation or decline, and ensured this by becoming the best tyrant possible.
Crucially, Leto II's death is a covert *assisted suicide*. He is surprised by the timing, because he did not see Hwi coming, but *he had the power to escape the assassination* and chose not to use it. For the maximum effect on the Scattering, he had to be killed, and so his death is an attack. But it is an attack he could defend against, did he so choose. He does not so choose; anti-prescience is ready within one generation, and so he lets the reins of power drop, secure in the knowledge that he has finished the millennia of suffering that he, and Paul before him, saw laid out before them, and that he has successfully made humanity unbreakable.
This was great.
I have some dumb questions that are probably going to annoy the hell out of the people here who know Dune backward and forward. But one of the virtues of this review was that it made the themes comprehensible and compelling to somebody who doesn't really know the Dune mythos at all. (I saw the David Lynch film many years ago, but that's it.) So humor me, if you will.
1) Remind me what's the connection between the sandworms and the spice? The sandworms more or less poop out spice, or something like that?
2) If people want spice, and sandworms make spice, but sandworms are also a pain for people to live around, why does anybody need or want to live on the sandworm planet? Why not just leave the planet to the sandworms, let them multiply and make spice, and do whatever you need to do to harvest the spice without establishing a permanent human settlement? Why are the battles over territorial control of the sandworm planet, rather than over some strategic chokepoint that controls access to it?
3) Is Leto a god? Is "god-emperor" just a title, in the same way that centuries of rulers in the Islamic world styled themselves as insan-i-kamil (the perfected person)? Or is the idea that he's actually a god who also happens to be an emperor?
I'd like to take issue with the following point: "Therein lies the secondary risk of AI; by creating an AI that coddles us, we run the risk of never advancing as a race again. As Dune points out, soft environments tend to weaken rather than reinforce."
This premise is often trotted out, and I don't think it's properly examined.
A) It's very hard to hold something as fluid as a race of intelligent people in stasis. I ask for proof that the human ambition for exploration, discovery and pushing boundaries can be quashed by being coddled. It seems more likely that coddling includes facilitating self-discovery and improvement than the alternative.
B) What's the inherent good in advancing? If you're imposing a harsh environment, you're hurting people. Why? If it is to enable a better future where people suffer less, then when you eventually reach that future you can go soft, right?
Finally, creating an AI that strives to "[identify] what it is that makes humans human and [amplify] those traits in a positive direction" seems like it's more likely to 'lock in' current conceptions of humanity and goodness than anything else. What ultimately needs to advance is not just humanity, but the conception of what humanity is. If anything, an AI that facilitates people in preference-driven self-modification seems better.
I love that people did fiction!
Decent review, and yes, God Emperor is definitely worth reading. It's gotta be the second best in the series.
But I want to strongly disagree with something that seems to be assumed in passing in both this review and the Dune books:
"success can make you soft," "soft environments tend to weaken rather than reinforce"
This is a classic tough guy trope. It sounds like wisdom. But I'm pretty sure it's complete nonsense. If the research shows us anything, it's that winners tend to win more. And common sense should tell us that we live in a much softer environment than any of our historical and prehistorical forebears, and yet we do much tougher things than they ever did (like live to 90, for example).
In fact, this kind of claim is often used to conflate at least three different things, and each one would need to be separately teased out and researched.
(1) Do individual people tend to become weaker given success/a soft environment? This is a question about human psychology. My feeling is that the answer is unambiguously no, but there's always room for argument.
(2) Do human cultures tend to develop into weaker forms given success/a soft environment? This is a question about cultural evolution, which is highly underspecified. I'm not sure I even know what a weak or strong culture is, so this question remains entirely open.
(3) Do evolving creatures tend to develop into weaker forms given success/a soft environment? I suspect this question is in fact much more context-dependent than it looks. We can fairly straightforwardly understand strength in this question to mean something like 'population resilience in the face of external environment shock,' but I suspect you'd just find a bunch of contradictory answers. An organism that's had it easy might have a high population, and they're harder to wipe out; an organism that's had it easy might have a narrow ecological niche (e.g. diet), so a highly specific environment shock might have a more devastating effect. I dunno if these factors make it a wash, but they're at least complex enough that I would reject any blanket statement like "success makes softness."
I gave up on this review about halfway through. The writing is too disorganized and rambling, which is not a style I enjoy reading.
I do find the concept of a "human that can threaten AI" to be quite interesting, though. From what I can tell, the answer (in the novel) seems to be a form of genetic engineering. But can anyone recommend me some books that explore this concept? I.e. in order to be competitive with machines, humans have to start becoming more ambitious with their "biological" status.
> Today, those thoughts are old hat, but this book came out in 1981.
And the meaning of this fact still hasn't really hit this subculture in the face hard enough yet. He wasn't before his time, he was _at_ his time.
I swear to God. A subculture of autodidacts who never really thought about how uneducated that made them, and what very real implications that had for their ignorance. Discovering those books now, late in the plot, believing they were the main character or at least knew him.
God played a cruel trick on the SFBA Rationalists.
That Mua'Dibs pun was awful and brilliant. I can only aspire to such wordsmithing. Kudos
As a casual aside, I wonder if Frank Herbert read Romance of the Three Kingdoms and got the idea of Hwi Noree from one of the female characters (Diaochan) because of the shared plot elements of an assassination plot against the ruler/emperor orchestrated by the opposition using a femme fatale as bait, involving a jealousy-induced betrayal by the emperor's right-hand man. Or maybe this is a more common plot device than I thought.
I was never fond of the "Golden Path" stuff from the second book onwards. It felt like it squandered the themes of the first novel, about the dangers of heroic narrative and being swept along by forces beyond your control. Then we get to the 2nd and 3rd books, and it turns out we really did need the Hero after all - in fact, without him, humanity is going to go extinct!
I do like "God Emperor of Dune", though. Weird book, and often pretty funny. There's a running gag about how every time someone finds out that Leto is "getting married" to Hwi, they immediately look at his giant worm-body for genitalia - he eventually thinks that maybe he should have some big appendance grafted on to shock them.
Not a fan of the last two books either. Only good things in those were Miles Teg, weird child form Miles Teg, that surprisingly explicit sex scene from "Heretics", and the genuinely sad parts where you see a planet's biosphere get destroyed (I remember thinking that would have been more interesting if the project had failed because of unseen parasites/wildlife/etc that undermined it).
I just...what? Totally bouncing off this one. There's too much inside-baseball, in both directions: I did read several of the __Dune__ series years and years ago, but wasn't all that interested in it, and definitely don't remember enough to understand all the references and implications here. And on the other side, either it's a really overstretched metaphor, or I'm *way* too ignorant about AI x-risk to see the connection. Just reads like excessive pattern-matching to me, the same way the review of __The Society of the Spectacle__ attempted to grant retroactive prescience credit to some dead French guy. Predicting the future is easy and common if you go looking for it...
Even aside from all that, I find it impossible to rank a book review of the lone fiction entry in an otherwise all-nonfiction field. Too hard to get over the (well-deserved imo) innate reflex to be really suspicious of Arguments from Fictional Evidence. And as the reviewer openly admits, this is a really unpopular book, even among those strictly evaluating it as typical scifi...
One last quibble:
>Nayla represents the balance of humanity’s nobility against its flaws
...who the hell is Nayla? That name isn't referenced anywhere else in the review.
Seeing Leto as a sort of AI is weird given Herbert's emphasis on human evolution.
The central theme of the entire Dune series is that natural selection ( Arrakis, Salusa Secundus) and artificial selection (Bene Gesserit, Siona) are a superior path to building machines (The Butlerian Jihad, Ix) or direct genetic engineering (Bene Tleilax). Leto has taken over the Bene Gesserit artificial selection plans and applied them until humanity surpassed him. That was his Golden Path. The successful assassination was Leto's own death-by-Siona long term plan.
This natural selection theme is also central to Herbert's The Dosadi Experiment where humans and aliens are isolated on a hostile planet for generations resulting in a population tougher and more capable than the general population softened by peace and prosperity
Which (ideally none) of the other books in the series should I read before I read this one?
Two things: 1. AI is explicitly "done with" in the Dune-iverse: There had been an AI-war/luddite revolt, the Butlerian Jihad. Humans won. THAT story is: history. Thus the main (?) idea of this review - the story of "GEoD" as an allegory/mirror/prescient 1981 version of "YUDvsAI" - seems off by a wide margin.
2. Loved the review. Well written. 35 years since I read Dune, appreciate the memory. :D
To balance the comments, I think the AI analogy is reasonable and well argued.
Nice ending of a review of a book that I will never read.
However, the ending begs the question of why so many are afraid of an AI killing us all. We all know that mankind must go extinct sooner or later, so why not go with a bang rather than a long drawn-out whimper?
In the foreseeable future, humanity’s lot is to live in societies dominated by the old. That is, in (hopefully) safe and pleasant societies (the way old people like it), but also stagnant and sclerotic societies. Africa will be the last continent with zest and youthful vigor, but the demographic transition is in full swing there also, and 100 years from now even Niger will probably look like Japan. The future is old take-no-risks societies, not steampunk or fun dystopian Mad Max scenarios. More than 90 of the 194 countries in the world already have fertility rates below 2,1 (reproduction level), and the 60+ age group is the fastest-growing population group everywhere.
Given this likely, whimperish future, why not welcome a superior-being AI that kills us off and replaces us, since such an AI will be cleverer, and also better able to further explore Cosmos – since AI machines are likely to be better suited to millennium-type space travel than living organisms.
...not that I think such an AI event is ever to materialize, but if one for the sake of argument assumes that the superior-AI-is-coming people are right: Hell, why not go out with a Bang, since go we sooner or later must.
From the footnote;
"Bene Gesserit were essentially witches who could draw on the ancestral memories of women in their ancestral line"
Yup.
"and they bred this ability into Paul, Leto’s father."
Nope, nope, noppity nope. This explicitly does not happen in Dune. Paul says "I'm not what they expected".
"As a bonus, they can also draw on male ancestor’s memories"
See above. This is apparently the actual point of the Bene Gesserit breeding program - to create a male version of a Reverend Mother, who has access to the Other Memories of the male line.
"as well as having a pretty good ability to predict the future and know what’s going on in the present."
Gawd.
The BG attempt to plan the future, not predict it, by examining the past - via Other Memory. They are aware that their methodology has a fatal flaw - they can only examine half of the past, hence the breeding program. Note what Paul's insight into the BG is in the testing scene early on in Dune, whilst the Atreides are still on Caladan - they do politics. Mohaim then says that the Spacing Guild does a form of higher mathematics.
Until Paul shows up, nobody is really aware of the capability of the spice to produce prescient visions.
These are fundamental errors in understanding the set up in Dune, let alone GEoD. As a consequence, it's difficult to take the review seriously.
"She is as refined as her father, with none of his domestication."
Who is her father?
"Nayla represents the balance of humanity’s nobility against its flaws"
Unlike the other characters mentioned in this paragraph, this is your first time referencing Nayla. Was Nayla supposed to have a section?
"if AI is inevitable then the dominance of a specific AI is likely"
I know Robin Hanson disagress with this. In Dune, it's because machines in general have been suppressed so there wouldn't be any other AIs to compete with the first to emerge.
I read up to the part where it mentioned AI risk and then I rolled my eyes and stopped reading - nothing against the review or the reviewer. I have really enjoyed this blog and SSC before it for several years, though I’m not really a part of the hardcore rationalist base. But I haven’t made the mental switch to prediction markets and AI risk with everyone else. Sometimes the prediction markets reveal something interesting, but I am unable to make myself care about AI risk beyond enjoying Asimov’s robot stories. I cant bring myself to believe that it poses a real and imminent threat, and I tire of reading about it. Maybe it’s just me.
1. Imo, you have to put the author's full name in the first paragraph of a book review.
2. I read Frank Herbert's trilogy in the late 70s, probably because of the release of Children of Dune. I have a recollection of picking up God Emperor when it came out and putting it down after a dozen pages. After all these years I can't recall whether that was because it bored me, or the difference between being in college and high school, or because I just had more important things to read. Thankfully, this review confirms I probably didn't miss much.
Is this book review superfluous or add much to the John Leanard NYTIMES review that came in 1981?https://www.nytimes.com/1981/04/27/books/books-of-the-times-104040.html
I have previously suggested that properly evaluating book reviews might well require reading the underlying original text (revealing my great books educated snobbery I guess.) But I'm also thinking that an even stronger case could be made for not reinventing the wheel. Why new book reviews at all when there are old book reviews available?
Is it even more problematic when basically the review threads are being "subcontracted" out? At least John Leonard had his name on the byline, was being paid, and probably has some copyright interest in his review.
I disagree with your characterization of the Two final books. I thought Chapterhouse Dune was basically an encounter with death, or rather a "final message before departing". Heretics was about the danger of autocratic control. But, of course, with any book what you see depends on what you bring to it.
With God Emperor you've certainly nailed major themes, though I disagree about it not having a plot. And I consider Dune Messiah to be the weakest book in the series. (I've only read it a couple of times.)
My takeaway from this review is, "We should build Optimus Prime then probably stop building autobots before we're sure humans alone could beat Decepticons," and I couldn't agree more.
Review-of-the-review: 8/10
During the last book review season I mistook Scott's review of 1001 Nights for a contest submission and was like "wow, this is creative and unusually good, I might vote for it". This review feels like an attempt to recapture that, uh, magic. It's not as successful-- the attempt to draw lessons about AGI from it comes off as forced-- but I still enjoyed it. It strikes about the right balance of snark and appreciation for discussing Herbert's work (and Yudkowsky's). On the other hand it has noticeable weak points: it fails to question Herbert's silly notions of how a world might operate (e.g. the "hard times / strong men" meme that acoup.blog's "Fremen Mirage" series addresses) and the presentation is rough around the edges. On the whole I don't expect to be voting for it but I enjoyed reading it and might actually give the book a try (I've read the first three). As always, many thanks for contributing!
Reading this comment section - I'm going to get murdered if I mispronounce anything, aren't I...
Hwi Noree was one-dimensional in the extreme...but Herbert seemed to acknowledge that and claimed that she was designed that way.
Her one-dimension was being like an uber-empathic mirror.
Which lets her understand Leto (at least at some level), and after his 3500 years of loneliness, that is like an intoxicating, addictive substance to him.
And Leto realizes that it is a trap, and damns the Ixians and Tleilaxu for doing it...but then falls into the trap anyways.
It doesn't make Hwi Noree a more interesting character, but it makes her role in the story more tolerable.
Also, I kind of think that Leto let himself fall into that trap only because he successfully set Siona on her path.
The 'Good AI that coddles us and thereby makes us weak' is pretty much the crux of The Culture series by Iain Banks, which I love almost as much as the Dune series.
I think Herbert was far more thought provoking, and I kind of recoil at the thought of being the 'pets' of AI like the non-AI citizens of The Culture are....but Banks' books are also excellent.
Interestingly (at least to me), in Dune the goal is specifically to help humans develop superhuman abilities and spread forever.
In The Culture, humans being modified to become so intelligent that they even somewhat rival AIs is strongly frowned-upon as unwise, if not impossible.
> Imagine Leto as a very big Big Yud (Eliezer Yudkowsky, rationalism’s original AI doom-sayer);
Some rich projection going on here: Leto is nothing like Yud.
Very enjoyable review!
We differ slightly in our reads of the main point of the book. I don't think it's a fragility-vs-robustness argument, but rather a fragility-vs-antifragility argument. The long time gap since the previous book lets him set up that robustness, in the end, doesn't matter. Even the Immortal God Emperors eventually get bored. Leto II has to keep cloning his Idahos just to have a reference point for his progress.
This isn't a juxtaposition of a humanity fragile to AI (baseline humans) versus robust to AI (humans with eugenic anti-AI weapons), but rather it's about the Scattering itself. Despite what Leto tells everyone except Siona, the Scattering is the real plan, the payoff of the Golden Path. That's why he lets her steal those documents at the beginning, so he can ensure an I Told You So many thousands of years in the future. Through Leto, Herbert articulates a view that homogenization (globalization?) and ease will make humanity fragile to *some* sort of existential threat, without any reasoning about what the existential threat would have to be. And Leto should think this; his father took over the Known Universe in a decade using only some superpowers and a childhood of training; and then Leto did the same thing a few years later! That's some serious tail risk. Herbert is making a Pareto/Taleb style argument for the experimentation itself, the Laboratory of the Scattering, as a hedge against an uncertain future. As Leto confides in Siona after making her drink from him, the point of the Golden Path is to make humanity rebel against it, scattering to the universe. In scattering humanity, in breeding in this hatred for sedentariness, Leto II believes he is securing a future for humanity by eliminating the possibility that any one future for humanity could dominate the wider human phenomenon. He's not condemned to be remembered as a mere dictator, he's actively playing that villain. He believes that will make people run so far and diversify so wide that they conquer the very idea of dictatorship. Yes, this strategy does protect against Butlerian AI - but not by being merely a little stronger in specific ways, but by the unpredictable variations and wonders that humanity will build for itself out of necessity and sheer churlishness. And, to cryptically hint at the later books, it works.
To use American mythological imagery, Leto II is King George III deliberately. He's creating a planned unplanned America, which will secure humanity against British-style stagnation. Or a Nero for Rome in Western imagery, letting Europe free to flourish free of Roman stagnation.
---------------------------------------
Also, it's true that the characters rant against rationalism, with the heroes being romantic and intuitive and sneering at the foibles of bureaucrats and other people who think they know things. Yet those same characters that the books lionize and who sneer at rationalism ... are often basically human computers, frequently are the product of long and deliberate breeding programs, and think in terms of conservation on the millennial scale. I think the books are a lot more ambivalent on rationalism, with their main complaint being people who get lost in the sauce.
>How much better would it then be if we could create an AI that restricts all other AI’s, but only as a secondary goal necessary to reach its primary objective of identifying what it is that makes humans human and amplifying those traits in a positive direction until we (as a still identifiably human race) can stand against AI’s on equal footing.
I mean why bother? Who cares at that point? A lot of this line of reasoning seems predicated on a value for human existence and mode of life that isn't really warranted.
It was never about AI. That theme followed from something Herbert encountered in his days, unfortunately something nobody seems to know much of these days. Look up his interviews on politics, Kennedy. Two core elements: strong man syndrome, and manipulation of participatory systems through automation. Herbert explored the impact of the Simulmatics Corporation.
Not thinking machines. But men embracing machines to be free, only to be enslaved by men owning the machines.
Herbert looked at humanity as a being. It's an organic perspective, an ecology of types and tests. Human behavioural biology meet human social psychology on a species level. The being continues to adapt, evolve, or it does. If the being adapts its environment to itself, the same problem follows as when the being destroys an environment. The analogy is of behaviour suitable for survival.
Diversification. The Golden Path was a constructed pressure valve set to burst. So that the being branches off in an ever expanding path of diversity. Never becoming subject to the pitfalls of conservative predispositions. Those leading only to status quo, providing a path of destruction of both being and environment.
AI. That's a theme the foundation rolled with simply because they were convinced to. By a breadwriter and a lobbyist. Ego did the rest.
this
I think that this might be the best review for no other reason than this: Mua’Dibs
I spit out my coffee and laughed even though it spattered on my screen.
Suprema! I read the entirety of Dune in the 90's and I still consider GE as setting the standard for epic scale novels. Your review is equally brilliant for mining the gems from Herbert's work. I still believe inspiration creates more insight than encumbered tour-de-force thought.
This novel was my favorite of the Dune series, though I understand why some people feel the opposite. A lot of Leto's ramblings are Frank's philosophical musings, which can get a bit mystical, or in his attempts to be poetic, end up being obscured or confusing.
I love a book that can make me think about it for years, and the more thought I put into this one, the more insights I uncover. Are they wholly mine, wholly Frank's insights, or something blended? Almost doesn't matter ... he succeeded in sparking my mind into motion.
Here's some of the better bits:
-Theme: Hard times make people strong. Sure, but there's something deeper, under the surface of that theme. Strength is always relative, and musings on the species interesting, but let's bring it down from the epic level of the human race to you personally. Frank is saying: You seek enough foresight and knowledge on your world to assert control. You think you want to escape uncertainty, and thereby find some level of peace. But humans have been evolved to be explorers, adventurers. We are at our best when forging ahead into the uncertainty, when adapting to the unknown. You will not be fulfilled by anything less.
-Leto isn't seeking to make humanity "stronger", he's seeking to adapt them to an upcoming threat he can see through prescience. The threat is prescience itself, mechanized or otherwise, locking humanity into a future course from which there is no escape. Humanity that escapes prescience itself has gained back what Leto already lost: free will. Therefore he intentionally acts like a predator on the entire species to force an adaptation against his prescience. He knows he has succeeded only when his prey can destroy him. In this, he completely succeeds.
-A predator must seem cruel to the prey, and Leto hates this fact and hates the personal cost of losing his humanity. He remembers past lives enduring such catastrophes as he must inflict to force humanity to adapt, so he knows the cost in suffering. He knows he will be hated for it. The pain of what he must do, has been doing for thousands of years makes him the best tragic hero in SF.
-Whether Leto is wrong, right, or his values just too alien, what he wants is for humanity to resist him. Those of you who hate him are having the exact reaction he wants for people to have to him. Duncan Idaho is a proxy for the reader. Frank keeps bringing you back to the same lessons, but instead of just telling you directly, he wants for you to unfold the mystery for yourself. Exactly how Leto treats Duncan. On some level it really feels arrogant and condescending, but you have to concede a point to Frank that this will get some people to think about it.
I'm in awe of how insightful and well-written this book review is (and other finalists as well). Why is amateur hour so much better than what the professionals produce?
That said,
"But if AI is inevitable then the dominance of a specific AI is likely;" -- I think literally everyone in the world but me who's interested in AI risk makes this assumption, and I think it's a terrible, terrible assumption, which blinds us to the best approaches to making AI "safe".
This assumption projects a human understanding of consciousness onto AIs. What does"singleton" even mean? Even the human brain doesn't implement a single person, in the sense of having a single consciousness which directs all of its cognitive computation. Our brains do a great deal of very clever computation which we're never aware of, and even that which we are aware of can dissociate into independent competing thought processes under times of extreme stress (or at least, I myself once subjectively experienced this).
A singleton would be massively distributed, and many of its sub-processes would be at least as complex as humans, and largely locally information-encapsulated, for efficiency of data transfer. Either there are supernatural spirits, or whatever gives rise to consciousness in our brains would give rise to consciousness in these sub-processes. There is no law of nature saying that the consciousness of an entity rules out the consciousness of its components. Or, perhaps there is such a law of "nature", if we read "nature" as "organic, evolved nature", which requires individuals sufficiently disconnected for competition between them to drive evolution. But it will not apply to AI.
It seems likely that the most-efficient structure for a singleton will be one which uses free-market and evolutionary competition among its sub-processes, delegating nearly all computation to very low-level processes. Such a singleton won't be a singleton at all. It would be more correct to think of that singleton as the United Nations, coordinating nations, which supervise states, which supervise counties, which supervise cities and districts, which supervise groups, which supervise conscious agents. If there's anything anywhere in that structure that will be conscious, I think it's most likely to be at a very *low* level, where agents process very local information at great depth. The higher-level processes must deal with data at such a high level of abstraction, and so physically dispersed, that if they experience consciousness, it will not be one that can be mapped onto physical space.
In any case, I don't think there's any justification for believing that the notion of a "singleton AI", conceived of as a single consciousness with a single will and set of values, is coherent, in the sense of corresponding to anything likely to be constructable and stable.
This means that we aren't justified in believing that an AI singleton would be a bad thing. It might be a new, very different society, containing moles of agents something like us, with consciousness, values, and desires.
And this is the important thing--figuring out how to guide AI development in a way that preserves consciousness, worthy values, and enough desires to keep things interesting. The idea that trying to preserve literal human life, not evolving or growing, but merely out-fucking death forever, is *altruistic* rather than *incredibly selfish*, is idiotic. Saying "Humanity above all!" is even worse than saying "Deutschland über alles!" It's just bigotry taken one level of abstraction further, and made orders of magnitude more evil for wanting to stop evolution.
Great. Now I’m going to go read this stupid book again. You made it sound way better than I remember it being. :)
Human worm hybrid? Yuk
"In the past, this tactic has resulted in various people realizing the necessity of Leto’s actions and joining his team. Siona, however, is not convinced."
That's wrong / someone has wrong memories about the book. Siona sees the Golden Path and agrees to Leto's plan - assassinating him is actually part of it, she agrees to it and commences the whole thing.
I don't know about winning the contest, but "It’s like Frank Herbert was worried you’d mistake him for the Reasonably Tough Emperor of Dune and over-corrected in the other direction" is a strong competitor to win the "best *sentence* in an ACX Book Review" contest.
Thanks reviewer! I already loved the book itself, yet never thought of your grid of reading. Fun and full of food for thoughts!