If you run this again next year, could you please make a statement along the lines of: “You do not need to write a 10,000+ word book review. A review that is just 3,000 words may be perfectly fine.”
I would even further say “Based on the types of books typically reviewed, 5,000 words should be most people’s targets. Brevity is a virtue.” However, the first statement is more neutral.
I'd rather read a tightly-written 6000-word review than a meandering 3000-word review, but all else being equal, the shorter review is probably better.
I don't know if it was mentioned in the guidelines, but "get an editor" is just good advice in general.
Length seems a quality criterium like any other. Why not let the market decide? If you find a book review too long, don't read it and don't vote for it.
I think there is some value in clarifying that, even though some of the best ACX/SSC posts historically have been very long, this doesn't require that your review is very long. I imagine some submitters just assume that long length is standard and expected.
I'd go further than that and impose an upper word limit. A 50,000 word novella is just not the same type of artistic creation as a 2000 word review. Judging them in the same category is unfair to both.
I think it's as silly to try to suggest a length for a book review as it is for a book (unless the advice relates to a very specific book). I would be happy reading a brilliant book review even if it was twice as long as the book it was reviewing.
It'd be interesting to see a post analysing whether review length, or review sentiment, or comment sentiment, or book popularity, or anything like that correlates with contest performance
Could link to the review analysis in next years contest announcement and let reviewers decide if they want it to influence their review
yeah I came here to say the same thing, not sure I'm going to be able to pick one to vote on but that's because there were several very strong candidates.
Good thing you get three choices! At least RCV makes it feel a little less high-stakes: it's pretty unlikely that your top two go head-to-head in the finals.
I've only read 6 of these so far, but I don't understand how the quality is so good. At least three of them rank as some of the best reviews I've ever read. I'll try to read the rest of the reviews before voting but its going to be hard to pick a favourite.
I'm trying to pick my top three, but I'm running into the problem that I didn't keep track of which ones I particularly liked or not. Looking at the posts again I have a vague notion of "oh yeah, I think I read this", but not much beyond that. I should keep notes or something for next time – assuming I remember to do that, either.
I swear I need to create my own scoring rubric for next year. There are so many dimensions: length, entertainment value, accuracy, knowledge gained, etc.
Yeah, I was keeping track on what my favorite is, but I didn't know I'd get to rank 3. I think I know what they are, but I'll have to do a reread about 4 before voting.
I ran into that problem last year, so this year I've been rating them as I go on a 10 point scale. 4 points for how interesting and well explained the subject is, 4 points for insights, critiques and further comments by the review author, and 2 points for quality of the writing itself.
I tried to put a mini-review in the comments of each, separated into sections on the review and on the book. But I find I only did about half of them. **sigh**
I want to softmax the likes on all the posts and then trade based on the output for some free mana, but I'm too lazy. If anyone else wants to try it please share your results.
Softmax with what base? Using the natural base results in the softmax being too "sharp" and if the likecounts on the manifold page are accurate, it would predict over 99.99% to one particular review winning.
I don't think the multimedia part was what put that in my top 3, it was the humor of the review. All the other reviews were very good, but imo, that review was the only one that had humor at the level of Scott's normal output. The Phoenix Wright game was part of it, but not the only reason it was funny.
Agreed, multimedia is imho one of those artistic cheat codes like "metafiction", where rubbing it on the thing makes it both better (usually) and more-acclaimed (always(?)). Homestuck and 17776 might not've gotten Wikipedia pages or die-hard fans if not for their multimedia elements, and it's hard to think of any phrase other than "artistic cheat code that magically makes it better!" when checking Wikipedia's list of metafictional works: https://en.wikipedia.org/wiki/List_of_metafictional_works
EDIT: To be clear, this is *not* a bad thing! It does *not* delegitimize any metafictional works, or your enjoyment of them! It's just part of a lens through which I view a lot of media in general, which I find useful for predictions and gears-level understanding.
I'm hoping it doesn't win because I'm pretty sure Scott wrote it, and while I do think his writing is excellent, it seems a shame to have one of the rare "non-Scott-centric" parts of the blog actually being written by Scott after all.
That depends do we think Scott is a liar? I believe him when he says he's not an entrant. If we don't believe that, then why are we even having a book review contest at all and soliciting entries? Scott could just dump them all and put up sixteen of his own reviews, and people who submitted a review that was dumped wouldn't know any different.
If he's going to sneak one of his own reviews in, why not have all the reviews be his?
But isn't that reading the tealeaves? Maybe this year he thought he could leave it out because by now we should know he doesn't enter his own competition.
Or maybe this is the SCANDAL that we need to derail his presidential campaign efforts! 😀 Corruption and shenanigans! Secret goings-on! Meddling with prediction markets to line his pockets!
He said he wasn't an entrant in previous years contests. I'm pretty sure he explicitly called out that he wasn't confirming or denying that he entered this years contest.
As for why, it would be a way to have his writing judged more objectively by not having his name immediately attached to it. Or maybe just for fun. It might be a bit awkward if he did enter and wins though. Perhaps he'll preemptively disqualify himself.
> 3: Last year people kept guessing that various reviews were by me, and I had to reassure you that no, I hadn’t secretly entered my own contest. This year I refuse to confirm or deny anything, so have fun speculating!
Personally I think he just did it for fun. Maybe he was trying out a slightly different voice, maybe he wanted to see if we could tell it was him, maybe he wanted to see if he could win. Obviously I don't think he's doing it for the prestige of winning, and certainly not the money. But I dunno. If I was him I'd find it quite an interesting experiment. :)
Just to clarify, are people using "ranked-choice voting" to mean "instant-runoff voting"? My understanding was that the former is any voting method where voters submit ranked preferences and includes borda count, instant runoff, single transferable vote, the ranked pairs condorcet method, and others.
Incidentally I prefer any ranked method to approval voting (which amounts to judging everything either "good" or "bad"; simplistic and unhealthy imo).
I have a question about the prediction market: why not look at it? In general, aren't markets supposed to be looked at? If markets would interfere with this competition, then what's the more general argument for thinking that prediction markets are good for lots of things?
I'd assume Scott doesn't want people voting for those that seem likelier to win. But I think it's just as likely people would see one of their favourites ranking low on the market and think "this one deserves more recognition and *needs* my vote".
1) Scott isn't collecting data, so far as I know. He's not trying to figure out who the world's best writer is. He's just running a competition for fun.
2) If independent data is more valuable, and people looking at a market makes data no longer independent, then isn't that an argument against (public) betting markets? Scott seems very keen on them - Mantic Monday and all that - so presumably he doesn't think they always have this negative effect on the value of data.
Re (2), the issue here is that the market prices can influence the object-level question of who wins (since people voting in the poll might rationally choose to vote for the top candidates where their votes have the most chance of making a difference), which is not true of most prediction markets. Arguably this is an issue with election polls in general. (As argued in https://bigthink.com/the-present/consequences-of-political-polling/?)
1. I'm running a competition for fun, but I still want to get people's unbiased opinion on which one is the best.
2. I don't think this is an argument against betting markets more than it's an argument against any kind of communication. When Dr. Fauci says "You should get a COVID vaccine", this is making it harder for people to research whether vaccines are good in a purely unbiased way, but it's still worth it. I think of prediction markets as potentially serving a similar goal of communicating consensus.
I'd like to see more fiction reviews next year. The audiences for fiction and non-fiction are probably a bit different, so maybe there could be two contests?
That's a good idea. Fiction reviews are also harder to pull-off in my opinion, since spoilers can be more of an issue than with a non-fiction review. A good fiction review will keep this fine balance between talking about the book in-depth and not revealing every plot detail.
1. Extremely high level this year. Could have picked 7 as "the best" by 'objective' criteria. This year I had to go: 'Which did I enjoy most' - 2. Maybe all finalists should get a small premium (300)? The uninitiated might not grant bragging rights to unrewarded runner-ups. - 3. This contest works so well, I would not change the format. Sticking to a book review is a rather helpful constraint - and some pieces showed: not much of a constraint; some mentioning several books, even several authors, some not much reviewing the book but the field - and if one is intrigued, one has a book to dig deeper into (and usu. one that got discussed in expert-circles). 4. If a loooong piece is not too long to make it to the finalists, it is: not too long. If too few vote the 10k word reviews as the winner, that vote is votum enough for authors to learn from. The educated mind - I do feel sorry for not voting for it. But it educated my mind. Somewhat. During the week it took to read. ;)
Yes the level was really high. I just skimmed through the whole lot again, and I don't even ask for my afternoon back.
I find it hard to rate the reviews on their quality though - I'd have to spend way more time than I have on them to separate that from the most obvious criteria: how much I care about the topic!
Assuming everyone has already voted before reading the comments, here's my approximate rating in descending order:
Guilty as stated. But then Scott is pro-manifold. And I really do not get that particular market. - I could have phrased it as a trigger warning, maybe. At least, I did not link to reddit's "Sneer Club" (honestly named, at least), where the same sick sneers at Scott can be found. (Hey, he at times links to the NYT even, though his usual stance is: “Écrasez l'infâme!”)
I generally agree that stocks are mostly stupid. I think, in rare cases, they can give out some information, but mostly they are just for fun. I made this "market" just for fun to see how it would go up when Scott does something cool and go down if Scott does something not cool. But I agree mostly stocks are a waste of time and I normally don't trade on them.
Appreciated. Answer and experiment. I consider it (as other "markets") failed; any bet on "Whill ACX reach over X subscribers/paid subscribers till D.M.YY?" should work better.
Investing in things which mean anything (like shares of a company, or bets) is so last century.
In this century people invest in Dogecoins and NFTs. When you invest in a Manifold permanent market, where does your play money go? It does not matter! It is all meaningless.
Permanent markets are a chance to bet on trends over time. If you think everyone likes Scott now but eventually they'll like him less you can short him. It does seem a little weird - I wouldn't mind seeing some analysis of the game theory of such a market.
I looked at them again to refresh my memory, and oof, the third book's arguments that machines will never ever be generally intellegent because complex things cannot be represented mathematically do not stand up well in September 2023.
(You might still vote for the summary because reviewing a book is not the same as agreeing with it)
Instead of voting for one’s favorite, I’d like to see a “which books did you buy and read” question. I think on some level a book review has failed if nobody is actually interested in the book afterwards
Not necessarily, some reviews are telling readers that a book is bad and you shouldn't waste your time reading it. It's a book 'review' not a book endorsement
Argh. This got unanchored from Jacob's "This is the hardest book review yet..." when I logged in. But yes, it was wonderfully difficult. Another great book review year.
I think the reviews this year didn't have as much of a sense of 'knock off version of scott's writing style without the wit or insight' that many reviews in previous years had.
If you run this again next year, could you please make a statement along the lines of: “You do not need to write a 10,000+ word book review. A review that is just 3,000 words may be perfectly fine.”
I would even further say “Based on the types of books typically reviewed, 5,000 words should be most people’s targets. Brevity is a virtue.” However, the first statement is more neutral.
I'd rather read a tightly-written 6000-word review than a meandering 3000-word review, but all else being equal, the shorter review is probably better.
I don't know if it was mentioned in the guidelines, but "get an editor" is just good advice in general.
I penalize verbose reviews in my ranking.
Length seems a quality criterium like any other. Why not let the market decide? If you find a book review too long, don't read it and don't vote for it.
I think there is some value in clarifying that, even though some of the best ACX/SSC posts historically have been very long, this doesn't require that your review is very long. I imagine some submitters just assume that long length is standard and expected.
EOC: "Why not let the market decide?"
yes
P.B.: "some value in clarifying...doesn't require that your review is very long"
and yes
I'd go further than that and impose an upper word limit. A 50,000 word novella is just not the same type of artistic creation as a 2000 word review. Judging them in the same category is unfair to both.
I think it's as silly to try to suggest a length for a book review as it is for a book (unless the advice relates to a very specific book). I would be happy reading a brilliant book review even if it was twice as long as the book it was reviewing.
It'd be interesting to see a post analysing whether review length, or review sentiment, or comment sentiment, or book popularity, or anything like that correlates with contest performance
Could link to the review analysis in next years contest announcement and let reviewers decide if they want it to influence their review
This is the hardest book review yet, I kept thinking I had my choice but every review was really amazing this year
yeah I came here to say the same thing, not sure I'm going to be able to pick one to vote on but that's because there were several very strong candidates.
Good thing you get three choices! At least RCV makes it feel a little less high-stakes: it's pretty unlikely that your top two go head-to-head in the finals.
Yes, but I just wrote down 1, 2 and 3 right away. And they were all great!
Yeah, I just picked three ones I liked out of a list of about 5-6 I'd consider top contenders.
+1
I've only read 6 of these so far, but I don't understand how the quality is so good. At least three of them rank as some of the best reviews I've ever read. I'll try to read the rest of the reviews before voting but its going to be hard to pick a favourite.
Njal Saga link points to the wrong post.
The link for Njal's Saga goes to Man's Search for Meaning instead.
I'm trying to pick my top three, but I'm running into the problem that I didn't keep track of which ones I particularly liked or not. Looking at the posts again I have a vague notion of "oh yeah, I think I read this", but not much beyond that. I should keep notes or something for next time – assuming I remember to do that, either.
Same. Not writing down my thoughts after reading the reviews was a big error. And I don't really fancy re-reading them all again.
I swear I need to create my own scoring rubric for next year. There are so many dimensions: length, entertainment value, accuracy, knowledge gained, etc.
Yeah I try to write at least a few words on my thoughts after I read each one or I would find voting pretty much impossible.
Yeah, I was keeping track on what my favorite is, but I didn't know I'd get to rank 3. I think I know what they are, but I'll have to do a reread about 4 before voting.
I ran into that problem last year, so this year I've been rating them as I go on a 10 point scale. 4 points for how interesting and well explained the subject is, 4 points for insights, critiques and further comments by the review author, and 2 points for quality of the writing itself.
I starred the email notifications of the ones I liked.
I tried to put a mini-review in the comments of each, separated into sections on the review and on the book. But I find I only did about half of them. **sigh**
I want to softmax the likes on all the posts and then trade based on the output for some free mana, but I'm too lazy. If anyone else wants to try it please share your results.
Love to help but don’t understand a word of this.
Softmax with what base? Using the natural base results in the softmax being too "sharp" and if the likecounts on the manifold page are accurate, it would predict over 99.99% to one particular review winning.
Tough choices. Now I have to ask myself if I want to skim these all again.
I just don't have strong enough preferences between them to only choose 3. I'll let everyone else decide who gets the official title
I loved it, but I’m sort of afraid Njal’s Saga will win and next year there will be a slew of imitators of the multimedia Phoenix Wright aspect of it.
I don't think the multimedia part was what put that in my top 3, it was the humor of the review. All the other reviews were very good, but imo, that review was the only one that had humor at the level of Scott's normal output. The Phoenix Wright game was part of it, but not the only reason it was funny.
Agreed! It was great as a whole. My fear is that people will think that the video was what made it great and not the writing.
Agreed, multimedia is imho one of those artistic cheat codes like "metafiction", where rubbing it on the thing makes it both better (usually) and more-acclaimed (always(?)). Homestuck and 17776 might not've gotten Wikipedia pages or die-hard fans if not for their multimedia elements, and it's hard to think of any phrase other than "artistic cheat code that magically makes it better!" when checking Wikipedia's list of metafictional works: https://en.wikipedia.org/wiki/List_of_metafictional_works
EDIT: To be clear, this is *not* a bad thing! It does *not* delegitimize any metafictional works, or your enjoyment of them! It's just part of a lens through which I view a lot of media in general, which I find useful for predictions and gears-level understanding.
I'm hoping it doesn't win because I'm pretty sure Scott wrote it, and while I do think his writing is excellent, it seems a shame to have one of the rare "non-Scott-centric" parts of the blog actually being written by Scott after all.
I'm not the only one who thinks he wrote it:
https://manifold.markets/ShakedKoplewitz/did-scott-write-the-njals-saga-book
That depends do we think Scott is a liar? I believe him when he says he's not an entrant. If we don't believe that, then why are we even having a book review contest at all and soliciting entries? Scott could just dump them all and put up sixteen of his own reviews, and people who submitted a review that was dumped wouldn't know any different.
If he's going to sneak one of his own reviews in, why not have all the reviews be his?
He hasn't said he's not an entrant this year. That part of the usual blurb is conspicuously missing.
But isn't that reading the tealeaves? Maybe this year he thought he could leave it out because by now we should know he doesn't enter his own competition.
Or maybe this is the SCANDAL that we need to derail his presidential campaign efforts! 😀 Corruption and shenanigans! Secret goings-on! Meddling with prediction markets to line his pockets!
He said he wasn't an entrant in previous years contests. I'm pretty sure he explicitly called out that he wasn't confirming or denying that he entered this years contest.
As for why, it would be a way to have his writing judged more objectively by not having his name immediately attached to it. Or maybe just for fun. It might be a bit awkward if he did enter and wins though. Perhaps he'll preemptively disqualify himself.
Matt has already mentioned it, but here is the post where Scott teased that he might enter:
https://www.astralcodexten.com/p/open-thread-274
> 3: Last year people kept guessing that various reviews were by me, and I had to reassure you that no, I hadn’t secretly entered my own contest. This year I refuse to confirm or deny anything, so have fun speculating!
Personally I think he just did it for fun. Maybe he was trying out a slightly different voice, maybe he wanted to see if we could tell it was him, maybe he wanted to see if he could win. Obviously I don't think he's doing it for the prestige of winning, and certainly not the money. But I dunno. If I was him I'd find it quite an interesting experiment. :)
The book review is Awesome! (if it ain't broke, don't fix it.)
I had no problem picking 1,2,3... my number 1 finished near the bottom, but 2 and 3 near the top.
I am a bit surprised by the prediction market results
Just to clarify, are people using "ranked-choice voting" to mean "instant-runoff voting"? My understanding was that the former is any voting method where voters submit ranked preferences and includes borda count, instant runoff, single transferable vote, the ranked pairs condorcet method, and others.
Incidentally I prefer any ranked method to approval voting (which amounts to judging everything either "good" or "bad"; simplistic and unhealthy imo).
Hmm. I feel the need to cast a protest vote of "entries were not up to the standard of previous contests". I didn't really care for most of these.
#7 Safe Enough .
I too loved lots of the reviews this year.
I have a question about the prediction market: why not look at it? In general, aren't markets supposed to be looked at? If markets would interfere with this competition, then what's the more general argument for thinking that prediction markets are good for lots of things?
I'd assume Scott doesn't want people voting for those that seem likelier to win. But I think it's just as likely people would see one of their favourites ranking low on the market and think "this one deserves more recognition and *needs* my vote".
Yes. Generally, independent data points are more informative/valuable than ones that have causally influenced each other.
But...
1) Scott isn't collecting data, so far as I know. He's not trying to figure out who the world's best writer is. He's just running a competition for fun.
2) If independent data is more valuable, and people looking at a market makes data no longer independent, then isn't that an argument against (public) betting markets? Scott seems very keen on them - Mantic Monday and all that - so presumably he doesn't think they always have this negative effect on the value of data.
This seems like a really good point, to me. Interested to see more responses on this thread.
Possibly related to the maxim "bet on facts, vote on values", although perhaps the book review quality is some mix of fact and values.
Re (2), the issue here is that the market prices can influence the object-level question of who wins (since people voting in the poll might rationally choose to vote for the top candidates where their votes have the most chance of making a difference), which is not true of most prediction markets. Arguably this is an issue with election polls in general. (As argued in https://bigthink.com/the-present/consequences-of-political-polling/?)
1. I'm running a competition for fun, but I still want to get people's unbiased opinion on which one is the best.
2. I don't think this is an argument against betting markets more than it's an argument against any kind of communication. When Dr. Fauci says "You should get a COVID vaccine", this is making it harder for people to research whether vaccines are good in a purely unbiased way, but it's still worth it. I think of prediction markets as potentially serving a similar goal of communicating consensus.
Thanks, Scott!
I'm not sure I agree with this, as I don't think people are "unbiased" to start with, but I understand what you mean. I'll ponder it some more.
I'd like to see more fiction reviews next year. The audiences for fiction and non-fiction are probably a bit different, so maybe there could be two contests?
That's a good idea. Fiction reviews are also harder to pull-off in my opinion, since spoilers can be more of an issue than with a non-fiction review. A good fiction review will keep this fine balance between talking about the book in-depth and not revealing every plot detail.
I like this idea. Things like mythology or medieval romances kind of straddle the border a bit at times, though.
1. Extremely high level this year. Could have picked 7 as "the best" by 'objective' criteria. This year I had to go: 'Which did I enjoy most' - 2. Maybe all finalists should get a small premium (300)? The uninitiated might not grant bragging rights to unrewarded runner-ups. - 3. This contest works so well, I would not change the format. Sticking to a book review is a rather helpful constraint - and some pieces showed: not much of a constraint; some mentioning several books, even several authors, some not much reviewing the book but the field - and if one is intrigued, one has a book to dig deeper into (and usu. one that got discussed in expert-circles). 4. If a loooong piece is not too long to make it to the finalists, it is: not too long. If too few vote the 10k word reviews as the winner, that vote is votum enough for authors to learn from. The educated mind - I do feel sorry for not voting for it. But it educated my mind. Somewhat. During the week it took to read. ;)
Yes the level was really high. I just skimmed through the whole lot again, and I don't even ask for my afternoon back.
I find it hard to rate the reviews on their quality though - I'd have to spend way more time than I have on them to separate that from the most obvious criteria: how much I care about the topic!
Assuming everyone has already voted before reading the comments, here's my approximate rating in descending order:
The weirdest
Man's search
Zuozhuan
Mind of a bee
The educated mind
Rise and fall
Njal's saga
Why nations fail
Safe enough
Marble cliffs
Lying for money
Secret govt
Cities and separatism
Public citizens
Why machines
The laws of trading
Laws of trading last? Very surprised, that was my favourite. Or does descending order mean descending from worst to best?
Manifold: What does this "market" even mean? (comments stink like reddit-sneer) https://manifold.markets/JustifieduseofFallibilism/scott-alexander-stock-permanent
I really wish people didn't spread things they say stink.
Guilty as stated. But then Scott is pro-manifold. And I really do not get that particular market. - I could have phrased it as a trigger warning, maybe. At least, I did not link to reddit's "Sneer Club" (honestly named, at least), where the same sick sneers at Scott can be found. (Hey, he at times links to the NYT even, though his usual stance is: “Écrasez l'infâme!”)
I generally agree that stocks are mostly stupid. I think, in rare cases, they can give out some information, but mostly they are just for fun. I made this "market" just for fun to see how it would go up when Scott does something cool and go down if Scott does something not cool. But I agree mostly stocks are a waste of time and I normally don't trade on them.
Appreciated. Answer and experiment. I consider it (as other "markets") failed; any bet on "Whill ACX reach over X subscribers/paid subscribers till D.M.YY?" should work better.
Investing in things which mean anything (like shares of a company, or bets) is so last century.
In this century people invest in Dogecoins and NFTs. When you invest in a Manifold permanent market, where does your play money go? It does not matter! It is all meaningless.
Oh my gosh, in the comments there really is somebody doing the "Scott Alexander spits in your face - NYT" bit for real.
Permanent markets are a chance to bet on trends over time. If you think everyone likes Scott now but eventually they'll like him less you can short him. It does seem a little weird - I wouldn't mind seeing some analysis of the game theory of such a market.
Thank you. So it is mad. Shorten Scott? Heaven and earth shall pass away, but his words shall not pass away.
See the section "Taking Stock" at https://www.astralcodexten.com/p/mantic-monday-1302023
Thank you! Seems, 'Manifold leadership IS totally resigned now to having stocks be meaningless Ponzi schemes'. Cool.
There should be a side betting market somewhere
I looked at them again to refresh my memory, and oof, the third book's arguments that machines will never ever be generally intellegent because complex things cannot be represented mathematically do not stand up well in September 2023.
(You might still vote for the summary because reviewing a book is not the same as agreeing with it)
Instead of voting for one’s favorite, I’d like to see a “which books did you buy and read” question. I think on some level a book review has failed if nobody is actually interested in the book afterwards
Not necessarily, some reviews are telling readers that a book is bad and you shouldn't waste your time reading it. It's a book 'review' not a book endorsement
Agreed. I asked for next year honorable mentions checkboxes so I can at least wave at the reviews that didn't quite make it into my top 3.
Argh. This got unanchored from Jacob's "This is the hardest book review yet..." when I logged in. But yes, it was wonderfully difficult. Another great book review year.
This is going to take me forever to read! Oh well, starting with number 1... fortunately I'm a huge Jane Jacobs fanboy.
I think the reviews this year didn't have as much of a sense of 'knock off version of scott's writing style without the wit or insight' that many reviews in previous years had.
The prediction market link isn't working for me.
Is this a geofence thing?