None of the venerable tests being accurate over 135 sounds like something that should be noticeable outside this community. Is this an accepted fact of IQ research, or if not how does that match with this? Is IQ just not normally distributed at the tails or what is going on?
I assume respondents were not asked how many tests have they taken. If a large proportion of respondents have taken more than 1 test, it seems fair to assume they are not reporting their average score but their highest score.
> When asked in a 2004 interview with The New York Times what his IQ is, Hawking gave a curt reply: "I have no idea. People who boast about their IQ are losers."
That point spread chart was illuminating. I've taken four IQ tests. When self-reporting my IQ I average my three highest. I don't like to mention my Mensa results which were significantly lower. ;-)
IQ testing is actually a pretty hard problem at the top (and bottom) ends. If you want to fit a nice bell curve, you better have your test cover a large part of the range, i.e., going down all the way to 100.
But if you're answering only 50-100 questions, someone with an IQ of 135+ is going to get 99-100% correct answers on such a small sample size and with such a small range of question difficulties.
So, tests given to high IQ people that claim to be accurate up to 160 or higher necessarily need to do much fancier (= error-prone) statistics to get significance all the way up there.
I thought speed was sometimes used as a technique to help tease apart scores at the top end.
Yes, if you just give everyone 50 questions then everyone above a certain level will just get 100%. But if you give a timed test and supply more questions than you'd expect even the best candidates to get through in the time, and Alice gets through 100 questions and gets 99% of them right, and Bob gets through 50 questions and gets 99% of them right, then Alice's IQ is higher.
IIRC, the in-person Mensa test I took used this technique.
I am anecdotally skeptical of this, as someone who tests WAY better when speed is taken into account but less well when speed is not the fulcrum of the test. Anecdotally my processing speed is not perfectly correlated with my overall ability to get hard problems correct.
I also dislike the speed method. Although I haven't been tested properly, I am temperamentally the kind of person who likes to take my time and double check things to make sure I get them right. I don't like to hurry, and I certainly don't like having to constantly evaluate whether I should pass on a difficult question in hopes of solving several subsequent questions in the time I would have spent figuring it out.
Processing speed is an interesting aspect of mental capacity, but I don't know if it's appropriate to roll it up into IQ or not.
Wouldn't a better method be to use the first 10 questions to determine the person's IQ quartile, then administer a test tuned for that quartile?
"Processing speed is an interesting aspect of mental capacity, but I don't know if it's appropriate to roll it up into IQ or not."
While we're on the subject of "IQ tests are not administered in the ways people intuit them as being administered"...yes, processing speed is a whole section on the Wechsler tests. One of the big four factors that goes equally into determining someone's full-scale IQ. It and working memory are the "least g-loaded", so people with very high scores on other factors frequently regress to the mean on them, lowering their FSIQs noticeably. (This is one of numerous reasons the traditionally quoted survey-IQs are implausibly high -- even if we select strongly on both verbal and nonverbal ability, and are genuinely doing that rather than primarily selecting on intellectual *interest*, enough people with super-high "general ability" scores will regress to the mean on "cognitive proficiency" to get not-astronomical FSIQs. I can't see a good argument that we're selecting for high CPI separately to high GAI.)
Oh, that’s fascinating. I had no idea. In my own subjective experience my cognitive processing (including working memory, if that makes sense to throw in there) are more SDs above the norm than than my general ability is. Is that a rarity? How do you tease out the two? I’ve never taken a true IQ test, so I don’t have a sense of what that looks like, and I didn’t follow your last sentence.
If you have taken enough IQ tests to compare their relative merits, they don't work on you anymore, because it's possible to learn how to take IQ tests over time.
Depends. The standard IQ tests have very large practice effects. The SAT is, or at least was, specifically designed to have a small practice effect, and will still work.
I think Raven's matrices is traditionally a timed subtest.
Digit recall span probably helps some in separating people at the high end, too.
Unfortunately, both of these are fairly trainable tests - they're probably quite reliable if you're testing 10-12 year olds to help decide whether they should qualify for the gifted program(*), but if you're dealing with adults who want to get a high score and are willing to do some preparation, they don't hold up so well.
(*) and of course even here we can imagine that parents might train their kids to help them get in, given enough knowledge. I suppose the kids themselves might even decide to do some online research, but if a 6th grader decides to do mnemonic exercise to improve their digit span recall, on their own initiative, it's probably a good idea to let them in the program regardless....
Yeah, definitely based on the couple online tests that were Ravens Matrices linked or adjacent, if I had someone to explain to me the 'trick' of "now in this question, what they want is...", then I would do better. You definitely could 'train' for some of these. So doing it for the first and only time is a good measure as to genuine smarts to be able to figure out cold what the problem is, but doing it any more than that, and you're training on how to answer, not how to figure it out from first principles.
I believe Raven's in particular is known for very large practice effects.
It's an open question to what degree this might be unintentionally driven by psychologists enjoying the feeling that they can answer the questions. I'd put better odds on it mostly being driven by laziness and the low stakes associated with IQ tests that are called "IQ tests".
> Alice gets through 100 questions and gets 99% of them right, and Bob gets through 50 questions and gets 99% of them right, then Alice's IQ is higher.
Or Alice prepared for the test and Bob didn't.
When I was younger, and I was taking tests, I made the foolish decision that I wanted to know my actual cognitive ability, so on principle I did not prepare for such tests. It would be "cheating." I was Bob.
I still tested top 1% on standardised nationalised tests, but not well enough to qualify for all the perks that come with being 1/1000 instead of 1/100.
Now, many years later, I'm kicking myself. I should have done anything to inflate those scores to the max. So many opportunities lost. Good on Alice and her family and peer group. She'll do well in life.
But meanwhile, the stats have us Bobs as lower-IQ. So be careful trying to draw conclusions there.
Your argument "proves too much". It works just as well against any form of IQ testing, whether timed or not (you can drill yourself on vocabulary, practice the matrices, etc, and some people do and some don't). It's not an argument for doing untimed rather than timed tests.
I have good news for you. I prove little, not even much, let alone "too much."
It is possible my mindset is a minority view, but I suspect that variations of it are common enough to show up in the data if accounted for.
In any case, it's just a hypothesis, and you can throw away whatever conclusions you might draw from it except that "Philo Vivero was a contra datapoint to this analysis."
The fallacy of "proving too much" isn't anything to do with "proof" in a mathematical sense or a forensic sense. It just means you're making an argument that would apply to a larger class of things than you intend.
For example, if you said "I don't use AI tools because it's imperative that my work be 100% error-free", that would "prove too much" because human work isn't 100% error-free so your argument would inadvertently rule out human work as well as AI work. (Of course there may be other good arguments against using AI, but I'm just saying that wouldn't be one of them.)
Here, you're presenting your objections as if they're objections to *timed* IQ tests specifically, but you haven't given any indication of why they wouldn't also apply to all IQ tests, or indeed all tests of any kind.
> I still tested top 1% on standardised nationalised tests, but not well enough to qualify for all the perks that come with being 1/1000 instead of 1/100.
In my experience, no such perks exist. What are you thinking of?
Well, if you score high enough on the PSAT you qualify for a National Merit Scholarship. Probably there are other scholarships that are linked to standardized test scores.
No, you don't. You qualify to apply for a National Merit Scholarship. You don't qualify for the scholarship itself.
The only thing your PSAT score will get you is "semifinalist" status. That's worth nothing; there are two levels above that, of which the top one ("scholar") gets scholarships.
...while not a serious issue in the current context, it's actually also important to have the test give a reasonable distribution all the way down to 65 or so as well...
> So, tests given to high IQ people that claim to be accurate up to 160 or higher necessarily need to do much fancier (= error-prone) statistics to get significance all the way up there.
This is not correct; they can use the same statistics. They need their questions to be more difficult.
Well, they need enough of their questions to be more difficult to detect differences in fractions of percentages at the top end, while still having enough of a difficulty gradient to detect the low end (i.e. so that it‘a not the case that nobody below 100 does no better than chance.) unless you’re talking about secondary tests of people who tested above a threshold in the first test.
> Well, they need enough of their questions to be more difficult to detect differences in fractions of percentages at the top end, while still having enough of a difficulty gradient to detect the low end
No, they don't. It's a test given to high IQ people; there's no need to have good resolution at the low end.
There's no theoretical obstacle to combining both purposes into one test, but it will, as a stylized fact, double the size of the test, since questions at the low end are totally uninformative at the high end and vice versa. Most tests won't do this because testees tend to give up partway through long tests. But again, that's not a problem with the statistics.
This means that tests that are intended for "general use" have poor resolution at the high end, and at the low end, because they're testing people in the middle and adding resolution outside the middle lowers accuracy in the places where they care about it. But "tests given to high IQ people that claim to be accurate up to 160 or higher" have no such problem. There is no reason for them to spend any effort on low-end resolution.
It is, as far as I can tell, an accepted fact of IQ research. I've encountered a few different figures for when IQ tests stop working reliably. For regular tests, anything above 130, basically. High-IQ-specific tests might, maybe, be able to get to 145.
But, 130 is kind of the ceiling.
For adults. For childhood IQ tests, which are testing something subtly different, the regular tests hit 180, and the high-IQ alternatives, I gather, hit 200, before the tests just get too unreliable. (But - and it's a huge but - this maximum decreases with age, because the number reflects something like precociousness, and you just have less and less room to be precocious the older you are.)
The problem, I gather, is that it's basically impossible to craft a set of problems such that IQ 145 people will reliably be able to answer them, but which IQ 130 people reliably won't.
It would be silly for IQ to not be normally distributed at the tails since it is a completely artificial unit. You can scale it however you want making the tails whatever you want. Normal is the one, "normal" (non-abnormal) choice.
I think the range restriction just refers to the idea that the farther out on the tails you get the less likely it’s measuring something and not just chance?
That seems likely. I was merely pointing out that if an IQ distribution has fat tails, that is entirely the fault of the people designing the test (and associated grading curve).
I suspect that the high emphasis that IQ tests place on speed and on short-term memory necessarily corrupts scores over 130. There are bound to be lots of dumb people who have great STM or short reaction times, and lots of smart people with poor STM or slow reaction time. These components are probably in the exam only because they correlate strongly with what we might call "actual smarts" in large samples. But I think they aren't what IQ tests should be looking for.
Yeah I was wondering if it isn't at least in part a change in the honesty level- when you get high enough, the number of legitimate scores gets smaller, and there's probably a subset of liars who just give a super high number rather than just adjusting upwards a bit.
Another issue is that the official in-person Mensa test is apparently not a standard IQ test with standard deviation of 15, and most people taking it don't realise this.
My Mensa certificate just says on it "This gives a true IQ" and doesn't indicate which test (e.g. Wechsler, Stanford-Binet, Cattell, etc) it was. So for the first few years of Scott's survey I just wrote the number from that certificate, which does sound pretty high. I've since inferred from the linked SSC discussion that it was probably Cattell and that the corresponding Wechsler and S-B numbers are quite a bit lower. But given that the survey rubric doesn't specify which test or indicate that some so-called "IQ tests" (including the one used by Mensa) use a differently-calibrated scale, and given that even the Mensa certificate itself doesn't indicate which test it used, I am not surprised that some commenters are reporting too-high scores, in good faith.
I took the Mensa test about ten years ago, and I remember being frustrated that they provided a [conversion table][1] from the Mensa test score to a percentile, and from percentile to IQ, but only at whole-percentile resolution -- making it near-useless within their target audience.
(I joined in the hope of finding IRL peers to talk to. It didn't work very well; it turns out that similar intelligence doesn't imply any particular common interests. Didn't help that most of the membership were twice my age.)
What exactly _is_ the Stanford-Binet test anyway? Or any other specific test?
Surely it can't be just a single test with a specific set of questions, or else you could take it twice and ace it.
But it also can't be a totally general formula for generating questions, because such a thing is pretty unimaginable and each question wouldn't be properly calibrated anyway.
So maybe it's a large collection of questions from which you choose a subset to give as an IQ test? But then surely all these questions have escaped out into the wild right now and are being carefully drilled into kids at cram schools?
The people on the lowest end of the IQ distribution probably can't work as click workers due to being illiterate, unable to follow instructions, or similar reasons. So you'd expect the average to be above 100 in that sample.
People generally only take IQ tests if they expect the result will either be very low, or very high. The "very low" group will be underrepresented in the sample as noted above. So we're left with the people who took an IQ test because they expected to get a high number (for gifted classes, skipping a grade, MENSA etc). The people whose IQ is actually ~100 haven't taken a test.
As for the SAT, people probably remember the exact score if they tried to get into some college program where it mattered, or if it was flatteringly high. If your score was really mid, there's no reason to recall it.
Tl;dr very low IQ people don't take online surveys, and middling IQ people don't know their score, leaving the high scoring people to sample.
I had the same thought. "Why do you assume that workers who are making money doing mental work have an average intelligence?"
It's like expecting people who carry boxes for a living to be of average strength, rather than being somewhat stronger (because of practice if nothing else, but also if you're not good at it and don't get better they will fire you).
"The people on the lowest end of the IQ distribution probably can't work as click workers due to being illiterate, unable to follow instructions, or similar reasons. So you'd expect the average to be above 100 in that sample."
This is true, but I'd also expect it to be true to an extent on the high end as well (for different reasons).
The US military, for example, has an ASVAB cutoff that essentially disqualifies the bottom 30% of the population from joining. This should push the IQ of the average military person above 100. But then I'd also stereo-typically expect that folks with Ivy League potential are less likely to join the military so that would push the average down. How much I can't say.
But I expect to see this sort of "exclude some percentage of the bottom" and "exclude a different percentage of the top" to show up a lot for everything except for the population as a whole. Doing something quantitative with it is tough.
ASVAB distributions of everyone who applied to the US military in fiscal year 2017, broken down by branch and with gender %, and with the population of the original norms (note: everyone in Cat V in the original norms seems to be misclassified as Unknown). For people unfamiliar with the ASVAB's structure: I is the highest, V is the lowest (and corresponds to lower borderline intellectual disability or lower, roughly), the IIIA/IIIB breakpoint is the 50th percentile, and IV or below (<30th percentile) is as noted basically a disqualification (some people in Cat IV can join, I think).
The first thing you notice is that women are WAY overrepresented in the lower categories. This isn't seen in the general population -- it's something conditioning on "women interested in joining the military".
"The first thing you notice is that women are WAY overrepresented in the lower categories. "
Can you point to a specific table? I'm looking at Table A-4 in the TOTAL DoD section and see 3.12% and 26.56% of the applicants in categories I and II. The female civilian population as a whole (right below) is 7.6% and 26.35%. These are close.
Also, maybe not relevant but this data is for "Active Component Enlistment" so probably excludes commissioned officers.
"Table A-4. Applicants 1 for Active Component Enlistment, FY17: by Armed Forces Qualification Test (AFQT) Category, Service, and Gender with Civilian Comparison Group". I was noticing the "percent females" metric -- that the lowest ASVAB categories consistently had a much higher proportion of women than the highest, though the exact distribution varied by branch. For the civilian norms, the percentages of women are obviously much higher in general, but the variance is more like Greater Male Variance in a normal population than the extreme distributions in the applicants.
This seems to be what happened to HS grades over the last 25-30 years.
Colleges: HS grades are a factor in admissions (and one that is someone amenable to lobbying)
Schools & parents: "Moar points than 4.0 for tougher classes!" and 4.0 -> 5.0 -> 6.0
College Admissions: ...and we're deducting those BS points.
My wife did one cycle in undergrad admissions at Dartmouth while I was at the end of grad school, to "understand the alumni better by understanding who and how we admit students." Her actual job then: raising money from alums. Normalization of grades and scores within Admissions: oh, yes.
I mean, admissions counselors are super transparent about this to students. But even going from the 4.0 to the 5.0 provides a numerical tool for weighting the difficulty of one’s class load, which is important and something that merely subtracting the points wouldn’t capture.
My daughter has had to explain moving from a rural school with a 4.0 max and few honors classes to a high-quality suburban school with a 5.0 scale and ALL the honors programs and the admissions people are like, “Yeah, we gotchoo, just drop a note in the additional info section.”
I’ve tried and failed repeatedly over 10 years to convince my now-16yo daughter that undergrad is an expensive country club that offers a vanity-metric credential, and that she can learn more and build skills faster operating outside the system (like the route a Thiel Fellow would take). She absorbed the perspective, but shushed me repeatedly with “fine, I get it, but damnit I’m going to be a doctor and there’s no hack I can find to circumvent the medical establishment gatekeepers. That vain credential is a pre-req to get through the first set of gates to heal people, so I gotta do that.”
The less extreme method is to study for and take a bunch of AP tests in high school, finish the first half of your degree at a community college, and transfer in to a four-year college as a junior.
This is a bit faster and a lot cheaper than spending all four years at a regular university, while still being a "regular" degree that should work as well as any other from the same four-year school as a prereq for a graduate program.
I got about 138 in an IQ score 20 years ago - pretty official as it was in a company and I was locked in a room. That’s my highest result. I haven’t tested since and I’m probably not going to. So I’m probably reporting my highest ever result there (well I know I am but I’m not sure what the others were, just that they were lower).
Anyway that’s what I would answer if asked, “in my last test I got 138” - which is true but almost certainly high balling it.
The last IQ test I took for which I received a score was in about the 3rd grade, when I scored (I think?) 133. I have no idea what test it was, or how much it ought to change, if any, over another 30-50 years.
The last IQ test I took I passed, as after answering something like 10 questions on an internet test, I declined to provide an email address to get my results.
That last bit about 1/30 people finding internet statistics essays interesting is a bit of a reach, I think. Maybe only 3% of people are into that, but they aren't necessarily going to be the top 3%.
I can believe there's a pretty strong selection effect, but so strong that somebody with 114 IQ (like 85th percentile) is just as likely in the sample as somebody with 140+ IQ (like 99.9th percentile) is pretty strong.
I feel, less than 3% are into that (unless it is Scott writing), but those who are: pretty sure there is near none not in the top 5%. (Also: A text about statistics will not appeal to the lower 90-97% of math-IQ - otoh: an ACX post will appeal to those high on verbal IQ: Scoring high on both makes for a high IQ overall, i.e. potential ACX-reader.) Math-IQ of 114 and "finding internet statistics essays interesting": I doubt it. Strongly.
The correlation between "any intellectual curiosity/trait-intellectualism measure we've ever managed to make" and IQ is pretty weak, and appears to only exist for elements of IQ tests that are more classically amenable to "being learned" (e.g. vocab, general knowledge). Whether the crystallized/fluid distinction is real is another thing ("fluid IQ" is the subset most impacted by the Flynn effect, so it's implausible it's *not* strongly learned), but at the very least, you don't land in "intellectual curiosity/interest/trait-intellectualism and ability are basically discussable together". I'd say most forms of recreational-essay-reading have a higher interest than ability threshold.
Interesting. Have a link? (google did not help). So, even normies and dumbies can be curious, one assumes. I'd agree: most forms of recreational-essay-reading have a higher interest than ability threshold. That is why the NYT has many more readers than Scott. ;) (Not the only reason, I admit. But is reading ACX/SSC as "recreational" as NYT/Hanania/CoHo/TheSun? Maybe with IQ 135 it is.)
I think you might be underestimating the degree to which you only ever see the top part of the IQ distribution.
I find https://link.springer.com/article/10.3758/s13428-012-0307-9/tables/1 pretty illuminating. It finds that eg only 3% of college students know that Budapest is the capital of Hungary, or that Euclid is the father of geometry. If you mostly hang out with people who would be able to answer those kinds of questions, you're only ever seeing the top 3% of general knowledge (not the same as the top 3% of IQ, but hopefully this serves as an intuition pump).
I'm amazed that so many people correctly answered "WHAT IS THE TERM FOR HITTING A VOLLEYBALL DOWN HARD INTO THE OPPONENT’S COURT?" How did 78.4% of respondents know the answer to that? Do Kent State and Colorado State have massively popular volleyball teams?
Well, I can tell you that I guessed "spike" as the answer to that question, and KM's comment appears to indicate that that's correct. I have never played volleyball outside of a school PE class, and not much then. I have never _watched_ volleyball at all.
One thing making this easy to guess is that my vocabulary of "special volleyball terminology" consists of just the one word.
No offense to you, but this is the most internet-intelligencia-bubble comment in this whole post. Expressing surprise that the capital of Hungary or the name of the Ancient Greek founder of a math discipline are not common knowledge, but that a common sports term is.
Once again, nothing against you personally, I think everyone on this site wildly underestimates how little the average person knows about... everything scholarly (https://xkcd.com/2501/), and how *much* they know about sports.
I'm not comparing the volleyball question to Budapest or Euler, I'm comparing it to "WHAT IS THE NAME FOR A MEDICAL DOCTOR WHO SPECIALIZES IN CUTTING THE BODY? (SURGEON)", which only 74.8% answered correctly. It's only a little surprising that 3% of people know their obscure European capitals, but it's very surprising that more people know volleyball terminology than know what a surgeon does.
I also don't think this is an internet-intelligencia-bubble thing, I think it's a me-being-a-different-demographic thing. Maybe Americans all know the correct volleyball terms because they play it so much. Or maybe they know it because the participants were recruited from introductory psychology courses and they played volleyball more recently.
I have an intuition that the number of people who can correctly answer "what does a surgeon do?" would be larger than the number of people who can correctly answer "what is the name for a medical doctor who specializes in cutting the body?"
For what it's worth, I was given basic instruction on 3 kinds of volleyball hits (bump, set, spike) in PE class in...elementary school, I think? Possibly middle school. (And it was a general public school, not something specialized.) The whole class played volleyball for perhaps a week or two and then moved on to some other activity.
I agree this knowledge seems more obscure than "what's a surgeon?" but I'm not surprised that it's common.
That's a pretty weirdly phrased question. I kind of did a double take on that one, trying to figure out if they could mean something more specific or technical than surgeon. I mean surgery isn't just cutting, it's also changing and reassembling stuff afterwards.
In a comment/explanation for one of the questions in his book "Coming Apart ", Charles Murray wrote this about underestimating how little of the bottom half of the IQ distribution smart people interact with:
"I use this question ['Have you ever had a close friend who could seldom get better than Cs in high school even if he or she tried hard?'] as a way of getting at the question I would like to ask, 'Have you ever had a close friend who would have scored below the national average on an IQ test?' I can’t ask that question, because readers who grew up in an upper-middle-class neighborhood or went to school with the children of the upper-middle class have no way of knowing what average means. The empirical case for that statement is given in detail elsewhere, but it may be summarized quickly. The typical mean IQ for students in schools that the children of the upper-middle class attend is around 115, compared to the national mean of 100. In such a school, almost all of the below-average students, the ones you thought of as the school’s dummies, actually were above the national average. Even if the students were arranged in a normal distribution around a mean of 115, only 11 percent of the students could be expected to have IQs under 100. But they probably weren’t normally distributed, especially at a private school that uses a floor of academic ability in its admission decisions. So if you went to upper-middle-class schools and think you had a good friend who was below the national IQ mean, and are right, it had to have been one of the students who was at the absolute bottom of academic ability."
Note that Murray may be using a value for IQ standard deviation a bit less than 15.
"I think you might be underestimating the degree to which you only ever see the top part of the IQ distribution."
Yes, I think lots of people, even smart ones, are very bad at understanding what "average" is. I teach at a private high school. I'd say the average ACT score of our graduates is probably somewhere around 26-27. I was teaching one of my sophomore honors classes a few days ago and someone said, "I'm stupid." They're kids, of course, so it's less surprising that they're in a bubble, but I had to point out that all of the kids in that class are well above average. They have to get into the school, get into honors classes, not flunk out of the school, and keep their grades up to stay in honors classes. In their social lives they basically never interact with a kid from one of our city's many, many failing public schools. They have no idea what "average" (outside of their bubble) really is. And even my colleagues who will complain about a dumb student are usually complaining about some one who ends up with a 19 or 20 on the ACT...in other words, basically average intelligence. (Admittedly, we're probably doing a better job of test prep with our students than a "bad" school, so someone with slightly below average intelligence might end up with an average score, but our "bad" students aren't bad in the overall scheme of things. We probably don't graduate anyone 1SD or more below the mean in intelligence.)
I mostly hang out with people who design analog power IC's for a living. I suspect that they are probably all at least 1 sigma above average. They are certainly not all 2 sigma above average. This is, nonetheless, one of the most challenging of engineering fields.
We all went to high school (most of us with near 1000 fellow students). Presumably these represented a fairly inclusive normal distribution. It was very clear who the brightest were all the way to >3sigma (ranking the middle is obviously more challenging). I don't think it is that difficult to judge, in person, whether someone is in the stratosphere of intelligence. Most of the time, it tends to be very obvious.
>WHAT IS THE NAME OF THE LARGE HAIRY SPIDER THAT LIVES NEAR BANANAS? (TARANTULA)
Slightly off topic, but as a spider enthusiast, I would have gotten this wrong. There is absolutely no association between Tarantulas and bananas in my head (indeed, they live globally, and a good portion live only in deserts where there are no bananas), and Tarantulas aren't even "true spiders". In contrast, there are several other species of true spiders that are literally called "banana spiders" because of their reputation for being found in shipments of bananas. Several of these are, in fact, large and hairy, namely Brazilian wandering spiders (and to a lesser degree, huntsmen).
And the authors of this test probably picked up that piece of misinformation from listening to Harry Belefonte's Day-O.
And this one.
WHICH PLANET WAS THE LAST TO BE DISCOVERED? (NEPTUNE)b
Really? When this test was first created (1980), Pluto was generally considered to be a planet and it was discovered after Neptune. The IAU reclassified it as a dwarf planet in the 2000s, but this answer was wrong when the test was written.
Also, I see two other booboos in this questionnaire.
Oh, yeah. You're right, that's definitely the connection I was missing - and it's a song that would have been much more in the forefront of a test-taker's pop culture knowledge in 1980.
A beautiful bunch of ripe banana
(Daylight come and me wan' go home)
Hide the deadly black tarantula
(Daylight come and me wan' go home)
However, that just bothers me more, since not only are tarantula bites almost exclusively not deadly, most aren't even medically significant.
More importantly, what percentage of people can identify the answers that the authors of this test got wrong? Although one of those four was correct at the time of its writing in 1980, it was no longer correct by the 1990s (so why do they ask it again in 2012?). And one answer was wrong when the test was originally written in 1980, but would now be considered technically correct after the mid-2000s. And two others are based on popular misinformation.
There may be more than four, though — full disclosure — none of the ones I spotted are sports-related questions. I don't know if the answers to any sports-related questions are wrong. I never followed sports. ;-)
I agree with the overall point that you're seeing usually the topmost percentage in many cases, and I expect that knowing various bits of knowledge positively correlates with IQ. However, I don't really find 3% surprising? There's a lot of memorization that goes on in early geography classes that kids just throw out of their mind, in my experience usually the stuff they remember specifically is things drawn to their attention in a more interesting manner (tv show, movie). I'd also find it completely plausible to not even have ever been told that Budapest was in Hungary — you definitely see it in movies, so I'd expect they know it *exists*, but rarely mentioned directly that it was in Hungary.
(Some of these bits are surprising in the opposite direction for me: 8% knowing capital of Jamaica? 20% know the actor name for Mary Poppins? However, there's definitely surprising bits, like only 58% knowing the answer to be chameleon)
Of course some of these datapoints are more surprising than others
The more of these questions I read, the more it sinks in just how arbitrary and idiosyncratic this "general knowledge" test could be. Questions about parts of typewriters and 1950s pop culture and 19th century literature and golf. I can picture the hairdo of the midcentury upper-middle-class white men they surveyed to normalize these questions. I would guess there are certain areas of human knowledge overrepresented here and thought to be generic, while other equally or more "significant" areas underrepresented or totally absent.
Yeah, for a "general knowledge" test to correlate with intelligence you really have to be using it on all the same demographic. That's somewhat true of other intelligence-related tests like the SAT, less for Ravens Matrices, but making one that's truly culture neutral (let alone testing whether it's culture neutral) seems impossible.
I wasn't consulting my normal distribution tables while I was typing the comment. An IQ of 142 would be 99.74th percentile, and an IQ of 114 would be 82.47th percentile, assuming mean 100 and standard deviation 15.
IQ test administrator here. The ones I use are WISC and WAIS usually. I’m here to add weight to the theory that above 135 scores are not as precise for the relevant groups, that is, adults and young adults reading this.
1) The WISC-V came out in the last seven years but even with the Flynn effect lagging as of late, it’s newer than the WAIS-IV, which has an update coming out this year. The SB is hopelessly old and I think its validity is suspect at this point. (Anecdotally, the final test item for WISC subtests “feel” harder than their adult equivalents.)
2) The WISC is used up to 17 years of age, and anecdotally speaking, that is, I’m not actually whipping out my norm conversion tables, the difference between a top possible score and a merely 130 score equivalent is sometimes a single item or the speed by which one answers a single item in a given subtest.
3) This ceiling effect is less true for younger ages, which makes sense because theoretically, even a six-year-old may be completing the same task as a 16-year-old.
4) This isn’t really a problem for the professionals typically using IQ tests as our main job is determining intellectual disabilities or specific learning disabilities. We’re rarely trying to tell super smart kids how smart they are. If you’re above 130, you qualify as gifted, and we don’t care anymore.
5) If you qualified for gifted (over 130), good for you. However, you probably qualified when you were super young and federal laws do not require that we retest you every three years like it does for disabilities. It’s quite possible that you may have had an intellectual growth spurt. We’d never know.
I have no idea what the general trend is, but I can say that my tested IQ went *up* by about 6 points between age 11 and 16. I suspect that some of that is just familiarity with the testing format, which is hard to control for.
That happens. I’m at the high school level so I see overall scores go up and down throughout the life span. Over five points is on the high side, but not unheard of. Another factor might be varied tools. I rarely hear the Woodcock mentioned outside of school psych circles, but it’s popular within the field.
I wonder about an obvious effect that wasn't mentioned: what happens when people take multiple IQ tests and report their highest score? For example, I've been accepted into Mensa, so scored about 135+ officially, but I guarantee that if I took 10 different IQ tests, in at least one of them (especially one involving language), I would score less than 125. I've even scored as low as 109 much earlier in my life.
Yeah, was about to note that the "well-behaved misestimation" sounds like "take samplemax of N tests", since this will produce a fairly-reliable systematic error.
Great point regarding language, remember Feynman's IQ score was "only" 125 yet he recorded "the highest score in the country by a large margin on the notoriously difficult Putnam mathematics competition exam, although he joined the MIT team on short notice and did not prepare for the test. He also reportedly had the highest scores on record on the math/physics graduate admission exams at Princeton." His mental mathematical hardware was Cleary off the charts but perhaps he had completely average verbal reasoning which kept his IQ score down.
It seems more likely that he didn't try when taking a school IQ test and then later repeated the anecdote about his score because he liked the reactions it got.
I remember reading an interview with Feynman's sister. She went back and found the original reports. It turns out he scored 122 (or 123--I think it depends on which interview).
IQ is, by definition, your score on an IQ test, something which is said to be highly consistent even though the are many explanations here that it is not. If he took only one IQ test then that score was his IQ forever. You may be imagining that he has some true, intrinsic IQ that matches his displayed intelligence, but again, other responses here explain why that is ill-defined at the level you'd expect him to have.
I think your point behaves the same for IQ and height. Do you think there's a disanalogy? If so, why?
(Adult scores on IQ tests are pretty consistent, but the connection between childhood IQ and adult IQ is lower. This is one of the reasons it's awkward we mostly have measurements of childhood IQ / academic performance.)
I got this right? a) If I claimed an IQ of 128: fine: most ACX readers should be (not sure, when reading some comments - but, hey, me probably 120+ and I AM much DUMBer than I wished)
b) If I claim 138 (I got as an early teen, due to age-bonus), then it gets loped off to 135 or 128 (138-10).
Why not just assume an IQ of 125 (or 128 - what does it matter, really)? And drop the question.
Or ask: 'What other blogs you read?' and adjust? SSC +5 /Zvi: +5 / Derek Lowe +8 / Hanania -10 / Pueyo -5 (yes, I pay subs. to Thomas P.)
TBF, In the pipeline added points should be counted only if you do not work in medicinal chem / organic chem / big pharma. It is pretty much required reading in our neck of the wood.
Thx for revealing Derek Lowe to me, but I disagree on Hanania, won't argue tho: that you gave him a swooping -10 means that he's probably your guilty pleasure.
I do not work in chem/pharma. In school, I gave up on salts. :) I found Derek before I found Scott; reading his "Things I won't work with" series was and is just out-of-this-world. Go for all of them! Pure pleasure, no guilt:
As for RH: yep, I subscribed. Easier on my poor English and very little brain than, say, Sam Kriss. Will I ever pay: Nope. Noah is better, Slow boring is better. Caplan is much better - his admiration for RH is one of his sillier ideas.
Hanania -10 is a really weird hip check. I read Hanania because Scott has linked to him occasionally and… I disagree with him on 70% of his priors, but there’s no doubt he’s super smart, articulate, and willing to consider when his team is wrong after thorough consideration.
I don’t see why that would make you classify a regular reader as “likely not smart”.
Ah, you got the joke. ;) Well: 125 minus 10 is 115. Which is still smart. Tucker Carlson would be a bigger minus (I saw him with Caplan and Lex F.). - Arnold Kling considers RH less than impressive. No doubt: RH is articulate. Even super smart. Less sure about his average reader (who is usu. just smart). Did I ever read an eye-opening idea first at RH?
Curious about the assumption that readers will hate on Mensa. Why the hate?
I joined for a year ages ago when I was new to a town. The monthly game night was a blast.
I’ve never taken an official IQ test but have gone off the fact that Mensa accepted me based on GRE scores which I figure puts me top 2%. And based on the incredibly unreliable observation of how often I think people are smarter than me, I don’t think I’m much, if any, over top 1%. It occurs to me that anyone doing a similar gut check on a test result over 140 may have a hard time due to sample size.
It seems more: If you take a Mensa test, you feel the need to verify you have a high IQ. Even though you failed in life. If you succeeded: no need to verify. - Thus: 'nerdy' losers do Mensa.
That would explain people quietly looking down on them. I think the need some people feel to be actively nasty about it is, at least in part, a status slapdown
It's the usual justification for mocking Mensa members, but it only explains the lack of respect, not the active antipathy. That's where status hierarchy enforcement comes in; these guys are 'losers' trying to confer high status upon themselves, which is a violation of the social order and triggers something like a disgust reaction. It also allows people to combine the fun of punching down with the sense of righteousness that comes with [doing something that, if you squint, kind of looks like] punching up.
Nobody hates on nerds harder than slightly different types of nerds.
Of course this thread is classic barber-pole behaviour. Class A nerds (Mensa members) brag about how smart they are, so Class A' nerds look down on them to prove how much better they are. And now in this thread, Class A'' nerds are looking down on the people who look down on Mensa members, to prove how much better *they* are.
And now look at me, I'm Class A''' and I'm looking down on those people too which proves I'm better than everybody. Luckily I am at the top of the barber pole and there is definitely no Class A'''' that would look down on me.
I don't know about modern Mensa, but there was rather a tradition of people in the 70s/80s who boasted of how smart they were and how superior this made them to you clods, and the basis for their assurance of their hyper-mega smarts was "I'm a member of Mensa".
Indeed; the punching (up or down) comes as a reaction to the implied sense of superiority/brag, and what makes it an easy target in particular is the implication in the brag that "I'm really smart but have done nothing of note with that intelligence, otherwise I would be talking about that."
Right, but we're the nerds here, we aren't going to be criticizing Mensa for being too nerdy and wanting that to be high-status.
I think criticism of Mensa by people smart enough to be in Mensa, comes from the fact that - aside from low-grade signalling - the main benefit of Mensa is that it provides you with a social group screened for high intelligence. But if you're *really* that smart, why aren't you working at MIT or FAANG or some other place where half the people you meet are really really smart already?
Which, sometimes, is a legitimate criticism - Mensa is going to appeal to the smart but unambitious who are satisfied being the brightest fish in a small, dull pond. But it's also going to appeal to the young woman whose family is only going to send her to Texas Women's University, and sure, Yale is in her future but for four years it's genuinely helpful that someone has gathered all the smart, lonely people at TWU into one bunch.
Mensa exists as a category exclusively to declare that its members (or at least people eligible to be its members) are better on some axis than other people, which many people dislike. If there were a Society for Hot People didn't allow anybody below +1 SD hotness or a Society for Rich People that didn't allow anyone below +1 SD wealth we'd think of them as prats and wonder who would join such a society.
I don’t know about that. Universities screen for intelligence. Are they hated for that? Also, most of the world groups people together with others like them. Is the issue that mensa is too honest about what they are doing? Vs the neighborhood only some people can afford to live in.
Unlike universities, Mensa's screening is an end in itself; It's literally exclusive for exclusivity's sake. And unlike other tribes, it's explicitly screening for (as Sei put it) "being better on some axis than other people."
But sure, being upfront about their reasons for being exclusive makes it more obvious.
Maybe Mensa is different where you live, that's entirely possible, different cultures are different after all. But where I live (The Netherlands) my experience has been that Mensa is not so much a "Organization for people with high IQ" as a "Organization people with high IQ who struggle because of that for one reason or another". It's not a country club, it's a support group.
Most people with very high IQs are surrounded by other people with high IQs. Family members, colleagues, university friends, etc. But not everybody is that lucky. I have an ex who was extremely bright, but she was the only in her family who ever went to university. In fact the only one in her entire extended family who ever did high school at more than the lowest tier. She got absolutely zero support at home for her academic pursuits. Not because her parents were abusive or anything, but because they just didn't understand.
For her and many people like her, being very intelligent is a source of alienation from friends and family. Having a way of meeting similar people is a godsend.
And similarly, most highly intelligent breeze through high school, and then go on to have good careers where they are surrounded by lots of people who are also intelligent. But again not everybody is so lucky. It's not uncommon for gifted kids to fail in school because they are just bored out of their minds all the time. Again, having a place where such issues are understood, and where you can find resources to deal with that, or just a sympathetic ear, is a godsend for some people.
This is exactly why I joined as a teenager (and why I haven't bothered to keep up my membership as an adult, because I now have other ways to find such people).
We've apparently shifted from valuing intelligence to valuing narcissism.
(Witness certain ex-presidents / reality TV thespians with marmalade complexions.)
But where and when does one test for I. Q.? I took the SAT, and scored higher as a junior than a senior, and the ACT, I think. But I never took or even knew anyone who took an I. Q. test.
> It’s a group that has, unfortunately, become frequented by underachievers from modest backgrounds, with extensive interest in playing games that prove they’re intelligent but who have little achievement otherwise.
There both good and bad reasons to hate Mensa. Bad reasons would include nerd bullying and intelligence denialism. Unfortunately, there are also some good reasons.
There is nothing wrong with choosing the subset of society with top 2% IQ per se. Actually, that would be awesome. The problem is that Mensa is *not* a representative sample of this group, because there is additional selection for "knows about Mensa" and "wants to be a Mensa member". Even worse, once this becomes generally known, the selection only becomes stronger, as now it is "knows that Mensa has problems, and wants to be a Mensa member anyway".
1) People *obsessed* with IQ are over-represented. Obviously, they are most likely to find out that Mensa exists and apply for membership. They have various crackpot theories about intelligence, and as usual among crackpots, they are not familiar with the mainstream psychological research and they don't care, they just want to share their own theory with everyone. They can't stop trying to guess every person's IQ based on extremely weak evidence. They can't stop talking about how the world would obviously be a much better place if people with high IQ could rule it. They refuse to admit that IQ is not everything, and you might also need actual knowledge, rationality, social skills, mental health, etc.
2) Generally, crackpots of all kinds love to use "I am smart enough to be in Mensa, therefore my theories are right, and the mainstream science is just too stupid to understand them". Almost every crackpot applies to Mensa... and 2% of them succeed, which in their eyes proves that they were right.
3) Also over-represented are lonely people, and people who despite high IQ somehow fail at life. Although these people have my sympathies individually, if too many of them come to the same place, it results in strong vibe of "this is a club for losers".
These things reinforce each other. For example, for the lonely people it is difficult to oppose the crackpots, because in their eyes, ostracism is the worst thing that could happen to anyone, and the idea of kicking someone out of Mensa feels too scary. But as a result, people who have better social options spontaneously leave Mensa, leaving mostly the loners and crackpots.
All this makes me very sad, because in my opinion the underlying idea is not bad, if we only could somehow overcome this selection for crackpots and social incompetents. I am not even saying that we need to get rid of them... only that they shouldn't be so horribly over-represented.
If you could just stop 1000 random people on a street, give them all an IQ test, select 20 with highest scores, invite them to a party, and kick out two or three of them who turn out to be insufferable for some reason... I think the result would be quite nice, and basically this is what Mensa originally wanted to be.
I also see value in measuring IQ explicitly, because there are many people who underestimate their own intelligence, because they do not match the stereotype. I made bets with a few people that if they try the Mensa test they will pass... first they laughed at me, but then all of them passed and were surprised. I hate to see people underestimate themselves.
In schools, many kids suffer because they are smarter than average, they understand the lessons quickly, and then they get bored. Ironically, many of them are considered stupid by their teachers... because they stop paying attention. Many high-IQ kids were sent to a psychologist to diagnose whether they might be mentally retarded.
Basically, if you happen to be highly intelligent, your environment makes a great difference. If you grow up in a family of supporting and academically successful parents, you probably have a great career ahead of you. On the other hand, if you are forced to live the average life, you will probably feel that you don't belong there, without understanding why; this often makes people depressed. It would be nice to have a standard way to detect the latter, and to give those people some alternative supporting environment.
If you make an environment that selects for high IQ indirectly, for example by measuring academic success, you do get a de-facto high-IQ society, without the problems of Mensa. The disadvantage is that you get a kind of monoculture. A group of students who study quantum physics is certainly full of smart people, but I think they would benefit from meeting people with similar levels of intelligence who do something else for a living. Less Wrong or ACX meetups are somewhat closer to this ideal, because they often connect smart people with different professions; but a truly random sample of high-IQ people would be much better.
"Curious about the assumption that readers will hate on Mensa. Why the hate?"
I will try (though I mostly don't think about Mensa).
If one thinks of Mensa as a mostly a social club for people who like to use their brains (which, I think, is how Mensans think of it) then an exclusion based on a test rather than candidate's actual ability and willingness to use their brains seems both wrong and strange.
The organization (I think) has sub-groups and clubs focused around things such as puzzle solving (Sudoku, etc). It is easy to imagine someone who is great at solving puzzles but doesn't meet the IQ standard being excluded while allowing someone who is objectively poor at solving puzzles (but still enjoys the challenge) being allowed to play as long as they score highly enough on IQ. This strikes many people as unjust/unfair.
Sei (below) suggests thinking of how we would consider a "Society for Hot People" that didn't allow in people less than 1 SD above average in hotness or a "Society for Rich People." But even these seem to make more sense than Mensa because the Hot People explicitly want to hang around other hot people (and are seen as shallow for valuing this so far above everything else) and the Rich People sometimes also want to hang around only other Rich People (which, while not a rich person, I have some sympathy for). But even the Rich People clubs usually filter on ability and willingness to pay rather than the actual size of one's bank account independent of ability to pay.
Mensa projects as filtering on the equivalent of money in the bank, but NOT on willingness to spend it (ie, deliver results of some sort at the meetings/events/whatever).
This may not be how Mensa actually IS and I'm pretty sure it is not how the Mensa members think of it, but I think this is how it presents to the outside world.
One of my aunts is a proud MENSA member and recommended it to me; she has spent a lot of her life in more sparsely populated parts of the country before the Internet was common, where groups like MENSA were a good way to meet other smart people.
I, on the other hand, grew up on the Internet, posting on forums where the average IQ is high enough that people write posts about "the average IQ can't actually be this high, can it?". When I thought about joining MENSA, it was explicitly as a "I want to meet more old people" move, since I already knew a ton of smart young people thru LessWrong.
I think it's a couple of things. First, as others in this chain have noted, there's the "filtering seemingly for the sake of filtering" that seems to support no particular purpose other than to support unfounded status claims.
To that, I'd also add, though, that "professed high-IQ man" is a common and distasteful part of internet message boards, and pretty much anybody who's a member of a community like this has had the privilege of dealing with self-professed-Mensa types in threads like https://www.astralcodexten.com/p/open-thread-240, which includes such gems as "[w]hat's the point in talking to people that don't have any intellectual achievements, eg contributions to human knowledge," "[p]eople like me have given you everything," and "[t]he minimum IQ for contribution to the knowledge of man is about 125... [p]eople under that level quite literally exist just to do labor for people above that leve[.]"
Does that represent all of Mensa? Of course not. Are the people who do have those opinions as vocal as they are distasteful drawn to Mensa like flies, and coloring people's perceptions of the high-IQ club accordingly with the nonsense they spout online? Unfortunately, very much yes.
Unrelated: My headcanon is that every time Scott repeats a word, it's a deliberate easter egg, and there's some kind of secret prize for the person who finds them all. That said,
> So are people just taking terrible Internet IQ tests that inflate their score **about about** 20 points?
When I was a psychology student, 30+ years ago, in a country far away from where I live now and from the US, we were told that an average IQ for *university students* then was about 120. This made rough sense considering the % of population that attended university there and then. I realize that it's not like that for the US now, but the idea that the audiences for those websites/your blog etc would be INCREDIBLY self selected by IQ feels plausible. Not 135+ on average obviously but certainly well above not just the population mean but even degree-level educated population.
I don't understand why you'd assume click workers would have population average IQs? To me it seems obvious that they'd be higher, because ANY group that engaged in any cognitive labour will have higher than average IQ. Not just because it would exclude those with the very lowest scores incapable of doing that labour but also because it'd exclude a lot of people with middling lowish to average scores who have less interest and inclination in such.
Also, self selection for even measuring IQ others mentioned and reporting the highest score: for example I'd likely report my 135+ Wechsler score from my student era (I was also subject to the one used by the US army historically, translated/normed of course but that was as a teenager and I don't trust that result) rather than 125+ Raven score (not just because it's higher but because I'm certain that the former measures "manifest" intelligence better). I'd not report middle aged English results (not because my vocabulary is worse, but because I've discovered that memory, especially digit memory, for me varies by 2 digits depending on language, both straight and backwards).
So I think selection bias in audience and reporting explains most of these results.
To put some numbers to this, see here: https://www.cremieux.xyz/p/education-isnt-what-it-used-to-be. It appears that in the US, mean IQ for college undergraduates is significantly below 110, and IQ for those with only an undergraduate degree is only about 102.
When you go from less than 5% of the population attending college to somewhere in the 30-40% range (not sure where it is now), the average IQ will go down, as it must.
Of course! That's the point of the article. Cremiuex notes in the article that if 40% of the population attends college, then even if they were perfectly selected as the top 40% in IQ, they would still only have a mean IQ of about 114.
You're missing an assumption that the original 5% came from the top end. If the average IQ was 100 before, expansion could take it in either direction and you'd predict no change. If it was 75 before, expansion would take it up.
That's roughly a good assumption though. Maybe not exactly the top 5%, but there's no remotely conceivable way that when 5% of the population were going to college that they were not at least significantly above average. There's no reason people with an IQ of 75 would have been going to college over smarter people, and there's no way that such people could have been competent enough for American society to have been as functional as it was.
Now I'm curious if you've been tested, and what it came up with. You don't *seem* below-average even in this crowd.
(my brother told me once that he sometimes brags to others that the rest of his family is in Mensa, but that he doesn't think he'd qualify. I think he is very very wrong).
Never formally tested, except for one time when I was about twenty and did an aptitude test (with about a gazillion others all crammed into one room) for a job application. Was informed I'd failed, and I have no idea how I scored or what the threshold they were looking for was.
It was the 80s in Ireland, there were many more applicants than vacancies for every job vacancy, and I think the "aptitude test" was just a filtering mechanism (it was a very basic job and didn't require much in the way of skills) to get the numbers for interview down to a manageable level. If the job hadn't already been filled internally, and this was just a way of meeting the requirements that "you must advertise job vacancies" but they had no intention of hiring outsiders (I did a few of those interviews in that period, too).
So I've only done online tests, ranging from the "what colour is a banana? if you said yellow, congratulations! you are smarter than 59.85% of other people who took this test!" type to the "we're doing a simpler version of a Ravens Matrices test which is a big legit IQ test so you know we're not scammers" types.
>Most people don’t read long articles online about statistics. 1/30 sounds like as good an estimate as any other for the sort of person who would find that comprehensible and interesting.
My estimate for the % of English readers on the web who read long articles about stats on this type of topic, with any degree of regularity, is 0.1% - 0.3%. My wife and quite a few of my smart friends are never (ever) going to read 80%-90% the types of things this community proactively seeks out and reads on a monthly basis. It's not reasonable IMO to convert this percentage of the population to the percentile threshold for an IQ cutoff, but for grins and giggles if that were a reasonable proxy we'd be back to the 138-145 range.
Yeah. First let me firmly set aside any sense of humility or shame. In my job, and in my education, I have interacted with a lot of really smart people, and have done so consistently for the last fifteen years or so. I have also been part of the rationalist community for almost that long. The people I meet at Less Wrong or SSC meetups tend to think, speak and behave like the very smartest of the smart people who I interact with in my career. Essentially *everyone* at a given Less Wrong meetup has a list of esoteric skills and interests, such that they would get along great with the smartest of the PhDs I work with. It isn’t subtle.
What contexts, other than LW meetups and STEM PhD programs, do you hang out in where people can have an informed conversation on either Nietzsche or quintessential models of cosmic inflation? Are we going to pretend this is not related to intelligence?
But those are not the only kind of great conversations to have -- the kind where people are informed about those things. Here are the topics of some of the best, most memorable conversations I have ever had:
-(with a close friend) Our deepest and craziest thoughts about self and consciousness and time.
-(with a close friend): WTF is wrong with the faculty at our grad program.
-(with 2 people I had met for the first time): Their work on maintenance of the Cathedral of St. John the Divine
-(with a schizophrenic): What it's like being him: ". . . the room gets liquid and I'm the aquarium they're all swimming in and what I want to know is how I go back to being just one fish."
Intelligence changes with age. You shouldn't weight on just education, at least age needs to figure in as well. Testing age will necessarily be lower than in the survey, SATs are usually done when people are at peak intelligence, so tests in ClearerThinking are likely to be post-peak and past tests likely to be closer to peak. Plus: Flynn effect, norms change.
Will you/Spencer share the ClearerThinking data? Would be useful for our research on norming from convenience samples.
>Crystallized intelligence typically increases gradually, stays relatively stable across most of adulthood, and then begins to decline after age 65. The exact peak age of cognitive skills remains elusive.
I've seen 40ish in a graphic somewhere; maybe it was extrapolating beyond what we're sure of.
About 2 months ago I read a headline: 5min (or some short) IQ tests work just as good as long IQ tests!
So I open Google and take the first test 5 min IQ test that popped up; I had to pay for my score when reached the end. I close that tab and search this time, very carefully- noting the word ‘free’ in the search results. Once again I get to the end of another free 5 min test, ‘We’ll send you an email with results’. The email I receive is a link to pay for results. I look through another 25 links, surely one must be free! I find a page “Free FREE FREE!” This is the one! I take the test, give me the results! 140? 155? It says I need to pay for results. but taking the /test/ was free…
I close all my tabs, and go for a walk.
Remembering my escapade 2 weeks prior (heaven knows I won’t tell a soul about *these* struggles) I doggedly find the headlined paper making the online buzz for 5min IQ tests. I download the software from the study -an old JavaScript program- which I have to debug because it’s ancient. I spend 20~ mins on this. Finally! I launch it. I take the test. I click ‘results’, and it launches me into a website where I can buy the full version of the software suite.
There's an observable phenomenon where people think about IQ as if it averaged 110-115, not 100. A "really smart" person has an !IQ around 140, and a "really dumb" person has an !IQ of, like, 98. (At its extremes, this is the fictional genius whose reported IQ is in the 170s or higher.) I saw a particularly prominent example of this process in a discussion of personality that referred to how a similar personality would "look different in someone with an IQ of 140 to someone with an IQ of 90". Those aren't symmetrical, but if you think about IQ how people tend to, you end up saying something like that.
(This isn't a range-restriction thing, exactly -- everyone does it.)
I've always thought accordingly that you'd be about right to estimate the average IQs of people taking the SSC/ACX/LW surveys as in the lower-mid 120s. This is extremely high! It is, in fact, about as high as what people are really intuiting when they think about someone with "an IQ of 140" -- the range of way-way-above-average people in the general population. It's also a plausible average *full-scale IQ* for a group of people who tend to be both smart and...eccentric. Subscales are pretty well-correlated in the general population (the "g factor"), but the correlations are not groundbreakingly large and many unusual people have sizable gaps between subscales. There are whole portions of the WISC/WAIS that are just "oh yeah, these are the sections that routinely break and lead to people's FSIQs being markedly off from some of their subdivisions".
Now, it'd be a bit of an exaggeration to say you can't test above an IQ of 135, in the same sense it'd be a bit of an exaggeration to say you can't test below an IQ of 65. We can definitely test in the mild intellectual disability range. Below that...hahahaha good question, someday we'll stop giving autistic kids "adaptive functioning tests" that dock points for stimming. I suspect a lot of the invalidity on *this particular metric* is due to things other than "the tests don't work that high", like "people are lying" or "people are giving an unrepresentative result" or "people misunderstood the results somehow" (given in a different comment) or "people have unusual subtest divisions, and the questions in the surveys didn't account for this" (were they matrices?). Probably all of these across many different people.
I’ve often noted this tendency in myself anecdotally and try to work around it—realizing that people with an 80 IQ are as common as those with 120 has gotten me to be more sympathetic with the slow person at the grocery checkout. They’re barely a standard deviation below the norm, and that’s not much at all.
> There's an observable phenomenon where people think about IQ as if it averaged 110-115, not 100. A "really smart" person has an !IQ around 140, and a "really dumb" person has an !IQ of, like, 98.
People are probably doing that because the average IQ of everyone they know is 110-115. I would consider a person with 98 IQ to be incredibly stupid. I would be right, in every meaningful sense of the condemnation - they would make laughably obvious mistakes at very high rates, show extremely poor performance on pretty much everything, etc., compared to anyone I'm in a position to compare them to. The lowest-performing member of my immediate family tests around the 90th percentile on standardized tests. Using the concordance that this post says we shouldn't use, that would be a 119 IQ.
I recall seeing some discussion of a result that found that, at elite universities, students who came from private schools had high opinions of what "average people" were able to accomplish, while students who came from public schools had sharply unflattering opinions. This tends to suggest that, if you tell people that a 98 IQ is average, people who are familiar with what kind of performance that involves will happily call it "really dumb", and people who aren't will say polite things about it.
Of the four schools I attended for a meaningful length of time in my primary/secondary ("grade school") educational career, three probably or definitely had average IQs below 100. I've lived in a lot of questionable neighbourhoods and met people from all walks of life. My academic research interests focus on things anticorrelated with IQ (e.g. particular genetic disorders). I am not from the upper-middle-class bubble and not range restricted. I...would not make the same post, let's put it that way. (There are very complex elements, some of which are *not flattering*, to discussing and highlighting range restriction as a non-range-restricted person. They are complex in wholly different ways.)
I carefully noted that this particular mistake is a different thing to range restriction, because it is. People of "evidently below average intelligence, when you talk to them" think about IQ this way. You notice it very prominently in fiction, where characters are represented as "smart" by having ludicrously high IQs. (You also notice it, if you're the smartest kid at a poor school, in how high people rhetorically estimate your IQ as.) People who score in the average range on IQ tests tend to think of their results as "really low", because we talk about IQ in a way that implies the 90-109 range is markedly below average rather than the interquartile range. (I have seen this, repeatedly.)
> You notice it very prominently in fiction, where characters are represented as "smart" by having ludicrously high IQs. (You also notice it, if you're the smartest kid at a poor school, in how high people rhetorically estimate your IQ as.)
I'm not really sure that this is informative of much. To quote Jonathan Coulton:
> People who score in the average range on IQ tests tend to think of their results as "really low", because we talk about IQ in a way that implies the 90-109 range is markedly below average rather than the interquartile range. (I have seen this, repeatedly.)
But this is a fairer point. However, you're explicitly marking this opinion as downstream of a select group that sets the tone for discussion of IQ.
Is that group range restricted?
------
You're right that my response is mostly on a point that isn't quite the one you were trying to make. If I read you correctly, you're saying that people think, because of some kind of cultural osmosis, that a 98 IQ is meaningfully below average.
But what I read you as saying was that people who say 98 IQ is "really dumb" are making a mistake. I don't think they are! I'm defending the idea that a 98 IQ is "really dumb". Most people are really dumb.
I do sometimes find this shocking, given how difficult I find it to do stuff everybody has to do, like budgeting and taxes and credit cards and health insurance. I realize some people just kind of fail at those, but most seem to do basically fine.
There is definitely a level of 'life smarts' or 'social smarts' that does not have to do with academic intelligence. If you're not book-smart but your parents/some other authority figure showed you how to figure out "this is how you do your taxes, this is how to balance a check book" and so forth, then you'll manage in life. The majority of people are "really dumb" by Mr. Watts metric, but they manage somehow to drive cars to their jobs and perform adequately enough to get paid, so they can then pay their bills, cook their meals, and raise their kids.
When you say "extremely high" and "way-way-above-average", please consider that in a city with million people, there are tens of thousands of people with IQ 120 or more, and thousands of people with IQ 140 or more.
High intelligence is not that rare in absolute numbers. It just seems so, because you will never see those thousands of people at the same place, as they are dispersed in a much larger crowd. But it would not be implausible to find out that ten of them read the same website.
Don't IQ tests also have an upper bound? Like you sit with a cognitive therapist or neurologist or whoever and they do several hours of different types of tests and then they report the score.
Every test has an upper bound, determined by getting everything correct. But "IQ tests" as a concept do not have an upper bound; you can have different tests that are informative at different levels of IQ.
If all you know about a person is that they were +2 SD on the SAT then on average they'll be less than +2 SD on an IQ test, but if you select a group of people for some other reason and they're +2 SD on the SAT on average then then you shouldn't necessarily expect them to be worse at IQ. After all, you could run the same regression argument in reverse (if all you know is that someone is +2 SD on an IQ test then on average they'll be less than +2 SD on the SAT), and the group can't be worse at IQ than at the SAT and also worse at the SAT than at IQ.
I'm also pretty skeptical of Spencer's attempt to norm an IQ test which was given to a convenience sample rather than a representative sample. I wouldn't expect controlling for 1 variable to fix that.
For example, suppose you had a complete dataset of every NBA player's height and their wingspan (distance between fingertip to fingertip with their arms outstretched). And you knew the distribution of height & wingspan for the general population (of 20-40 year-old men); conveniently they are normal distributions with a perfectly linear relationship and a correlation of r=.85.
Then you could ask the question: are NBA players more extreme by height or by wingspan (in SD above average)? I don't know the answer; without having the data it seems like it could go either way. Height and arm length are both useful for basketball so probably the sample is heavily selected for both. Maybe it's really close, e.g. +3.60 SD in height and +3.62 SD in wingspan.
Now, suppose that you didn't have the wingspan data for NBA players, just their heights. But you knew the relationship between height & wingspan in the general population, so you could set up a regression equation (or a table) which tells you the typical wingspan for a given height. With a correlation of r=.85, on average people who are +1 SD in height are +.85 SD in wingspan, +2 SD height ---> +1.7 SD in wingspan, etc. If the average player is +3.60 SD in height, this height to wingspan conversion would give you an estimate of +3.06 SD for the average player's wingspan. So very noticeably more extreme in height than in wingspan, since wingspan estimates have been regressed towards the mean.
In other words, using this regression equation to estimate wingspans is assuming that NBA players' heights are more extreme than their wingspans. But we don't know that; it could go either way. And if we were missing the height data and had only the wingspan data, then we could run the regression the other way around and estimate that heights are less extreme than wingspans. e.g., If the average player is +3.62 SD in wingspan then the wingspan to height conversion would give you an estimate of +3.077 SD for the average player's height.
This linear regression approach is the wrong tool for the job.
My mum was highly into IQ so I ended up taking an absolute tonne of IQ tests as a child, and I have fold memories of the one I got 225 on, but then on further investigation it turned up I'd added up my scores wrong so, you know, actually didn't. But if you count that one out, I remember getting scores between 105 and 175 from various tests, so, I mean, clearly they were not all super reliable. If I wanted, I could just take the highest (except the clearly erroneous 225) and walk around being like "yeah I tested at 175 IQ as a child" and it wouldn't be explicitly false but I'm not sure it would be honest. I do often suspect something like this is going on in these self-reports.
> I remember getting scores between 105 and 175 from various tests, so, I mean, clearly they were not all super reliable.
Unrelated technical note - in psychometric jargon, "reliability" is the property that when you take the same test twice, you get similar scores, the correlation of a test with itself. It isn't the property that different tests get similar scores; that would be validity. (Technically, validity is the property that the test measures "the thing you want to measure", not "the same thing as some other tests", but at least these concepts are related to each other - a collection of tests with high validity must also have a high correlation with each other,† since they are all correctly measuring the same value.)
† In some sense. To actually have a high correlation, they'll _also_ need high reliability, or you'll need to be measuring the correlation between tests in terms of the average value over a large number of testings.
"in psychometric jargon, "reliability" is the property that when you take the same test twice, you get similar scores, the correlation of a test with itself. It isn't the property that different tests get similar scores; that would be validity."
Sounds somewhat like the distinction between precision versus accuracy:
Some quick looking into it suggests that reliability and precision are the same thing, but validity is a composite of precision and accuracy. I overqualified my footnote - high validity automatically means high reliability.
I found a book once that made this point in the context of complaining about personnel selection - that the amount your selection method is able to achieve ("validity", the usefulness of the method) is limited by its reliability - but I don't want to spend the time right now to look that up.
> These are implausibly high. Only 1/200 people has an IQ of 138 or higher. 1/50 people have IQ 130, but the ClearerThinking survey used crowdworkers (eg Mechanical Turk) who should be totally average.
There's a huge availability bias in saying that those numbers are implausibly high. Scott, most people in his bubble, and probably a majority of the blog readers - they just don't interact with people below average IQ. Practically... never?
To quote Scott himself when talking about how well-selected his tribe is:
> And I don’t have a single one of those people in my social circle. It’s not because I’m deliberately avoiding them; I’m pretty live-and-let-live politically, I wouldn’t ostracize someone just for some weird beliefs. And yet, even though I probably know about a hundred fifty people, I am pretty confident that not one of them is creationist. Odds of this happening by chance? 1/2^150 = 1/10^45 = approximately the chance of picking a particular atom if you are randomly selecting among all the atoms on Earth.
Yeah, practically never. I don't know about ClearThinking, but the LessWrong numbers feel pretty much spot on. And even the bit about mturk workers being "totally average" - it's honestly a bit hilarious, and comes from never having to explain a below-average person concepts of equivalent complexity as mturk.
And once the large bias towards high IQ is explained, all that's left is disparity between various ways to measure it. Which yeah, is an interesting statistical exercise and I actually read and enjoyed the post. But, like I said, not a huge mystery.
I've interacted with tons of people of below-average IQ, including many people with outright intellectual disabilities, in both egalitarian contexts and otherwise. (I did not grow up in the upper-middle-class bubble.) The idea that the real average of respondents is that high is instantly risible. Absolutely way out of the bounds of plausibility. Same for the more extreme sorts of claims about "what people with IQs below X are like" (e.g. the famous hypotheticals 4chan post).
I notice consistently that people who share this background, or variants of it, are the people who find the high results and the "did you know people with low IQs don't have object permanence?????" claims most implausible. That's not to say there aren't very serious range restriction considerations -- everyone with a variant of this background knows it firsthand, too -- but these *particular* arguments don't seem to correlate well with out-of-range experience.
Why do you find it so risible? I'm trying to find some objective checks, and I find a couple that to me seem to make sense. I don't know my IQ but I'd estimate it in the 120s (software developer, previous star student etc), and LessWrong is, by and large, above me. I understand most conversations, but the effort is enough to not make me a regular reader.
Also 1/200 is... enough? It's over 1 million only from US. That's the population of a small country, easily sufficient to populate a small-ish forum.
None of those is an argument that the average IQ actually is that high, but at least I don't find it ridiculous. Accounting for some bias of people over estimating it or or bragging a bit... it sounds sane. Why does it seem not so to you? Am I missing obvious checks in the opposite direction?
It's a tiny minority, on a trait that readership of ratsphere writing definitely selects for, but doesn't select "significantly harder for than being in the upper echelons of academia". Ratsphere-adjacency selects on a number of uncommon characteristics, of which IQ is one, and probably not sufficient by itself; the threshold for intellectual curiosity/interest/trait-intellectualism is quite likely *higher* than that for raw ability, and every measure we've ever made of that trait has a pretty weak correlation with IQ. (I might believe not-super-far-below-99th-percentile averages for that.) 99.4th percentile IQ averages are just wackily high compared to every other group-IQ-average we have, including groups strongly selected for intellectual ability and (sometimes) interest.
(This is before getting into things like "disproportionately many people self-report large gaps between cognitive abilities, which depress full-scale IQ on an IQ test proper". I suspect one of the contributing factors -- albeit a small one -- to the ultra-high survey reports are people taking their highest subtest as "most representative".)
So you're coming from a selection perspective, and I come from a filter perspective. You're right that IQ is far from being a main driver of people going to those places. I completely agree with you here. My argument is that for a large portion of the people who would be interested in them, the effort to follow and participate is working as a self-filter. And it makes sense from my subjective experience - there are quite a few places where I would very much like to spend more time, but... it's simply a matter of mental energy. Not capacity per se, but effort per time spent.
For example I very much enjoy DataSecretsLox, but I find that I avoid long threads and actually spend a lot less time there than I'd like. It's not a matter of being able to comprehend notions (though occasionally some comments are over my head as well), but simply of how much text I can consume per hour spent there. I'm regularly amazed of how much information Scott is able to absorb - which is a separate skill from how good a thinker he is.
So I'm suggesting that higher IQ comes also with something like higher endurance for intellectual tasks, and that's a bigger filter than pure skill.
Curious as to why you found very high average IQ implausible for LessWrong readers? Personally, I'd be stunned if it **wasn't** an amalgam of the literal smartest people in the world.
College professors are usually 120, a group with IQ 138 would be to college professors as college professors are to the average person. I don't want to say that no group achieves that, it just seems pretty extreme.
That's an interesting point. Is it possible there's minimal measurable difference in actual brainpower between 120 and 140 and the practical difference is not as vast as it sounds? I suppose I have a lot more thinking to do on this topic because I am in the "Very High IQ" bracket and I can tell you, I'm no Von Neumann.
If I think of some high achieving academic acquaintances, they don't immediately stand out as smarter than me, they're all just uniformly willing to play politics and work within a system that sickens me. Don't think there's a correlation there between IQ but I guess there could be.
1. The "practical difference" between 120 and 140 should, on paper, be equivalent to that between 60 and 80. This is to say it should be very significant indeed. This should-on-paper hold true as you add and subtract further, though realistically you'll quickly start to bump up against what can possibly be tested.
2. The correlation between IQ and the "clinical impression" of someone's intelligence is weaker than #1 implies. This is the paradox of "learning how seriously to take IQ" -- it measures an important and meaningful thing that does not perfectly correspond to how we think of "how smart someone is". Over the general population, every subscale on an IQ test correlates pretty well with every other subscale on an IQ test, which correlate pretty well with your impression of how smart someone is. On the individual level, particularly at extremes or for unusual populations, this is less guaranteed.
("What does it mean for something to correlate with another thing well, but not perfectly?" is a great accidental mini-intelligence-test. Surprisingly many people fail it.)
That the impact of IQ should scale linearly seems not necessarily true. It could scale exponentially or it could scale sub-linearly. Suppose IQ amounted to time required to solve a problem. As you progressively solve the problem in half of half of half the time, it becomes far less significant. On the other hand, if it were additional dimensions of complexity one was capable of holding in one's head at a time (it isn't) it would scale far more than linearly. My feeling, is that it IQ is either related to the trainability of "System 1," or just to a readiness to notice parallels or analogies in new learnings while quickly dispositioning them as either valuable or not. (or possibly, that is what I have most efficiently trained my "System 1" to do....
I'm no IQ expert but my understanding was that an IQ score of a particular number is defined by reference to where it is in the (presumed normal) distribution of IQ scores.
In other words, some random website tells me that an IQ of 140 means you are smarter than all but 0.3830% of people. They presumably got that number from the CDF of the normal function ... having an IQ of 140 *by definition* means that you are smarter than all but 0.3830% of people. It doesn't mean that you are 20 "brainpower units" smarter than someone with an IQ of 120, or something like that.
So I don't think this:
> a group with IQ 138 would be to college professors as college professors are to the average person
really means anything. The only thing that you can say about (1) the difference between 140 IQ people and 120 IQ people, and (2) the difference between 120 IQ people and 100 IQ people, is that the number of people *in between* the IQ levels you're comparing, in each case, corresponds to 20/15 = 1.33 standard deviations worth of people for a normal distribution with mean 100 and standard deviation 15 (which isn't even a set number of people).
"College professors" isn't the filter you think it is. A group with IQ 138 would be to the average person as high-end physics professors are to the average person.
"College professors" are extremely heterogenous and are often selected for political reasons rather than academic reasons.
Yeah, I also wanted to ask "which college are we talking about?", because some college professors are... uhm... not very smart.
Smarter than a random guy on a street, sure, but to find a group that is smarter than them as much as they are smarter than the random guy on the street... I think a group of students from a prestigious STEM school would probably be like that. Which sounds similar to Less Wrong readers.
Something in your calibration feels so wrong that I have a problem finding words. Which sadly would make this comment quite useless, so here is my best attempt...
For starters, perhaps don't think of *STEM* colleges. Instead, imagine the kind of people who read "Transgressing the Boundaries: Towards a Transformative Hermeneutics of Quantum Gravity" and think: "wow, this guy is so smart", because they are too stupid to understand that they were just reading word salad. Because their own best attempts to produce science are not too different.
Or maybe just don't think of the *elite* colleges, and consider the average ones, which are more numerous. I have met a math professor who didn't understand math, and who only gave good grades to students who solved the problem exactly the way it was taught at his lecture, because if it was solved slightly differently, he couldn't tell whether the solution was right or wrong. I have met a professor of economics who couldn't tell whether "10, 12, 14" was supposed to be a rising or a falling line on the graph.
IQ 120 basically means: has the ability to talk about complex topics. "Talk about" does not imply deep understanding. IQ 130 is Mensa. Less Wrong is definitely much smarter place than Mensa.
I guess you probably know a few very smart college professors who are far from being typical representatives of their profession.
> imagine the kind of people who read "Transgressing the Boundaries: Towards a Transformative Hermeneutics of Quantum Gravity" and think: "wow, this guy is so smart",
Well, the guy who wrote that paper is in fact pretty smart. /jk
That doesn't sound so unreasonable. Consider that:
1) Professors are also selected for ability to focus intently on research, which is a different ability but also contributes to perceived formidability
2) When we interact with professors, it's usually in their field of specialization
3) That's the average of all college professors. When we picture a professor, it's probably from a top school and a more g-loaded department
Wjem I compare academic articles and lesswrong posts, I expect the articles to show more effort and diligence but the lesswrong posts to show better thinking.
Hm... this is just my n=1 anecdote, based on being a professor in STEM, having taken the WAIS-IV in adulthood, joining MENSA, joining the Triple Nine Society, and attending SSC meetups over the past few years. But in my impression, the typical SSC meetup member is absolutely smarter than the professors and MENSAns. I say this based on the depth and breadth of conversations, the "polymath" abilities, and the interest in deep dives on obscure topics. People I know who are professors, doctors, or lawyers, generally spend their free time on typical leisurely things like golf or gossiping at the country club, and seem to think people who enjoy philosophy or statistics or deep-dives are weird. They do not seem to have explored subjects beyond what they were directly required to learn from a textbook. At work, I feel like I know significantly more about my subject, and consider nuances and connections, more than other professors. At MENSA nights, I felt rather bored and dreary. But I always feel intellectually stimulated, invigorated, and challenged at the SSC meetups. This is only my impression, but, yes, I do think the difference between the intense community here and the professors I work with is absolutely as large as the difference between professors and the average person. *Perhaps* the difference between typical "smart" people and the community here is not entirely due to intelligence, but also due to the personality trait Openness? But overall I would not be at all surprised if the average IQ of readers here is 130-138. In fact, I'd be surprised if it were lower. It sounds like Scott's main reason for re-scaling the self-reported scores is that the reported results seem "implausibly high" to him. But I can say with certainty that I have never talked with someone outside of SSC readers who seemed interested in (or even aware of) the types of topics investigated here. The idea that readers of this site are in the top 1% of the IQ range seems very likely to me.
Whenever I come across voluntary surveys of IQ, David Mitchell's rant about who tries to become a member of MENSA and who does not, comes to mind. It's about selection effects. Not particularly relevant in this context perhaps, but it is a fun rant, as are all of David Mitchells' rants:
I made one IQ test in my.life where I scored "below 130" (don't remember the exact number, since 130 was a cutoff for something) and one, later, with 135.
If I am asked for IQ number, I now report 130.
So for ppl who did multiple IQ tests, we have an upwards bias (even if asking for the most recent one, because people redo it more likely if they're unsatisfied with the last), whereas "doing live testing" gives us an unbiased snapshot, including things like "I slept bad and my coffee machine broke and my brother is dieing" unconcentrated for some)
I sometimes wonder: what would be the mean IQ if everybody was allowed to submit the best test result throughout their lives? 120?
That depends on how many tests people take. Your measured IQ is equal to your true IQ plus measurement error, generally conceived of as normally-distributed noise. If the noise is normally-distributed, there is no bound on its value, so with an infinite number of retests you can achieve an infinitely high IQ.
In reality, you're only going to take a finite number of tests, but it remains true that the more tests you take, the higher your high water mark is expected to be.
Who cares. I understand that you collect this information for your own uses and that a lot of people take tests offered by the internet because they want to know how smart they are but we don’t ask people what their IQ is and we suppress laughter when someone volunteers that information.
Nor does it predict success in life—life being more than professional attainment or wealth. Of course admission to elite universities requires greater facility with mathematics and language. And physicians and engineers are all pretty smart—but to me here is the key question. Two surgeons will have the same IQ but one will be outstanding and the other a disaster. A doctor with a lower IQ than their colleagues may be the better diagnostician of the bunch.
Let’s move from technical professions to business and law. The ability to relate to all sorts of people is a far more valuable attribute than one’s IQ. In some instances and to some extent it may be inversely proportional.
I love your brilliance, but I think this post was dumb.
> I love your brilliance, but I think this post was dumb.
This would be a more compelling insult if you showed any awareness of any of the standard counterpoints to the myths you repeat, or even any awareness that there *might* be counterpoints that you are unaware of but Scott is and why he doesn't appreciate your stunning insights like "did you know some doctors are better than others but may not be smarter?". As your comment stands, it mostly shows your inability to relate to other people.
Gwern, I think Scott got the joke: my calling an essay on IQ, dumb. Besides, who am I for Scott to feel insulted by any opinion I offer. But to answer your critique of my comment, a few observations.
First. What I like about Scott’s articles is that he respects his audience and doesn’t dumb down his discussions. But his Substack is subscribed by numerous ordinary people like myself who find his analysis of ideas and issues cogent even if we can’t always follow his math. Unlike you I do not presume to know whether that was his intent when he created his blog, but I suspect that he knows a great deal about those who read his posts.
Second. My objection to his discussion is that he doesn’t relate his analysis to anything in the real world. The only thing he says is why he collects this information. If his post was directed only at specialists, he should have published it in an academic journal.
Finally. If there are counter arguments that undermine or refute my comments, educate me. I don’t have any problem with someone explaining why i am wrong.
> And physicians and engineers are all pretty smart—but to me here is the key question. Two surgeons will have the same IQ but one will be outstanding and the other a disaster. A doctor with a lower IQ than their colleagues may be the better diagnostician of the bunch.
Hey, guess what? IQ correlates extremely strongly with job performance. More strongly than almost anything else. Expecting a 100% correspondance between two things is simply an isolated demand for rigor (something you should be well aquainted with as a reader of Scott's). There's almost no non-trivial 100% correlations between things, and if we threw out anything less than a 100% correlation then science as we know it would not exist.
>Two surgeons will have the same IQ but one will be outstanding and the other a disaster. A doctor with a lower IQ than their colleagues may be the better diagnostician of the bunch.
Don't call people "dumb" when you lack the basic ability to think about things statistically.
Is it possible that a lower IQ doctor is better than higher IQ doctors? Certainly.
Is this important? Only if we there isn't a strong *correlation* between job performance and IQ, such that predicting a person's performance from their IQ wasn't possible. In reality, it is very possible and without knowing anything else about your doctor, you always want a higher IQ doctor over a lower IQ doctor.
It's like saying height is irrelevant for basketball because some short people are great at basketball. Muggsy Bogues at his peak was better than almost everyone on earth at basketball despite being only 1.6m tall. And yet, if you bet on whether the much taller than average kids at a school would be more likely to make it to the NBA than the shorter than average kids, you would be a fool not to bet on the taller kids.
Mensa advertises that 2/3 of people get in with “prior evidence” and don’t have to take the Mensa test. They also only accept SAT scores older than ‘86 IIRC.
I took the Mensa test about 15 years ago and they also administered the Wonderlic beforehand. Possibly for renorming. When I received my results it was 130+ (or maybe 132+) because that was the limit of the test. I would be suspicious of anyone claiming to get a very high score on the Mensa test.
Non Mensa related: I found a bootleg copy of Ravens and it also capped out in the low 130s. There’s a separate step 2 if you want to test higher.
"They also only accept SAT scores older than ‘86 IIRC." Yes, it's easier today to get a higher SAT score, so the date you took it matters as to the correlation with IQ.
Most college acceptance tests are no longer accepted. This matches my understanding that the folks making these tests have been consciously making them less g-loaded (e.g. removing the analogies portion of the SAT verbal). But it is interesting to see Mensa spell it out without *quite* spelling it out.
Given the community here I might get a hit on this and have always been fairly curious as to the experience of others: Anyone else get wildly different math and verbal scores? I was put in special education the first week of the first grade because I had a lot of difficulty making myself understood. I used to put that all on some panic stuff I had going on but now I’m less confident that was the biggest reason. Even after I eventually learned to translate my head universe into communicating with other people there was still a pretty significant gap. If I’m reaching back I think I got a 35 in the math section of the ACT and maybe 29 in verbal? I’m always able to look at something and figure out how to fix it much, much quicker than I’m able to figure out how to explain how I know to someone else.
Yeah, if I remember right, my ACT was 22 math, 30 science, 32 English, 36 reading (this was 1994.) I'm not sure that reflects any "natural" ability at math I might have; math just wasn't interesting/didn't click for me in high school. It's funny; I was skipped ahead a grade while you were in special ed, and you were probably brighter than I was. Shows how weird the system is.
I'd probably do quite a bit better on the math now, assuming I studied for it. Math "clicks" for me a lot better now. Not that I could jump in to calculus or anything, but all the algebra and trig stuff I missed I could probably pass now. At least going by how able I am to help my kids with their high school algebra homework.
The combination of high math and low verbal reasoning scores is a pretty common trait in autistic people, probably also reasonably common in people who might have a dyslexia diagnosis or people who have subclinical traits in that direction. My personal gap is processing speed - I completely maxed out a couple sections of the last IQ test I took, but my processing speed is dead-nuts average.
ACX/SSC-only people often won't get this, but in Lesswrong and AI safety-oriented culture, being merely one-in-300 is pretty terrible news; it means you were the smartest kid in almost every classroom growing up, but now you are median, and the people doing the lion's share of impact on Earth are the one-in-10,000 types.
Why would anyone read your post/paper at all, or hire you for their org, if they can spend that time reading/hiring a one-in-10,000 person instead?
Because even in intellectual endeavors, there are other things that matter than pure intelligence. Other traits, like creativity, ability/willingness to look at one’s own (or one’s social circle’s!) ideas critically, to slog through lots of boring parts… just for three top-of-the mind examples from a probably very long list. Not to mention domain knowledge, including deep knowledge that one can only get from experience or by having an unusual degree of specific intrinsic interest… I’m more likely to read someone’s work because it’s *different* than other peoples’ than because it’s doing ‘the same thing, but better.’
Is this not likely to be reporting bias/selection effects?
- If you think you are average will you bother getting an IQ test?
- If you did a test and got an average IQ are you likely to fill in a form about it or will you only bother if you scored high and want to boast a bit?
So it could be that only the people who suspect they are in the highest rungs will have had and IQ test, and of those the ones who scored highest are more likely to complete the surveys as a boast and you get very skewed selection effects
I had a tested IQ of 138...when I was 10, on the WISC. This raises three obvious lines of questioning.
1. Does age-adjusted IQ at 10 correspond to an equally high adult IQ, and older adult IQ?
2. IQ declines over time, probably due to decades of substance (ab)use, head injuries, illness, and most importantly general aging. How does that skew the data?
3. Would someone with an IQ within a standard deviation of the mean be more or less likely to be sent to a fancy psychiatrist in the inner city to take the WISC/WAIS/etc.?
I think I'd expect to get 115 to 125 on the WAIS as an adult. My memory is shot, I'm slower than I used to be, and I struggle with some mental tasks I used to find easy. I remember getting somewhere around 120ish on the least-disreputable online IQ tests. It's not clear to me that "my IQ was X when it was tested" means "my IQ is X" and I think time is likely a bias.
It would also be interesting to see if the WISC was unusually biased compared to adult tests, and interesting to collect data on age of measurement. Maybe everyone has a higher tested IQ because most of them were either taking an age-adjusted test as a child, or in the 18-25 age range when their IQ should have been at its highest point.
re: " My memory is shot, I'm slower than I used to be, and I struggle with some mental tasks I used to find easy."
I also wonder if many notice this as they age? I felt my mind was fairly quick and strong up through about age 30, but started noticing a subtle dropoff after that, and now in my 50s I can without hesitation say my mind does not have close to the "horsepower" it used to have. I guess it's possible I'm assessing wrongly and misinterpreting past ease/difficulty, but I'm more than 50% on this.
Your experience matches my own impression perfectly.
In addition, I had a near-eidetic memory until about age 30, and I was interested (and dismayed) to start to experience this 'forgetting stuff I meant to remember' thing that others talked about.
I tested out at around 135 when I was younger but would bet dollars to donuts that I'd test lower than that now.
I also feel that I take longer to learn new (complex) ideas and processes. But on the upside I also feel that I draw more widely from other disciplines and can see functional implications and outcomes faster and more accurately.
Psychometricians noticed that scores on a lot of different types of cognitive test are correlated. The general intelligence factor, g, which IQ tests are intended to measure, is the correlated component of those scores. But it turns out that the relationship between scores on different types of cognitive tests is heteroskedatic— they become less correlated as you go further up the scale. This also means that if different assessments of g include different items, their correlations will drop at the high end. More profoundly, it implies that IQ-type metrics get less useful as predictors of performance on specific tasks after you get past a certain threshold. So, “reported IQs over 135 are less meaningful” is a good heuristic.
Have you considered just asking your readers to take a well-normed IQ test that directly reports the results to you? Like, I’m pretty sure you can just pay Wonderlic to administer an IQ test to whoever you want (I don’t know how well normed they are, but it’s got to be better than the existing situation).
SAT is a US thing. As an Australian I don't have one. I didn't read the article in detail but maybe that affected the scoring -- non-US-educated either not reporting or guestimating.
If you were to suggest that only the smartest 1/200 or even 1/10,000 people are the ones that read this blog/have taken Mensa tests/iq tests, I would absolutely believe you. The high level of reading / logical reasoning to both understand and enjoy this style of writing is such a strong filter.
I know we avoid complex language to make the blog more accessible to anyone, but to enjoy it is another story.
I’d assume based on my peer group that people with an average IQ, don’t read at all and mostly scroll tiktok. If they do read it is very simple books/novels.
In lieu of replying "the correlation between intellectual ability and intellectual curiosity is very weak" to everyone who says something like this, I'm going to tell a story about a case study I once read.
It was on a man with a particular genetic disorder, which is associated with a characteristic "behavioural phenotype" -- distinctive shared personality traits between many people with the syndrome -- and with often-severe intellectual disability, but he had been spared from the latter. He had his share of social and emotional problems, but was an avid reader from a young age, consistently performing well above average in language and literature-related classes at school. His reading choices trended towards the "Great American Novel"-types; his favourite authors were Faulkner (not a man known for his terseness or simplicity!) and Steinbeck.
His WAIS full-scale IQ was 85. His verbal IQ was quite a bit higher -- that is, 117 (his other scales were all 70ish). This is an extreme skew, but there are neurotypes where multi-SD skews above and below the mean are far less remarkable than in the general population (e.g. autism).
Even if we take "a verbal IQ of 115 or thereabouts is probably the minimum possible to appreciate Faulkner", which is an extreme reading of this (because intellectual interest is weakly correlated with intellectual ability!), you don't land somewhere that gets you an average FSIQ nearly as high as reported in these surveys. Ratsphere bloggers are quite a bit more accessible than most of Faulkner's work; I am unconvinced the level of verbal and logical reasoning needed to appreciate them is multiple SDs above that needed to appreciate him, particularly for posts that are more about generally-accessible subjects (e.g. politics) than technical or scientific ones.
“I assume based on my peer group that people with an average IQ, don’t read at all and mostly scroll tiktok. If they do read it is very simple books/novels.”
This is such an absurd statement, I can only assume bbqturtle was being facetious.
People with an average IQ don’t read at all?
Mostly scroll TikTok?
They only read simple books/ novels?
Just how much intelligence do you need to read and enjoy “A Take of Two Cities?” Or “For Whom The Bell Tolls?” Hamlet is only interesting to people with an IQ of 110 but not 100?
In 1950 there were only about 100,000 books in print. Today there are well over a million. Who’s reading all those books.
Ok, let us assume that these statements are not so absurd to be debatable. I only asked for some proof.
Of course, reading bbqturtle’s statement over again, maybe I misread her statement, maybe she is describing their peer group. In which case I would say, they’re hanging out with the wrong crowd.
To what extent do these "IQ" tests remain correlated with wide-spectrum achievement at the 130+ level? The scholarly paper has 116 students, there's no way there are enough data to base a claim that these tests are measuring general intelligence at the tails. Not a test-repeatability question but a test-to-IQ question.
Having spent my life in places where a 1400 SAT would be considered kinda low, there is clearly not an expontential-tail distribution where the 1500 SAT people are so obviously less intelligent than the 1600 people, even on specific things. And the "true genius" types are less likely to have general intelligence that you would call g, and instead seem exceptional in a particular area.
The concept of Bell-curve IQ describing general intelligence at genius levels should go IMO. Taking with it all the implications for godlike AGI.
My 1985 SAT scores - 710 math, 720 verbal - were good enough to get me into Mensa - 98th %ile - and Colloquy - 99.5th %ile - but not the triple 9 societies. I estimate my IQ as 3 SD up, ~140. A friend of my mom's, Stephanie Creamer, was getting her doctorate in industrial psych around 1980; consequently, I took A LOT of IQ tests, and usually hit around there, though my scores trended slightly upward as I kept practicing.
Parenthetically, the Raven matrices were the only times - standard and advanced - that I broke 150; Tanny was so awestruck she took us to Cafe Provencale.
Jensens point is very reasonable but Im not sure its right for your application. If you have genpop looking at an internet site to convert their SAT to IQ, you should use his calculation. If you have a sample thats selected from genpop for their SAT, then his calculation will also give their average IQ correctly. On the other hand, if you have a sample selcted for IQ, and then you measure their SAT, and then you apply his calculation, you will underestimate IQ.
So, if your survey sample is unusually smart, what you should do depends on why you think that is: If its selected mostly for SAT, use Jensens calculation. If its selected mostly for IQ, you need to do the opposite correction, and if its about in the middle, then the original percentile calculation is about right.
I left a comment about this same issue. I don't understand why the concordance is supposed to be wrong.
If you have a sample that's selected for their (above-average) SAT scores, and you measure their SAT scores, the new average will be lower. That's regression to the mean.
If you have a sample that's selected for their above-average SAT scores, and you measure their IQ, the new average will be lower. That's regression. Regression to the mean? That appears to be a philosophical question.
But if we're willing to use people's measured SAT score as a point estimate of their SAT score, why aren't we willing to use it as a point estimate of their IQ? If you want the true value, you need to do an adjustment in both cases.
"People who took the SAT but “don’t remember” their score have tested-IQ 104"
But...I don't remember my score. It was 45 years ago! I know it had to be above 1200 combined, because in those days California was legally required to let you into a campus of the University of California if your GPA was over 3.2 and your SAT was over 1200, and if you met several other requirements like X number of math, science, and foreign language classes. And I remember thinking "Great! UC is legally required to admit me, I can exhale." And indeed I did go to Cal.
I can’t find the hard copy of the Al Franken book I’m going to reference right now so I’ll have to paraphrase from memory.
Apparently one of the first questions Ted Cruz likes to ask when he meets a person is “What’s your IQ?” If the person doesn’t know he asks “Well, what is your SAT score?”
Franken goes on to say “I like Ted Cruz more than anyone else in the Senate does, and I hate Ted Cruz.”
I really like Al, he’s a funny guy and was a great senator IMO and his being driven out of the Senate was uncalled for, again IMO.
I still get his emails from him with the greeting “Dear person who still opens my emails.”
He still works for Democratic candidates and sells merch on his web site to that end. One item is a coffee cup with a pretty good caricature, done by Al, of Cruz on one side and Al’s withering opinion on the other.
The SAT is a test Americans take before going to college. In the old days, it included a section called Verbal; now that section is called Reading. If you took the SAT, what was your score on this section?"
When I first read the draft of this article, I was impressed by how far Scott could adjust downward the IQ scores without resorting to calling the people liars, but I still think that the drop in correlation above 135 is probably mostly due to people lying. They either lie big or don't lie at all!
I thought the SAT question on the survey might serve as a test: one would expect that fewer people would input a very high incorrect score (because their reading comprehension is better), but that if liars lie big, then the opposite would happen—more people would input an incorrect high score.
As a rough test, I decided to divide the scores between 801 and 1600 by 2 and compared them with the other scores by 10-point intervals.
I then looked at what fraction of people answered incorrectly. Unfortunately, this didn't tell me much except that there's no clear trend to my eye.
> I still think that the drop in correlation above 135 is probably mostly due to people lying
There's always a drop in correlation whenever you restrict a variable.
> one would expect that fewer people would input a very high incorrect score (because their reading comprehension is better), but that if liars lie big, then the opposite would happen—more people would input an incorrect high score.
This doesn't make sense; the motivation for claiming an impossible score cannot be that you're trying to fool someone.
>>This doesn't make sense; the motivation for claiming an impossible score cannot be that you're trying to fool someone.
I see that this was not as clear as in should be. I am restricting the statement only to people who answered the questions in a rage of 801 and 1600. By very high incorrect score I mean >1500 as compared to, say, 1200.
Edit:
Let me restate it.
If someone answers >800, I assume they have put in their total score instead of just the verbal score. I think people who are good at reading comprehension are less likely to make this mistake.
So if we compare the ratio of the number of people who answered 1600 vs 800 with the ratio of people who answered 1200 vs 600, I’d expect the 1600 vs 80 ratio to be lower. As the scores get higher, the number of people making this mistake gets lower.
But, if people are lying about having a very high score, then that ratio would be artificially high at the very high scores.
"I'm a little disturbed at the claim above that several dozen people asserted their score on the verbal section was 1500 or more."
I went to high school with (at least) two people who scored above 1500 on the SAT before the 1994 "re-centering." If we adjust scores to the post 1994 scale then I know a few more who could clear 1500. This isn't several dozen, but it is at least four. From a pretty small peer-group (I know lots more people but don't routinely exchange SAT scores).
No. At the time and in my part of the country the ACT was the only test that mattered for college entrance in my region. Scores on tests also seemed to matter much less than they do now. For my friends they were more of a hurdle - "hey, if you want to go to college go to the city on a certain date and take this test." Some of us got lost on the way. ;-)
You are reminding me of a classic Far Side cartoon. The little fella trying to enter ‘Midvale School for the Gifted’ is pushing strenuously on the entry door clearly labeled ‘Pull’.
I didn't take the survey so maybe the question accounted for this, but when I took the SAT there were separate reading and writing sections with a maximum of 800 each, for a maximum possible score of 2400. If Scott *only* asked about reading, and didn't ask for separate writing scores, it's possible people were simply adding their reading and writing results together for a total verbal score, which could easily have exceeded 800.
"even official IQ tests are gobbledygook over 135"
This.
How do you make a test for someone smarter than you are? At some point, you're running into the doomsday scenario for AI, where it runs rings around you.
I assume IQ test makers tend to be smart people. But how smart, and how many of them are there?
> How do you make a test for someone smarter than you are?
1. You find the smartest people in generation i to write the test for generation i+1. "What interview questions do they ask at Google?" (back in the Old Days)
2. You write lots of questions, and determine empirically which ones are hard and which ones aren't.
3. You spend way more time than they get to spend.
4. You have a team of specialists, each of whom writes questions for their specialty, and then you combine them for the general test.
5. You find competitive scenarios and do relative rankings; you don't need to be as clever as the best chess-players or go-players in order to invent chess or go.
How do you find the smartest people in generation i? Get them from the ones who wrote the test for generation i-1 and pay them enough to write more tests as opposed to the thousand other things they could be doing with their time.
"Determine empirically which ones are hard". I guess that's by testing a lot of people with them and seeing which ones get solved least? I suppose you would have to keep up with algorithms in a number of fields.
How does time relate to IQ? How do you calibrate?
And yet, there's some evidence that as you get smarter, it tends to be in one or two fields, rather than generally.
I think that last is probably about all you can do: determine relative ranking. Sticking a number on it is more grading based on those who take the test and then curve fitting, which gets worse as you march further out on the right hand side.
And yet, IQ, like height, cannot actually be a bell curve, as it is necessarily anchored on the left. You can't have negative IQ, like you can't have negative height.
Sure, but this is not relevant to the bulk of the distribution for height, and I don't think it's relevant for IQ either.
[The choice to force IQ to be a normal distribution--instead of lognormal or uniform or w/e--seems like a good guess to me, but the sort of thing that needs to be empirically justified, and I think doesn't really hold up. Height is more complicated than a normal distribution in inches, and IQ probably is as well; sadly there's not an equivalent to 'inches' for intelligence.]
>How do you make a test for someone smarter than you are?
Easy. Writing questions is much, much easier than solving them. I can create a very complex pattern progression by changing some element or elements in each subsequent picture. But if I were presented with this progression without ever seeing it before, it does not mean I would have any hope of identifying what the progression is based on and what would come next in a reasonable time frame.
I can generate all kinds of questions in this way, then test them on enough people, and if most people can answer it correctly then you've made it too easy, if nobody can answer it you've made it too hard and so on.
Or to use a completely trivial example - I could make an arithmetic test that would be challenging for people much smarter than me. I could use a calculator to solve sqrt(5,721) and then ask people to solve it without a calculator and compare it to the answer I got from a calculator.
I think that your method of considering the distributions could use refinement. If one just assumes that nobody below 100 IQ is involved in reading less wrong then the new mean for the half-normal distribution is 112 100+(sqrt(2)*sigma/sqrt(pi)). Essentially, eliminating a significant bottom portion of the distribution substantially skews the mean without the need to assume an inordinate number of high IQ individuals.
This article is especially fascinating to me because there needs to be another possibly: testing bias(?)
I'm old enough that I've taken: the GRE, GMAT, SAT and ACT. My respective scores were 2200, 780, 1410 and 34. I also took the Wunderlic for a job and scored 49.
So this should mean I'm some sort of brianiac eh?
Nope. When my IQ was actually tested - in a rigid setting - it came out as 111. Not dumb but definitely just... bright normal, which I think is the actual term ascribed to me.
So what gives? Outliers exist but I am highly skeptical that I'm the only person out there who aced a bunch of standardized tests only to actually be told: hey, you're bright but not exactly Einstein
> It looks like up to about 140, self-reported IQ and normed IQ rise together, and then the relationship breaks down. Sure enough, looking at the subset of self-reported IQ scores below 140, the correlation with tested IQ rises to .6, and looking at the subset above 140, the correlation is nonsignificant at -0.02. I don’t want to assert that the breakpoint is exactly 140, but I do think the test stops working somewhere in the 130 - 140 range.
Didn't ...you... already write a post about this being an illusion? Pointing out that the reason for a weaker correlation between IQ and salary in the subset of people with very high IQs than overall is simply that there is no other possibility, and that in fact in a model where we know the relationship between IQ and salary is constant, we observe exactly the same thing? Why is the same argument that's invalid as applied to IQ-vs-salary valid as applied to IQ-vs-IQ_in_the_past?
------
I think I've seen Emil Kirkegaard make the same point as in #1 - that you shouldn't convert SAT scores to IQs on a percentile-to-percentile basis, because the correlation between them is only the stylized 0.8 that's expected between two different IQ tests, and not 1.
But I don't really understand why this is an argument against doing the percentile-to-percentile conversion. The correlation between the Wechsler and the Stanford-Binet is also a stylized 0.8, because they're both IQ tests. But we don't go around saying that measured Stanford-Binet IQs are incorrect (they're correct by definition!) because they need to be regressed by the correlation to Wechsler, which is the True Arbiter of IQ, and we especially don't simultaneously say that, even though Stanford-Binet IQs are too high because in reality they need to be regressed by the correlation to Wechsler, Wechsler IQs are 𝗮𝗹𝘀𝗼 too high, because they need to be regressed by the correlation to Stanford-Binet!
We do say that if someone has a high measured IQ, their actual IQ is most likely lower, because of regression to the mean. This also applies to SAT scores, which are the same thing. But that argument suggests, to me, that percentile-to-percentile is the 𝗿𝗶𝗴𝗵𝘁 way to do concordance. What am I missing?
IQ is a normalized distribution so for every person with an IQ of 140 there is a person with an IQ of 60. Someone with an IQ of 60 is not going to be taking these kinds of quizzes while someone with an IQ of 140 is very likely to. I’d even expect people with an IQ of 100 to be very unlikely to be interested in these kinds of surveys. If you cut off the left tail of the IQ distribution, of course your average will be higher than expected.
I think that if people take IQ tests, they take them as kids because they're being tested for "are you really bright/is something going on as to why you're not doing well in school and/or socialising with your peers?"
And thus I think people as adults may be reporting child IQ test results, which I don't know if one is related to the other accurately. Yes you were very smart for your age when you were twelve, but are you still very smart for your age now you're thirty?
Anyway, I cling proudly to my Completely Reliable, Honestly! Online IQ Score result of 98 😁
I do think it's fairly accurate because it was based (they said) on Ravens Matrices and I am *hopeless* at pattern recognition, shape rotation, or maths. Given that the tests were all pattern recognition, shape rotation, and maths, this result seems legit to me.
I do wonder what it's like to be able to look at a sequence of numbers and *immediately* see that "The pattern is X, Y, Z so the next digit should be A". But I'll never know, no more than I'll ever know what it's like to be a bird or a whale or a sloth, happily clinging to a branch in a South American jungle as the lichen grows on my fur because I am so inactive.
This makes me think that we're moving to a new age of calculating "intelligence", where now we're in the Information Age of AI And Computers, it's all based on mathematical ability rather than verbal. We're moving between C. P. Snow's Two Cultures, where before to be considered 'smart' was judged on the basis of your cultural and literacy attainments, but today it's based on "can you even code, bro?":
" 'A good many times I have been present at gatherings of people who, by the standards of the traditional culture, are thought highly educated and who have with considerable gusto been expressing their incredulity at the illiteracy of scientists. Once or twice I have been provoked and have asked the company how many of them could describe the Second Law of Thermodynamics. The response was cold: it was also negative. Yet I was asking something which is the scientific equivalent of: Have you read a work of Shakespeare's? I now believe that if I had asked an even simpler question – such as, What do you mean by mass, or acceleration, which is the scientific equivalent of saying, Can you read? – not more than one in ten of the highly educated would have felt that I was speaking the same language. So the great edifice of modern physics goes up, and the majority of the cleverest people in the western world have about as much insight into it as their Neolithic ancestors would have had.'
...Snow's Rede Lecture condemned the British educational system as having, since the Victorian era, over-rewarded the humanities (especially Latin and Greek) at the expense of scientific and engineering education, despite such achievements having been so decisive in winning the Second World War for the Allies. This in practice deprived British elites (in politics, administration, and industry) of adequate preparation to manage the modern scientific world. By contrast, Snow said, German and American schools sought to prepare their citizens equally in the sciences and humanities, and better scientific teaching enabled these countries' rulers to compete more effectively in a scientific age."
I'm one of the dinosaurs of the Neolithic who is verbally able, but totally lost now in the era of you small nimble mammals with your maths manipulation abilities.
>This makes me think that we're moving to a new age of calculating "intelligence", where now we're in the Information Age of AI And Computers, it's all based on mathematical ability rather than verbal. We're moving between C. P. Snow's Two Cultures, where before to be considered 'smart' was judged on the basis of your cultural and literacy attainments, but today it's based on "can you even code, bro?":
I'm not speaking of IQ tests, I mean as Snow puts it, people who were regarded as well-educated, cultured, successful and hence bright, had no idea of basic scientific facts or principles.
Nowadays we're swinging more to the "Shakespeare was over-rated" side where the good jobs are for the smart people who can manipulate numbers in some manner.
I feel like income is a much better proxy for what we value in “intelligence”. Obviously Jeff Bezos isn’t necessarily smarter than Scott Aaronson but they’re definitely in the same league or else Bezos wouldn’t have been in the position he’s in today. This is perfectly captured in the “if you’re so smart why aren’t you so rich” quote.
So… isn’t it better to just ask for peoples incomes, stack rank them against people of their age and place of residence and call that “IQ”?
Yes, definitely. There are hundreds of thousands of people capable of composing music and singing as well as Taylor Swift. There are hundreds of people with a #1 song. There's only a handful of people with a successful career in music spanning for more than a decade.
I think there's a typo in your last paragraph - 1/30 people don't have an IQ above 128 (z-score of 0.9974). I think 1/30 is roughly the number for a score of 120 (z-score of 0.9772).
Why even bother with some IQ test of adults who are gainfully employed? Of what value is it other than perhaps for self gratification for some who see via internet test/survey that they score high on an IQ test. Me: Could care less as having multiple degrees and financial and career success I find it all a bit silly. Yes, for the very young it might be of help for seeing those with very very high IQ get the proper degree of educational nourishment.
I think the only IQ test that matters is supposedly literate adults being able to know the difference between "could care less " and "couldn't care less".
What life circumstances lead a cognitively normal person to get an IQ test? This seems completely foreign to me, like something I only hear about on the internet. The only real-life person I know who has gotten an IQ test is my older brother, who is has an intellectual disability.
I guess maybe for a job? I had to take the Wonderlic once for a job. But that's not exactly an IQ test, I don't think.
Many people wondering if they have ADHD or a learning disability go or are sent to a neuropsychologist for individual testing. Testing is also done if brain damage or brain disease is suspected.
Maybe you like this club, too, but for those of us who think Beautiful People is silly, do you think it’s also because we’re simply jealous? Or do you think it might be because creating highly exclusive social clubs that rigorously reject members based on a largely heritable trait is slightly off-putting?
I don’t see much difference between Mensa and Beautiful People. They both have every right to exist but it is a bit much to also expect the vast majority of us who are ugly and stupid to think they’re cool. And I can certainly appreciate why even many of the gorgeous and brilliant folks out there might not like such clubs.
There's lots of possibilities. I don't hate Mensa--I actually joined it for a bit and then never went to any meetings so my membership lapsed--but I do have ambient "lack of excitement about Mensa" that I think is related to the hate. My guess is almost all of it is secondhand.
Some guesses:
* Adverse selection. Smart people with friend groups don't need to go to generic meetings, but smart people without friend groups do.
* Directness. Smart people like to be around other smart people, and there are two ways to go about that: indirectly ("I'm a college professor and I hang around my colleagues") and directly ("I have a generic job but I'm a genius, and I would like to hang around other people who are also geniuses."), and a lot of people are down on directness.
* Implied lack of sociability to be able to cross intellectual gaps. (People might think the opinion "I don't want to be friends with people IQ 125 or lower" correlates with other things they dislike.)
* Sour grapes. About as many people probably thought they could get into Mensa and then turned out to have IQs in the 125-130 range as actually got into Mensa.
Because it's the 'High IQ underachiever club'. Most high IQ people distinguish themselves through the application of their intelligence to their work, not from the mere fact of having a high IQ.
> The LessWrong average was 1446, corresponding to IQ 140.
I think you're using the pre-1994 SAT correspondence, but most of the people responding to the survey probably took the SAT after 1994. As stated on that link, they made a change then that dramatically compressed the range; as a result, I don't think the SAT is that useful for assessing IQs above ~120.
Re. "the ClearerThinking survey used crowdworkers (eg Mechanical Turk) who should be totally average.": Why do you think that?
Mechanical turk workers have a lot of filters on them. First, they must have a computer and an Internet connection in their homes. In India, this is a big filter. They must be non-conformist enough to risk getting this weird new kind of job; that's another filter. Then they must apply for the job and pass. Then, once they're on the job, they compete for tasks, especially high-paying tasks. And your results might be enriched in mechanical turk workers who searched for that job because already knew about these surveys from being involved in LW or ClearerThinking.
As a child I tested 144 and my SAT was 1380. I can name poker hands and explain the infield fly rule but I know next to nothing about pineapples or how to build a table. People are too obsessed with these measurements, and I have a theory that that is holding back human progress.
I occurs to me that one way of establishing a norm could be another test a lot of people take: the ASVAB. It isn't hindered by the "smarter people take it more often" like the SAT. It probably is hindered a bit because it probably isn't taken by many people at the upper and lower bounds. The AFQT score from it is a literal percentile and the score most people remember.
As a data point , my AFQT was 99 and my SAT was 1400. According to the correlations listed above and the percentiles at iqcomparisonsite.com if AFQT is an accurate percentile, that apparently puts my IQ at somewhere between 135 and 137.
The first point you make, about regression, should apply equally in both directions. If SAT and IQ are correlated at .8, then using either one to predict the other should result in regression to the mean. So if IQ scores are higher than you should expect by regression from SAT scores, because they are at the same percentile, then by the same token, SAT score are higher than you should expect by regression from IQ scores.
More likely, both of these average reported scores are being dragged up by some third factor (which might be one or more of the other things you mention) rather than one of them leading and the other being dragged.
"and least likely of all to optimize their lies for getting IQ/SAT correspondences right."
That's comparing averages, not individual scores, so consistent with people getting the correspondence badly wrong but their errors roughly averaging out.
One possibility you didn't mention is people getting their IQ from a test done when they were children, with the score being literally mental age over physical age, not the norming of adult tests. The only IQ figure I have for myself was due to a fellow high school student copying the school's list of students and IQs and showing it around. I was at that school from kindergarten on, so the score could be me at ten and only weak evidence of my adult IQ.
The only time I’ve ever bothered to get my IQ officially tested was back when I was 18 - I was very interested in IQ and IQ tests, I practiced on all the online tests and similar games (anyone remember Brain Age on Nintendo DS?), and I prided myself on being “smart” a lot more than I do now, so I gave it all the preparation (nicotine!) and effort I could.
If I took the test today, I bet I’d score 10 points lower. This effect surely doesn’t account for a whole standard deviation, but I bet there’s some factor where people test their IQs when they’re at their “smartest”, narrowly construed. The idealized IQ test may be impossible to prepare for, and remain constant over a person’s life, but in real life that’s definitely not the case.
I don't really care. When I was well above fifty years old I got the result of an IQ test I did as a teenager and reported that here. After forty years of more or less moderate drinking, I should score significantly lower now.
Just so you know, internet iq tests can be terrible in the other direction. Once I took an obviously sarcastic IQ test on a humor site that asked a bunch of ridiculous questions and trick questions, then reported that my IQ was 7.
Gwern, I think Scott got the joke: my calling an essay on IQ, dumb. Besides, who am I for Scott to feel insulted by any opinion I offer. But to answer your critique of my comment, a few observations.
First. What I like about Scott’s articles is that he respects his audience and doesn’t dumb down his discussions. But his Substack is subscribed by numerous ordinary people like myself who find his analysis of ideas and issues cogent even if we can’t always follow his math. Unlike you I do not presume to know whether that was his intent when he created his blog, but I suspect that he knows a great deal about those who read his posts.
Second. My objection to his discussion is that he doesn’t relate his analysis to anything in the real world. The only thing he says is why he collects this information. If his post was directed only at specialists, he should have published it in an academic journal.
Finally. If there are counter arguments that undermine or refute my comments, educate me. I don’t have any problem with someone explaining why i am wrong.
Sebastian's corrections to the SAT are wrong in your context.
If I sampled a random person in the population and told you their SAT was +2std above average, then yes, Sebastian is right that this shouldn't correspond to IQ 130 due to the imperfect correlation.
But if I first fixed a secret target IQ level L (maybe 120 or 130, you don't know it, let's say I picked it from a prior that's uniform(110,140)), and then I sampled people who have IQ levels close to L (say, L plus some error) and told you the average SAT score, you should very much NOT be regressing that down.
The second situation is basically your situation: you want to estimate the average IQ of a fixed group like LessWrongers, and that fixed group is expected to have IQ distribution which is centered at L>>100. You should not be regressing the SAT scores like that!
That thread became a meme due to how bad the IQ questions are. If you believe IQ is real and important, read that thread and see if you change your mind :)
(I do believe IQ is real and important, just less important than IQcels think. Even I had to do a double-take at those questions, though. It reads like parody. Check it out.)
Speaking of people underestimating how dumb the average person is, there was this opinion piece on CNN ( https://www.cnn.com/2024/03/19/opinions/zone-of-interest-holocaust-movie-rutland/index.html ) complaining how the movie "The Zone of Interest" (a Holocaust film) was humanizing Nazis and ignoring the suffering of the Jews. That alone is deserving of mockery for blatantly missing the point of the film, but then there's this paragraph:
> “The Zone of Interest” is rather tedious as a film. It barely has a plot, and the conversations and daily routines are repetitious. Several scenes will leave viewers confused, such as the one where Höss finds a jawbone while fishing in the river and drags his kids out of the water. I would not have known what was happening except I had previously read in a review that there are supposedly human remains being dumped in the river.
<
...I still can't come to terms with the fact that a human being actually wrote those sentences. Like, first he had to be stupid enough to not realize the obvious implications of human remains being in the river in a fucking Holocaust movie, then he had to be stupid enough to actually admit that he didn't understand that obvious implication without realizing how absurdly stupid it made him look. And this isn't just some guy CNN found on the street, he's a COLLEGE PROFESSOR. (And according to the editor, "an expert in the politics of contemporary Russia".) ...All of this is legitimately making me want to kill myself. How the hell am I the same species as this person?
"An audience member saw connections to the “Don’t say gay” law in Florida. But the Holocaust happened to the Jews, at the hands of the Nazis and their collaborators, and its specificity should not be diluted into a general meditation on the banality of evil. "
" Several scenes will leave viewers confused, such as the one where Höss finds a jawbone while fishing in the river and drags his kids out of the water"
What the hell? Isn't that a *normal* reaction? You brought the kids down to the riverbank for a nice day out, they're swimming while you're fishing, and the next thing you're pulling bits of human out of the water? Of course you're going to go "Holy mackerel!" and tell the kids to get out and get dressed.
Would Mr. Reviewer just calmly keep on fishing in the way of "I wonder if I can get the full set of pieces?"
EDIT: Okay, being kind, maybe Mr. Reviewer means "it was not adequately explained why there would be a jawbone in the river". But they do sound like they wanted a scene where the Bad Nazis backed a dump truck full of corpses to the water and went "Achtung! We will evilly dispose of the remains of the Jewish victims we evilly murdered, bwa-ha-ha! Because we're Nazis and Nazis are evil!" as they tipped the bodies in.
The only formal IQ test results I have are from a test I took as a child. Such tests are notoriously unreliable, and that was a long time ago, but if that's what you ask for, that's what you're going to get. I assume many others are in the same boat.
I am sure someone mentioned this, but I couldn’t find it in the comments: in 1995 the College Board did a major recentering of the SAT which moved Verbal scores massively. A 730V (looking in mirror) before 1995 was equivalent to 800 after recentering. 780 went to 800. So any mapping to IQ must specify which scales is used.
While I concede that IQ measurements are mildly interesting for social research, I for one can not fathom making a fetish out of one's IQ score. Given that getting a measurement of IQ is actual work, much more involved than measuring one's height or weight, I would argue that in most cases where people set out to measure their IQ there is some degree of fetish involved.
In my social circles nobody has ever brought up their IQ. If anyone did, I think it would be met similarly to them mentioning them the length of their penis -- incredulity at someone going through all the troubles of measuring that and then having the poor taste to mention it.
Personally, I don't feel that my intelligence is a bottleneck in most endeavors I might undertake. Real life is not like D&D where you can just win your battles by throwing INT-based fireballs. Take it from me, in real life, having dumpstats like Conscientiousness, Drive, non-Depression (same as Drive?), Communication and so on really limits what you can achieve, more like a few extra points of IQ would likely help. (Not that I would know, nobody ever cast Fox's Cunning on me.)
Jeez, every man I know well enough to discuss such things has gone to the trouble of measuring the length and girth of his penis -- but yeah, few talk about their results. I agree with you about disclosing IQ score, if one somehow has managed to get one. It's poor taste. It's tacky. If the score is high, it's extremely braggy and rude to tell people the number. So disclosing IQ is sort of like disclosing your net financial worth. If the number is low or moderate, people feel that it's just weird to disclose it -- and if it's high it's braggy. It think some of people's recoil from a high value in either area is because everyone would like to feel as though the number is not that important, but privately feels like it is. So hearing it kind of ties people up in knots. But I agree with you that in most endeavors, if you a bright enough to understand the field, then success depends less on whatever IQ points you have above the minimum and more on how fascinated you are with the field, how inventive, conscientious, competitive, likable, persuasive you are.
Bryan Caplan's comment on X: "Very interesting piece on SAT --> IQ conversions. My own view is that the SAT *is* an IQ test. The fact that it only correlates at .85 with official IQ tests shows nothing, because that's roughly the correlation between different IQ tests, too."
As someone in the category where I've been administered exactly one respectable IQ test in my life and reported it on your past surveys, I'd suggest asking for "age when tested". I was tested as a child and there may be interesting effects with regression to the mean in adulthood, cognitive decline over decades, Flynn effect, etc., that you could examine if you're doing a large survey and ask for "age when tested" as well as current age. (Also I for one would state "WISC-R" if asked, but I don't know if asking for the test type would yield good data in general.)
And I suspect I'm in a common category. Probably nearly everyone who takes a respectable IQ test is being tested either by a school psychologist or by Mensa.
(Finally, I echo various people's point that "only the smartest people know the result of a respectable IQ test performed on them" is probably even stronger than "only the smartest people remember their SAT score". In normal life, there's no reason to administer an IQ test without some existing suspicion that the subject is either unusually smart or unusually dumb, but then dumb people rarely end up taking the ACX survey.)
There is no evidence, nor could there be any evidence, that IQ correlates to job performance if for no other reason than outcomes are determined by so many factors that such an evaluation could not be valid.
And what is the measure of performance? Certainly there are many occupations where a higher IQ is necessary. Anything involving the application of advance mathematics, or chemistry or physics. But performance? Just look at Boeing. When it was run by engineers and not by bean counters, doors weren’t falling off of planes. But I would bet that some of those bean counters have higher IQs than their engineer predecessors.
Selecting a doctor by their IQ would be more than reckless. I am sure you are aware that the best predictor of a successful outcome in surgery are the number and frequency of surgeries performed.
Consider an area I know something about. Expert witnesses. A prepared lawyer of lesser intelligence can destroy an expert of far greater intelligence in every field of expertise. Happens all the time. I’ve done it, and my wife can attest to the fact that I’m no genius.
How many truly smart people were taken in by Elizabeth Smart. In fact there is a real correlation between people of high intelligence and academic achievement falling victim to con artists. I know, I represent the con artists.
Show me the studies that support your contention and I will explain why they are flawed. Or maybe even made up as some very, very, very supposedly smart professors of medicine at the most elite institutions have been caught doing of late. So what is the measure of performance when people of high IQ achieve success by cheating. Pardon me if I hesitate to choose them over my University of Miami trained Internist.
> But I would bet that some of those bean counters have higher IQs than their engineer predecessors.
This would be unlikely. Engineering degrees are hard and business degrees considerably easier.
> Selecting a doctor by their IQ would be more than reckless. I am sure you are aware that the best predictor of a successful outcome in surgery are the number and frequency of surgeries performed.
Learning on the job is controlled for, regardless of IQ everybody has to do it. Nevertheless although you wouldn’t use an IQ test to select a doctor medical students have high IQs.
For what it's worth, TWO of the legit smartest people I have worked with were (by job description, in reality the way the filled their role was far wider) bean counters...
And it is quite rare that I feel clearly beaten in that dimension in real life.
That said, starting off your comment with "I refuse to believe that evidence could exist" makes it... not very motivating to show you evidence? Like, if you insist on being wrong, I'm not going to stop you, but I _am_ going to view you with a mixture of pity and scorn.
Looking back at my earlier comment I don’t see any place where I said, “I refuse to believe that evidence could exist.” I did say there could be no valid evidence of a correlation between IQ and performance because there are so many other variables involved. The question is, does the Hunter-Schmidt article justify your pity and scorn? [If somewhere else I said that I refuse to believe…that would be asinine and I would be deserving of far worse than mere scorn]
The H-S study demonstrates a relationship between GMA and success. Success as indicated by positions attained, money earned, performance reviews. BUT, it also demonstrates a relationship between success and consciousness. [If you will permit me to subscribe dedication and hard work for consciousness, I think I can make my argument clearer]
Now, it seems to my lay mind, that we should be able to take IQ tests and predict which 10 year olds will have the highest incomes in a random a group of 1000 10 year olds. But it won’t tell us, which ones will actually be better at their jobs, because that is a function of their dedication to their job and the effort they put into their work.
So I concede that every kind of job requires a certain minimum IQ. But within that range, should a board of directors pick the candidate with the highest IQ to be the CEO. Doesn’t a host of other factors go into which person in the range will be the most successful at their task.
How did you intend me to interpret "nor could there be any evidence"?
On the main point: I think you have swapped out the claim "IQ correlates to job performance" with the claim "IQ is the same as job performance." The relationship they demonstrate between GMA and success is correlation. Does that mean the smartest person is the person with the best job performance? Only if the correlation is 1, and we know it's less than 1.
Moreover, variable A being correlated with variable B does not stop variable B from being correlated with variable C. If you're actually hiring people, you want to compare candidates based on as much information as you can obtai, which will include intelligence and conscientiousness and many other smaller features.
When I took a Mensa IQ test they told us two interesting things:
1. The test is most accurate around the 130 mark. It's what they're interested in and what it's optimized for. They said to not take low scores too seriously. (And it doesn't test above 145)
2. About 50% of test takers score above the cutoff.
I previously figured there was a selection effect, but I didn't expect it to be so large.
```even official IQ tests are gobbledygook over 135.```
Of course they are. It would be outright shocking if there were commonly available (or even uncommonly available) tests which could distinguish IQ at that range.
Suppose you have a group of 50 people. 30 of them have IQs of 100. 15 have IQs of 115. 4 have IQs of 130. And exactly one has an IQ of 145.
Your job is to sort them all into their correct group. All of them are available to you at all times, and will takes as many tests as you want, to the best of their ability, under ideal conditions every time.
Outside of this group you have access to arbitrarily many people of IQs 100, 115, etc. whose IQs are known to you, so you can norm the tests to them.
The best way to write the test is to have three groups of questions. Questions which an IQ 115+ reliably can answer, but IQ <115 reliably can't, questions that IQ 130+ can answer, and you get the idea.
But herein lies the problem. If you're not at least IQ 145, you can't write, or even *conceive* of a question that an IQ 145 person can reliably answer, but a less intelligent person can't. If IQ means anything at all, it means that.
So if you're an IQ 130 person, the best you can do is sort out the 100s, and 115s, and then everyone else is in the 130+ bucket.
There are ways to figure out through interaction that someone is smarter than you. And you could even figure out which of two people, both of whom are smarter than you, is the smarter of the pair.
But you can't write test questions that will distinguish the two, even under ideal circumstances.
You (the 130 person in this example) could write questions at the upper limit of your own ability and then administer a timed test with the time short enough that you yourself couldn't complete the questions in the time limit. Then you can rank the people smarter than you based on how many of the questions they complete within the time limit, on the assumption that if a question is really hard for you it's probably relatively easy for them (just as a question that really taxes an IQ 100 person would be relatively easy for you and you could answer it more quickly than they could).
There are all kinds of clever tricks you can do, but empirically, they haven't borne fruit, and I have a guess as to why.
Every trick you use adds noise, inversely proportional to how good the trick is.
For example, timed tests can't discriminate between people who would get the right answer given 15% more time, and people would who wouldn't get the right answer given arbitrarily more time, but I think we can agree the former is 'smarter'.
So we would expect most tests designed by ~130 IQ people to distinguish people smarter than themselves to have low reliability. And in practice, that's exactly what we see.
There's also the question of how many ~145 IQ people there are actually are, and what that means.
A typical 130 IQ person can do everything a typical 115 IQ person can do cognitively, and a fair bit more besides. But I strongly suspect that the typical 145 IQ person has deficits, and those deficits are likely to vary widely.
And there is no such thing as a typical 160 IQ person.
I think it's fairly uncontroversial to say that there is more variance in the 95th percentile IQ cohort, in terms of brain function, than in the 65th percentile cohort.
The mechanisms that produce this are fairly easy to figure out, if you take it as a given that the evolutionary optimal IQ is the average.
Has anyone mentioned the possibility that SAT scores may stop correlating with IQ above 135 not because the IQ scores are gobbledegook, but because teenagers with IQ above 135 are using their brains pretty differently than their peers, leading to all sorts of confounding academic and social issues, and this might lead to a wider range of SAT outcomes, including testing worse than a peer with an IQ of 125?
"It looks like up to about 140, self-reported IQ and normed IQ rise together, and then the relationship breaks down. Sure enough, looking at the subset of self-reported IQ scores below 140, the correlation with tested IQ rises to .6, and looking at the subset above 140, the correlation is nonsignificant at -0.02. I don’t want to assert that the breakpoint is exactly 140, but I do think the test stops working somewhere in the 130 - 140 range."
There seems to be an inferential step here saying that in general, if you reapply the correlation calculation after taking a cutoff, a decrease is indicative of the test failing to measure something past that point. This is not in general true, as even in the case of applying a cutoff to randomly generated data for at some fixed correlation, this will reduce the correlation coefficient of the remaining data (on average). This is especially true when the cutoff removes more of the original datapoints or the data is sparse past the cutoff, which just from eyeballing the graph, this seems to do to a great deal. It may be the case that the test fails around this area, but this calculation provides little evidence in that direction.
Note to self, add an additional 10-15 points when giving my IQ to Scott. And if he leaves a comment saying he will subtract an additional 15 points in my case, I will need to add 15 more. In fact he may read this and say nothing, so it's best to add another 15 just in case...
People today are such dummies. They can't even start fires in the wilderness from scratch, navigate by the stars, grow their own food, or memorize and recite epic poetry like Beowulf or The Odyssey. All this newfangled technology, like this "writing" thing, is making morons of us all.
What's wrong with assuming averages of 130s or even 140s? Have you ever been on the forums of Mensa, Triple nine, or any other high-IQ socities? If you read the kind of thoughts people have on there, you will probably be disappointed. The comment section here is way better. I don't think we're overestimaing IQ scores, I think we're overestimating how intelligent people with, say, 130 IQ actually are.
You're too humble. How many people have IQs one or two standard deviations above you? If this is the middle of the pyramid, then where the hell is the top? Can anyone name a single book written by somebody "truly intelligent"?
It's pretty common practice to take the SAT multiple times. In fact, not only that, but colleges allow you to submit your best score from each section, even if those sections are across multiple attempts.
Let's say Bob took the SAT twice. On his first attempt he got verbal 600 and math 700. On his second attempt he got verbal 550 and math 720. He would apply to schools with a 600/720. What score do you think he's likely to give when he knows someone is using it to estimate his IQ?
I actually think the percentile-matching thing is potentially reasonable!
On the one hand, if you trust the SAT scores, think your readers are otherwise typical, and want an estimate of the IQ scores, the regression thing is correct. But ACX readers are probably not filtered strongly by SAT, with no additional selection for IQ.
On the other hand, if you want to test the hypothesis "ACX readers are as high-IQ as they claim and otherwise typical", in a sort of Bayesian way, by computing implied SAT scores and comparing to reported scores, you should actually do the regression the other way, which (I think) produces implied SAT scores *lower* than reported, and makes the IQ reports seem easily believable or even an underestimate.
And if you want to test the hypothesis "ACX readers are selected for both high IQ and high SAT", with similar amount of selection for both, matching percentiles is at least halfway reasonable. (One version of this hypothesis: readers are selected for high g, and IQ and SAT are two different noisy measures of g with similar noise. But in that case, I'd guess SAT is noisier and we should assume ACX readers are higher-percentile on IQ.)
I'm not really sure what you should actually do here, though, except note that "ACX readers are selected for some factor-of-intelligence which is independently correlated with both IQ and SAT" is consistent with the numbers.
(At the very least, if a population reports being comparably high-percentile on two different tests, you probably shouldn't immediately assume they're lying about one of them!)
On the gripping hand: the main use case for that website -- "I know my SAT score, how high is my probable IQ?" -- has exactly the flaw you describe, with exactly the solution you describe.
[disclaimer: haven't read the rest of the post yet]
In my Big Kink Survey I ask if people have ever had an officially-administered IQ test; if they say yes, they get a question where I ask what their score was. The average reported IQ for this is around 128. I forget my sample for this exactly (would have to load up the giant data again which takes forever) but iirc it was around 30k people.
The people taking this are like... US-based young very-online people, mostly coming from tiktok. Not intellectual culture at all.
"Third, it has a ceiling effect: you can’t get more education than being a PhD, so it will underestimate the intelligence of a sample where a significant percent of subjects have PhDs or other advanced degrees."
This is wrong. Professional degree holders tend to be smarter than holders of adjacent PhDs. It's a bit complicated because physics PhDs are obviously smarter than lawyers, but MD>hardsci PhD>Lawyer>humanities/socsci PhD. So you could education norm to higher levels by separating out professional degrees and/or types of PhD.
None of the venerable tests being accurate over 135 sounds like something that should be noticeable outside this community. Is this an accepted fact of IQ research, or if not how does that match with this? Is IQ just not normally distributed at the tails or what is going on?
Is there some other test that's used above 130?
Well for one: The measurement error of IQ tests greatly increases in the high ranges. Erik Hoel describes it well in one of his posts:
https://www.theintrinsicperspective.com/p/your-iq-isnt-160-no-ones-is
I assume respondents were not asked how many tests have they taken. If a large proportion of respondents have taken more than 1 test, it seems fair to assume they are not reporting their average score but their highest score.
Snap. See my post below.
IIRC I reported my average score in 2015, but that's only weak evidence about the average LWer.
I love that Stephen Hawking quote:
> When asked in a 2004 interview with The New York Times what his IQ is, Hawking gave a curt reply: "I have no idea. People who boast about their IQ are losers."
Literally, utterly, totally irrelevant.
:-)
That point spread chart was illuminating. I've taken four IQ tests. When self-reporting my IQ I average my three highest. I don't like to mention my Mensa results which were significantly lower. ;-)
IQ testing is actually a pretty hard problem at the top (and bottom) ends. If you want to fit a nice bell curve, you better have your test cover a large part of the range, i.e., going down all the way to 100.
But if you're answering only 50-100 questions, someone with an IQ of 135+ is going to get 99-100% correct answers on such a small sample size and with such a small range of question difficulties.
So, tests given to high IQ people that claim to be accurate up to 160 or higher necessarily need to do much fancier (= error-prone) statistics to get significance all the way up there.
I thought speed was sometimes used as a technique to help tease apart scores at the top end.
Yes, if you just give everyone 50 questions then everyone above a certain level will just get 100%. But if you give a timed test and supply more questions than you'd expect even the best candidates to get through in the time, and Alice gets through 100 questions and gets 99% of them right, and Bob gets through 50 questions and gets 99% of them right, then Alice's IQ is higher.
IIRC, the in-person Mensa test I took used this technique.
I am anecdotally skeptical of this, as someone who tests WAY better when speed is taken into account but less well when speed is not the fulcrum of the test. Anecdotally my processing speed is not perfectly correlated with my overall ability to get hard problems correct.
I also dislike the speed method. Although I haven't been tested properly, I am temperamentally the kind of person who likes to take my time and double check things to make sure I get them right. I don't like to hurry, and I certainly don't like having to constantly evaluate whether I should pass on a difficult question in hopes of solving several subsequent questions in the time I would have spent figuring it out.
Processing speed is an interesting aspect of mental capacity, but I don't know if it's appropriate to roll it up into IQ or not.
Wouldn't a better method be to use the first 10 questions to determine the person's IQ quartile, then administer a test tuned for that quartile?
"Processing speed is an interesting aspect of mental capacity, but I don't know if it's appropriate to roll it up into IQ or not."
While we're on the subject of "IQ tests are not administered in the ways people intuit them as being administered"...yes, processing speed is a whole section on the Wechsler tests. One of the big four factors that goes equally into determining someone's full-scale IQ. It and working memory are the "least g-loaded", so people with very high scores on other factors frequently regress to the mean on them, lowering their FSIQs noticeably. (This is one of numerous reasons the traditionally quoted survey-IQs are implausibly high -- even if we select strongly on both verbal and nonverbal ability, and are genuinely doing that rather than primarily selecting on intellectual *interest*, enough people with super-high "general ability" scores will regress to the mean on "cognitive proficiency" to get not-astronomical FSIQs. I can't see a good argument that we're selecting for high CPI separately to high GAI.)
Oh, that’s fascinating. I had no idea. In my own subjective experience my cognitive processing (including working memory, if that makes sense to throw in there) are more SDs above the norm than than my general ability is. Is that a rarity? How do you tease out the two? I’ve never taken a true IQ test, so I don’t have a sense of what that looks like, and I didn’t follow your last sentence.
If you have taken enough IQ tests to compare their relative merits, they don't work on you anymore, because it's possible to learn how to take IQ tests over time.
Depends. The standard IQ tests have very large practice effects. The SAT is, or at least was, specifically designed to have a small practice effect, and will still work.
I think Raven's matrices is traditionally a timed subtest.
Digit recall span probably helps some in separating people at the high end, too.
Unfortunately, both of these are fairly trainable tests - they're probably quite reliable if you're testing 10-12 year olds to help decide whether they should qualify for the gifted program(*), but if you're dealing with adults who want to get a high score and are willing to do some preparation, they don't hold up so well.
(*) and of course even here we can imagine that parents might train their kids to help them get in, given enough knowledge. I suppose the kids themselves might even decide to do some online research, but if a 6th grader decides to do mnemonic exercise to improve their digit span recall, on their own initiative, it's probably a good idea to let them in the program regardless....
Yeah, definitely based on the couple online tests that were Ravens Matrices linked or adjacent, if I had someone to explain to me the 'trick' of "now in this question, what they want is...", then I would do better. You definitely could 'train' for some of these. So doing it for the first and only time is a good measure as to genuine smarts to be able to figure out cold what the problem is, but doing it any more than that, and you're training on how to answer, not how to figure it out from first principles.
I believe Raven's in particular is known for very large practice effects.
It's an open question to what degree this might be unintentionally driven by psychologists enjoying the feeling that they can answer the questions. I'd put better odds on it mostly being driven by laziness and the low stakes associated with IQ tests that are called "IQ tests".
> Alice gets through 100 questions and gets 99% of them right, and Bob gets through 50 questions and gets 99% of them right, then Alice's IQ is higher.
Or Alice prepared for the test and Bob didn't.
When I was younger, and I was taking tests, I made the foolish decision that I wanted to know my actual cognitive ability, so on principle I did not prepare for such tests. It would be "cheating." I was Bob.
I still tested top 1% on standardised nationalised tests, but not well enough to qualify for all the perks that come with being 1/1000 instead of 1/100.
Now, many years later, I'm kicking myself. I should have done anything to inflate those scores to the max. So many opportunities lost. Good on Alice and her family and peer group. She'll do well in life.
But meanwhile, the stats have us Bobs as lower-IQ. So be careful trying to draw conclusions there.
Your argument "proves too much". It works just as well against any form of IQ testing, whether timed or not (you can drill yourself on vocabulary, practice the matrices, etc, and some people do and some don't). It's not an argument for doing untimed rather than timed tests.
I have good news for you. I prove little, not even much, let alone "too much."
It is possible my mindset is a minority view, but I suspect that variations of it are common enough to show up in the data if accounted for.
In any case, it's just a hypothesis, and you can throw away whatever conclusions you might draw from it except that "Philo Vivero was a contra datapoint to this analysis."
The fallacy of "proving too much" isn't anything to do with "proof" in a mathematical sense or a forensic sense. It just means you're making an argument that would apply to a larger class of things than you intend.
For example, if you said "I don't use AI tools because it's imperative that my work be 100% error-free", that would "prove too much" because human work isn't 100% error-free so your argument would inadvertently rule out human work as well as AI work. (Of course there may be other good arguments against using AI, but I'm just saying that wouldn't be one of them.)
Here, you're presenting your objections as if they're objections to *timed* IQ tests specifically, but you haven't given any indication of why they wouldn't also apply to all IQ tests, or indeed all tests of any kind.
Perks?
> I still tested top 1% on standardised nationalised tests, but not well enough to qualify for all the perks that come with being 1/1000 instead of 1/100.
In my experience, no such perks exist. What are you thinking of?
Well, if you score high enough on the PSAT you qualify for a National Merit Scholarship. Probably there are other scholarships that are linked to standardized test scores.
No, you don't. You qualify to apply for a National Merit Scholarship. You don't qualify for the scholarship itself.
The only thing your PSAT score will get you is "semifinalist" status. That's worth nothing; there are two levels above that, of which the top one ("scholar") gets scholarships.
> i.e., going down all the way to 100.
...while not a serious issue in the current context, it's actually also important to have the test give a reasonable distribution all the way down to 65 or so as well...
> So, tests given to high IQ people that claim to be accurate up to 160 or higher necessarily need to do much fancier (= error-prone) statistics to get significance all the way up there.
This is not correct; they can use the same statistics. They need their questions to be more difficult.
Well, they need enough of their questions to be more difficult to detect differences in fractions of percentages at the top end, while still having enough of a difficulty gradient to detect the low end (i.e. so that it‘a not the case that nobody below 100 does no better than chance.) unless you’re talking about secondary tests of people who tested above a threshold in the first test.
> Well, they need enough of their questions to be more difficult to detect differences in fractions of percentages at the top end, while still having enough of a difficulty gradient to detect the low end
No, they don't. It's a test given to high IQ people; there's no need to have good resolution at the low end.
There's no theoretical obstacle to combining both purposes into one test, but it will, as a stylized fact, double the size of the test, since questions at the low end are totally uninformative at the high end and vice versa. Most tests won't do this because testees tend to give up partway through long tests. But again, that's not a problem with the statistics.
This means that tests that are intended for "general use" have poor resolution at the high end, and at the low end, because they're testing people in the middle and adding resolution outside the middle lowers accuracy in the places where they care about it. But "tests given to high IQ people that claim to be accurate up to 160 or higher" have no such problem. There is no reason for them to spend any effort on low-end resolution.
The WAIS IV goes up pretty high. I think I saw something in the 180 range at the bottom of the score->IQ table I wasn’t supposed to see.
It is, as far as I can tell, an accepted fact of IQ research. I've encountered a few different figures for when IQ tests stop working reliably. For regular tests, anything above 130, basically. High-IQ-specific tests might, maybe, be able to get to 145.
But, 130 is kind of the ceiling.
For adults. For childhood IQ tests, which are testing something subtly different, the regular tests hit 180, and the high-IQ alternatives, I gather, hit 200, before the tests just get too unreliable. (But - and it's a huge but - this maximum decreases with age, because the number reflects something like precociousness, and you just have less and less room to be precocious the older you are.)
The problem, I gather, is that it's basically impossible to craft a set of problems such that IQ 145 people will reliably be able to answer them, but which IQ 130 people reliably won't.
Interesting
So as they get older, smart people become more and more postcocious.
It would be silly for IQ to not be normally distributed at the tails since it is a completely artificial unit. You can scale it however you want making the tails whatever you want. Normal is the one, "normal" (non-abnormal) choice.
I think the range restriction just refers to the idea that the farther out on the tails you get the less likely it’s measuring something and not just chance?
That seems likely. I was merely pointing out that if an IQ distribution has fat tails, that is entirely the fault of the people designing the test (and associated grading curve).
I suspect that the high emphasis that IQ tests place on speed and on short-term memory necessarily corrupts scores over 130. There are bound to be lots of dumb people who have great STM or short reaction times, and lots of smart people with poor STM or slow reaction time. These components are probably in the exam only because they correlate strongly with what we might call "actual smarts" in large samples. But I think they aren't what IQ tests should be looking for.
Yeah I was wondering if it isn't at least in part a change in the honesty level- when you get high enough, the number of legitimate scores gets smaller, and there's probably a subset of liars who just give a super high number rather than just adjusting upwards a bit.
Another issue is that the official in-person Mensa test is apparently not a standard IQ test with standard deviation of 15, and most people taking it don't realise this.
This was discussed back on SSC at https://slatestarcodex.com/2018/01/14/ot93-giant-threadwood/#comment-588588 and https://slatestarcodex.com/2018/11/21/impending-survey-discussion-thread/#comment-693849.
My Mensa certificate just says on it "This gives a true IQ" and doesn't indicate which test (e.g. Wechsler, Stanford-Binet, Cattell, etc) it was. So for the first few years of Scott's survey I just wrote the number from that certificate, which does sound pretty high. I've since inferred from the linked SSC discussion that it was probably Cattell and that the corresponding Wechsler and S-B numbers are quite a bit lower. But given that the survey rubric doesn't specify which test or indicate that some so-called "IQ tests" (including the one used by Mensa) use a differently-calibrated scale, and given that even the Mensa certificate itself doesn't indicate which test it used, I am not surprised that some commenters are reporting too-high scores, in good faith.
I took the Mensa test about ten years ago, and I remember being frustrated that they provided a [conversion table][1] from the Mensa test score to a percentile, and from percentile to IQ, but only at whole-percentile resolution -- making it near-useless within their target audience.
(I joined in the hope of finding IRL peers to talk to. It didn't work very well; it turns out that similar intelligence doesn't imply any particular common interests. Didn't help that most of the membership were twice my age.)
[1]: https://web.archive.org/web/20160329100009/http://www.us.mensa.org/join/testing/scoreevaluation/testscoreconversion/
What exactly _is_ the Stanford-Binet test anyway? Or any other specific test?
Surely it can't be just a single test with a specific set of questions, or else you could take it twice and ace it.
But it also can't be a totally general formula for generating questions, because such a thing is pretty unimaginable and each question wouldn't be properly calibrated anyway.
So maybe it's a large collection of questions from which you choose a subset to give as an IQ test? But then surely all these questions have escaped out into the wild right now and are being carefully drilled into kids at cram schools?
Some thoughts:
The people on the lowest end of the IQ distribution probably can't work as click workers due to being illiterate, unable to follow instructions, or similar reasons. So you'd expect the average to be above 100 in that sample.
People generally only take IQ tests if they expect the result will either be very low, or very high. The "very low" group will be underrepresented in the sample as noted above. So we're left with the people who took an IQ test because they expected to get a high number (for gifted classes, skipping a grade, MENSA etc). The people whose IQ is actually ~100 haven't taken a test.
As for the SAT, people probably remember the exact score if they tried to get into some college program where it mattered, or if it was flatteringly high. If your score was really mid, there's no reason to recall it.
Tl;dr very low IQ people don't take online surveys, and middling IQ people don't know their score, leaving the high scoring people to sample.
I had the same thought. "Why do you assume that workers who are making money doing mental work have an average intelligence?"
It's like expecting people who carry boxes for a living to be of average strength, rather than being somewhat stronger (because of practice if nothing else, but also if you're not good at it and don't get better they will fire you).
"The people on the lowest end of the IQ distribution probably can't work as click workers due to being illiterate, unable to follow instructions, or similar reasons. So you'd expect the average to be above 100 in that sample."
This is true, but I'd also expect it to be true to an extent on the high end as well (for different reasons).
The US military, for example, has an ASVAB cutoff that essentially disqualifies the bottom 30% of the population from joining. This should push the IQ of the average military person above 100. But then I'd also stereo-typically expect that folks with Ivy League potential are less likely to join the military so that would push the average down. How much I can't say.
But I expect to see this sort of "exclude some percentage of the bottom" and "exclude a different percentage of the top" to show up a lot for everything except for the population as a whole. Doing something quantitative with it is tough.
I saw something *really* interesting recently. I don't remember where I first saw it, but search-engine-ing for the same table, it's Table A-4 here:
https://www.cna.org/pop-rep/2017/appendixa/appendixa.pdf
ASVAB distributions of everyone who applied to the US military in fiscal year 2017, broken down by branch and with gender %, and with the population of the original norms (note: everyone in Cat V in the original norms seems to be misclassified as Unknown). For people unfamiliar with the ASVAB's structure: I is the highest, V is the lowest (and corresponds to lower borderline intellectual disability or lower, roughly), the IIIA/IIIB breakpoint is the 50th percentile, and IV or below (<30th percentile) is as noted basically a disqualification (some people in Cat IV can join, I think).
The first thing you notice is that women are WAY overrepresented in the lower categories. This isn't seen in the general population -- it's something conditioning on "women interested in joining the military".
"The first thing you notice is that women are WAY overrepresented in the lower categories. "
Can you point to a specific table? I'm looking at Table A-4 in the TOTAL DoD section and see 3.12% and 26.56% of the applicants in categories I and II. The female civilian population as a whole (right below) is 7.6% and 26.35%. These are close.
Also, maybe not relevant but this data is for "Active Component Enlistment" so probably excludes commissioned officers.
"Table A-4. Applicants 1 for Active Component Enlistment, FY17: by Armed Forces Qualification Test (AFQT) Category, Service, and Gender with Civilian Comparison Group". I was noticing the "percent females" metric -- that the lowest ASVAB categories consistently had a much higher proportion of women than the highest, though the exact distribution varied by branch. For the civilian norms, the percentages of women are obviously much higher in general, but the variance is more like Greater Male Variance in a normal population than the extreme distributions in the applicants.
Good thoughts
> or just subtract 10-15 points from whatever you give me.
Ok, great, I'll just add 10-15 points to what I report next time around.
This seems to be what happened to HS grades over the last 25-30 years.
Colleges: HS grades are a factor in admissions (and one that is someone amenable to lobbying)
Schools & parents: "Moar points than 4.0 for tougher classes!" and 4.0 -> 5.0 -> 6.0
College Admissions: ...and we're deducting those BS points.
My wife did one cycle in undergrad admissions at Dartmouth while I was at the end of grad school, to "understand the alumni better by understanding who and how we admit students." Her actual job then: raising money from alums. Normalization of grades and scores within Admissions: oh, yes.
I mean, admissions counselors are super transparent about this to students. But even going from the 4.0 to the 5.0 provides a numerical tool for weighting the difficulty of one’s class load, which is important and something that merely subtracting the points wouldn’t capture.
My daughter has had to explain moving from a rural school with a 4.0 max and few honors classes to a high-quality suburban school with a 5.0 scale and ALL the honors programs and the admissions people are like, “Yeah, we gotchoo, just drop a note in the additional info section.”
Best of luck to your daughter!
I’ve tried and failed repeatedly over 10 years to convince my now-16yo daughter that undergrad is an expensive country club that offers a vanity-metric credential, and that she can learn more and build skills faster operating outside the system (like the route a Thiel Fellow would take). She absorbed the perspective, but shushed me repeatedly with “fine, I get it, but damnit I’m going to be a doctor and there’s no hack I can find to circumvent the medical establishment gatekeepers. That vain credential is a pre-req to get through the first set of gates to heal people, so I gotta do that.”
It *is* possible to hack undergrad degrees...
https://degreeforum.miraheze.org/
People have gotten into law and grad school doing this. Not sure about med school.
The less extreme method is to study for and take a bunch of AP tests in high school, finish the first half of your degree at a community college, and transfer in to a four-year college as a junior.
This is a bit faster and a lot cheaper than spending all four years at a regular university, while still being a "regular" degree that should work as well as any other from the same four-year school as a prereq for a graduate program.
Sounds to me like she's right.
I thought of this. Once I add 15 points to my 185 IQ the number will be too 'round' and I'll be deleted from the set.
I got about 138 in an IQ score 20 years ago - pretty official as it was in a company and I was locked in a room. That’s my highest result. I haven’t tested since and I’m probably not going to. So I’m probably reporting my highest ever result there (well I know I am but I’m not sure what the others were, just that they were lower).
Anyway that’s what I would answer if asked, “in my last test I got 138” - which is true but almost certainly high balling it.
The last IQ test I took for which I received a score was in about the 3rd grade, when I scored (I think?) 133. I have no idea what test it was, or how much it ought to change, if any, over another 30-50 years.
The last IQ test I took I passed, as after answering something like 10 questions on an internet test, I declined to provide an email address to get my results.
what would it even mean to "fail" an IQ test?
That last bit about 1/30 people finding internet statistics essays interesting is a bit of a reach, I think. Maybe only 3% of people are into that, but they aren't necessarily going to be the top 3%.
I can believe there's a pretty strong selection effect, but so strong that somebody with 114 IQ (like 85th percentile) is just as likely in the sample as somebody with 140+ IQ (like 99.9th percentile) is pretty strong.
I feel, less than 3% are into that (unless it is Scott writing), but those who are: pretty sure there is near none not in the top 5%. (Also: A text about statistics will not appeal to the lower 90-97% of math-IQ - otoh: an ACX post will appeal to those high on verbal IQ: Scoring high on both makes for a high IQ overall, i.e. potential ACX-reader.) Math-IQ of 114 and "finding internet statistics essays interesting": I doubt it. Strongly.
The correlation between "any intellectual curiosity/trait-intellectualism measure we've ever managed to make" and IQ is pretty weak, and appears to only exist for elements of IQ tests that are more classically amenable to "being learned" (e.g. vocab, general knowledge). Whether the crystallized/fluid distinction is real is another thing ("fluid IQ" is the subset most impacted by the Flynn effect, so it's implausible it's *not* strongly learned), but at the very least, you don't land in "intellectual curiosity/interest/trait-intellectualism and ability are basically discussable together". I'd say most forms of recreational-essay-reading have a higher interest than ability threshold.
Interesting. Have a link? (google did not help). So, even normies and dumbies can be curious, one assumes. I'd agree: most forms of recreational-essay-reading have a higher interest than ability threshold. That is why the NYT has many more readers than Scott. ;) (Not the only reason, I admit. But is reading ACX/SSC as "recreational" as NYT/Hanania/CoHo/TheSun? Maybe with IQ 135 it is.)
I think you might be underestimating the degree to which you only ever see the top part of the IQ distribution.
I find https://link.springer.com/article/10.3758/s13428-012-0307-9/tables/1 pretty illuminating. It finds that eg only 3% of college students know that Budapest is the capital of Hungary, or that Euclid is the father of geometry. If you mostly hang out with people who would be able to answer those kinds of questions, you're only ever seeing the top 3% of general knowledge (not the same as the top 3% of IQ, but hopefully this serves as an intuition pump).
Top 3% of college students, in 2012. Sounds more like top 1% of general population.
I'm amazed that so many people correctly answered "WHAT IS THE TERM FOR HITTING A VOLLEYBALL DOWN HARD INTO THE OPPONENT’S COURT?" How did 78.4% of respondents know the answer to that? Do Kent State and Colorado State have massively popular volleyball teams?
They might have had mandatory volleyball practice in middle- and high-school P.E. classes.
Indeed. In German it is "Schmetterball" - also in (table-)Tennis.
FWIW, people who play volleyball nowadays (at least in my neck of the woods) will look at you funny if you say "spike." They just say "hit."
Well, I can tell you that I guessed "spike" as the answer to that question, and KM's comment appears to indicate that that's correct. I have never played volleyball outside of a school PE class, and not much then. I have never _watched_ volleyball at all.
One thing making this easy to guess is that my vocabulary of "special volleyball terminology" consists of just the one word.
No offense to you, but this is the most internet-intelligencia-bubble comment in this whole post. Expressing surprise that the capital of Hungary or the name of the Ancient Greek founder of a math discipline are not common knowledge, but that a common sports term is.
Once again, nothing against you personally, I think everyone on this site wildly underestimates how little the average person knows about... everything scholarly (https://xkcd.com/2501/), and how *much* they know about sports.
I'm not comparing the volleyball question to Budapest or Euler, I'm comparing it to "WHAT IS THE NAME FOR A MEDICAL DOCTOR WHO SPECIALIZES IN CUTTING THE BODY? (SURGEON)", which only 74.8% answered correctly. It's only a little surprising that 3% of people know their obscure European capitals, but it's very surprising that more people know volleyball terminology than know what a surgeon does.
I also don't think this is an internet-intelligencia-bubble thing, I think it's a me-being-a-different-demographic thing. Maybe Americans all know the correct volleyball terms because they play it so much. Or maybe they know it because the participants were recruited from introductory psychology courses and they played volleyball more recently.
I have an intuition that the number of people who can correctly answer "what does a surgeon do?" would be larger than the number of people who can correctly answer "what is the name for a medical doctor who specializes in cutting the body?"
For what it's worth, I was given basic instruction on 3 kinds of volleyball hits (bump, set, spike) in PE class in...elementary school, I think? Possibly middle school. (And it was a general public school, not something specialized.) The whole class played volleyball for perhaps a week or two and then moved on to some other activity.
I agree this knowledge seems more obscure than "what's a surgeon?" but I'm not surprised that it's common.
That's a pretty weirdly phrased question. I kind of did a double take on that one, trying to figure out if they could mean something more specific or technical than surgeon. I mean surgery isn't just cutting, it's also changing and reassembling stuff afterwards.
Other weird ones: nobody at all knew who wrote The Brothers Karamazov?
But 40% of students know that Marlboro was the first brand of cigarette to have a flip-top box? Why do they know this?
I wonder what percentage would have guessed "Malboro" as the answer to any question of the form "what was the first brand of cigarette to do X?"
At this point I wonder what percentage of people know what "Marlboro" is. (The actual test will be multiple-choice, though.)
If the test were multiple-choice, I wouldn't expect any questions to have success rates as low as 3%
I got both of those purely based on reference class - "name a cigarette brand" and "name a Russian novelist".
I think know who wrote it - the same guy that wrote Crime and Punishment - I just can't spell his name. :P
In a comment/explanation for one of the questions in his book "Coming Apart ", Charles Murray wrote this about underestimating how little of the bottom half of the IQ distribution smart people interact with:
"I use this question ['Have you ever had a close friend who could seldom get better than Cs in high school even if he or she tried hard?'] as a way of getting at the question I would like to ask, 'Have you ever had a close friend who would have scored below the national average on an IQ test?' I can’t ask that question, because readers who grew up in an upper-middle-class neighborhood or went to school with the children of the upper-middle class have no way of knowing what average means. The empirical case for that statement is given in detail elsewhere, but it may be summarized quickly. The typical mean IQ for students in schools that the children of the upper-middle class attend is around 115, compared to the national mean of 100. In such a school, almost all of the below-average students, the ones you thought of as the school’s dummies, actually were above the national average. Even if the students were arranged in a normal distribution around a mean of 115, only 11 percent of the students could be expected to have IQs under 100. But they probably weren’t normally distributed, especially at a private school that uses a floor of academic ability in its admission decisions. So if you went to upper-middle-class schools and think you had a good friend who was below the national IQ mean, and are right, it had to have been one of the students who was at the absolute bottom of academic ability."
Note that Murray may be using a value for IQ standard deviation a bit less than 15.
"I think you might be underestimating the degree to which you only ever see the top part of the IQ distribution."
Yes, I think lots of people, even smart ones, are very bad at understanding what "average" is. I teach at a private high school. I'd say the average ACT score of our graduates is probably somewhere around 26-27. I was teaching one of my sophomore honors classes a few days ago and someone said, "I'm stupid." They're kids, of course, so it's less surprising that they're in a bubble, but I had to point out that all of the kids in that class are well above average. They have to get into the school, get into honors classes, not flunk out of the school, and keep their grades up to stay in honors classes. In their social lives they basically never interact with a kid from one of our city's many, many failing public schools. They have no idea what "average" (outside of their bubble) really is. And even my colleagues who will complain about a dumb student are usually complaining about some one who ends up with a 19 or 20 on the ACT...in other words, basically average intelligence. (Admittedly, we're probably doing a better job of test prep with our students than a "bad" school, so someone with slightly below average intelligence might end up with an average score, but our "bad" students aren't bad in the overall scheme of things. We probably don't graduate anyone 1SD or more below the mean in intelligence.)
I mostly hang out with people who design analog power IC's for a living. I suspect that they are probably all at least 1 sigma above average. They are certainly not all 2 sigma above average. This is, nonetheless, one of the most challenging of engineering fields.
We all went to high school (most of us with near 1000 fellow students). Presumably these represented a fairly inclusive normal distribution. It was very clear who the brightest were all the way to >3sigma (ranking the middle is obviously more challenging). I don't think it is that difficult to judge, in person, whether someone is in the stratosphere of intelligence. Most of the time, it tends to be very obvious.
>WHAT IS THE NAME OF THE LARGE HAIRY SPIDER THAT LIVES NEAR BANANAS? (TARANTULA)
Slightly off topic, but as a spider enthusiast, I would have gotten this wrong. There is absolutely no association between Tarantulas and bananas in my head (indeed, they live globally, and a good portion live only in deserts where there are no bananas), and Tarantulas aren't even "true spiders". In contrast, there are several other species of true spiders that are literally called "banana spiders" because of their reputation for being found in shipments of bananas. Several of these are, in fact, large and hairy, namely Brazilian wandering spiders (and to a lesser degree, huntsmen).
And the authors of this test probably picked up that piece of misinformation from listening to Harry Belefonte's Day-O.
And this one.
WHICH PLANET WAS THE LAST TO BE DISCOVERED? (NEPTUNE)b
Really? When this test was first created (1980), Pluto was generally considered to be a planet and it was discovered after Neptune. The IAU reclassified it as a dwarf planet in the 2000s, but this answer was wrong when the test was written.
Also, I see two other booboos in this questionnaire.
> Harry Belefonte's Day-O
Oh, yeah. You're right, that's definitely the connection I was missing - and it's a song that would have been much more in the forefront of a test-taker's pop culture knowledge in 1980.
A beautiful bunch of ripe banana
(Daylight come and me wan' go home)
Hide the deadly black tarantula
(Daylight come and me wan' go home)
However, that just bothers me more, since not only are tarantula bites almost exclusively not deadly, most aren't even medically significant.
Technically it's not correct now either. The current answer would be some exoplanet, and change all the time.
More importantly, what percentage of people can identify the answers that the authors of this test got wrong? Although one of those four was correct at the time of its writing in 1980, it was no longer correct by the 1990s (so why do they ask it again in 2012?). And one answer was wrong when the test was originally written in 1980, but would now be considered technically correct after the mid-2000s. And two others are based on popular misinformation.
There may be more than four, though — full disclosure — none of the ones I spotted are sports-related questions. I don't know if the answers to any sports-related questions are wrong. I never followed sports. ;-)
I agree with the overall point that you're seeing usually the topmost percentage in many cases, and I expect that knowing various bits of knowledge positively correlates with IQ. However, I don't really find 3% surprising? There's a lot of memorization that goes on in early geography classes that kids just throw out of their mind, in my experience usually the stuff they remember specifically is things drawn to their attention in a more interesting manner (tv show, movie). I'd also find it completely plausible to not even have ever been told that Budapest was in Hungary — you definitely see it in movies, so I'd expect they know it *exists*, but rarely mentioned directly that it was in Hungary.
(Some of these bits are surprising in the opposite direction for me: 8% knowing capital of Jamaica? 20% know the actor name for Mary Poppins? However, there's definitely surprising bits, like only 58% knowing the answer to be chameleon)
Of course some of these datapoints are more surprising than others
I think a lot of the movie stuff would have been more common knowledge 50 years ago
The more of these questions I read, the more it sinks in just how arbitrary and idiosyncratic this "general knowledge" test could be. Questions about parts of typewriters and 1950s pop culture and 19th century literature and golf. I can picture the hairdo of the midcentury upper-middle-class white men they surveyed to normalize these questions. I would guess there are certain areas of human knowledge overrepresented here and thought to be generic, while other equally or more "significant" areas underrepresented or totally absent.
Yeah, for a "general knowledge" test to correlate with intelligence you really have to be using it on all the same demographic. That's somewhat true of other intelligence-related tests like the SAT, less for Ravens Matrices, but making one that's truly culture neutral (let alone testing whether it's culture neutral) seems impossible.
Transparent self-promotion here but I made a blogpost with some simulation/math to illustrate how truly severe the selection has to be to end up with the self-reported distribution of IQ scores: https://unconfusion.substack.com/p/i-dont-think-slatestarcodex-readers
> 114 IQ (like 85th percentile)
This is correct (well, 82nd, but a close match to several rules of thumb whose accumulated error accounts for the difference), which makes this:
> 140+ IQ (like 99.9th percentile)
odd. 145 IQ would be 99.85th percentile; 140 is not going to approach 99.9.
I wasn't consulting my normal distribution tables while I was typing the comment. An IQ of 142 would be 99.74th percentile, and an IQ of 114 would be 82.47th percentile, assuming mean 100 and standard deviation 15.
IQ test administrator here. The ones I use are WISC and WAIS usually. I’m here to add weight to the theory that above 135 scores are not as precise for the relevant groups, that is, adults and young adults reading this.
1) The WISC-V came out in the last seven years but even with the Flynn effect lagging as of late, it’s newer than the WAIS-IV, which has an update coming out this year. The SB is hopelessly old and I think its validity is suspect at this point. (Anecdotally, the final test item for WISC subtests “feel” harder than their adult equivalents.)
2) The WISC is used up to 17 years of age, and anecdotally speaking, that is, I’m not actually whipping out my norm conversion tables, the difference between a top possible score and a merely 130 score equivalent is sometimes a single item or the speed by which one answers a single item in a given subtest.
3) This ceiling effect is less true for younger ages, which makes sense because theoretically, even a six-year-old may be completing the same task as a 16-year-old.
4) This isn’t really a problem for the professionals typically using IQ tests as our main job is determining intellectual disabilities or specific learning disabilities. We’re rarely trying to tell super smart kids how smart they are. If you’re above 130, you qualify as gifted, and we don’t care anymore.
5) If you qualified for gifted (over 130), good for you. However, you probably qualified when you were super young and federal laws do not require that we retest you every three years like it does for disabilities. It’s quite possible that you may have had an intellectual growth spurt. We’d never know.
I'd be curious if there's later-age IQ score data for folks who tested IQ 130+ as children. This is motivated by Lubinski et al's work eg
https://cdn.vanderbilt.edu/vu-my/wp-content/uploads/sites/826/2013/02/14084609/Article-Bernstein-PS-2019.pdf or
https://cdn.vanderbilt.edu/vu-my/wp-content/uploads/sites/826/2013/02/14084621/Article-PS-Makel-et-al-2016-II.pdf
I have no idea what the general trend is, but I can say that my tested IQ went *up* by about 6 points between age 11 and 16. I suspect that some of that is just familiarity with the testing format, which is hard to control for.
That happens. I’m at the high school level so I see overall scores go up and down throughout the life span. Over five points is on the high side, but not unheard of. Another factor might be varied tools. I rarely hear the Woodcock mentioned outside of school psych circles, but it’s popular within the field.
There’d have to be a reason to do it. I’m not sure what that could be beyond construct validation.
Do you happen to know if there are any online IQ tests that are actually reliable?
I don’t. Sorry!
I wonder about an obvious effect that wasn't mentioned: what happens when people take multiple IQ tests and report their highest score? For example, I've been accepted into Mensa, so scored about 135+ officially, but I guarantee that if I took 10 different IQ tests, in at least one of them (especially one involving language), I would score less than 125. I've even scored as low as 109 much earlier in my life.
Yeah, was about to note that the "well-behaved misestimation" sounds like "take samplemax of N tests", since this will produce a fairly-reliable systematic error.
Great point regarding language, remember Feynman's IQ score was "only" 125 yet he recorded "the highest score in the country by a large margin on the notoriously difficult Putnam mathematics competition exam, although he joined the MIT team on short notice and did not prepare for the test. He also reportedly had the highest scores on record on the math/physics graduate admission exams at Princeton." His mental mathematical hardware was Cleary off the charts but perhaps he had completely average verbal reasoning which kept his IQ score down.
Seems unlikely given that he was also a popular author
It seems more likely that he didn't try when taking a school IQ test and then later repeated the anecdote about his score because he liked the reactions it got.
No evidence for this beyond his self-report
I remember reading an interview with Feynman's sister. She went back and found the original reports. It turns out he scored 122 (or 123--I think it depends on which interview).
Not according to any of his students ever 😆
I think there's something sort of delusional about mapping "he scored 125 on an IQ test once" to "his IQ was 125"
Like, sure, his height was 4 feet tall too. Oh, you wanted his _adult_ height? I was talking about when he was a child.
IQ is, by definition, your score on an IQ test, something which is said to be highly consistent even though the are many explanations here that it is not. If he took only one IQ test then that score was his IQ forever. You may be imagining that he has some true, intrinsic IQ that matches his displayed intelligence, but again, other responses here explain why that is ill-defined at the level you'd expect him to have.
I think your point behaves the same for IQ and height. Do you think there's a disanalogy? If so, why?
(Adult scores on IQ tests are pretty consistent, but the connection between childhood IQ and adult IQ is lower. This is one of the reasons it's awkward we mostly have measurements of childhood IQ / academic performance.)
We have no proof that that was his score
We have no idea what test he actually took
We have no proof he tried his best at it
This is ONE singular data point
And yet every midwit is 100% convinced this completely refutes the entire field of intelligence testing. It's asinine.
I got this right? a) If I claimed an IQ of 128: fine: most ACX readers should be (not sure, when reading some comments - but, hey, me probably 120+ and I AM much DUMBer than I wished)
b) If I claim 138 (I got as an early teen, due to age-bonus), then it gets loped off to 135 or 128 (138-10).
Why not just assume an IQ of 125 (or 128 - what does it matter, really)? And drop the question.
Or ask: 'What other blogs you read?' and adjust? SSC +5 /Zvi: +5 / Derek Lowe +8 / Hanania -10 / Pueyo -5 (yes, I pay subs. to Thomas P.)
TBF, In the pipeline added points should be counted only if you do not work in medicinal chem / organic chem / big pharma. It is pretty much required reading in our neck of the wood.
PS: for many good posts on AI in chemistry from a professional standpoint - https://practicalcheminformatics.blogspot.com/
Thx for revealing Derek Lowe to me, but I disagree on Hanania, won't argue tho: that you gave him a swooping -10 means that he's probably your guilty pleasure.
I do not work in chem/pharma. In school, I gave up on salts. :) I found Derek before I found Scott; reading his "Things I won't work with" series was and is just out-of-this-world. Go for all of them! Pure pleasure, no guilt:
https://www.science.org/content/blog-post/things-i-won-t-work-dimethylcadmium
As for RH: yep, I subscribed. Easier on my poor English and very little brain than, say, Sam Kriss. Will I ever pay: Nope. Noah is better, Slow boring is better. Caplan is much better - his admiration for RH is one of his sillier ideas.
Hanania -10 is a really weird hip check. I read Hanania because Scott has linked to him occasionally and… I disagree with him on 70% of his priors, but there’s no doubt he’s super smart, articulate, and willing to consider when his team is wrong after thorough consideration.
I don’t see why that would make you classify a regular reader as “likely not smart”.
Ah, you got the joke. ;) Well: 125 minus 10 is 115. Which is still smart. Tucker Carlson would be a bigger minus (I saw him with Caplan and Lex F.). - Arnold Kling considers RH less than impressive. No doubt: RH is articulate. Even super smart. Less sure about his average reader (who is usu. just smart). Did I ever read an eye-opening idea first at RH?
If you get through all of Zvi's I'll give you +5 on reading speed, that's for sure
Curious about the assumption that readers will hate on Mensa. Why the hate?
I joined for a year ages ago when I was new to a town. The monthly game night was a blast.
I’ve never taken an official IQ test but have gone off the fact that Mensa accepted me based on GRE scores which I figure puts me top 2%. And based on the incredibly unreliable observation of how often I think people are smarter than me, I don’t think I’m much, if any, over top 1%. It occurs to me that anyone doing a similar gut check on a test result over 140 may have a hard time due to sample size.
> Curious about the assumption that readers will hate on Mensa. Why the hate?
Maybe this is too cynical, but I think it largely comes down to "hey these nerds are claiming status they don't deserve, let's bully them".
It seems more: If you take a Mensa test, you feel the need to verify you have a high IQ. Even though you failed in life. If you succeeded: no need to verify. - Thus: 'nerdy' losers do Mensa.
Personally I did Mensa because I was a lonely nerdy teenager trying to find people I could relate to (this was before widespread internet access).
That would explain people quietly looking down on them. I think the need some people feel to be actively nasty about it is, at least in part, a status slapdown
This: you don't feel the need for validation if you're already doing a job that self evidently requires you to be intelligent.
It's the usual justification for mocking Mensa members, but it only explains the lack of respect, not the active antipathy. That's where status hierarchy enforcement comes in; these guys are 'losers' trying to confer high status upon themselves, which is a violation of the social order and triggers something like a disgust reaction. It also allows people to combine the fun of punching down with the sense of righteousness that comes with [doing something that, if you squint, kind of looks like] punching up.
Yeah, I get general anti nerd sentiment. But Scott seems to expect this of his readers - a pretty nerdy bunch themselves.
Nobody hates on nerds harder than slightly different types of nerds.
Of course this thread is classic barber-pole behaviour. Class A nerds (Mensa members) brag about how smart they are, so Class A' nerds look down on them to prove how much better they are. And now in this thread, Class A'' nerds are looking down on the people who look down on Mensa members, to prove how much better *they* are.
And now look at me, I'm Class A''' and I'm looking down on those people too which proves I'm better than everybody. Luckily I am at the top of the barber pole and there is definitely no Class A'''' that would look down on me.
I don't know about modern Mensa, but there was rather a tradition of people in the 70s/80s who boasted of how smart they were and how superior this made them to you clods, and the basis for their assurance of their hyper-mega smarts was "I'm a member of Mensa".
Indeed; the punching (up or down) comes as a reaction to the implied sense of superiority/brag, and what makes it an easy target in particular is the implication in the brag that "I'm really smart but have done nothing of note with that intelligence, otherwise I would be talking about that."
Right, but we're the nerds here, we aren't going to be criticizing Mensa for being too nerdy and wanting that to be high-status.
I think criticism of Mensa by people smart enough to be in Mensa, comes from the fact that - aside from low-grade signalling - the main benefit of Mensa is that it provides you with a social group screened for high intelligence. But if you're *really* that smart, why aren't you working at MIT or FAANG or some other place where half the people you meet are really really smart already?
Which, sometimes, is a legitimate criticism - Mensa is going to appeal to the smart but unambitious who are satisfied being the brightest fish in a small, dull pond. But it's also going to appeal to the young woman whose family is only going to send her to Texas Women's University, and sure, Yale is in her future but for four years it's genuinely helpful that someone has gathered all the smart, lonely people at TWU into one bunch.
Mensa exists as a category exclusively to declare that its members (or at least people eligible to be its members) are better on some axis than other people, which many people dislike. If there were a Society for Hot People didn't allow anybody below +1 SD hotness or a Society for Rich People that didn't allow anyone below +1 SD wealth we'd think of them as prats and wonder who would join such a society.
Check of hypothesis: do Societies for Rich People exist, and do we view people who join them as prats?
Answer: Yes, country clubs exist and we view people who join them as prats.
Hypothesis supported.
I don’t know about that. Universities screen for intelligence. Are they hated for that? Also, most of the world groups people together with others like them. Is the issue that mensa is too honest about what they are doing? Vs the neighborhood only some people can afford to live in.
Unlike universities, Mensa's screening is an end in itself; It's literally exclusive for exclusivity's sake. And unlike other tribes, it's explicitly screening for (as Sei put it) "being better on some axis than other people."
But sure, being upfront about their reasons for being exclusive makes it more obvious.
Hard disagree.
Maybe Mensa is different where you live, that's entirely possible, different cultures are different after all. But where I live (The Netherlands) my experience has been that Mensa is not so much a "Organization for people with high IQ" as a "Organization people with high IQ who struggle because of that for one reason or another". It's not a country club, it's a support group.
Most people with very high IQs are surrounded by other people with high IQs. Family members, colleagues, university friends, etc. But not everybody is that lucky. I have an ex who was extremely bright, but she was the only in her family who ever went to university. In fact the only one in her entire extended family who ever did high school at more than the lowest tier. She got absolutely zero support at home for her academic pursuits. Not because her parents were abusive or anything, but because they just didn't understand.
For her and many people like her, being very intelligent is a source of alienation from friends and family. Having a way of meeting similar people is a godsend.
And similarly, most highly intelligent breeze through high school, and then go on to have good careers where they are surrounded by lots of people who are also intelligent. But again not everybody is so lucky. It's not uncommon for gifted kids to fail in school because they are just bored out of their minds all the time. Again, having a place where such issues are understood, and where you can find resources to deal with that, or just a sympathetic ear, is a godsend for some people.
This is exactly why I joined as a teenager (and why I haven't bothered to keep up my membership as an adult, because I now have other ways to find such people).
We've apparently shifted from valuing intelligence to valuing narcissism.
(Witness certain ex-presidents / reality TV thespians with marmalade complexions.)
But where and when does one test for I. Q.? I took the SAT, and scored higher as a junior than a senior, and the ACT, I think. But I never took or even knew anyone who took an I. Q. test.
https://www.cremieux.xyz/p/mensa-the-above-average-iq-society
> It’s a group that has, unfortunately, become frequented by underachievers from modest backgrounds, with extensive interest in playing games that prove they’re intelligent but who have little achievement otherwise.
There both good and bad reasons to hate Mensa. Bad reasons would include nerd bullying and intelligence denialism. Unfortunately, there are also some good reasons.
There is nothing wrong with choosing the subset of society with top 2% IQ per se. Actually, that would be awesome. The problem is that Mensa is *not* a representative sample of this group, because there is additional selection for "knows about Mensa" and "wants to be a Mensa member". Even worse, once this becomes generally known, the selection only becomes stronger, as now it is "knows that Mensa has problems, and wants to be a Mensa member anyway".
1) People *obsessed* with IQ are over-represented. Obviously, they are most likely to find out that Mensa exists and apply for membership. They have various crackpot theories about intelligence, and as usual among crackpots, they are not familiar with the mainstream psychological research and they don't care, they just want to share their own theory with everyone. They can't stop trying to guess every person's IQ based on extremely weak evidence. They can't stop talking about how the world would obviously be a much better place if people with high IQ could rule it. They refuse to admit that IQ is not everything, and you might also need actual knowledge, rationality, social skills, mental health, etc.
2) Generally, crackpots of all kinds love to use "I am smart enough to be in Mensa, therefore my theories are right, and the mainstream science is just too stupid to understand them". Almost every crackpot applies to Mensa... and 2% of them succeed, which in their eyes proves that they were right.
3) Also over-represented are lonely people, and people who despite high IQ somehow fail at life. Although these people have my sympathies individually, if too many of them come to the same place, it results in strong vibe of "this is a club for losers".
These things reinforce each other. For example, for the lonely people it is difficult to oppose the crackpots, because in their eyes, ostracism is the worst thing that could happen to anyone, and the idea of kicking someone out of Mensa feels too scary. But as a result, people who have better social options spontaneously leave Mensa, leaving mostly the loners and crackpots.
All this makes me very sad, because in my opinion the underlying idea is not bad, if we only could somehow overcome this selection for crackpots and social incompetents. I am not even saying that we need to get rid of them... only that they shouldn't be so horribly over-represented.
If you could just stop 1000 random people on a street, give them all an IQ test, select 20 with highest scores, invite them to a party, and kick out two or three of them who turn out to be insufferable for some reason... I think the result would be quite nice, and basically this is what Mensa originally wanted to be.
I also see value in measuring IQ explicitly, because there are many people who underestimate their own intelligence, because they do not match the stereotype. I made bets with a few people that if they try the Mensa test they will pass... first they laughed at me, but then all of them passed and were surprised. I hate to see people underestimate themselves.
In schools, many kids suffer because they are smarter than average, they understand the lessons quickly, and then they get bored. Ironically, many of them are considered stupid by their teachers... because they stop paying attention. Many high-IQ kids were sent to a psychologist to diagnose whether they might be mentally retarded.
Basically, if you happen to be highly intelligent, your environment makes a great difference. If you grow up in a family of supporting and academically successful parents, you probably have a great career ahead of you. On the other hand, if you are forced to live the average life, you will probably feel that you don't belong there, without understanding why; this often makes people depressed. It would be nice to have a standard way to detect the latter, and to give those people some alternative supporting environment.
If you make an environment that selects for high IQ indirectly, for example by measuring academic success, you do get a de-facto high-IQ society, without the problems of Mensa. The disadvantage is that you get a kind of monoculture. A group of students who study quantum physics is certainly full of smart people, but I think they would benefit from meeting people with similar levels of intelligence who do something else for a living. Less Wrong or ACX meetups are somewhat closer to this ideal, because they often connect smart people with different professions; but a truly random sample of high-IQ people would be much better.
"Curious about the assumption that readers will hate on Mensa. Why the hate?"
I will try (though I mostly don't think about Mensa).
If one thinks of Mensa as a mostly a social club for people who like to use their brains (which, I think, is how Mensans think of it) then an exclusion based on a test rather than candidate's actual ability and willingness to use their brains seems both wrong and strange.
The organization (I think) has sub-groups and clubs focused around things such as puzzle solving (Sudoku, etc). It is easy to imagine someone who is great at solving puzzles but doesn't meet the IQ standard being excluded while allowing someone who is objectively poor at solving puzzles (but still enjoys the challenge) being allowed to play as long as they score highly enough on IQ. This strikes many people as unjust/unfair.
Sei (below) suggests thinking of how we would consider a "Society for Hot People" that didn't allow in people less than 1 SD above average in hotness or a "Society for Rich People." But even these seem to make more sense than Mensa because the Hot People explicitly want to hang around other hot people (and are seen as shallow for valuing this so far above everything else) and the Rich People sometimes also want to hang around only other Rich People (which, while not a rich person, I have some sympathy for). But even the Rich People clubs usually filter on ability and willingness to pay rather than the actual size of one's bank account independent of ability to pay.
Mensa projects as filtering on the equivalent of money in the bank, but NOT on willingness to spend it (ie, deliver results of some sort at the meetings/events/whatever).
This may not be how Mensa actually IS and I'm pretty sure it is not how the Mensa members think of it, but I think this is how it presents to the outside world.
EDIT:
Derp, this was intended as a reply in the main thread. Moving.
I think there's also a generational thing.
One of my aunts is a proud MENSA member and recommended it to me; she has spent a lot of her life in more sparsely populated parts of the country before the Internet was common, where groups like MENSA were a good way to meet other smart people.
I, on the other hand, grew up on the Internet, posting on forums where the average IQ is high enough that people write posts about "the average IQ can't actually be this high, can it?". When I thought about joining MENSA, it was explicitly as a "I want to meet more old people" move, since I already knew a ton of smart young people thru LessWrong.
I think it's a couple of things. First, as others in this chain have noted, there's the "filtering seemingly for the sake of filtering" that seems to support no particular purpose other than to support unfounded status claims.
To that, I'd also add, though, that "professed high-IQ man" is a common and distasteful part of internet message boards, and pretty much anybody who's a member of a community like this has had the privilege of dealing with self-professed-Mensa types in threads like https://www.astralcodexten.com/p/open-thread-240, which includes such gems as "[w]hat's the point in talking to people that don't have any intellectual achievements, eg contributions to human knowledge," "[p]eople like me have given you everything," and "[t]he minimum IQ for contribution to the knowledge of man is about 125... [p]eople under that level quite literally exist just to do labor for people above that leve[.]"
Does that represent all of Mensa? Of course not. Are the people who do have those opinions as vocal as they are distasteful drawn to Mensa like flies, and coloring people's perceptions of the high-IQ club accordingly with the nonsense they spout online? Unfortunately, very much yes.
Unrelated: My headcanon is that every time Scott repeats a word, it's a deliberate easter egg, and there's some kind of secret prize for the person who finds them all. That said,
> So are people just taking terrible Internet IQ tests that inflate their score **about about** 20 points?
double brood of cicadas.
When I was a psychology student, 30+ years ago, in a country far away from where I live now and from the US, we were told that an average IQ for *university students* then was about 120. This made rough sense considering the % of population that attended university there and then. I realize that it's not like that for the US now, but the idea that the audiences for those websites/your blog etc would be INCREDIBLY self selected by IQ feels plausible. Not 135+ on average obviously but certainly well above not just the population mean but even degree-level educated population.
I don't understand why you'd assume click workers would have population average IQs? To me it seems obvious that they'd be higher, because ANY group that engaged in any cognitive labour will have higher than average IQ. Not just because it would exclude those with the very lowest scores incapable of doing that labour but also because it'd exclude a lot of people with middling lowish to average scores who have less interest and inclination in such.
Also, self selection for even measuring IQ others mentioned and reporting the highest score: for example I'd likely report my 135+ Wechsler score from my student era (I was also subject to the one used by the US army historically, translated/normed of course but that was as a teenager and I don't trust that result) rather than 125+ Raven score (not just because it's higher but because I'm certain that the former measures "manifest" intelligence better). I'd not report middle aged English results (not because my vocabulary is worse, but because I've discovered that memory, especially digit memory, for me varies by 2 digits depending on language, both straight and backwards).
So I think selection bias in audience and reporting explains most of these results.
.
>I realize that it's not like that for the US now
To put some numbers to this, see here: https://www.cremieux.xyz/p/education-isnt-what-it-used-to-be. It appears that in the US, mean IQ for college undergraduates is significantly below 110, and IQ for those with only an undergraduate degree is only about 102.
When you go from less than 5% of the population attending college to somewhere in the 30-40% range (not sure where it is now), the average IQ will go down, as it must.
Of course! That's the point of the article. Cremiuex notes in the article that if 40% of the population attends college, then even if they were perfectly selected as the top 40% in IQ, they would still only have a mean IQ of about 114.
You're missing an assumption that the original 5% came from the top end. If the average IQ was 100 before, expansion could take it in either direction and you'd predict no change. If it was 75 before, expansion would take it up.
That's roughly a good assumption though. Maybe not exactly the top 5%, but there's no remotely conceivable way that when 5% of the population were going to college that they were not at least significantly above average. There's no reason people with an IQ of 75 would have been going to college over smarter people, and there's no way that such people could have been competent enough for American society to have been as functional as it was.
Oh, I agree that the assumption is correct; I'm just pointing out that this argument doesn't transfer to other contexts.
Honestly, every time I read these IQ self-report posts, I wonder "How the hell did I sneak in here under false pretences?"
I'll just be over there, hiding under the table in the shadowy corner of the room so I'm not discovered and shooed out with a broom 😁
Now I'm curious if you've been tested, and what it came up with. You don't *seem* below-average even in this crowd.
(my brother told me once that he sometimes brags to others that the rest of his family is in Mensa, but that he doesn't think he'd qualify. I think he is very very wrong).
Never formally tested, except for one time when I was about twenty and did an aptitude test (with about a gazillion others all crammed into one room) for a job application. Was informed I'd failed, and I have no idea how I scored or what the threshold they were looking for was.
It was the 80s in Ireland, there were many more applicants than vacancies for every job vacancy, and I think the "aptitude test" was just a filtering mechanism (it was a very basic job and didn't require much in the way of skills) to get the numbers for interview down to a manageable level. If the job hadn't already been filled internally, and this was just a way of meeting the requirements that "you must advertise job vacancies" but they had no intention of hiring outsiders (I did a few of those interviews in that period, too).
So I've only done online tests, ranging from the "what colour is a banana? if you said yellow, congratulations! you are smarter than 59.85% of other people who took this test!" type to the "we're doing a simpler version of a Ravens Matrices test which is a big legit IQ test so you know we're not scammers" types.
>Most people don’t read long articles online about statistics. 1/30 sounds like as good an estimate as any other for the sort of person who would find that comprehensible and interesting.
My estimate for the % of English readers on the web who read long articles about stats on this type of topic, with any degree of regularity, is 0.1% - 0.3%. My wife and quite a few of my smart friends are never (ever) going to read 80%-90% the types of things this community proactively seeks out and reads on a monthly basis. It's not reasonable IMO to convert this percentage of the population to the percentile threshold for an IQ cutoff, but for grins and giggles if that were a reasonable proxy we'd be back to the 138-145 range.
*grins and giggles*
Hands off my coprolalia, bandit!
Yeah. First let me firmly set aside any sense of humility or shame. In my job, and in my education, I have interacted with a lot of really smart people, and have done so consistently for the last fifteen years or so. I have also been part of the rationalist community for almost that long. The people I meet at Less Wrong or SSC meetups tend to think, speak and behave like the very smartest of the smart people who I interact with in my career. Essentially *everyone* at a given Less Wrong meetup has a list of esoteric skills and interests, such that they would get along great with the smartest of the PhDs I work with. It isn’t subtle.
What contexts, other than LW meetups and STEM PhD programs, do you hang out in where people can have an informed conversation on either Nietzsche or quintessential models of cosmic inflation? Are we going to pretend this is not related to intelligence?
...It's probably more correlated with autism.
Only to the extent that people in this subculture use “autistic” as a euphemism for “evidently smart.”
But those are not the only kind of great conversations to have -- the kind where people are informed about those things. Here are the topics of some of the best, most memorable conversations I have ever had:
-(with a close friend) Our deepest and craziest thoughts about self and consciousness and time.
-(with a close friend): WTF is wrong with the faculty at our grad program.
-(with 2 people I had met for the first time): Their work on maintenance of the Cathedral of St. John the Divine
-(with a schizophrenic): What it's like being him: ". . . the room gets liquid and I'm the aquarium they're all swimming in and what I want to know is how I go back to being just one fish."
Intelligence changes with age. You shouldn't weight on just education, at least age needs to figure in as well. Testing age will necessarily be lower than in the survey, SATs are usually done when people are at peak intelligence, so tests in ClearerThinking are likely to be post-peak and past tests likely to be closer to peak. Plus: Flynn effect, norms change.
Will you/Spencer share the ClearerThinking data? Would be useful for our research on norming from convenience samples.
SATs are not at peak intelligence; fluid is around 21 and crystallised around 40.
That can be factored in, of course, but so can decline with age.
I thought crystallised peaked at around 60, not 40?
WP:
>Crystallized intelligence typically increases gradually, stays relatively stable across most of adulthood, and then begins to decline after age 65. The exact peak age of cognitive skills remains elusive.
I've seen 40ish in a graphic somewhere; maybe it was extrapolating beyond what we're sure of.
About 2 months ago I read a headline: 5min (or some short) IQ tests work just as good as long IQ tests!
So I open Google and take the first test 5 min IQ test that popped up; I had to pay for my score when reached the end. I close that tab and search this time, very carefully- noting the word ‘free’ in the search results. Once again I get to the end of another free 5 min test, ‘We’ll send you an email with results’. The email I receive is a link to pay for results. I look through another 25 links, surely one must be free! I find a page “Free FREE FREE!” This is the one! I take the test, give me the results! 140? 155? It says I need to pay for results. but taking the /test/ was free…
I close all my tabs, and go for a walk.
Remembering my escapade 2 weeks prior (heaven knows I won’t tell a soul about *these* struggles) I doggedly find the headlined paper making the online buzz for 5min IQ tests. I download the software from the study -an old JavaScript program- which I have to debug because it’s ancient. I spend 20~ mins on this. Finally! I launch it. I take the test. I click ‘results’, and it launches me into a website where I can buy the full version of the software suite.
I laughed, thanks!
After this description, I think you'll agree that short IQ tests work as well as long ones!
There's an observable phenomenon where people think about IQ as if it averaged 110-115, not 100. A "really smart" person has an !IQ around 140, and a "really dumb" person has an !IQ of, like, 98. (At its extremes, this is the fictional genius whose reported IQ is in the 170s or higher.) I saw a particularly prominent example of this process in a discussion of personality that referred to how a similar personality would "look different in someone with an IQ of 140 to someone with an IQ of 90". Those aren't symmetrical, but if you think about IQ how people tend to, you end up saying something like that.
(This isn't a range-restriction thing, exactly -- everyone does it.)
I've always thought accordingly that you'd be about right to estimate the average IQs of people taking the SSC/ACX/LW surveys as in the lower-mid 120s. This is extremely high! It is, in fact, about as high as what people are really intuiting when they think about someone with "an IQ of 140" -- the range of way-way-above-average people in the general population. It's also a plausible average *full-scale IQ* for a group of people who tend to be both smart and...eccentric. Subscales are pretty well-correlated in the general population (the "g factor"), but the correlations are not groundbreakingly large and many unusual people have sizable gaps between subscales. There are whole portions of the WISC/WAIS that are just "oh yeah, these are the sections that routinely break and lead to people's FSIQs being markedly off from some of their subdivisions".
Now, it'd be a bit of an exaggeration to say you can't test above an IQ of 135, in the same sense it'd be a bit of an exaggeration to say you can't test below an IQ of 65. We can definitely test in the mild intellectual disability range. Below that...hahahaha good question, someday we'll stop giving autistic kids "adaptive functioning tests" that dock points for stimming. I suspect a lot of the invalidity on *this particular metric* is due to things other than "the tests don't work that high", like "people are lying" or "people are giving an unrepresentative result" or "people misunderstood the results somehow" (given in a different comment) or "people have unusual subtest divisions, and the questions in the surveys didn't account for this" (were they matrices?). Probably all of these across many different people.
I’ve often noted this tendency in myself anecdotally and try to work around it—realizing that people with an 80 IQ are as common as those with 120 has gotten me to be more sympathetic with the slow person at the grocery checkout. They’re barely a standard deviation below the norm, and that’s not much at all.
> There's an observable phenomenon where people think about IQ as if it averaged 110-115, not 100. A "really smart" person has an !IQ around 140, and a "really dumb" person has an !IQ of, like, 98.
People are probably doing that because the average IQ of everyone they know is 110-115. I would consider a person with 98 IQ to be incredibly stupid. I would be right, in every meaningful sense of the condemnation - they would make laughably obvious mistakes at very high rates, show extremely poor performance on pretty much everything, etc., compared to anyone I'm in a position to compare them to. The lowest-performing member of my immediate family tests around the 90th percentile on standardized tests. Using the concordance that this post says we shouldn't use, that would be a 119 IQ.
I recall seeing some discussion of a result that found that, at elite universities, students who came from private schools had high opinions of what "average people" were able to accomplish, while students who came from public schools had sharply unflattering opinions. This tends to suggest that, if you tell people that a 98 IQ is average, people who are familiar with what kind of performance that involves will happily call it "really dumb", and people who aren't will say polite things about it.
Of the four schools I attended for a meaningful length of time in my primary/secondary ("grade school") educational career, three probably or definitely had average IQs below 100. I've lived in a lot of questionable neighbourhoods and met people from all walks of life. My academic research interests focus on things anticorrelated with IQ (e.g. particular genetic disorders). I am not from the upper-middle-class bubble and not range restricted. I...would not make the same post, let's put it that way. (There are very complex elements, some of which are *not flattering*, to discussing and highlighting range restriction as a non-range-restricted person. They are complex in wholly different ways.)
I carefully noted that this particular mistake is a different thing to range restriction, because it is. People of "evidently below average intelligence, when you talk to them" think about IQ this way. You notice it very prominently in fiction, where characters are represented as "smart" by having ludicrously high IQs. (You also notice it, if you're the smartest kid at a poor school, in how high people rhetorically estimate your IQ as.) People who score in the average range on IQ tests tend to think of their results as "really low", because we talk about IQ in a way that implies the 90-109 range is markedly below average rather than the interquartile range. (I have seen this, repeatedly.)
> You notice it very prominently in fiction, where characters are represented as "smart" by having ludicrously high IQs. (You also notice it, if you're the smartest kid at a poor school, in how high people rhetorically estimate your IQ as.)
I'm not really sure that this is informative of much. To quote Jonathan Coulton:
𝘒𝘦𝘯𝘦𝘴𝘢𝘸 𝘔𝘰𝘶𝘯𝘵𝘢𝘪𝘯 𝘓𝘢𝘯𝘥𝘪𝘴 𝘸𝘢𝘴 𝘢 𝘣𝘢𝘥 𝘮𝘰𝘵𝘩𝘦𝘳𝘧𝘶𝘤𝘬𝘦𝘳
𝘏𝘦 𝘸𝘢𝘴 𝘴𝘦𝘷𝘦𝘯𝘵𝘦𝘦𝘯 𝘧𝘦𝘦𝘵 𝘵𝘢𝘭𝘭, 𝘩𝘦 𝘩𝘢𝘥 𝘢 𝘩𝘶𝘯𝘥𝘳𝘦𝘥 𝘢𝘯𝘥 𝘧𝘪𝘧𝘵𝘺 𝘸𝘪𝘷𝘦𝘴
> People who score in the average range on IQ tests tend to think of their results as "really low", because we talk about IQ in a way that implies the 90-109 range is markedly below average rather than the interquartile range. (I have seen this, repeatedly.)
But this is a fairer point. However, you're explicitly marking this opinion as downstream of a select group that sets the tone for discussion of IQ.
Is that group range restricted?
------
You're right that my response is mostly on a point that isn't quite the one you were trying to make. If I read you correctly, you're saying that people think, because of some kind of cultural osmosis, that a 98 IQ is meaningfully below average.
But what I read you as saying was that people who say 98 IQ is "really dumb" are making a mistake. I don't think they are! I'm defending the idea that a 98 IQ is "really dumb". Most people are really dumb.
And most people are really slow compared to Usain Bolt. This is not meaningful except perhaps if you're running for the bus.
Can that really dumb 98 IQ person do the same stratospherically elevated job as my really hugely big brained family? No.
Can they hold down a job, earn money, pay bills, and do normal everyday tasks of life? Yes.
I do sometimes find this shocking, given how difficult I find it to do stuff everybody has to do, like budgeting and taxes and credit cards and health insurance. I realize some people just kind of fail at those, but most seem to do basically fine.
There is definitely a level of 'life smarts' or 'social smarts' that does not have to do with academic intelligence. If you're not book-smart but your parents/some other authority figure showed you how to figure out "this is how you do your taxes, this is how to balance a check book" and so forth, then you'll manage in life. The majority of people are "really dumb" by Mr. Watts metric, but they manage somehow to drive cars to their jobs and perform adequately enough to get paid, so they can then pay their bills, cook their meals, and raise their kids.
When you say "extremely high" and "way-way-above-average", please consider that in a city with million people, there are tens of thousands of people with IQ 120 or more, and thousands of people with IQ 140 or more.
High intelligence is not that rare in absolute numbers. It just seems so, because you will never see those thousands of people at the same place, as they are dispersed in a much larger crowd. But it would not be implausible to find out that ten of them read the same website.
Don't IQ tests also have an upper bound? Like you sit with a cognitive therapist or neurologist or whoever and they do several hours of different types of tests and then they report the score.
Every test has an upper bound, determined by getting everything correct. But "IQ tests" as a concept do not have an upper bound; you can have different tests that are informative at different levels of IQ.
This way of doing the SAT to IQ conversion looks wrong for reasons that were explained on the subreddit.
https://www.reddit.com/r/slatestarcodex/comments/1avcld5/i_still_dont_think_slate_star_codex_readers_have/krccacl/
If all you know about a person is that they were +2 SD on the SAT then on average they'll be less than +2 SD on an IQ test, but if you select a group of people for some other reason and they're +2 SD on the SAT on average then then you shouldn't necessarily expect them to be worse at IQ. After all, you could run the same regression argument in reverse (if all you know is that someone is +2 SD on an IQ test then on average they'll be less than +2 SD on the SAT), and the group can't be worse at IQ than at the SAT and also worse at the SAT than at IQ.
I'm also pretty skeptical of Spencer's attempt to norm an IQ test which was given to a convenience sample rather than a representative sample. I wouldn't expect controlling for 1 variable to fix that.
For example, suppose you had a complete dataset of every NBA player's height and their wingspan (distance between fingertip to fingertip with their arms outstretched). And you knew the distribution of height & wingspan for the general population (of 20-40 year-old men); conveniently they are normal distributions with a perfectly linear relationship and a correlation of r=.85.
Then you could ask the question: are NBA players more extreme by height or by wingspan (in SD above average)? I don't know the answer; without having the data it seems like it could go either way. Height and arm length are both useful for basketball so probably the sample is heavily selected for both. Maybe it's really close, e.g. +3.60 SD in height and +3.62 SD in wingspan.
Now, suppose that you didn't have the wingspan data for NBA players, just their heights. But you knew the relationship between height & wingspan in the general population, so you could set up a regression equation (or a table) which tells you the typical wingspan for a given height. With a correlation of r=.85, on average people who are +1 SD in height are +.85 SD in wingspan, +2 SD height ---> +1.7 SD in wingspan, etc. If the average player is +3.60 SD in height, this height to wingspan conversion would give you an estimate of +3.06 SD for the average player's wingspan. So very noticeably more extreme in height than in wingspan, since wingspan estimates have been regressed towards the mean.
In other words, using this regression equation to estimate wingspans is assuming that NBA players' heights are more extreme than their wingspans. But we don't know that; it could go either way. And if we were missing the height data and had only the wingspan data, then we could run the regression the other way around and estimate that heights are less extreme than wingspans. e.g., If the average player is +3.62 SD in wingspan then the wingspan to height conversion would give you an estimate of +3.077 SD for the average player's height.
This linear regression approach is the wrong tool for the job.
^ this
My mum was highly into IQ so I ended up taking an absolute tonne of IQ tests as a child, and I have fold memories of the one I got 225 on, but then on further investigation it turned up I'd added up my scores wrong so, you know, actually didn't. But if you count that one out, I remember getting scores between 105 and 175 from various tests, so, I mean, clearly they were not all super reliable. If I wanted, I could just take the highest (except the clearly erroneous 225) and walk around being like "yeah I tested at 175 IQ as a child" and it wouldn't be explicitly false but I'm not sure it would be honest. I do often suspect something like this is going on in these self-reports.
> I remember getting scores between 105 and 175 from various tests, so, I mean, clearly they were not all super reliable.
Unrelated technical note - in psychometric jargon, "reliability" is the property that when you take the same test twice, you get similar scores, the correlation of a test with itself. It isn't the property that different tests get similar scores; that would be validity. (Technically, validity is the property that the test measures "the thing you want to measure", not "the same thing as some other tests", but at least these concepts are related to each other - a collection of tests with high validity must also have a high correlation with each other,† since they are all correctly measuring the same value.)
† In some sense. To actually have a high correlation, they'll _also_ need high reliability, or you'll need to be measuring the correlation between tests in terms of the average value over a large number of testings.
"in psychometric jargon, "reliability" is the property that when you take the same test twice, you get similar scores, the correlation of a test with itself. It isn't the property that different tests get similar scores; that would be validity."
Sounds somewhat like the distinction between precision versus accuracy:
https://asana.com/resources/accuracy-vs-precision
Some quick looking into it suggests that reliability and precision are the same thing, but validity is a composite of precision and accuracy. I overqualified my footnote - high validity automatically means high reliability.
I found a book once that made this point in the context of complaining about personnel selection - that the amount your selection method is able to achieve ("validity", the usefulness of the method) is limited by its reliability - but I don't want to spend the time right now to look that up.
Fair enough! I mean validity, then, not reliability. I guess one of them could have been valid, but I can't tell which one.
I honestly fail to see the mystery here.
> These are implausibly high. Only 1/200 people has an IQ of 138 or higher. 1/50 people have IQ 130, but the ClearerThinking survey used crowdworkers (eg Mechanical Turk) who should be totally average.
There's a huge availability bias in saying that those numbers are implausibly high. Scott, most people in his bubble, and probably a majority of the blog readers - they just don't interact with people below average IQ. Practically... never?
To quote Scott himself when talking about how well-selected his tribe is:
> And I don’t have a single one of those people in my social circle. It’s not because I’m deliberately avoiding them; I’m pretty live-and-let-live politically, I wouldn’t ostracize someone just for some weird beliefs. And yet, even though I probably know about a hundred fifty people, I am pretty confident that not one of them is creationist. Odds of this happening by chance? 1/2^150 = 1/10^45 = approximately the chance of picking a particular atom if you are randomly selecting among all the atoms on Earth.
Yeah, practically never. I don't know about ClearThinking, but the LessWrong numbers feel pretty much spot on. And even the bit about mturk workers being "totally average" - it's honestly a bit hilarious, and comes from never having to explain a below-average person concepts of equivalent complexity as mturk.
And once the large bias towards high IQ is explained, all that's left is disparity between various ways to measure it. Which yeah, is an interesting statistical exercise and I actually read and enjoyed the post. But, like I said, not a huge mystery.
I've interacted with tons of people of below-average IQ, including many people with outright intellectual disabilities, in both egalitarian contexts and otherwise. (I did not grow up in the upper-middle-class bubble.) The idea that the real average of respondents is that high is instantly risible. Absolutely way out of the bounds of plausibility. Same for the more extreme sorts of claims about "what people with IQs below X are like" (e.g. the famous hypotheticals 4chan post).
I notice consistently that people who share this background, or variants of it, are the people who find the high results and the "did you know people with low IQs don't have object permanence?????" claims most implausible. That's not to say there aren't very serious range restriction considerations -- everyone with a variant of this background knows it firsthand, too -- but these *particular* arguments don't seem to correlate well with out-of-range experience.
Why do you find it so risible? I'm trying to find some objective checks, and I find a couple that to me seem to make sense. I don't know my IQ but I'd estimate it in the 120s (software developer, previous star student etc), and LessWrong is, by and large, above me. I understand most conversations, but the effort is enough to not make me a regular reader.
Also 1/200 is... enough? It's over 1 million only from US. That's the population of a small country, easily sufficient to populate a small-ish forum.
None of those is an argument that the average IQ actually is that high, but at least I don't find it ridiculous. Accounting for some bias of people over estimating it or or bragging a bit... it sounds sane. Why does it seem not so to you? Am I missing obvious checks in the opposite direction?
It's a tiny minority, on a trait that readership of ratsphere writing definitely selects for, but doesn't select "significantly harder for than being in the upper echelons of academia". Ratsphere-adjacency selects on a number of uncommon characteristics, of which IQ is one, and probably not sufficient by itself; the threshold for intellectual curiosity/interest/trait-intellectualism is quite likely *higher* than that for raw ability, and every measure we've ever made of that trait has a pretty weak correlation with IQ. (I might believe not-super-far-below-99th-percentile averages for that.) 99.4th percentile IQ averages are just wackily high compared to every other group-IQ-average we have, including groups strongly selected for intellectual ability and (sometimes) interest.
(This is before getting into things like "disproportionately many people self-report large gaps between cognitive abilities, which depress full-scale IQ on an IQ test proper". I suspect one of the contributing factors -- albeit a small one -- to the ultra-high survey reports are people taking their highest subtest as "most representative".)
So you're coming from a selection perspective, and I come from a filter perspective. You're right that IQ is far from being a main driver of people going to those places. I completely agree with you here. My argument is that for a large portion of the people who would be interested in them, the effort to follow and participate is working as a self-filter. And it makes sense from my subjective experience - there are quite a few places where I would very much like to spend more time, but... it's simply a matter of mental energy. Not capacity per se, but effort per time spent.
For example I very much enjoy DataSecretsLox, but I find that I avoid long threads and actually spend a lot less time there than I'd like. It's not a matter of being able to comprehend notions (though occasionally some comments are over my head as well), but simply of how much text I can consume per hour spent there. I'm regularly amazed of how much information Scott is able to absorb - which is a separate skill from how good a thinker he is.
So I'm suggesting that higher IQ comes also with something like higher endurance for intellectual tasks, and that's a bigger filter than pure skill.
Curious as to why you found very high average IQ implausible for LessWrong readers? Personally, I'd be stunned if it **wasn't** an amalgam of the literal smartest people in the world.
College professors are usually 120, a group with IQ 138 would be to college professors as college professors are to the average person. I don't want to say that no group achieves that, it just seems pretty extreme.
That's an interesting point. Is it possible there's minimal measurable difference in actual brainpower between 120 and 140 and the practical difference is not as vast as it sounds? I suppose I have a lot more thinking to do on this topic because I am in the "Very High IQ" bracket and I can tell you, I'm no Von Neumann.
If I think of some high achieving academic acquaintances, they don't immediately stand out as smarter than me, they're all just uniformly willing to play politics and work within a system that sickens me. Don't think there's a correlation there between IQ but I guess there could be.
Broad impressions formed over time...
1. The "practical difference" between 120 and 140 should, on paper, be equivalent to that between 60 and 80. This is to say it should be very significant indeed. This should-on-paper hold true as you add and subtract further, though realistically you'll quickly start to bump up against what can possibly be tested.
2. The correlation between IQ and the "clinical impression" of someone's intelligence is weaker than #1 implies. This is the paradox of "learning how seriously to take IQ" -- it measures an important and meaningful thing that does not perfectly correspond to how we think of "how smart someone is". Over the general population, every subscale on an IQ test correlates pretty well with every other subscale on an IQ test, which correlate pretty well with your impression of how smart someone is. On the individual level, particularly at extremes or for unusual populations, this is less guaranteed.
("What does it mean for something to correlate with another thing well, but not perfectly?" is a great accidental mini-intelligence-test. Surprisingly many people fail it.)
That the impact of IQ should scale linearly seems not necessarily true. It could scale exponentially or it could scale sub-linearly. Suppose IQ amounted to time required to solve a problem. As you progressively solve the problem in half of half of half the time, it becomes far less significant. On the other hand, if it were additional dimensions of complexity one was capable of holding in one's head at a time (it isn't) it would scale far more than linearly. My feeling, is that it IQ is either related to the trainability of "System 1," or just to a readiness to notice parallels or analogies in new learnings while quickly dispositioning them as either valuable or not. (or possibly, that is what I have most efficiently trained my "System 1" to do....
I'm no IQ expert but my understanding was that an IQ score of a particular number is defined by reference to where it is in the (presumed normal) distribution of IQ scores.
In other words, some random website tells me that an IQ of 140 means you are smarter than all but 0.3830% of people. They presumably got that number from the CDF of the normal function ... having an IQ of 140 *by definition* means that you are smarter than all but 0.3830% of people. It doesn't mean that you are 20 "brainpower units" smarter than someone with an IQ of 120, or something like that.
So I don't think this:
> a group with IQ 138 would be to college professors as college professors are to the average person
really means anything. The only thing that you can say about (1) the difference between 140 IQ people and 120 IQ people, and (2) the difference between 120 IQ people and 100 IQ people, is that the number of people *in between* the IQ levels you're comparing, in each case, corresponds to 20/15 = 1.33 standard deviations worth of people for a normal distribution with mean 100 and standard deviation 15 (which isn't even a set number of people).
"College professors" isn't the filter you think it is. A group with IQ 138 would be to the average person as high-end physics professors are to the average person.
"College professors" are extremely heterogenous and are often selected for political reasons rather than academic reasons.
Yeah, I also wanted to ask "which college are we talking about?", because some college professors are... uhm... not very smart.
Smarter than a random guy on a street, sure, but to find a group that is smarter than them as much as they are smarter than the random guy on the street... I think a group of students from a prestigious STEM school would probably be like that. Which sounds similar to Less Wrong readers.
😲
Something in your calibration feels so wrong that I have a problem finding words. Which sadly would make this comment quite useless, so here is my best attempt...
For starters, perhaps don't think of *STEM* colleges. Instead, imagine the kind of people who read "Transgressing the Boundaries: Towards a Transformative Hermeneutics of Quantum Gravity" and think: "wow, this guy is so smart", because they are too stupid to understand that they were just reading word salad. Because their own best attempts to produce science are not too different.
Or maybe just don't think of the *elite* colleges, and consider the average ones, which are more numerous. I have met a math professor who didn't understand math, and who only gave good grades to students who solved the problem exactly the way it was taught at his lecture, because if it was solved slightly differently, he couldn't tell whether the solution was right or wrong. I have met a professor of economics who couldn't tell whether "10, 12, 14" was supposed to be a rising or a falling line on the graph.
IQ 120 basically means: has the ability to talk about complex topics. "Talk about" does not imply deep understanding. IQ 130 is Mensa. Less Wrong is definitely much smarter place than Mensa.
I guess you probably know a few very smart college professors who are far from being typical representatives of their profession.
> imagine the kind of people who read "Transgressing the Boundaries: Towards a Transformative Hermeneutics of Quantum Gravity" and think: "wow, this guy is so smart",
Well, the guy who wrote that paper is in fact pretty smart. /jk
(I know what you mean, and I agree with you.)
That doesn't sound so unreasonable. Consider that:
1) Professors are also selected for ability to focus intently on research, which is a different ability but also contributes to perceived formidability
2) When we interact with professors, it's usually in their field of specialization
3) That's the average of all college professors. When we picture a professor, it's probably from a top school and a more g-loaded department
Wjem I compare academic articles and lesswrong posts, I expect the articles to show more effort and diligence but the lesswrong posts to show better thinking.
Googling suggests that physicists average about 130. I don't know if this helps or not.
Hm... this is just my n=1 anecdote, based on being a professor in STEM, having taken the WAIS-IV in adulthood, joining MENSA, joining the Triple Nine Society, and attending SSC meetups over the past few years. But in my impression, the typical SSC meetup member is absolutely smarter than the professors and MENSAns. I say this based on the depth and breadth of conversations, the "polymath" abilities, and the interest in deep dives on obscure topics. People I know who are professors, doctors, or lawyers, generally spend their free time on typical leisurely things like golf or gossiping at the country club, and seem to think people who enjoy philosophy or statistics or deep-dives are weird. They do not seem to have explored subjects beyond what they were directly required to learn from a textbook. At work, I feel like I know significantly more about my subject, and consider nuances and connections, more than other professors. At MENSA nights, I felt rather bored and dreary. But I always feel intellectually stimulated, invigorated, and challenged at the SSC meetups. This is only my impression, but, yes, I do think the difference between the intense community here and the professors I work with is absolutely as large as the difference between professors and the average person. *Perhaps* the difference between typical "smart" people and the community here is not entirely due to intelligence, but also due to the personality trait Openness? But overall I would not be at all surprised if the average IQ of readers here is 130-138. In fact, I'd be surprised if it were lower. It sounds like Scott's main reason for re-scaling the self-reported scores is that the reported results seem "implausibly high" to him. But I can say with certainty that I have never talked with someone outside of SSC readers who seemed interested in (or even aware of) the types of topics investigated here. The idea that readers of this site are in the top 1% of the IQ range seems very likely to me.
Thus we can only return to the status quo of using social status and citations as proxies for measuring extreme intelligence.
Whenever I come across voluntary surveys of IQ, David Mitchell's rant about who tries to become a member of MENSA and who does not, comes to mind. It's about selection effects. Not particularly relevant in this context perhaps, but it is a fun rant, as are all of David Mitchells' rants:
https://www.youtube.com/watch?v=qPMKqyaXtHI
I made one IQ test in my.life where I scored "below 130" (don't remember the exact number, since 130 was a cutoff for something) and one, later, with 135.
If I am asked for IQ number, I now report 130.
So for ppl who did multiple IQ tests, we have an upwards bias (even if asking for the most recent one, because people redo it more likely if they're unsatisfied with the last), whereas "doing live testing" gives us an unbiased snapshot, including things like "I slept bad and my coffee machine broke and my brother is dieing" unconcentrated for some)
I sometimes wonder: what would be the mean IQ if everybody was allowed to submit the best test result throughout their lives? 120?
That depends on how many tests people take. Your measured IQ is equal to your true IQ plus measurement error, generally conceived of as normally-distributed noise. If the noise is normally-distributed, there is no bound on its value, so with an infinite number of retests you can achieve an infinitely high IQ.
In reality, you're only going to take a finite number of tests, but it remains true that the more tests you take, the higher your high water mark is expected to be.
Do I know you?
Who cares. I understand that you collect this information for your own uses and that a lot of people take tests offered by the internet because they want to know how smart they are but we don’t ask people what their IQ is and we suppress laughter when someone volunteers that information.
Nor does it predict success in life—life being more than professional attainment or wealth. Of course admission to elite universities requires greater facility with mathematics and language. And physicians and engineers are all pretty smart—but to me here is the key question. Two surgeons will have the same IQ but one will be outstanding and the other a disaster. A doctor with a lower IQ than their colleagues may be the better diagnostician of the bunch.
Let’s move from technical professions to business and law. The ability to relate to all sorts of people is a far more valuable attribute than one’s IQ. In some instances and to some extent it may be inversely proportional.
I love your brilliance, but I think this post was dumb.
> I love your brilliance, but I think this post was dumb.
This would be a more compelling insult if you showed any awareness of any of the standard counterpoints to the myths you repeat, or even any awareness that there *might* be counterpoints that you are unaware of but Scott is and why he doesn't appreciate your stunning insights like "did you know some doctors are better than others but may not be smarter?". As your comment stands, it mostly shows your inability to relate to other people.
Gwern, I think Scott got the joke: my calling an essay on IQ, dumb. Besides, who am I for Scott to feel insulted by any opinion I offer. But to answer your critique of my comment, a few observations.
First. What I like about Scott’s articles is that he respects his audience and doesn’t dumb down his discussions. But his Substack is subscribed by numerous ordinary people like myself who find his analysis of ideas and issues cogent even if we can’t always follow his math. Unlike you I do not presume to know whether that was his intent when he created his blog, but I suspect that he knows a great deal about those who read his posts.
Second. My objection to his discussion is that he doesn’t relate his analysis to anything in the real world. The only thing he says is why he collects this information. If his post was directed only at specialists, he should have published it in an academic journal.
Finally. If there are counter arguments that undermine or refute my comments, educate me. I don’t have any problem with someone explaining why i am wrong.
Jon May
>My objection to his discussion is that he doesn’t relate his analysis to anything in the real world.
Most people here know and understand the sigificance, to some degree, of intelligence.
> And physicians and engineers are all pretty smart—but to me here is the key question. Two surgeons will have the same IQ but one will be outstanding and the other a disaster. A doctor with a lower IQ than their colleagues may be the better diagnostician of the bunch.
Hey, guess what? IQ correlates extremely strongly with job performance. More strongly than almost anything else. Expecting a 100% correspondance between two things is simply an isolated demand for rigor (something you should be well aquainted with as a reader of Scott's). There's almost no non-trivial 100% correlations between things, and if we threw out anything less than a 100% correlation then science as we know it would not exist.
>Two surgeons will have the same IQ but one will be outstanding and the other a disaster. A doctor with a lower IQ than their colleagues may be the better diagnostician of the bunch.
Don't call people "dumb" when you lack the basic ability to think about things statistically.
Is it possible that a lower IQ doctor is better than higher IQ doctors? Certainly.
Is this important? Only if we there isn't a strong *correlation* between job performance and IQ, such that predicting a person's performance from their IQ wasn't possible. In reality, it is very possible and without knowing anything else about your doctor, you always want a higher IQ doctor over a lower IQ doctor.
It's like saying height is irrelevant for basketball because some short people are great at basketball. Muggsy Bogues at his peak was better than almost everyone on earth at basketball despite being only 1.6m tall. And yet, if you bet on whether the much taller than average kids at a school would be more likely to make it to the NBA than the shorter than average kids, you would be a fool not to bet on the taller kids.
Some notes on Mensa:
Mensa advertises that 2/3 of people get in with “prior evidence” and don’t have to take the Mensa test. They also only accept SAT scores older than ‘86 IIRC.
I took the Mensa test about 15 years ago and they also administered the Wonderlic beforehand. Possibly for renorming. When I received my results it was 130+ (or maybe 132+) because that was the limit of the test. I would be suspicious of anyone claiming to get a very high score on the Mensa test.
Non Mensa related: I found a bootleg copy of Ravens and it also capped out in the low 130s. There’s a separate step 2 if you want to test higher.
"They also only accept SAT scores older than ‘86 IIRC." Yes, it's easier today to get a higher SAT score, so the date you took it matters as to the correlation with IQ.
"They [Mensa] also only accept SAT scores older than ‘86 IIRC. "
Early 1994.
https://www.us.mensa.org/join/testscores/qualifying-test-scores/
Most college acceptance tests are no longer accepted. This matches my understanding that the folks making these tests have been consciously making them less g-loaded (e.g. removing the analogies portion of the SAT verbal). But it is interesting to see Mensa spell it out without *quite* spelling it out.
Given the community here I might get a hit on this and have always been fairly curious as to the experience of others: Anyone else get wildly different math and verbal scores? I was put in special education the first week of the first grade because I had a lot of difficulty making myself understood. I used to put that all on some panic stuff I had going on but now I’m less confident that was the biggest reason. Even after I eventually learned to translate my head universe into communicating with other people there was still a pretty significant gap. If I’m reaching back I think I got a 35 in the math section of the ACT and maybe 29 in verbal? I’m always able to look at something and figure out how to fix it much, much quicker than I’m able to figure out how to explain how I know to someone else.
Yeah, if I remember right, my ACT was 22 math, 30 science, 32 English, 36 reading (this was 1994.) I'm not sure that reflects any "natural" ability at math I might have; math just wasn't interesting/didn't click for me in high school. It's funny; I was skipped ahead a grade while you were in special ed, and you were probably brighter than I was. Shows how weird the system is.
I'd probably do quite a bit better on the math now, assuming I studied for it. Math "clicks" for me a lot better now. Not that I could jump in to calculus or anything, but all the algebra and trig stuff I missed I could probably pass now. At least going by how able I am to help my kids with their high school algebra homework.
The combination of high math and low verbal reasoning scores is a pretty common trait in autistic people, probably also reasonably common in people who might have a dyslexia diagnosis or people who have subclinical traits in that direction. My personal gap is processing speed - I completely maxed out a couple sections of the last IQ test I took, but my processing speed is dead-nuts average.
ACX/SSC-only people often won't get this, but in Lesswrong and AI safety-oriented culture, being merely one-in-300 is pretty terrible news; it means you were the smartest kid in almost every classroom growing up, but now you are median, and the people doing the lion's share of impact on Earth are the one-in-10,000 types.
Why would anyone read your post/paper at all, or hire you for their org, if they can spend that time reading/hiring a one-in-10,000 person instead?
Because even in intellectual endeavors, there are other things that matter than pure intelligence. Other traits, like creativity, ability/willingness to look at one’s own (or one’s social circle’s!) ideas critically, to slog through lots of boring parts… just for three top-of-the mind examples from a probably very long list. Not to mention domain knowledge, including deep knowledge that one can only get from experience or by having an unusual degree of specific intrinsic interest… I’m more likely to read someone’s work because it’s *different* than other peoples’ than because it’s doing ‘the same thing, but better.’
Of course, let's ignore the part where these things are highly correlated with intelligence.
Because there are a pretty limited number of 1-in-10k people in the world and everyone wants them, so they're probably out of your budget.
Is this not likely to be reporting bias/selection effects?
- If you think you are average will you bother getting an IQ test?
- If you did a test and got an average IQ are you likely to fill in a form about it or will you only bother if you scored high and want to boast a bit?
So it could be that only the people who suspect they are in the highest rungs will have had and IQ test, and of those the ones who scored highest are more likely to complete the surveys as a boast and you get very skewed selection effects
I had a tested IQ of 138...when I was 10, on the WISC. This raises three obvious lines of questioning.
1. Does age-adjusted IQ at 10 correspond to an equally high adult IQ, and older adult IQ?
2. IQ declines over time, probably due to decades of substance (ab)use, head injuries, illness, and most importantly general aging. How does that skew the data?
3. Would someone with an IQ within a standard deviation of the mean be more or less likely to be sent to a fancy psychiatrist in the inner city to take the WISC/WAIS/etc.?
I think I'd expect to get 115 to 125 on the WAIS as an adult. My memory is shot, I'm slower than I used to be, and I struggle with some mental tasks I used to find easy. I remember getting somewhere around 120ish on the least-disreputable online IQ tests. It's not clear to me that "my IQ was X when it was tested" means "my IQ is X" and I think time is likely a bias.
It would also be interesting to see if the WISC was unusually biased compared to adult tests, and interesting to collect data on age of measurement. Maybe everyone has a higher tested IQ because most of them were either taking an age-adjusted test as a child, or in the 18-25 age range when their IQ should have been at its highest point.
re: " My memory is shot, I'm slower than I used to be, and I struggle with some mental tasks I used to find easy."
I also wonder if many notice this as they age? I felt my mind was fairly quick and strong up through about age 30, but started noticing a subtle dropoff after that, and now in my 50s I can without hesitation say my mind does not have close to the "horsepower" it used to have. I guess it's possible I'm assessing wrongly and misinterpreting past ease/difficulty, but I'm more than 50% on this.
Your experience matches my own impression perfectly.
In addition, I had a near-eidetic memory until about age 30, and I was interested (and dismayed) to start to experience this 'forgetting stuff I meant to remember' thing that others talked about.
I tested out at around 135 when I was younger but would bet dollars to donuts that I'd test lower than that now.
I also feel that I take longer to learn new (complex) ideas and processes. But on the upside I also feel that I draw more widely from other disciplines and can see functional implications and outcomes faster and more accurately.
Part of the problem here is
Psychometricians noticed that scores on a lot of different types of cognitive test are correlated. The general intelligence factor, g, which IQ tests are intended to measure, is the correlated component of those scores. But it turns out that the relationship between scores on different types of cognitive tests is heteroskedatic— they become less correlated as you go further up the scale. This also means that if different assessments of g include different items, their correlations will drop at the high end. More profoundly, it implies that IQ-type metrics get less useful as predictors of performance on specific tasks after you get past a certain threshold. So, “reported IQs over 135 are less meaningful” is a good heuristic.
Have you considered just asking your readers to take a well-normed IQ test that directly reports the results to you? Like, I’m pretty sure you can just pay Wonderlic to administer an IQ test to whoever you want (I don’t know how well normed they are, but it’s got to be better than the existing situation).
SAT is a US thing. As an Australian I don't have one. I didn't read the article in detail but maybe that affected the scoring -- non-US-educated either not reporting or guestimating.
If you were to suggest that only the smartest 1/200 or even 1/10,000 people are the ones that read this blog/have taken Mensa tests/iq tests, I would absolutely believe you. The high level of reading / logical reasoning to both understand and enjoy this style of writing is such a strong filter.
I know we avoid complex language to make the blog more accessible to anyone, but to enjoy it is another story.
I’d assume based on my peer group that people with an average IQ, don’t read at all and mostly scroll tiktok. If they do read it is very simple books/novels.
In lieu of replying "the correlation between intellectual ability and intellectual curiosity is very weak" to everyone who says something like this, I'm going to tell a story about a case study I once read.
It was on a man with a particular genetic disorder, which is associated with a characteristic "behavioural phenotype" -- distinctive shared personality traits between many people with the syndrome -- and with often-severe intellectual disability, but he had been spared from the latter. He had his share of social and emotional problems, but was an avid reader from a young age, consistently performing well above average in language and literature-related classes at school. His reading choices trended towards the "Great American Novel"-types; his favourite authors were Faulkner (not a man known for his terseness or simplicity!) and Steinbeck.
His WAIS full-scale IQ was 85. His verbal IQ was quite a bit higher -- that is, 117 (his other scales were all 70ish). This is an extreme skew, but there are neurotypes where multi-SD skews above and below the mean are far less remarkable than in the general population (e.g. autism).
Even if we take "a verbal IQ of 115 or thereabouts is probably the minimum possible to appreciate Faulkner", which is an extreme reading of this (because intellectual interest is weakly correlated with intellectual ability!), you don't land somewhere that gets you an average FSIQ nearly as high as reported in these surveys. Ratsphere bloggers are quite a bit more accessible than most of Faulkner's work; I am unconvinced the level of verbal and logical reasoning needed to appreciate them is multiple SDs above that needed to appreciate him, particularly for posts that are more about generally-accessible subjects (e.g. politics) than technical or scientific ones.
(case study if anyone is interested is https://www.sciencedirect.com/science/article/abs/pii/S089085670963820X -- paywalled)
That’s the mast elitist, arrogant, snobbish comment i have heard today. Do you have any evidence based data to support your assertion.
It's snobbish to suggest that people who are more intelligence have greater intelligence? Interesting.
That’s not what I said.
And you know it.
This is what I was responding to:
“I assume based on my peer group that people with an average IQ, don’t read at all and mostly scroll tiktok. If they do read it is very simple books/novels.”
This is such an absurd statement, I can only assume bbqturtle was being facetious.
People with an average IQ don’t read at all?
Mostly scroll TikTok?
They only read simple books/ novels?
Just how much intelligence do you need to read and enjoy “A Take of Two Cities?” Or “For Whom The Bell Tolls?” Hamlet is only interesting to people with an IQ of 110 but not 100?
In 1950 there were only about 100,000 books in print. Today there are well over a million. Who’s reading all those books.
Ok, let us assume that these statements are not so absurd to be debatable. I only asked for some proof.
Of course, reading bbqturtle’s statement over again, maybe I misread her statement, maybe she is describing their peer group. In which case I would say, they’re hanging out with the wrong crowd.
To what extent do these "IQ" tests remain correlated with wide-spectrum achievement at the 130+ level? The scholarly paper has 116 students, there's no way there are enough data to base a claim that these tests are measuring general intelligence at the tails. Not a test-repeatability question but a test-to-IQ question.
Having spent my life in places where a 1400 SAT would be considered kinda low, there is clearly not an expontential-tail distribution where the 1500 SAT people are so obviously less intelligent than the 1600 people, even on specific things. And the "true genius" types are less likely to have general intelligence that you would call g, and instead seem exceptional in a particular area.
The concept of Bell-curve IQ describing general intelligence at genius levels should go IMO. Taking with it all the implications for godlike AGI.
My 1985 SAT scores - 710 math, 720 verbal - were good enough to get me into Mensa - 98th %ile - and Colloquy - 99.5th %ile - but not the triple 9 societies. I estimate my IQ as 3 SD up, ~140. A friend of my mom's, Stephanie Creamer, was getting her doctorate in industrial psych around 1980; consequently, I took A LOT of IQ tests, and usually hit around there, though my scores trended slightly upward as I kept practicing.
Parenthetically, the Raven matrices were the only times - standard and advanced - that I broke 150; Tanny was so awestruck she took us to Cafe Provencale.
>The Biggest SAT → IQ Conversion Site Is Wrong
Jensens point is very reasonable but Im not sure its right for your application. If you have genpop looking at an internet site to convert their SAT to IQ, you should use his calculation. If you have a sample thats selected from genpop for their SAT, then his calculation will also give their average IQ correctly. On the other hand, if you have a sample selcted for IQ, and then you measure their SAT, and then you apply his calculation, you will underestimate IQ.
So, if your survey sample is unusually smart, what you should do depends on why you think that is: If its selected mostly for SAT, use Jensens calculation. If its selected mostly for IQ, you need to do the opposite correction, and if its about in the middle, then the original percentile calculation is about right.
I left a comment about this same issue. I don't understand why the concordance is supposed to be wrong.
If you have a sample that's selected for their (above-average) SAT scores, and you measure their SAT scores, the new average will be lower. That's regression to the mean.
If you have a sample that's selected for their above-average SAT scores, and you measure their IQ, the new average will be lower. That's regression. Regression to the mean? That appears to be a philosophical question.
But if we're willing to use people's measured SAT score as a point estimate of their SAT score, why aren't we willing to use it as a point estimate of their IQ? If you want the true value, you need to do an adjustment in both cases.
"People who took the SAT but “don’t remember” their score have tested-IQ 104"
But...I don't remember my score. It was 45 years ago! I know it had to be above 1200 combined, because in those days California was legally required to let you into a campus of the University of California if your GPA was over 3.2 and your SAT was over 1200, and if you met several other requirements like X number of math, science, and foreign language classes. And I remember thinking "Great! UC is legally required to admit me, I can exhale." And indeed I did go to Cal.
But I don't remember my exact score.
Only tangentially related
I can’t find the hard copy of the Al Franken book I’m going to reference right now so I’ll have to paraphrase from memory.
Apparently one of the first questions Ted Cruz likes to ask when he meets a person is “What’s your IQ?” If the person doesn’t know he asks “Well, what is your SAT score?”
Franken goes on to say “I like Ted Cruz more than anyone else in the Senate does, and I hate Ted Cruz.”
I really like Al, he’s a funny guy and was a great senator IMO and his being driven out of the Senate was uncalled for, again IMO.
I still get his emails from him with the greeting “Dear person who still opens my emails.”
He still works for Democratic candidates and sells merch on his web site to that end. One item is a coffee cup with a pretty good caricature, done by Al, of Cruz on one side and Al’s withering opinion on the other.
I have one. Ted Cruz rubs me the wrong way too.
I'm curious how you think your analysis here works with your viewpoint here? https://www.astralcodexten.com/p/selection-bias-is-a-fact-of-life
Given that IQ kind of -is- a rank order question, I don't think the problem #3 is the problem you are presenting it as.
This was a question from the 2022 ACX Survey:
"SAT score verbal/reading
The SAT is a test Americans take before going to college. In the old days, it included a section called Verbal; now that section is called Reading. If you took the SAT, what was your score on this section?"
Several dozen respondents put in a value >1500.
Am I the little the only one here who took the ACT?
They're not exclusive. I took both.
I'm a little disturbed at the claim above that several dozen people asserted their score on the verbal section was 1500 or more.
Inaccurate statements on the internet? Not terribly surprising. I just shrug.
When I first read the draft of this article, I was impressed by how far Scott could adjust downward the IQ scores without resorting to calling the people liars, but I still think that the drop in correlation above 135 is probably mostly due to people lying. They either lie big or don't lie at all!
I thought the SAT question on the survey might serve as a test: one would expect that fewer people would input a very high incorrect score (because their reading comprehension is better), but that if liars lie big, then the opposite would happen—more people would input an incorrect high score.
As a rough test, I decided to divide the scores between 801 and 1600 by 2 and compared them with the other scores by 10-point intervals.
I then looked at what fraction of people answered incorrectly. Unfortunately, this didn't tell me much except that there's no clear trend to my eye.
> I still think that the drop in correlation above 135 is probably mostly due to people lying
There's always a drop in correlation whenever you restrict a variable.
> one would expect that fewer people would input a very high incorrect score (because their reading comprehension is better), but that if liars lie big, then the opposite would happen—more people would input an incorrect high score.
This doesn't make sense; the motivation for claiming an impossible score cannot be that you're trying to fool someone.
>>This doesn't make sense; the motivation for claiming an impossible score cannot be that you're trying to fool someone.
I see that this was not as clear as in should be. I am restricting the statement only to people who answered the questions in a rage of 801 and 1600. By very high incorrect score I mean >1500 as compared to, say, 1200.
Edit:
Let me restate it.
If someone answers >800, I assume they have put in their total score instead of just the verbal score. I think people who are good at reading comprehension are less likely to make this mistake.
So if we compare the ratio of the number of people who answered 1600 vs 800 with the ratio of people who answered 1200 vs 600, I’d expect the 1600 vs 80 ratio to be lower. As the scores get higher, the number of people making this mistake gets lower.
But, if people are lying about having a very high score, then that ratio would be artificially high at the very high scores.
"I'm a little disturbed at the claim above that several dozen people asserted their score on the verbal section was 1500 or more."
I went to high school with (at least) two people who scored above 1500 on the SAT before the 1994 "re-centering." If we adjust scores to the post 1994 scale then I know a few more who could clear 1500. This isn't several dozen, but it is at least four. From a pretty small peer-group (I know lots more people but don't routinely exchange SAT scores).
Why are you disturbed?
SAT verbal maximum is 800.
That you.
I missed the "verbal section" was 1500 or more and read it as Math+Verbal.
Yeah, 1500 for verbal doesn't work :-)
It does if you're math score is very low, HO HO HO
No. At the time and in my part of the country the ACT was the only test that mattered for college entrance in my region. Scores on tests also seemed to matter much less than they do now. For my friends they were more of a hurdle - "hey, if you want to go to college go to the city on a certain date and take this test." Some of us got lost on the way. ;-)
You are reminding me of a classic Far Side cartoon. The little fella trying to enter ‘Midvale School for the Gifted’ is pushing strenuously on the entry door clearly labeled ‘Pull’.
I didn't take the survey so maybe the question accounted for this, but when I took the SAT there were separate reading and writing sections with a maximum of 800 each, for a maximum possible score of 2400. If Scott *only* asked about reading, and didn't ask for separate writing scores, it's possible people were simply adding their reading and writing results together for a total verbal score, which could easily have exceeded 800.
Could be
https://www.astralcodexten.com/p/ssc-survey-results-on-schooling-types/comment/12018048
"even official IQ tests are gobbledygook over 135"
This.
How do you make a test for someone smarter than you are? At some point, you're running into the doomsday scenario for AI, where it runs rings around you.
I assume IQ test makers tend to be smart people. But how smart, and how many of them are there?
> How do you make a test for someone smarter than you are?
1. You find the smartest people in generation i to write the test for generation i+1. "What interview questions do they ask at Google?" (back in the Old Days)
2. You write lots of questions, and determine empirically which ones are hard and which ones aren't.
3. You spend way more time than they get to spend.
4. You have a team of specialists, each of whom writes questions for their specialty, and then you combine them for the general test.
5. You find competitive scenarios and do relative rankings; you don't need to be as clever as the best chess-players or go-players in order to invent chess or go.
How do you find the smartest people in generation i? Get them from the ones who wrote the test for generation i-1 and pay them enough to write more tests as opposed to the thousand other things they could be doing with their time.
"Determine empirically which ones are hard". I guess that's by testing a lot of people with them and seeing which ones get solved least? I suppose you would have to keep up with algorithms in a number of fields.
How does time relate to IQ? How do you calibrate?
And yet, there's some evidence that as you get smarter, it tends to be in one or two fields, rather than generally.
I think that last is probably about all you can do: determine relative ranking. Sticking a number on it is more grading based on those who take the test and then curve fitting, which gets worse as you march further out on the right hand side.
And yet, IQ, like height, cannot actually be a bell curve, as it is necessarily anchored on the left. You can't have negative IQ, like you can't have negative height.
> like you can't have negative height.
Sure, but this is not relevant to the bulk of the distribution for height, and I don't think it's relevant for IQ either.
[The choice to force IQ to be a normal distribution--instead of lognormal or uniform or w/e--seems like a good guess to me, but the sort of thing that needs to be empirically justified, and I think doesn't really hold up. Height is more complicated than a normal distribution in inches, and IQ probably is as well; sadly there's not an equivalent to 'inches' for intelligence.]
>How do you make a test for someone smarter than you are?
Easy. Writing questions is much, much easier than solving them. I can create a very complex pattern progression by changing some element or elements in each subsequent picture. But if I were presented with this progression without ever seeing it before, it does not mean I would have any hope of identifying what the progression is based on and what would come next in a reasonable time frame.
I can generate all kinds of questions in this way, then test them on enough people, and if most people can answer it correctly then you've made it too easy, if nobody can answer it you've made it too hard and so on.
Or to use a completely trivial example - I could make an arithmetic test that would be challenging for people much smarter than me. I could use a calculator to solve sqrt(5,721) and then ask people to solve it without a calculator and compare it to the answer I got from a calculator.
I think that your method of considering the distributions could use refinement. If one just assumes that nobody below 100 IQ is involved in reading less wrong then the new mean for the half-normal distribution is 112 100+(sqrt(2)*sigma/sqrt(pi)). Essentially, eliminating a significant bottom portion of the distribution substantially skews the mean without the need to assume an inordinate number of high IQ individuals.
This article is especially fascinating to me because there needs to be another possibly: testing bias(?)
I'm old enough that I've taken: the GRE, GMAT, SAT and ACT. My respective scores were 2200, 780, 1410 and 34. I also took the Wunderlic for a job and scored 49.
So this should mean I'm some sort of brianiac eh?
Nope. When my IQ was actually tested - in a rigid setting - it came out as 111. Not dumb but definitely just... bright normal, which I think is the actual term ascribed to me.
So what gives? Outliers exist but I am highly skeptical that I'm the only person out there who aced a bunch of standardized tests only to actually be told: hey, you're bright but not exactly Einstein
Should crowdworkers come in at a little above 100 because people with really low IQs aren't likely to be crowdworkers?
" the ClearerThinking survey used crowdworkers (eg Mechanical Turk) who should be totally average"
Why would you expect people who seek out online crowdwork to be "totally average?"
> It looks like up to about 140, self-reported IQ and normed IQ rise together, and then the relationship breaks down. Sure enough, looking at the subset of self-reported IQ scores below 140, the correlation with tested IQ rises to .6, and looking at the subset above 140, the correlation is nonsignificant at -0.02. I don’t want to assert that the breakpoint is exactly 140, but I do think the test stops working somewhere in the 130 - 140 range.
Didn't ...you... already write a post about this being an illusion? Pointing out that the reason for a weaker correlation between IQ and salary in the subset of people with very high IQs than overall is simply that there is no other possibility, and that in fact in a model where we know the relationship between IQ and salary is constant, we observe exactly the same thing? Why is the same argument that's invalid as applied to IQ-vs-salary valid as applied to IQ-vs-IQ_in_the_past?
------
I think I've seen Emil Kirkegaard make the same point as in #1 - that you shouldn't convert SAT scores to IQs on a percentile-to-percentile basis, because the correlation between them is only the stylized 0.8 that's expected between two different IQ tests, and not 1.
But I don't really understand why this is an argument against doing the percentile-to-percentile conversion. The correlation between the Wechsler and the Stanford-Binet is also a stylized 0.8, because they're both IQ tests. But we don't go around saying that measured Stanford-Binet IQs are incorrect (they're correct by definition!) because they need to be regressed by the correlation to Wechsler, which is the True Arbiter of IQ, and we especially don't simultaneously say that, even though Stanford-Binet IQs are too high because in reality they need to be regressed by the correlation to Wechsler, Wechsler IQs are 𝗮𝗹𝘀𝗼 too high, because they need to be regressed by the correlation to Stanford-Binet!
We do say that if someone has a high measured IQ, their actual IQ is most likely lower, because of regression to the mean. This also applies to SAT scores, which are the same thing. But that argument suggests, to me, that percentile-to-percentile is the 𝗿𝗶𝗴𝗵𝘁 way to do concordance. What am I missing?
IQ is a normalized distribution so for every person with an IQ of 140 there is a person with an IQ of 60. Someone with an IQ of 60 is not going to be taking these kinds of quizzes while someone with an IQ of 140 is very likely to. I’d even expect people with an IQ of 100 to be very unlikely to be interested in these kinds of surveys. If you cut off the left tail of the IQ distribution, of course your average will be higher than expected.
I think that if people take IQ tests, they take them as kids because they're being tested for "are you really bright/is something going on as to why you're not doing well in school and/or socialising with your peers?"
And thus I think people as adults may be reporting child IQ test results, which I don't know if one is related to the other accurately. Yes you were very smart for your age when you were twelve, but are you still very smart for your age now you're thirty?
Anyway, I cling proudly to my Completely Reliable, Honestly! Online IQ Score result of 98 😁
I do think it's fairly accurate because it was based (they said) on Ravens Matrices and I am *hopeless* at pattern recognition, shape rotation, or maths. Given that the tests were all pattern recognition, shape rotation, and maths, this result seems legit to me.
I do wonder what it's like to be able to look at a sequence of numbers and *immediately* see that "The pattern is X, Y, Z so the next digit should be A". But I'll never know, no more than I'll ever know what it's like to be a bird or a whale or a sloth, happily clinging to a branch in a South American jungle as the lichen grows on my fur because I am so inactive.
This makes me think that we're moving to a new age of calculating "intelligence", where now we're in the Information Age of AI And Computers, it's all based on mathematical ability rather than verbal. We're moving between C. P. Snow's Two Cultures, where before to be considered 'smart' was judged on the basis of your cultural and literacy attainments, but today it's based on "can you even code, bro?":
https://en.wikipedia.org/wiki/The_Two_Cultures
" 'A good many times I have been present at gatherings of people who, by the standards of the traditional culture, are thought highly educated and who have with considerable gusto been expressing their incredulity at the illiteracy of scientists. Once or twice I have been provoked and have asked the company how many of them could describe the Second Law of Thermodynamics. The response was cold: it was also negative. Yet I was asking something which is the scientific equivalent of: Have you read a work of Shakespeare's? I now believe that if I had asked an even simpler question – such as, What do you mean by mass, or acceleration, which is the scientific equivalent of saying, Can you read? – not more than one in ten of the highly educated would have felt that I was speaking the same language. So the great edifice of modern physics goes up, and the majority of the cleverest people in the western world have about as much insight into it as their Neolithic ancestors would have had.'
...Snow's Rede Lecture condemned the British educational system as having, since the Victorian era, over-rewarded the humanities (especially Latin and Greek) at the expense of scientific and engineering education, despite such achievements having been so decisive in winning the Second World War for the Allies. This in practice deprived British elites (in politics, administration, and industry) of adequate preparation to manage the modern scientific world. By contrast, Snow said, German and American schools sought to prepare their citizens equally in the sciences and humanities, and better scientific teaching enabled these countries' rulers to compete more effectively in a scientific age."
I'm one of the dinosaurs of the Neolithic who is verbally able, but totally lost now in the era of you small nimble mammals with your maths manipulation abilities.
>This makes me think that we're moving to a new age of calculating "intelligence", where now we're in the Information Age of AI And Computers, it's all based on mathematical ability rather than verbal. We're moving between C. P. Snow's Two Cultures, where before to be considered 'smart' was judged on the basis of your cultural and literacy attainments, but today it's based on "can you even code, bro?":
That's not how IQ tests worked then or now.
I'm not speaking of IQ tests, I mean as Snow puts it, people who were regarded as well-educated, cultured, successful and hence bright, had no idea of basic scientific facts or principles.
Nowadays we're swinging more to the "Shakespeare was over-rated" side where the good jobs are for the smart people who can manipulate numbers in some manner.
I feel like income is a much better proxy for what we value in “intelligence”. Obviously Jeff Bezos isn’t necessarily smarter than Scott Aaronson but they’re definitely in the same league or else Bezos wouldn’t have been in the position he’s in today. This is perfectly captured in the “if you’re so smart why aren’t you so rich” quote.
So… isn’t it better to just ask for peoples incomes, stack rank them against people of their age and place of residence and call that “IQ”?
Do you think Taylor Swift is in the same league as Scott Aaronson?
Yes, definitely. There are hundreds of thousands of people capable of composing music and singing as well as Taylor Swift. There are hundreds of people with a #1 song. There's only a handful of people with a successful career in music spanning for more than a decade.
She definitely has better looking legs in my opinion.
Funny thing to say about that -- Aaronson, famously, has "an IQ of 106".
https://scottaaronson.blog/?p=3473
(This is, per the post, due to a particularly extreme pattern of subtest variance in early childhood.)
I think there's a typo in your last paragraph - 1/30 people don't have an IQ above 128 (z-score of 0.9974). I think 1/30 is roughly the number for a score of 120 (z-score of 0.9772).
Why even bother with some IQ test of adults who are gainfully employed? Of what value is it other than perhaps for self gratification for some who see via internet test/survey that they score high on an IQ test. Me: Could care less as having multiple degrees and financial and career success I find it all a bit silly. Yes, for the very young it might be of help for seeing those with very very high IQ get the proper degree of educational nourishment.
I think the only IQ test that matters is supposedly literate adults being able to know the difference between "could care less " and "couldn't care less".
Old eyes did not see what was typed; but thank you for being an A hole.
Hopefully that was typed clearly.
What life circumstances lead a cognitively normal person to get an IQ test? This seems completely foreign to me, like something I only hear about on the internet. The only real-life person I know who has gotten an IQ test is my older brother, who is has an intellectual disability.
I guess maybe for a job? I had to take the Wonderlic once for a job. But that's not exactly an IQ test, I don't think.
I’ve taken a few mental shape rotation and ‘What’s the next number in this series’ tests for programming jobs. That kind of makes sense.
Some researchers offered to administer IQ tests to members of my dorm for free and some members accepted.
Intelligence researchers pay people to take tests, and they want representative samples of the population.
I'm curious about this as well.
Many people wondering if they have ADHD or a learning disability go or are sent to a neuropsychologist for individual testing. Testing is also done if brain damage or brain disease is suspected.
Why do people hate Mensa?
Probably partially simple jealousy.
Probably partially because the people that go in for this stuff in a big way can seem a tad pretentious.
Just human nature.
>Probably partially simple jealousy.
I found a Mensa for beautiful people: https://www.beautifulpeople.com/about
Maybe you like this club, too, but for those of us who think Beautiful People is silly, do you think it’s also because we’re simply jealous? Or do you think it might be because creating highly exclusive social clubs that rigorously reject members based on a largely heritable trait is slightly off-putting?
I don’t see much difference between Mensa and Beautiful People. They both have every right to exist but it is a bit much to also expect the vast majority of us who are ugly and stupid to think they’re cool. And I can certainly appreciate why even many of the gorgeous and brilliant folks out there might not like such clubs.
¯\_(ツ)_/¯
Human beings. Who knows? Unless one is preternaturally self aware most of us don’t really know what drives our emotions.
A regular meditation practice helped me to get to the bottom of some of my own but not all of them.
There's lots of possibilities. I don't hate Mensa--I actually joined it for a bit and then never went to any meetings so my membership lapsed--but I do have ambient "lack of excitement about Mensa" that I think is related to the hate. My guess is almost all of it is secondhand.
Some guesses:
* Adverse selection. Smart people with friend groups don't need to go to generic meetings, but smart people without friend groups do.
* Directness. Smart people like to be around other smart people, and there are two ways to go about that: indirectly ("I'm a college professor and I hang around my colleagues") and directly ("I have a generic job but I'm a genius, and I would like to hang around other people who are also geniuses."), and a lot of people are down on directness.
* Implied lack of sociability to be able to cross intellectual gaps. (People might think the opinion "I don't want to be friends with people IQ 125 or lower" correlates with other things they dislike.)
* Sour grapes. About as many people probably thought they could get into Mensa and then turned out to have IQs in the 125-130 range as actually got into Mensa.
Because it's the 'High IQ underachiever club'. Most high IQ people distinguish themselves through the application of their intelligence to their work, not from the mere fact of having a high IQ.
People don't like status grabs.
> The LessWrong average was 1446, corresponding to IQ 140.
I think you're using the pre-1994 SAT correspondence, but most of the people responding to the survey probably took the SAT after 1994. As stated on that link, they made a change then that dramatically compressed the range; as a result, I don't think the SAT is that useful for assessing IQs above ~120.
Re. "the ClearerThinking survey used crowdworkers (eg Mechanical Turk) who should be totally average.": Why do you think that?
Mechanical turk workers have a lot of filters on them. First, they must have a computer and an Internet connection in their homes. In India, this is a big filter. They must be non-conformist enough to risk getting this weird new kind of job; that's another filter. Then they must apply for the job and pass. Then, once they're on the job, they compete for tasks, especially high-paying tasks. And your results might be enriched in mechanical turk workers who searched for that job because already knew about these surveys from being involved in LW or ClearerThinking.
As a child I tested 144 and my SAT was 1380. I can name poker hands and explain the infield fly rule but I know next to nothing about pineapples or how to build a table. People are too obsessed with these measurements, and I have a theory that that is holding back human progress.
I occurs to me that one way of establishing a norm could be another test a lot of people take: the ASVAB. It isn't hindered by the "smarter people take it more often" like the SAT. It probably is hindered a bit because it probably isn't taken by many people at the upper and lower bounds. The AFQT score from it is a literal percentile and the score most people remember.
As a data point , my AFQT was 99 and my SAT was 1400. According to the correlations listed above and the percentiles at iqcomparisonsite.com if AFQT is an accurate percentile, that apparently puts my IQ at somewhere between 135 and 137.
The first point you make, about regression, should apply equally in both directions. If SAT and IQ are correlated at .8, then using either one to predict the other should result in regression to the mean. So if IQ scores are higher than you should expect by regression from SAT scores, because they are at the same percentile, then by the same token, SAT score are higher than you should expect by regression from IQ scores.
More likely, both of these average reported scores are being dragged up by some third factor (which might be one or more of the other things you mention) rather than one of them leading and the other being dragged.
"and least likely of all to optimize their lies for getting IQ/SAT correspondences right."
That's comparing averages, not individual scores, so consistent with people getting the correspondence badly wrong but their errors roughly averaging out.
One possibility you didn't mention is people getting their IQ from a test done when they were children, with the score being literally mental age over physical age, not the norming of adult tests. The only IQ figure I have for myself was due to a fellow high school student copying the school's list of students and IQs and showing it around. I was at that school from kindergarten on, so the score could be me at ten and only weak evidence of my adult IQ.
I have no problem with all the discussion around the inaccuracy of test scores collected in a uncontrolled manner.
But also: is it really so hard to believe that the audiences for LessWrong and ClearerThinking have a selectivity of at least 1/200?
I would be surprised if it wasn't even higher.
The only time I’ve ever bothered to get my IQ officially tested was back when I was 18 - I was very interested in IQ and IQ tests, I practiced on all the online tests and similar games (anyone remember Brain Age on Nintendo DS?), and I prided myself on being “smart” a lot more than I do now, so I gave it all the preparation (nicotine!) and effort I could.
If I took the test today, I bet I’d score 10 points lower. This effect surely doesn’t account for a whole standard deviation, but I bet there’s some factor where people test their IQs when they’re at their “smartest”, narrowly construed. The idealized IQ test may be impossible to prepare for, and remain constant over a person’s life, but in real life that’s definitely not the case.
I don't really care. When I was well above fifty years old I got the result of an IQ test I did as a teenager and reported that here. After forty years of more or less moderate drinking, I should score significantly lower now.
Just so you know, internet iq tests can be terrible in the other direction. Once I took an obviously sarcastic IQ test on a humor site that asked a bunch of ridiculous questions and trick questions, then reported that my IQ was 7.
Gwern, I think Scott got the joke: my calling an essay on IQ, dumb. Besides, who am I for Scott to feel insulted by any opinion I offer. But to answer your critique of my comment, a few observations.
First. What I like about Scott’s articles is that he respects his audience and doesn’t dumb down his discussions. But his Substack is subscribed by numerous ordinary people like myself who find his analysis of ideas and issues cogent even if we can’t always follow his math. Unlike you I do not presume to know whether that was his intent when he created his blog, but I suspect that he knows a great deal about those who read his posts.
Second. My objection to his discussion is that he doesn’t relate his analysis to anything in the real world. The only thing he says is why he collects this information. If his post was directed only at specialists, he should have published it in an academic journal.
Finally. If there are counter arguments that undermine or refute my comments, educate me. I don’t have any problem with someone explaining why i am wrong.
Jon May
Sebastian's corrections to the SAT are wrong in your context.
If I sampled a random person in the population and told you their SAT was +2std above average, then yes, Sebastian is right that this shouldn't correspond to IQ 130 due to the imperfect correlation.
But if I first fixed a secret target IQ level L (maybe 120 or 130, you don't know it, let's say I picked it from a prior that's uniform(110,140)), and then I sampled people who have IQ levels close to L (say, L plus some error) and told you the average SAT score, you should very much NOT be regressing that down.
The second situation is basically your situation: you want to estimate the average IQ of a fixed group like LessWrongers, and that fixed group is expected to have IQ distribution which is centered at L>>100. You should not be regressing the SAT scores like that!
Also, I just realized that ClearerThinking's survey is the same one from which Spencer Greenberg posted this thread:
https://twitter.com/SpencrGreenberg/status/1767207372777308464
That thread became a meme due to how bad the IQ questions are. If you believe IQ is real and important, read that thread and see if you change your mind :)
(I do believe IQ is real and important, just less important than IQcels think. Even I had to do a double-take at those questions, though. It reads like parody. Check it out.)
Speaking of people underestimating how dumb the average person is, there was this opinion piece on CNN ( https://www.cnn.com/2024/03/19/opinions/zone-of-interest-holocaust-movie-rutland/index.html ) complaining how the movie "The Zone of Interest" (a Holocaust film) was humanizing Nazis and ignoring the suffering of the Jews. That alone is deserving of mockery for blatantly missing the point of the film, but then there's this paragraph:
> “The Zone of Interest” is rather tedious as a film. It barely has a plot, and the conversations and daily routines are repetitious. Several scenes will leave viewers confused, such as the one where Höss finds a jawbone while fishing in the river and drags his kids out of the water. I would not have known what was happening except I had previously read in a review that there are supposedly human remains being dumped in the river.
<
...I still can't come to terms with the fact that a human being actually wrote those sentences. Like, first he had to be stupid enough to not realize the obvious implications of human remains being in the river in a fucking Holocaust movie, then he had to be stupid enough to actually admit that he didn't understand that obvious implication without realizing how absurdly stupid it made him look. And this isn't just some guy CNN found on the street, he's a COLLEGE PROFESSOR. (And according to the editor, "an expert in the politics of contemporary Russia".) ...All of this is legitimately making me want to kill myself. How the hell am I the same species as this person?
Another stupid part:
"An audience member saw connections to the “Don’t say gay” law in Florida. But the Holocaust happened to the Jews, at the hands of the Nazis and their collaborators, and its specificity should not be diluted into a general meditation on the banality of evil. "
" Several scenes will leave viewers confused, such as the one where Höss finds a jawbone while fishing in the river and drags his kids out of the water"
What the hell? Isn't that a *normal* reaction? You brought the kids down to the riverbank for a nice day out, they're swimming while you're fishing, and the next thing you're pulling bits of human out of the water? Of course you're going to go "Holy mackerel!" and tell the kids to get out and get dressed.
Would Mr. Reviewer just calmly keep on fishing in the way of "I wonder if I can get the full set of pieces?"
EDIT: Okay, being kind, maybe Mr. Reviewer means "it was not adequately explained why there would be a jawbone in the river". But they do sound like they wanted a scene where the Bad Nazis backed a dump truck full of corpses to the water and went "Achtung! We will evilly dispose of the remains of the Jewish victims we evilly murdered, bwa-ha-ha! Because we're Nazis and Nazis are evil!" as they tipped the bodies in.
The only formal IQ test results I have are from a test I took as a child. Such tests are notoriously unreliable, and that was a long time ago, but if that's what you ask for, that's what you're going to get. I assume many others are in the same boat.
I am sure someone mentioned this, but I couldn’t find it in the comments: in 1995 the College Board did a major recentering of the SAT which moved Verbal scores massively. A 730V (looking in mirror) before 1995 was equivalent to 800 after recentering. 780 went to 800. So any mapping to IQ must specify which scales is used.
Be careful that you don't look too deep Scott, you wouldn't want to discover the IQ score at which people start selecting *out* of our community. ;-)
While I concede that IQ measurements are mildly interesting for social research, I for one can not fathom making a fetish out of one's IQ score. Given that getting a measurement of IQ is actual work, much more involved than measuring one's height or weight, I would argue that in most cases where people set out to measure their IQ there is some degree of fetish involved.
In my social circles nobody has ever brought up their IQ. If anyone did, I think it would be met similarly to them mentioning them the length of their penis -- incredulity at someone going through all the troubles of measuring that and then having the poor taste to mention it.
Personally, I don't feel that my intelligence is a bottleneck in most endeavors I might undertake. Real life is not like D&D where you can just win your battles by throwing INT-based fireballs. Take it from me, in real life, having dumpstats like Conscientiousness, Drive, non-Depression (same as Drive?), Communication and so on really limits what you can achieve, more like a few extra points of IQ would likely help. (Not that I would know, nobody ever cast Fox's Cunning on me.)
Jeez, every man I know well enough to discuss such things has gone to the trouble of measuring the length and girth of his penis -- but yeah, few talk about their results. I agree with you about disclosing IQ score, if one somehow has managed to get one. It's poor taste. It's tacky. If the score is high, it's extremely braggy and rude to tell people the number. So disclosing IQ is sort of like disclosing your net financial worth. If the number is low or moderate, people feel that it's just weird to disclose it -- and if it's high it's braggy. It think some of people's recoil from a high value in either area is because everyone would like to feel as though the number is not that important, but privately feels like it is. So hearing it kind of ties people up in knots. But I agree with you that in most endeavors, if you a bright enough to understand the field, then success depends less on whatever IQ points you have above the minimum and more on how fascinated you are with the field, how inventive, conscientious, competitive, likable, persuasive you are.
FWIW, I've never tried to measure my penis, though I did think about doing so sometimes when I was younger as people often talked about penis length.
Bryan Caplan's comment on X: "Very interesting piece on SAT --> IQ conversions. My own view is that the SAT *is* an IQ test. The fact that it only correlates at .85 with official IQ tests shows nothing, because that's roughly the correlation between different IQ tests, too."
As someone in the category where I've been administered exactly one respectable IQ test in my life and reported it on your past surveys, I'd suggest asking for "age when tested". I was tested as a child and there may be interesting effects with regression to the mean in adulthood, cognitive decline over decades, Flynn effect, etc., that you could examine if you're doing a large survey and ask for "age when tested" as well as current age. (Also I for one would state "WISC-R" if asked, but I don't know if asking for the test type would yield good data in general.)
And I suspect I'm in a common category. Probably nearly everyone who takes a respectable IQ test is being tested either by a school psychologist or by Mensa.
(Finally, I echo various people's point that "only the smartest people know the result of a respectable IQ test performed on them" is probably even stronger than "only the smartest people remember their SAT score". In normal life, there's no reason to administer an IQ test without some existing suspicion that the subject is either unusually smart or unusually dumb, but then dumb people rarely end up taking the ACX survey.)
There is no evidence, nor could there be any evidence, that IQ correlates to job performance if for no other reason than outcomes are determined by so many factors that such an evaluation could not be valid.
And what is the measure of performance? Certainly there are many occupations where a higher IQ is necessary. Anything involving the application of advance mathematics, or chemistry or physics. But performance? Just look at Boeing. When it was run by engineers and not by bean counters, doors weren’t falling off of planes. But I would bet that some of those bean counters have higher IQs than their engineer predecessors.
Selecting a doctor by their IQ would be more than reckless. I am sure you are aware that the best predictor of a successful outcome in surgery are the number and frequency of surgeries performed.
Consider an area I know something about. Expert witnesses. A prepared lawyer of lesser intelligence can destroy an expert of far greater intelligence in every field of expertise. Happens all the time. I’ve done it, and my wife can attest to the fact that I’m no genius.
How many truly smart people were taken in by Elizabeth Smart. In fact there is a real correlation between people of high intelligence and academic achievement falling victim to con artists. I know, I represent the con artists.
Show me the studies that support your contention and I will explain why they are flawed. Or maybe even made up as some very, very, very supposedly smart professors of medicine at the most elite institutions have been caught doing of late. So what is the measure of performance when people of high IQ achieve success by cheating. Pardon me if I hesitate to choose them over my University of Miami trained Internist.
> But I would bet that some of those bean counters have higher IQs than their engineer predecessors.
This would be unlikely. Engineering degrees are hard and business degrees considerably easier.
> Selecting a doctor by their IQ would be more than reckless. I am sure you are aware that the best predictor of a successful outcome in surgery are the number and frequency of surgeries performed.
Learning on the job is controlled for, regardless of IQ everybody has to do it. Nevertheless although you wouldn’t use an IQ test to select a doctor medical students have high IQs.
For what it's worth, TWO of the legit smartest people I have worked with were (by job description, in reality the way the filled their role was far wider) bean counters...
And it is quite rare that I feel clearly beaten in that dimension in real life.
What’s a bean counter in this scenario?
Here's a study: https://wminsk-shared-docs.s3.amazonaws.com/Schmidt_Hunter_2004.pdf
That said, starting off your comment with "I refuse to believe that evidence could exist" makes it... not very motivating to show you evidence? Like, if you insist on being wrong, I'm not going to stop you, but I _am_ going to view you with a mixture of pity and scorn.
Looking back at my earlier comment I don’t see any place where I said, “I refuse to believe that evidence could exist.” I did say there could be no valid evidence of a correlation between IQ and performance because there are so many other variables involved. The question is, does the Hunter-Schmidt article justify your pity and scorn? [If somewhere else I said that I refuse to believe…that would be asinine and I would be deserving of far worse than mere scorn]
The H-S study demonstrates a relationship between GMA and success. Success as indicated by positions attained, money earned, performance reviews. BUT, it also demonstrates a relationship between success and consciousness. [If you will permit me to subscribe dedication and hard work for consciousness, I think I can make my argument clearer]
Now, it seems to my lay mind, that we should be able to take IQ tests and predict which 10 year olds will have the highest incomes in a random a group of 1000 10 year olds. But it won’t tell us, which ones will actually be better at their jobs, because that is a function of their dedication to their job and the effort they put into their work.
So I concede that every kind of job requires a certain minimum IQ. But within that range, should a board of directors pick the candidate with the highest IQ to be the CEO. Doesn’t a host of other factors go into which person in the range will be the most successful at their task.
How did you intend me to interpret "nor could there be any evidence"?
On the main point: I think you have swapped out the claim "IQ correlates to job performance" with the claim "IQ is the same as job performance." The relationship they demonstrate between GMA and success is correlation. Does that mean the smartest person is the person with the best job performance? Only if the correlation is 1, and we know it's less than 1.
Moreover, variable A being correlated with variable B does not stop variable B from being correlated with variable C. If you're actually hiring people, you want to compare candidates based on as much information as you can obtai, which will include intelligence and conscientiousness and many other smaller features.
When I took a Mensa IQ test they told us two interesting things:
1. The test is most accurate around the 130 mark. It's what they're interested in and what it's optimized for. They said to not take low scores too seriously. (And it doesn't test above 145)
2. About 50% of test takers score above the cutoff.
I previously figured there was a selection effect, but I didn't expect it to be so large.
```even official IQ tests are gobbledygook over 135.```
Of course they are. It would be outright shocking if there were commonly available (or even uncommonly available) tests which could distinguish IQ at that range.
Suppose you have a group of 50 people. 30 of them have IQs of 100. 15 have IQs of 115. 4 have IQs of 130. And exactly one has an IQ of 145.
Your job is to sort them all into their correct group. All of them are available to you at all times, and will takes as many tests as you want, to the best of their ability, under ideal conditions every time.
Outside of this group you have access to arbitrarily many people of IQs 100, 115, etc. whose IQs are known to you, so you can norm the tests to them.
The best way to write the test is to have three groups of questions. Questions which an IQ 115+ reliably can answer, but IQ <115 reliably can't, questions that IQ 130+ can answer, and you get the idea.
But herein lies the problem. If you're not at least IQ 145, you can't write, or even *conceive* of a question that an IQ 145 person can reliably answer, but a less intelligent person can't. If IQ means anything at all, it means that.
So if you're an IQ 130 person, the best you can do is sort out the 100s, and 115s, and then everyone else is in the 130+ bucket.
There are ways to figure out through interaction that someone is smarter than you. And you could even figure out which of two people, both of whom are smarter than you, is the smarter of the pair.
But you can't write test questions that will distinguish the two, even under ideal circumstances.
You (the 130 person in this example) could write questions at the upper limit of your own ability and then administer a timed test with the time short enough that you yourself couldn't complete the questions in the time limit. Then you can rank the people smarter than you based on how many of the questions they complete within the time limit, on the assumption that if a question is really hard for you it's probably relatively easy for them (just as a question that really taxes an IQ 100 person would be relatively easy for you and you could answer it more quickly than they could).
There are all kinds of clever tricks you can do, but empirically, they haven't borne fruit, and I have a guess as to why.
Every trick you use adds noise, inversely proportional to how good the trick is.
For example, timed tests can't discriminate between people who would get the right answer given 15% more time, and people would who wouldn't get the right answer given arbitrarily more time, but I think we can agree the former is 'smarter'.
So we would expect most tests designed by ~130 IQ people to distinguish people smarter than themselves to have low reliability. And in practice, that's exactly what we see.
There's also the question of how many ~145 IQ people there are actually are, and what that means.
A typical 130 IQ person can do everything a typical 115 IQ person can do cognitively, and a fair bit more besides. But I strongly suspect that the typical 145 IQ person has deficits, and those deficits are likely to vary widely.
And there is no such thing as a typical 160 IQ person.
I think it's fairly uncontroversial to say that there is more variance in the 95th percentile IQ cohort, in terms of brain function, than in the 65th percentile cohort.
The mechanisms that produce this are fairly easy to figure out, if you take it as a given that the evolutionary optimal IQ is the average.
Has anyone mentioned the possibility that SAT scores may stop correlating with IQ above 135 not because the IQ scores are gobbledegook, but because teenagers with IQ above 135 are using their brains pretty differently than their peers, leading to all sorts of confounding academic and social issues, and this might lead to a wider range of SAT outcomes, including testing worse than a peer with an IQ of 125?
"It looks like up to about 140, self-reported IQ and normed IQ rise together, and then the relationship breaks down. Sure enough, looking at the subset of self-reported IQ scores below 140, the correlation with tested IQ rises to .6, and looking at the subset above 140, the correlation is nonsignificant at -0.02. I don’t want to assert that the breakpoint is exactly 140, but I do think the test stops working somewhere in the 130 - 140 range."
There seems to be an inferential step here saying that in general, if you reapply the correlation calculation after taking a cutoff, a decrease is indicative of the test failing to measure something past that point. This is not in general true, as even in the case of applying a cutoff to randomly generated data for at some fixed correlation, this will reduce the correlation coefficient of the remaining data (on average). This is especially true when the cutoff removes more of the original datapoints or the data is sparse past the cutoff, which just from eyeballing the graph, this seems to do to a great deal. It may be the case that the test fails around this area, but this calculation provides little evidence in that direction.
You can see this in the following python script:
import numpy as np
import matplotlib.pyplot as plt
from scipy.stats import pearsonr
def generate_correlated_data(n, corr_target):
X = np.random.normal(0, 1, n)
Y = corr_target * X + np.random.normal(0, np.sqrt(1 - corr_target**2), n)
return X, Y
def calculate_correlation(X, Y):
corr, _ = pearsonr(X, Y)
return corr
def run_and_plot_correlations(n, corr_target, cutoff, iterations):
correlations_before = []
correlations_after = []
for _ in range(iterations):
X, Y = generate_correlated_data(n, corr_target)
corr_before = calculate_correlation(X, Y)
correlations_before.append(corr_before)
idx = X > cutoff
if idx.sum() > 1:
corr_after = calculate_correlation(X[idx], Y[idx])
correlations_after.append(corr_after)
else:
correlations_after.append(None)
plt.figure(figsize=(10, 6))
plt.scatter(range(iterations), correlations_before, alpha=0.6, label='Correlation Before Cutoff')
plt.scatter(range(iterations), correlations_after, color='red', alpha=0.6, label='Correlation After Cutoff')
plt.xlabel('Iteration')
plt.ylabel('Correlation')
plt.title('Correlation Before and After Cutoff Across Iterations')
plt.legend()
plt.show()
n = 100
corr_target = 0.6
cutoff_value = 2
iterations = 100
# Run da sim
run_and_plot_correlations(n, corr_target, cutoff_value, iterations)
Note to self, add an additional 10-15 points when giving my IQ to Scott. And if he leaves a comment saying he will subtract an additional 15 points in my case, I will need to add 15 more. In fact he may read this and say nothing, so it's best to add another 15 just in case...
People today are such dummies. They can't even start fires in the wilderness from scratch, navigate by the stars, grow their own food, or memorize and recite epic poetry like Beowulf or The Odyssey. All this newfangled technology, like this "writing" thing, is making morons of us all.
What's wrong with assuming averages of 130s or even 140s? Have you ever been on the forums of Mensa, Triple nine, or any other high-IQ socities? If you read the kind of thoughts people have on there, you will probably be disappointed. The comment section here is way better. I don't think we're overestimaing IQ scores, I think we're overestimating how intelligent people with, say, 130 IQ actually are.
You're too humble. How many people have IQs one or two standard deviations above you? If this is the middle of the pyramid, then where the hell is the top? Can anyone name a single book written by somebody "truly intelligent"?
Another point re: SAT score / IQ conversions.
It's pretty common practice to take the SAT multiple times. In fact, not only that, but colleges allow you to submit your best score from each section, even if those sections are across multiple attempts.
Let's say Bob took the SAT twice. On his first attempt he got verbal 600 and math 700. On his second attempt he got verbal 550 and math 720. He would apply to schools with a 600/720. What score do you think he's likely to give when he knows someone is using it to estimate his IQ?
I actually think the percentile-matching thing is potentially reasonable!
On the one hand, if you trust the SAT scores, think your readers are otherwise typical, and want an estimate of the IQ scores, the regression thing is correct. But ACX readers are probably not filtered strongly by SAT, with no additional selection for IQ.
On the other hand, if you want to test the hypothesis "ACX readers are as high-IQ as they claim and otherwise typical", in a sort of Bayesian way, by computing implied SAT scores and comparing to reported scores, you should actually do the regression the other way, which (I think) produces implied SAT scores *lower* than reported, and makes the IQ reports seem easily believable or even an underestimate.
And if you want to test the hypothesis "ACX readers are selected for both high IQ and high SAT", with similar amount of selection for both, matching percentiles is at least halfway reasonable. (One version of this hypothesis: readers are selected for high g, and IQ and SAT are two different noisy measures of g with similar noise. But in that case, I'd guess SAT is noisier and we should assume ACX readers are higher-percentile on IQ.)
I'm not really sure what you should actually do here, though, except note that "ACX readers are selected for some factor-of-intelligence which is independently correlated with both IQ and SAT" is consistent with the numbers.
(At the very least, if a population reports being comparably high-percentile on two different tests, you probably shouldn't immediately assume they're lying about one of them!)
On the gripping hand: the main use case for that website -- "I know my SAT score, how high is my probable IQ?" -- has exactly the flaw you describe, with exactly the solution you describe.
[disclaimer: haven't read the rest of the post yet]
In my Big Kink Survey I ask if people have ever had an officially-administered IQ test; if they say yes, they get a question where I ask what their score was. The average reported IQ for this is around 128. I forget my sample for this exactly (would have to load up the giant data again which takes forever) but iirc it was around 30k people.
The people taking this are like... US-based young very-online people, mostly coming from tiktok. Not intellectual culture at all.
"Third, it has a ceiling effect: you can’t get more education than being a PhD, so it will underestimate the intelligence of a sample where a significant percent of subjects have PhDs or other advanced degrees."
This is wrong. Professional degree holders tend to be smarter than holders of adjacent PhDs. It's a bit complicated because physics PhDs are obviously smarter than lawyers, but MD>hardsci PhD>Lawyer>humanities/socsci PhD. So you could education norm to higher levels by separating out professional degrees and/or types of PhD.